The transition toward sustainable energy on a global scale is hindered by the intermittent nature of renewables and the high costs of achieving energy independence for off-grid commercial systems. This study develops a deep reinforcement learning (DRL) solution for the optimal management of a hybrid hydrogen-battery energy storage system in a microgrid based on renewables. Taking a study-based approach, drawing on a case study from a commercial hospitality center, the control challenge was modelled as a Markov Decision Process. For the implementation, the Soft Actor-Critic (SAC) learning algorithm was used to regulate power transitions among photovoltaic generation units, batteries, and a hydrogen chain comprising electrolyzers, storage, and fuel cells. The SAC learning-based controller was constructed for minimizing the cost of operations through a reward function that takes into account the degradation costs of batteries as well as the transition costs for seamless operational changes. The SAC-learning-based controller was validated on a high-resolution dataset for a year and was compared with a rule-based controller. It was found that the SAC controller is capable of reducing costs by 2.0%, producing a much smoother transition in power, as well as more intelligent usage of the two storage systems. These findings pave the way for exploration of the DRL for a reliable and affordable transition toward commercial microgrids.

Energy management, Deep Reinforcement learning, Hybrid energy storage, Hydrogen, Battery, Microgrid, Soft actor-critic algorithm

[1] M.S. Su’ait, N.A. Ludin, K. Sopian, Introduction to Renewable Energy Technologies and Energy Transition Strategies. ACS symposium Series, 1449, 2025: 1–14. https://doi.org/10.1021/bk-2025-1499.ch001
[2] S. Bielecki, T. Skoczkowski, M. Wołowicz, Microgrids as a Tool for Energy Self-Sufficiency. Sensors, 25(21), 2025: 6707. https://doi.org/10.3390/s25216707
[3] P. Cosgrove, T. Roulstone, S. Zachary, Intermittency and Periodicity in Net-Zero Renewable Energy Systems with Storage. Renewable Energy, 212, 2023: 299–307. https://doi.org/10.1016/j.renene.2023.04.135
[4] P. Parvizi, M. Jalilian, A.M. Amidi, M.R. Zangeneh, J.R. Riba, From Present Innovations to Future Potential: The Promising Journey of Lithium-Ion Batteries. Micromachines, 16(2), 2025: 194. https://doi.org/10.3390/mi16020194
[5] R. Areola, A.A. Adebiyi, K. Moloi, Integrated Energy Storage Systems for Enhanced Grid Efficiency: A Comprehensive Review of Technologies and Applications. Energies, 18(7), 2025: 1848. https://doi.org/10.3390/en18071848
[6] R.Y. Abdallah, M.F. Shaaban, A.H. Osman, A. Ali, K. Obaideen, L. Albasha, Synergizing Gas and Electric Systems Using Power-To-Hydrogen: Integrated Solutions for Clean and Sustainable Energy Networks. Smart Cities, 8(3), 2025: 81.
https://doi.org/10.3390/smartcities8030081
[7] Y. Zhao, H. Li, C. Wan, D. Du, B. Chen, Microgrid Energy Management Considering Energy Storage Degradation Cost. Batteries, 11(5), 2025: 169. https://doi.org/10.3390/batteries11050169
[8] M.S. Endiz, G. Gökkuş, A.E. Coşgun, H. Demir, A Review of Traditional and Advanced MPPT Approaches for PV Systems Under Uniformly Insolation and Partially Shaded Conditions. Applied Sciences, 15(3), 2025: 1031.
https://doi.org/10.3390/app15031031
[9] H. Liu, H. Wang, M. Yu, Y. Wang, Y. Luo, Long Short-Term Memory–Model Predictive Control Speed Prediction-Based Double Deep Q-Network Energy Management for Hybrid Electric Vehicle to Enhanced Fuel Economy. Sensors, 25(9), 2025: 2784. https://doi.org/10.3390/s25092784
[10] L. Jia, Y. Pei, Recent advances in Multi-Agent Reinforcement Learning for Intelligent Automation and Control of Water Environment Systems. Machines, 13(6), 2025: 503. https://doi.org/10.3390/machines13060503
[11] G. Muriithi, S. Chowdhury, Deep Q‐network Application for Optimal Energy Management in a Grid‐Tied Solar PV‐Battery Microgrid. The Journal of Engineering, 2022(4), 2022: 422–441. https://doi.org/10.1049/tje2.12128
[12] A. Bäumler, A. Benterki, J. Meng, T. Azib, M. Boukhnifer, Energy Management Strategy based on Reinforcement Learning for Fuel Cell Hybrid Vehicle with a New Reward Function Approach. 2023 IEEE Vehicle Power and Propulsion Conference, 24-27 October 2023, Milan, Italy, pp.1–6.
[13] E. Mohammadi, D. Ortiz-Arroyo, A.A. Hansen, M. Stokholm-Bjerregaard, S. Gros, A.S. Anand, P. Durdevic. Application of Soft Actor-Critic Algorithms in Optimizing Wastewater Treatment with Time Delays Integration. Expert Systems with Applications, 277, 2025: 127180. https://doi.org/10.1016/j.eswa.2025.127180
[14] X. Lü, S. Li, X. He, C. Xie, S. He, Y. Xu, J. Fang, M. Zhang, X. Yang, Hybrid Electric Vehicles: A Review of Energy Management Strategies Based on Model Predictive Control. Journal of Energy Storage, 56, 2022: 106112.
https://doi.org/10.1016/j.est.2022.106112
[15] A. Dosa, O.A. Olanrewaju, F. Mora-Camino, A Comprehensive Review of Hybrid Renewable Microgrids: Key Design Parameters, Optimization Techniques, and the Role of Demand Response in Enhancing System Flexibility. Energies, 18(19), 2025: 5154. https://doi.org/10.3390/en18195154
[16] M.A.P. Ramos, A. Busboom, Systematic Review of Reinforcement Learning in Process Industries: A Contextual and Taxonomic Approach. Applied Sciences, 15(24), 2025: 12904. https://doi.org/10.3390/app152412904
[17] M. Aatabe, W. Jenkal, M.I. Mosaad, S.A. Hussien, Stochastic Control for Sustainable Hydrogen Generation in Standalone PV–Battery–PEM Electrolyzer Systems. Energies, 18(15), 2025: 3899. https://doi.org/10.3390/en18153899
[18] X. Liu, H. Qi, S. Jia, Y. Guo, Y. Liu, Recent Advances in Optimization Methods for Machine Learning: A Systematic Review. Mathematics, 13(13), 2025: 2210. https://doi.org/10.3390/math13132210
[19] P. Michailidis, I. Michailidis, F. Minelli, H.H. Coban, E. Kosmatopoulos, Model Predictive Control for Smart Buildings: Applications and Innovations in Energy Management. Buildings, 15(18), 2025: 3298.
https://doi.org/10.3390/buildings15183298
[20] J. Xu, Efficiency-Oriented Model Predictive Control: A Novel MPC Strategy to Optimize the Global Process Performance. Sensors, 24(17), 2024: 5732. https://doi.org/10.3390/s24175732
[21] Z. Zhang, Z. Wu, D. Rincon, P. Christofides, Real-Time Optimization and Control of Nonlinear Processes Using Machine Learning. Mathematics, 7(10), 2019: 890. https://doi.org/10.3390/math7100890
[22] M. Gautam, Deep Reinforcement Learning for Resilient Power and Energy Systems: Progress, Prospects, and Future Avenues. Electricity, 4(4), 2023: 336–380. https://doi.org/10.3390/electricity4040020
[23] S. Meziane, T. Ryad, Y.O. Assolami, T.M. Aljohani, A Hierarchical Two-Layer MPC-Supervised strategy for Efficient Inverter-Based Small Microgrid Operation. Sustainability, 17(19), 2025: 8729. https://doi.org/10.3390/su17198729
[24] S. Dai, P. Shen, W. Deng, Q. Yu, Hydrogen Energy in Electrical Power Systems: A Review and Future Outlook. Electronics, 13(17), 2024: 3370. https://doi.org/10.3390/electronics13173370
[25] D. Deltetto, D. Coraci, G. Pinto, M. S. Piscitelli, A. Capozzoli, Exploring the Potentialities of Deep Reinforcement Learning for Incentive-Based Demand Response in a Cluster of Small Commercial Buildings. Energies, 14(10), 2021: 2933.
https://doi.org/10.3390/en14102933
[26] A.V. Waghmare, V.P. Singh, T. Varshney, P. Saanjevikumar, A Systematic Review of Reinforcement Learning-Based Control for Microgrids: Trends, Challenges, and Emerging Algorithms. Discover Applied Sciences, 7, 2025: 939.
https://doi.org/10.1007/s42452-025-07529-6
[27] J. Ye, X. Wang, Q. Hua, L. Sun, Deep Reinforcement Learning Based Energy Management of a Hybrid Electricity-Heat-Hydrogen Energy System with Demand Response. Energy, 305, 2024: 131874. https://doi.org/10.1016/j.energy.2024.131874
[28] M.H. Alabdullah, M.A. Abido, Microgrid Energy Management using Deep Q-Network Reinforcement Learning. Alexandria Engineering Journal, 61(11), 2022: 9069–9078. https://doi.org/10.1016/j.aej.2022.02.042
[29] T. Liang, L. Chai, X. Cao, J. Tan, Y. Jing, L. Lv, Real-time Optimization of Large-Scale Hydrogen Production Systems Using Off-Grid Renewable Energy: Scheduling Strategy Based on Deep Reinforcement Learning. Renewable Energy, 224, 2024: 120177. https://doi.org/10.1016/j.renene.2024.120177
[30] Y. Zheng, J. Jia, D. An, Energy management for Microgrids with Hybrid Hydrogen-Battery Storage: A Reinforcement Learning Framework Integrated Multi-Objective Dynamic Regulation. Processes, 13(8), 2025: 2558.
https://doi.org/10.3390/pr13082558
[31] W. Zhu, S. Wen, Q. Zhao, B. Zhang, Y. Huang, M. Zhu, Deep Reinforcement Learning Based Optimal Operation of Low-Carbon Island Microgrid with High Renewables and Hybrid Hydrogen–Energy Storage System. Journal of Marine Science and Engineering, 13(2), 2025: 225. https://doi.org/10.3390/jmse13020225
[32] K. Kumar, S. Kwon, S. Bae, Deep Reinforcement Learning-Based Control Strategy for Integration of a Hybrid Energy Storage System in Microgrid. Journal of Energy Storage, 108(13), 2025: 114936. https://doi.org/10.1016/j.est.2024.114936