Deep Reinforcement Learning–Based IoT Energy Management System for Smart Cities and Green Microgrids

Abstract

The integration of urbanization trends, increased use of renewables, the advent of electric vehicles, and the advent of distributed battery energy storage systems (BESS) are causing considerable operational issues within smart city power systems. Classic rule-based and optimization-based operational energy management strategies face challenges stemming from dynamic demand fluctuations, intermittent renewables, and uncertain power systems which ultimately increases operational costs, peak demand, and carbon emissions. The purpose of this study is to present an IoT Energy Management System (EMS) driven by Deep Reinforcement Learning (DRL) (real-time, autonomous, and adaptive control of distributed energy resources) technology for smart cities and green microgrids. The proposed solution will be tested using an urban microgrid for a 24-hour horizon under changing weather and demand conditions. The urban microgrid will be composed of IoT-enabled smart meters, renewable energy sources, electric vehicle (EV) charging stations, and battery energy storage systems (BESS) interconnected through a centralized cloud-edge infrastructure. As states change in real-time (load demand, renewable generation, state of charge of the battery, and energy price), the Deep Q-Network (DQN) Agent will optimize and learn battery discharge scheduling policies. The reward function of the agent is designed to minimize energy costs, peak demand, and carbon emissions while maximizing the use of renewable energy sources. Results of our experiments show noticeable enhancements over the existing scheduling and rule-based control methods. The DRL system exhibits 23.8\% peak load reduction, 19.6\% operational energy cost, 27.4\% increased utilization of renewable energy, and 21.2\% decrease in CO2 emissions, with a grid reliability equal to or greater than 98\% in supply adequacy. Also, the system is capable of adapting to load changes in less than a second, allowing for real-time flexibility and increasing the system's adaptability and robustness. From the results, it can be concluded that the provided IoT energy management and control system, based on Deep Reinforced Learning, is a smart, flexible, and cost-effective approach for the energy management of future smart grids and green microgrids, making it a valuable asset for the resilient urban energy systems and the sustainable energy future of the planet.