Distributed Optimization for Distribution Grids With Stochastic DER Using Multi-Agent Deep Reinforcement Learning

Abstract

This article develops a special decomposition methodology for the traditional optimal power flow which facilitates optimal integration of stochastic distributed energy resources in power distribution systems. The resulting distributed optimal power flow algorithm reduces the computational complexity of the conventional linear programming approach while avoiding the challenges associated with the stochastic nature of the energy resources and loads. It does so using machine learning algorithms employed for two crucial tasks. First, two proposed algorithms, Dynamic Distributed Multi-Microgrid and Monte Carlo Tree Search based Reinforcement Learning, constitute dynamic microgrids of network nodes to confirm the electric power transaction optimality. Second, the optimal distributed energy resources are obtained by the proposed deep reinforcement learning method named Multi Leader-Follower Actors under Centralized Critic. It accelerates conventional linear programming approach by considering a reduced set of resources and their constraints. The proposed method is demonstrated through a real-time balancing electricity market constructed over the IEEE 123-bus system and enhanced using price signals based on distribution locational marginal prices. This application clearly shows the ability of the new approach to effectively coordinate multiple distribution system entities while maintaining system security constraints.