Abstract:

This paper proposes a reformulation of a Mixed-Integer Programming model into a Decentralized Markov Decision Process model. It solves using a Multi-Agent Reinforcement Learning algorithm to address the scalability issues of large-scale smart grid systems. The Decentralized Markov Decision Process model uses centralized training and distributed execution: agents are trained using a unique actor network for each agent and a shared critic network. Then, agents execute actions independently from other agents to reduce computation time. The performance of the Multi-Agent Reinforcement Learning model is assessed under different configurations of customers and electric vehicles and compared to the results from deep reinforcement learning and three heuristic algorithms. The simulation results demonstrate that the Multi-Agent Reinforcement Learning algorithm can reduce simulation time significantly compared to deep reinforcement learning, genetic algorithm, particle swarm optimization, and the artificial fish swarm algorithm. The superior performance of the proposed method indicates that it may be a realistic solution for large-scale implementation.