Abstract:

In this paper, the authors propose a charging coordination system based on Reinforcement Learning using an artificial neural network as a function approximator. Taking into account the baseload present in the power grid, a central agent creates forward-looking, coordinated charging schedules for an electric vehicle fleet of any size. In contrast to optimization-based charging strategies, system dynamics such as future arrivals, departures, and energy consumption do not have to be known beforehand. The authors implement and compare a range of parameter variants that differ in terms of the reward function and prioritized experience. Subsequently, they use a case study to compare Reinforcement Learning algorithm with several other charging strategies. The Reinforcement Learning-based charging coordination system is shown to perform very well. All electric vehicles have enough energy for their next trip on departure and charging is carried out almost exclusively during the load valleys at night. Compared with an uncontrolled charging strategy, the Reinforcement Learning algorithm reduces the variance of the total load by 65%. The performance of Reinforcement Learning concept comes close to that of an optimization-based charging strategy. However, an optimization algorithm needs to know certain information beforehand, such as the vehicle's departure time and its energy requirement on arriving at the charging station. The novel Reinforcement Learning-based charging coordination system therefore offers a flexible, easily adaptable, and scalable approach for an electric vehicle fleet under realistic operating conditions. The results of this study show that Reinforcement Learning based coordination systems are, especially in real-time applications and environments with many uncertainties, a reasonable and extremely promising alternative to optimization approaches.