Abstract:

This paper presents a performance analysis of reinforcement learning-based energy management strategy (EMS) in plug-in hybrid electric vehicles (PHEV) and fuel cell electric vehicles (FCEV), highlighting the promising nature of this approach in managing energy in electric vehicles with multiple power sources, despite the lack of systematic examination of its essential elements in previous studies. The performance analysis is developed in four aspects: algorithm, perception and decision granularity, hyperparameters, and reward function. The results show that the off-policy algorithm effectively develops a more fuel-efficient solution within the complete driving cycle compared with other algorithms. Improving perception and decision granularity reduces the frequency of tabular-based policy updates but better balances battery power and fuel consumption. Setting a high initial SOC in training will effectively improve the performance of RL-based EMS. The construction of an equivalent energy loss reward function for RL-based EMS based on instantaneous state of charge (SOC) variation should be approached with caution. This approach is highly sensitive to parameters and is more likely to lead to violations of SOC constraints. In contrast, an equivalent energy reward function based on overall SOC variation is a safer alternative.