Economic and policy factors are driving the continuous increase in the adoption and usage of electrical vehicles (EVs). However, despite being a cleaner alternative to combustion engine vehicles, EVs have negative impacts on the lifespan of microgrid equipment and energy balance due to increased power demand and the timing of their usage. In our view grid management should leverage on EVs scheduling flexibility to support local network balancing through active participation in demand response programs. In this paper, we propose a model-free solution, leveraging Deep Q-Learning to schedule the charging and discharging activities of EVs within a microgrid to align with a target energy profile provided by the distribution system operator. We adapted the Bellman Equation to assess the value of a state based on specific rewards for EV scheduling actions and used a neural network to estimate Q-values for available actions and the epsilon-greedy algorithm to balance exploitation and exploration to meet the target energy profile. The results are promising showing that the proposed solution can effectively schedule the EVs charging and discharging actions to align with the target profile with a Person coefficient of 0.99, handling effective EVs scheduling situations that involve dynamicity given by the e-mobility features, relying only on data with no knowledge of EVs and microgrid dynamics.

翻译：经济与政策因素正推动电动汽车采用率和使用量持续增长。然而，尽管作为内燃机车辆的清洁替代方案，电动汽车因电力需求增加和使用时段特征，对微电网设备寿命和能量平衡产生负面影响。我们认为，电网管理应利用电动汽车调度的灵活性，通过积极参与需求响应项目来支持本地网络平衡。本文提出一种无模型解决方案，利用深度Q学习调度微电网内电动汽车的充放电活动，使其与配电系统运营商提供的目标能量曲线对齐。我们改进贝尔曼方程，基于电动汽车调度动作的特定奖励评估状态价值，采用神经网络估计可用动作的Q值，并运用ε-贪婪算法平衡利用与探索以匹配目标能量曲线。结果表明，该方案能有效调度电动汽车充放电动作与目标曲线对齐，皮尔逊系数达0.99，仅依赖数据而不依赖电动汽车和微电网动态先验知识，即可处理电动出行特性带来的动态性调度场景。