Cao, J, Harrold, D, Fan, Z, Morstyn, T, Healey, D and Li, K (2020) Deep Reinforcement Learning Based Energy Storage Arbitrage With Accurate Lithium-ion Battery Degradation Model. IEEE Transactions on Smart Grid, 11 (5). pp. 4513-4521. ISSN 1949-3053

[thumbnail of final_submitted_energy_storage_arbitrage_using_DRL (7).pdf]
final_submitted_energy_storage_arbitrage_using_DRL (7).pdf - Accepted Version

Download (11MB) | Preview


Accurate estimation of battery degradation cost is one of the main barriers for battery participating on the energy arbitrage market. This paper addresses this problem by using a model-free deep reinforcement learning (DRL) method to optimize the battery energy arbitrage considering an accurate battery degradation model. Firstly, the control problem is formulated as a Markov Decision Process (MDP). Then a noisy network based deep reinforcement learning approach is proposed to learn an optimized control policy for storage charging/discharging strategy. To address the uncertainty of electricity price, a hybrid Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM) model is adopted to predict the price for the next day. Finally, the proposed approach is tested on the the historical UK wholesale electricity market prices. The results compared with model based Mixed Integer Linear Programming (MILP) have demonstrated the effectiveness and performance of the proposed framework.

Item Type: Article
Additional Information: The final version of this article with all relevant information can be found at;
Uncontrolled Keywords: Energy storage, Energy arbitrage, Battery degradation, Deep reinforcement learning, Noisy Networks.
Subjects: T Technology > T Technology (General)
T Technology > TA Engineering (General). Civil engineering (General)
Divisions: Faculty of Natural Sciences > School of Geography, Geology and the Environment
Depositing User: Symplectic
Date Deposited: 22 Jul 2020 12:10
Last Modified: 10 Feb 2021 16:07

Actions (login required)

View Item
View Item