Harrold, DJB, Cao, J and Fan, Z (2022) Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning. Energy, 238 (Part C). ISSN 0360-5442

[thumbnail of 2106.06061v1.pdf] Text
2106.06061v1.pdf - Accepted Version
Restricted to Repository staff only until 8 September 2023.
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (1MB)


As the world seeks to become more sustainable, intelligent solutions are needed to increase the penetration of renewable energy. In this paper, the model-free deep reinforcement learning algorithm Rainbow Deep Q-Networks is used to control a battery in a small microgrid to perform energy arbitrage and more efficiently utilise solar and wind energy sources. The grid operates with its own demand and renewable generation based on a dataset collected at Keele University, as well as using dynamic energy pricing from a real wholesale energy market. Four scenarios are tested including using demand and price forecasting produced with local weather data. The algorithm and its subcomponents are evaluated against two continuous control benchmarks with Rainbow able to outperform all other method. This research shows the importance of using the distributional approach for reinforcement learning when working with complex environments and reward functions, as well as how it can be used to visualise and contextualise the agent's behaviour for real-world applications.

Item Type: Article
Additional Information: The final version of this article and all relevant information related to it, including copyrights, can be found online at; https://www.sciencedirect.com/science/article/pii/S0360544221022064?via%3Dihub
Uncontrolled Keywords: Actor-critic methods; Deep Q-Networks; Demand response; Microgrids; Renewable energy
Subjects: Q Science > Q Science (General)
T Technology > T Technology (General)
Divisions: Faculty of Natural Sciences > School of Computing and Mathematics
Related URLs:
Depositing User: Symplectic
Date Deposited: 19 Oct 2021 11:27
Last Modified: 30 Nov 2021 14:59
URI: https://eprints.keele.ac.uk/id/eprint/10153

Actions (login required)

View Item
View Item