Harrold, DJB, Cao, J and Fan, Z (2022) Renewable energy integration and microgrid energy trading using multi-agent deep reinforcement learning. Applied Energy, 318 (119151). ISSN 0306-2619

[thumbnail of 1-s2.0-S0306261922005256-main.pdf]
1-s2.0-S0306261922005256-main.pdf - Published Version

Download (5MB) | Preview


To reduce global greenhouse gas emissions, the world must find intelligent solutions to maximise the utilisation of carbon-free renewable energy sources. In this paper, multi-agent reinforcement learning is used to control a microgrid in a mixed cooperative and competitive setting. The agents observe fluctuating energy demand, dynamic wholesale energy prices, and intermittent renewable energy sources to control a hybrid energy storage system to maximise the utilisation of the renewables to reduce the energy costs of the grid. In addition, an aggregator agent trades with external microgrids competing against one another and the aggregator to reduce their own energy bills. For this, the algorithm deep deterministic policy gradients (DDPG) and multi-agent DDPG (MADDPG) are used to compare the use of a single global controller versus multiple distributed agents, along with the single and multi-agent variants of distributional DDPG (D3PG) and twin delayed DDPG (TD3). The research found it is significantly more profitable for the primary microgrid to sell energy on its own terms rather than selling back to the utility grid, and is also beneficial for the external microgrids as they also reduce their own energy bills. The methods that produced the greatest profits were the multi-agent approaches where each agent has its own reward function based on the principle of marginal contribution from game theory. The multi-agent approaches were better able to evaluate their performance controlling individual components of the environment which allowed them to develop their own unique policies for the different types of energy storage system.

Item Type: Article
Additional Information: © 2022 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
Uncontrolled Keywords: Deep deterministic policy gradients; Energy arbitrage; Hybrid energy storage systems; Multi-agent systems; Renewable energy
Subjects: T Technology > T Technology (General)
T Technology > TD Environmental technology. Sanitary engineering
Depositing User: Symplectic
Date Deposited: 31 May 2022 14:44
Last Modified: 21 Jul 2022 12:37
URI: https://eprints.keele.ac.uk/id/eprint/10979

Actions (login required)

View Item
View Item