Neuroevolution of bipedal locomotion: algorithmic, balance penalty and morphological improvements for improved robustness and performance

Abstract

Bipedalism is theorised to have emerged in humans in order to enable endurance running and tool use via the hands. It is one of the most complex styles of locomotion, with agents typically having a high center of mass and two feet very close together. It is therefore particularly difficult to build smooth cyclic motion on top of this instability.

Existing neuro-evolutionary methods for bipedalism involve the use of a central pattern generator or passive dynamics. A bipedal walker designed by Solomon et al was able to walk on rough terrain with a set of simple linear neural network controllers. I utilise control cost enhancements alongside additional elitism to initialise the walking agents in 3D without the 2D bootstrapping required by the original. The model is shown to be capable of walking in 2D and 3D, and of turning and walking toward a target point.

Improving another leading bipedal system, two fitness-function enhancements are applied to the 3D Humanoid-v1 walking task using a replica of the Salimans et al evolution strategy system. The first enhancement reduces control cost and the second enhancement penalises poor balance. Individually, each enhancement results in improved fitness and life-like gaits. Combining the two enhancements produces gaits that are more robust to noise in their actions according to statistical significance tests.

After producing single-turn behaviour in the previous work, agent fitness in the Solomon et al system is found to be improved according to a statistical signi cance test, by evolving agents alongside morphologies resembling a baby albatross. Pursuit-based turning behaviour is produced in the evolved albatross agents. The agents are required to pursue a target point as it moves further away and back and forth across the x-axis. This task produces bipedal agents capable of making four consecutive turns in pursuit over a short time period.