Fuzzy Double Deep Q-Network-Based Gait Pattern Controller for Humanoid Robots

In this article, the adaptive-network-based fuzzy inference system (ANFIS) is combined with the double deep Q-network (DDQN) to realize a fuzzy DDQN (FDDQN) such that a humanoid robot can generate a linear inverted pendulum model-based gait pattern in real time. The FDDQN not only allows the humanoid robot to correct the gait pattern instantly but also improves its stability. The proposed scheme is designed and implemented in a toddler-sized humanoid robot called Louis. First, four pressure sensors are installed on the bottom of the sole and one inertial measurement unit is set up on the trunk of the robot. A wireless communication chip is employed to transfer the data to a computer to determine the required parameters for the robot. Next, a control system based on the Linux operating system is developed. The values of the center of pressure and acceleration obtained with the ANFIS are adopted to train the DDQN. The proposed neural network comprises four layers, and the model is cautiously selected to avoid overfitting. The proposed scheme is verified using a robot simulator and then real-time-tested on Louis. The experimental results indicate that the FDDQN can provide the robot timely feedback during walking as well as helps it in adjusting the gait pattern independently. The balancing of the robot through effective dynamic feedback is similar to the balancing ability of an infant learning to walk.