Fuzzy Double Deep Q-Network-Based Gait Pattern Controller for Humanoid Robots

Tzuu Hseng S. Li, Ping Huan Kuo, Lin Han Chen, Chia Ching Hung, Po Chien Luan, Hao Ping Hsu, Chien Hsin Chang, Yi Ting Hsieh, Wen Hsun Lin

Research output: Contribution to journalArticlepeer-review

9 Citations (Scopus)


In this article, the adaptive-network-based fuzzy inference system (ANFIS) is combined with the double deep Q-network (DDQN) to realize a fuzzy DDQN (FDDQN) such that a humanoid robot can generate a linear inverted pendulum model-based gait pattern in real time. The FDDQN not only allows the humanoid robot to correct the gait pattern instantly but also improves its stability. The proposed scheme is designed and implemented in a toddler-sized humanoid robot called Louis. First, four pressure sensors are installed on the bottom of the sole and one inertial measurement unit is set up on the trunk of the robot. A wireless communication chip is employed to transfer the data to a computer to determine the required parameters for the robot. Next, a control system based on the Linux operating system is developed. The values of the center of pressure and acceleration obtained with the ANFIS are adopted to train the DDQN. The proposed neural network comprises four layers, and the model is cautiously selected to avoid overfitting. The proposed scheme is verified using a robot simulator and then real-time-tested on Louis. The experimental results indicate that the FDDQN can provide the robot timely feedback during walking as well as helps it in adjusting the gait pattern independently. The balancing of the robot through effective dynamic feedback is similar to the balancing ability of an infant learning to walk.

Original languageEnglish
Pages (from-to)147-161
Number of pages15
JournalIEEE Transactions on Fuzzy Systems
Issue number1
Publication statusPublished - 2022 Jan 1

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics


Dive into the research topics of 'Fuzzy Double Deep Q-Network-Based Gait Pattern Controller for Humanoid Robots'. Together they form a unique fingerprint.

Cite this