Deep Neural Network Based Emotion Recognition System for Humanoid Robot

  • 蔡 定男

Student thesis: Master's Thesis


It is crucial for robots to recognize human emotions during the interaction between human and robot Therefore this thesis proposes an emotion recognition system for a humanoid robot The robot is equipped with a camera in order to capture the image of the user's face and the goal is for the robot to respond appropriately according to the user's emotion which is recognized by our system The emotion recognition system based on a deep neural network learns the six basic emotions including happiness anger disgust fear sadness and surprise The whole structure of the system consists of four steps: the first step takes advantage of a convolutional neural network to extract visual features by learning on a great amount of static images; the second step utilizes a long short-term memory recurrent neural network to figure out the relationship between the transformation of facial expressions in image sequences and the six basic emotions; the third step combines the advantages of both CNN and LSTMs by integrating them into our model; the last step but not least improves the performance of the emotion recognition system by using transfer learning which is a method to transfer the knowledge of related but different problems Finally the performance of the proposed system is verified by leave-one-out cross validation and is compared with other models Then the proposed system is applied to the interaction between human and robot to demonstrate the practicability of this system
Date of Award2018 Jul 18
Original languageEnglish
SupervisorTzuu-Hseng S. Li (Supervisor)

Cite this