Deep Learning in HCI: Case Studies of 3D Content Recommendation and Gesture Interaction

  • 潘 則佑

Student thesis: Doctoral Thesis


Computers have become indispensable in our daily life and Human-Computer Interaction (HCI) which studies how people interact with computers has drawn a lot of attention of researchers Nowadays many researches focus on two topics in HCI i e how human use hand gestures as input to intuitively interact with computers and what kinds of visual content can provide better human experience In this dissertation we dig into these two research topics by the Double Diamond design thinking method To be more precise we discover real world problems define the target applications develop the corresponding methodologies and deliver the systems to the target user Two reliable gesture interaction models are introduced and validated with a sports referee signal training application Moreover a 3D content recommendation method is proposed to provide harmonic visual experience for the user Human gestures can be generally divided into two categories i e large motion gestures and subtle motion gestures In the past most HCI systems utilize only large motion gestures or only subtle motion gestures Moreover since it is difficult to develop a robust gesture recognition method most systems merely provide few kinds of simple gestures for interaction To provide more intuitive and various gestures for interaction we investigate how to develop a robust recognition model for wearable hybrid and multi-channel sensors by using deep learning technology The first part of this dissertation will introduce two gesture recognition models which are designed based on Deep Belief Networks (DBN) Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) We apply the two proposed models to recognize the sports referee signals which involve both large motion and subtle motion gestures and design a real-time system for sports referee training Visual perception provides vital clues for human cognition which highly affects the user experience in HCI systems In applications of augmented/virtual reality 3D content with harmonic visual quality will bring remarkable user experience In the second part of this dissertation we propose a 3D content recommendation method based on Triplet CNN which can evaluate the compatibility of each pair of 3D models and give proper suggestions for the content editor We take 3D furniture recommendation system as an example to evaluate the proposed style-based Triplet CNN model
Date of Award2018 Aug 20
Original languageEnglish
SupervisorMin-Chun Hu (Supervisor)

Cite this