Dialog State Tracking and action selection using deep learning mechanism for interview coaching

Ming Hsiang Su, Kun Yi Huang, Tsung Hsien Yang, Kuan Jung Lai, Chung-Hsien Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

The best way to prepare for an interview is to review the different types of possible interview questions you will be asked during an interview and practice responding to questions. An interview coaching system tries to simulate an interviewer to provide mock interview practice simulation sessions for the users. The traditional interview coaching systems provide some feedbacks, including facial preference, head nodding, response time, speaking rate, and volume, to let users know their own performance in the mock interview. But most of these systems are trained with insufficient dialog data and provide the pre-designed interview questions. In this study, we propose an approach to dialog state tracking and action selection based on deep learning methods. First, the interview corpus in this study is collected from 12 participants, and is annotated with dialog states and actions. Next, a long-short term memory and an artificial neural network are employed to predict dialog states and the Deep RL is adopted to learn the relation between dialog states and actions. Finally, the selected action is used to generate the interview question for interview practice. To evaluate the proposed method in action selection, an interview coaching system is constructed. Experimental results show the effectiveness of the proposed method for dialog state tracking and action selection.

Original languageEnglish
Title of host publicationProceedings of the 2016 International Conference on Asian Language Processing, IALP 2016
EditorsMinghui Dong, Chung-Hsien Wu, Yanfeng Lu, Haizhou Li, Yuen-Hsien Tseng, Liang-Chih Yu, Lung-Hao Lee
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages6-9
Number of pages4
ISBN (Electronic)9781509009213
DOIs
Publication statusPublished - 2017 Mar 10
Event20th International Conference on Asian Language Processing, IALP 2016 - Tainan, Taiwan
Duration: 2016 Nov 212016 Nov 23

Publication series

NameProceedings of the 2016 International Conference on Asian Language Processing, IALP 2016

Other

Other20th International Conference on Asian Language Processing, IALP 2016
CountryTaiwan
CityTainan
Period16-11-2116-11-23

Fingerprint

coaching
dialogue
Neural networks
Feedback
interview
learning
Deep learning
Long short-term memory
learning method
neural network
speaking

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Linguistics and Language
  • Artificial Intelligence

Cite this

Su, M. H., Huang, K. Y., Yang, T. H., Lai, K. J., & Wu, C-H. (2017). Dialog State Tracking and action selection using deep learning mechanism for interview coaching. In M. Dong, C-H. Wu, Y. Lu, H. Li, Y-H. Tseng, L-C. Yu, & L-H. Lee (Eds.), Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016 (pp. 6-9). [7875922] (Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IALP.2016.7875922
Su, Ming Hsiang ; Huang, Kun Yi ; Yang, Tsung Hsien ; Lai, Kuan Jung ; Wu, Chung-Hsien. / Dialog State Tracking and action selection using deep learning mechanism for interview coaching. Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016. editor / Minghui Dong ; Chung-Hsien Wu ; Yanfeng Lu ; Haizhou Li ; Yuen-Hsien Tseng ; Liang-Chih Yu ; Lung-Hao Lee. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 6-9 (Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016).
@inproceedings{5f3f3341217147ed8a3ffe67739e8ebb,
title = "Dialog State Tracking and action selection using deep learning mechanism for interview coaching",
abstract = "The best way to prepare for an interview is to review the different types of possible interview questions you will be asked during an interview and practice responding to questions. An interview coaching system tries to simulate an interviewer to provide mock interview practice simulation sessions for the users. The traditional interview coaching systems provide some feedbacks, including facial preference, head nodding, response time, speaking rate, and volume, to let users know their own performance in the mock interview. But most of these systems are trained with insufficient dialog data and provide the pre-designed interview questions. In this study, we propose an approach to dialog state tracking and action selection based on deep learning methods. First, the interview corpus in this study is collected from 12 participants, and is annotated with dialog states and actions. Next, a long-short term memory and an artificial neural network are employed to predict dialog states and the Deep RL is adopted to learn the relation between dialog states and actions. Finally, the selected action is used to generate the interview question for interview practice. To evaluate the proposed method in action selection, an interview coaching system is constructed. Experimental results show the effectiveness of the proposed method for dialog state tracking and action selection.",
author = "Su, {Ming Hsiang} and Huang, {Kun Yi} and Yang, {Tsung Hsien} and Lai, {Kuan Jung} and Chung-Hsien Wu",
year = "2017",
month = "3",
day = "10",
doi = "10.1109/IALP.2016.7875922",
language = "English",
series = "Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "6--9",
editor = "Minghui Dong and Chung-Hsien Wu and Yanfeng Lu and Haizhou Li and Yuen-Hsien Tseng and Liang-Chih Yu and Lung-Hao Lee",
booktitle = "Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016",
address = "United States",

}

Su, MH, Huang, KY, Yang, TH, Lai, KJ & Wu, C-H 2017, Dialog State Tracking and action selection using deep learning mechanism for interview coaching. in M Dong, C-H Wu, Y Lu, H Li, Y-H Tseng, L-C Yu & L-H Lee (eds), Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016., 7875922, Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016, Institute of Electrical and Electronics Engineers Inc., pp. 6-9, 20th International Conference on Asian Language Processing, IALP 2016, Tainan, Taiwan, 16-11-21. https://doi.org/10.1109/IALP.2016.7875922

Dialog State Tracking and action selection using deep learning mechanism for interview coaching. / Su, Ming Hsiang; Huang, Kun Yi; Yang, Tsung Hsien; Lai, Kuan Jung; Wu, Chung-Hsien.

Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016. ed. / Minghui Dong; Chung-Hsien Wu; Yanfeng Lu; Haizhou Li; Yuen-Hsien Tseng; Liang-Chih Yu; Lung-Hao Lee. Institute of Electrical and Electronics Engineers Inc., 2017. p. 6-9 7875922 (Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Dialog State Tracking and action selection using deep learning mechanism for interview coaching

AU - Su, Ming Hsiang

AU - Huang, Kun Yi

AU - Yang, Tsung Hsien

AU - Lai, Kuan Jung

AU - Wu, Chung-Hsien

PY - 2017/3/10

Y1 - 2017/3/10

N2 - The best way to prepare for an interview is to review the different types of possible interview questions you will be asked during an interview and practice responding to questions. An interview coaching system tries to simulate an interviewer to provide mock interview practice simulation sessions for the users. The traditional interview coaching systems provide some feedbacks, including facial preference, head nodding, response time, speaking rate, and volume, to let users know their own performance in the mock interview. But most of these systems are trained with insufficient dialog data and provide the pre-designed interview questions. In this study, we propose an approach to dialog state tracking and action selection based on deep learning methods. First, the interview corpus in this study is collected from 12 participants, and is annotated with dialog states and actions. Next, a long-short term memory and an artificial neural network are employed to predict dialog states and the Deep RL is adopted to learn the relation between dialog states and actions. Finally, the selected action is used to generate the interview question for interview practice. To evaluate the proposed method in action selection, an interview coaching system is constructed. Experimental results show the effectiveness of the proposed method for dialog state tracking and action selection.

AB - The best way to prepare for an interview is to review the different types of possible interview questions you will be asked during an interview and practice responding to questions. An interview coaching system tries to simulate an interviewer to provide mock interview practice simulation sessions for the users. The traditional interview coaching systems provide some feedbacks, including facial preference, head nodding, response time, speaking rate, and volume, to let users know their own performance in the mock interview. But most of these systems are trained with insufficient dialog data and provide the pre-designed interview questions. In this study, we propose an approach to dialog state tracking and action selection based on deep learning methods. First, the interview corpus in this study is collected from 12 participants, and is annotated with dialog states and actions. Next, a long-short term memory and an artificial neural network are employed to predict dialog states and the Deep RL is adopted to learn the relation between dialog states and actions. Finally, the selected action is used to generate the interview question for interview practice. To evaluate the proposed method in action selection, an interview coaching system is constructed. Experimental results show the effectiveness of the proposed method for dialog state tracking and action selection.

UR - http://www.scopus.com/inward/record.url?scp=85017260844&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85017260844&partnerID=8YFLogxK

U2 - 10.1109/IALP.2016.7875922

DO - 10.1109/IALP.2016.7875922

M3 - Conference contribution

T3 - Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016

SP - 6

EP - 9

BT - Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016

A2 - Dong, Minghui

A2 - Wu, Chung-Hsien

A2 - Lu, Yanfeng

A2 - Li, Haizhou

A2 - Tseng, Yuen-Hsien

A2 - Yu, Liang-Chih

A2 - Lee, Lung-Hao

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Su MH, Huang KY, Yang TH, Lai KJ, Wu C-H. Dialog State Tracking and action selection using deep learning mechanism for interview coaching. In Dong M, Wu C-H, Lu Y, Li H, Tseng Y-H, Yu L-C, Lee L-H, editors, Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016. Institute of Electrical and Electronics Engineers Inc. 2017. p. 6-9. 7875922. (Proceedings of the 2016 International Conference on Asian Language Processing, IALP 2016). https://doi.org/10.1109/IALP.2016.7875922