Response Generation in a Dialogue System from Unstructured Data Using Semantic Dependency Pair Model

  • 林 吾軒

Student thesis: Master's Thesis


In recent years as spoken dialogue systems have been successfully applied to personal voice assistant services such as Apple Siri and Google Now spoken language question-answering (QA) systems are becoming the next stage of intelligent human-machine interfaces In the past most of QA systems focused on keyword or sentence matching between the query sentence and the question-answer pairs (QA pairs) However these systems are unable to provide appropriate answers to the query as the query does not appear in the question-answer pair database This thesis presents an augmented dialogue system to eliminate the above problems When the user make a query the system will first try to find the most suitable answer to the user questions from the QA pair set based on QA matching When the system is unable to find a suitable answer the system will use a pre-trained event model and QA semantic dependency pair model to help extract the suitable answer from unstructured documents collected from related websites as the response to the user In unstructured document processing question answering on negative events was selected as the research domain in this thesis We first collected unstructured articles in the chat rooms and discussion boards from psychological consultation websites and segment the articles into fragments based on the event model Supervised Latent Dirichlet allocation and delta Bayesian Information Criterion are employed for event detection and segmentation Second we use the CKIP Probabilistic Context-free Grammar to parse the questions and all the corresponding answer segments to obtain the semantic dependency graph of questions and answers Finally we pair the words and their semantic dependency pair in semantic dependency graph into a set of semantic dependency and create a matrix with the correlation between question semantic dependency and answer semantic dependency extracted from the unstructured data for response comparison The answer segment with the highest matching score is selected as the response to the user query Performance on the proposed method was evaluated K-Fold cross validation and user satisfaction test were employed and comparison to the traditional vector space model was also performed The accuracy of the dialogue system with semantic dependency pair model (SDPM-based system) achieved 98 8% and the accuracy of the traditional vector space model was only 23 4% In user satisfaction test the satisfaction score of the SDPM-based system is higher than the traditional system Experimental results show that the system with semantic dependency pair model obtains better results for response generation and user satisfaction
Date of Award2015 Aug 24
Original languageEnglish
SupervisorChung-Hsien Wu (Supervisor)

Cite this