Focus Masking Attention for Abstractive Summarization

Abstract

Text summarization condenses the input text while preserving semantic This natural language processing technique can apply to many fields For example to summarize daily news can rapidly extract key points of news which reduces the reading time for readers Summarization system can be divided to extractive and abstractive summarization All words in generated summary comes from input text for extractive summarization The model will capture critical information from input text and generate output sequence by natural language generation for abstractive summarization Recent research train abstractive summarization via sequence-to-sequence learning with encoder-decoder structure Even though this structure can effectively learn the corresponding relationship between input and output text it still has room for improvement on content selection This thesis tries to add a content selector to sequence-to-sequence model and apply the content selection results to model by “focus masking attention” mechanism The content selector combines the sequence tagging and keywords extraction Focus masking attention reweights the copy attention distribution for decoder to improve results The experiment results show that our approach get effective improvement on both word-level and sentence-level reweighting Combining both levels reweighting improves more than only applying any single level adjustment

Date of Award	2019
Original language	English
Supervisor	Hung-Yu Kao (Supervisor)