LDA based semi-supervised learning from streaming short text

Ji De Chen, Hung Yu Kao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

With the rapidly growing of real-time social media, like Twitter, many users share and discuss their interest topics through such platforms. Hashtag is a type of metadata tag which allows users to annotate their topics of tweets. For research usage, for example, hashtags can help the performance of event detection by observing the trend of hashtags. Although Twitter grows rapidly, hashtag growth is not as expected. Our dataset shows that there are less than 20% of all tweets containing hashtags. We think that it is caused by that most users may have no idea what hashtags are suitable for tweets they post. If we can recommend suitable hashtags to users, it can be one of the solutions to solve the problem of low usage rate of hashtag. Hashtag recommendation belongs to supervised learning problem. More labeled data for training the learning model can get higher performance in prediction. However, labeled data in hashtag recommendation is not so much due to low usage rate of hashtag. Thus, we want to exploit unlabeled data, i.e. non-hashtag tweets, to solve this problem. Now we have large amount of unlabeled data, but directly adding all non-hashtag tweets may not be helpful to train the model. To overcome this issue, we apply the weight-updating mechanisms to filter out the useless parts of non-hashtag tweets. These mechanisms also have to consider the temporal characteristics of hashtag due to the real-time nature of Twitter. The experimental results in this research show that adding non-hashtag tweets to extend original training data outperforms baseline methods which only exploit labeled data to train the model.

Original languageEnglish
Title of host publicationProceedings of the 2015 IEEE International Conference on Data Science and Advanced Analytics, DSAA 2015
EditorsGabriella Pasi, James Kwok, Osmar Zaiane, Patrick Gallinari, Eric Gaussier, Longbing Cao
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781467382731
DOIs
Publication statusPublished - 2015 Dec 2
EventIEEE International Conference on Data Science and Advanced Analytics, DSAA 2015 - Paris, France
Duration: 2015 Oct 192015 Oct 21

Publication series

NameProceedings of the 2015 IEEE International Conference on Data Science and Advanced Analytics, DSAA 2015

Other

OtherIEEE International Conference on Data Science and Advanced Analytics, DSAA 2015
CountryFrance
CityParis
Period15-10-1915-10-21

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Information Systems and Management
  • Information Systems

Fingerprint Dive into the research topics of 'LDA based semi-supervised learning from streaming short text'. Together they form a unique fingerprint.

Cite this