Cross-lingual document representation and semantic similarity measure: A fuzzy set and rough set based approach

Hsun Hui Huang, Yau Hwang Kuo

Research output: Contribution to journalArticlepeer-review

36 Citations (Scopus)

Abstract

As cross-lingual information retrieval is attracting increasing attention, tools that measure cross-lingual semantic similarity between documents are becoming desirable. In this paper, two aspects of cross-lingual semantic document similarity measures are investigated: One is document representation, and the other is the formulation of similarity measures. Fuzzy set and rough set theories are applied to capture the inherently fuzzy relationships among concepts expressed by natural languages. Our approach first develops a language-independent sense-level document representation based on the fuzzy set model to reduce the barrier between different languages and further explores the fuzzyrough hybrid approach to obtain a more robust macrosense-level document representation through the partitioning of the integrated sense association network of the document collection into macrosenses. Then, Tverskys notion of similarity and the F1 measure on information retrieval are adopted to formulate, respectively, two document similarity measures with fuzzy set operations on the two proposed document representations. The effectiveness of our approach is demonstrated by its success rate in identifying the English translations to their corresponding Chinese documents in a collection of ChineseEnglish parallel documents. Moreover, the proposed approach can be easily extended to process documents in other languages. It is believed that the proposed representations, along with the similarity measures, will enable more effective text mining processes.

Original languageEnglish
Article number5549886
Pages (from-to)1098-1111
Number of pages14
JournalIEEE Transactions on Fuzzy Systems
Volume18
Issue number6
DOIs
Publication statusPublished - 2010 Dec 1

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Cross-lingual document representation and semantic similarity measure: A fuzzy set and rough set based approach'. Together they form a unique fingerprint.

Cite this