Abstract
As cross-lingual information retrieval is attracting increasing attention, tools that measure cross-lingual semantic similarity between documents are becoming desirable. In this paper, two aspects of cross-lingual semantic document similarity measures are investigated: One is document representation, and the other is the formulation of similarity measures. Fuzzy set and rough set theories are applied to capture the inherently fuzzy relationships among concepts expressed by natural languages. Our approach first develops a language-independent sense-level document representation based on the fuzzy set model to reduce the barrier between different languages and further explores the fuzzyrough hybrid approach to obtain a more robust macrosense-level document representation through the partitioning of the integrated sense association network of the document collection into macrosenses. Then, Tverskys notion of similarity and the F1 measure on information retrieval are adopted to formulate, respectively, two document similarity measures with fuzzy set operations on the two proposed document representations. The effectiveness of our approach is demonstrated by its success rate in identifying the English translations to their corresponding Chinese documents in a collection of ChineseEnglish parallel documents. Moreover, the proposed approach can be easily extended to process documents in other languages. It is believed that the proposed representations, along with the similarity measures, will enable more effective text mining processes.
Original language | English |
---|---|
Article number | 5549886 |
Pages (from-to) | 1098-1111 |
Number of pages | 14 |
Journal | IEEE Transactions on Fuzzy Systems |
Volume | 18 |
Issue number | 6 |
DOIs | |
Publication status | Published - 2010 Dec 1 |
All Science Journal Classification (ASJC) codes
- Control and Systems Engineering
- Computational Theory and Mathematics
- Artificial Intelligence
- Applied Mathematics