A new separation measure for improving the effectiveness of validity indices

Shihong Yue, Jeen Shing Wang, Teresa Wu, Huaxiang Wang

研究成果: Article同行評審

27 引文 斯高帕斯(Scopus)

摘要

Many validity indices have been proposed for quantitatively assessing the performance of clustering algorithms. One limitation of existing indices is their lack of generalizability, due to their dependence on the specific algorithms and structures of the data space. To handle large-scale datasets with arbitrary structures, this research study proposes a new cluster separation measure for improving the effectiveness of existing validity indices. This is achieved by partitioning the original data space into a grid-based structure which allows the introduction of a new measurement for assessing the true data distribution between any two clusters instead of the distance between the two cluster prototypes. To validate the effectiveness of the proposed separation measure, we adopt two commonly used validity indices, the Davies-Bouldin's function (DB) and Tibshirani's Gap statistic (GS). These indices are denoted as R-DB-1 and R-GS-1 for clusters with sphere-shaped structures and R-DB-2 and R-GS-2 for irregular-shaped structures. This integration enables the indices to evaluate both partitional algorithms and hierarchical algorithms. Partitional algorithms including C-Means (CM), Fuzzy C-Means (FCM), and hierarchical algorithms, including DBSCAN and CLIQUE, are used to test the performance of the new indices. Two synthetic datasets with spherical structures and four synthetic datasets with irregular shapes are first compared. Five real datasets from the UCI machine learning repository are then used to further test the measure's performance. The experimental results provide evidence that the new indices outperform the original indices.

原文English
頁(從 - 到)748-764
頁數17
期刊Information sciences
180
發行號5
DOIs
出版狀態Published - 2010 三月 1

All Science Journal Classification (ASJC) codes

  • 軟體
  • 控制與系統工程
  • 理論電腦科學
  • 電腦科學應用
  • 資訊系統與管理
  • 人工智慧

指紋

深入研究「A new separation measure for improving the effectiveness of validity indices」主題。共同形成了獨特的指紋。

引用此