Linear Approximation of F-Measure for the Performance Evaluation of Classification Algorithms on Imbalanced Data Sets

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

Accuracy is a popular measure for evaluating the performance of classification algorithms tested on ordinary data sets. When a data set is imbalanced, F-measure will be a better choice than accuracy for this purpose. Since F-measure is calculated as the harmonic mean of recall and precision, it is difficult to find the sampling distribution of F-measure for evaluating classification algorithms. Since the values of recall and precision are dependent, their joint distribution is assumed to follow a bivariate normal distribution in this study. When the evaluation method is k-fold cross validation, a linear approximation approach is proposed to derive the sampling distribution of F-measure. This approach is used to design methods for comparing the performance of two classification algorithms tested on single or multiple imbalanced data sets. The methods are tested on ten imbalanced data sets to demonstrate their effectiveness. The weight of recall provided by this linear approximation approach can help us to analyze the characteristics of classification algorithms.

Original languageEnglish
Pages (from-to)753-763
Number of pages11
JournalIEEE Transactions on Knowledge and Data Engineering
Volume34
Issue number2
DOIs
Publication statusPublished - 2022 Feb 1

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics

Fingerprint

Dive into the research topics of 'Linear Approximation of F-Measure for the Performance Evaluation of Classification Algorithms on Imbalanced Data Sets'. Together they form a unique fingerprint.

Cite this