Accuracy is a popular measure for evaluating the performance of classification algorithms tested on ordinary data sets. When a data set is imbalanced, F-measure will be a better choice than accuracy for this purpose. Since F-measure is calculated as the harmonic mean of recall and precision, it is difficult to find the sampling distribution of F-measure for evaluating classification algorithms. Since the values of recall and precision are dependent, their joint distribution is assumed to follow a bivariate normal distribution in this study. When the evaluation method is k-fold cross validation, a linear approximation approach is proposed to derive the sampling distribution of F-measure. This approach is used to design methods for comparing the performance of two classification algorithms tested on single or multiple imbalanced data sets. The methods are tested on ten imbalanced data sets to demonstrate their effectiveness. The weight of recall provided by this linear approximation approach can help us to analyze the characteristics of classification algorithms.
|Number of pages||11|
|Journal||IEEE Transactions on Knowledge and Data Engineering|
|Publication status||Published - 2022 Feb 1|
All Science Journal Classification (ASJC) codes
- Information Systems
- Computer Science Applications
- Computational Theory and Mathematics