Reliable Accuracy Estimates from k-Fold Cross Validation

Tzu Tsung Wong, Po Yang Yeh

研究成果: Article同行評審

9 引文 斯高帕斯(Scopus)

摘要

It is popular to evaluate the performance of classification algorithms by k-fold cross validation. A reliable accuracy estimate will have a relatively small variance, and several studies therefore suggested to repeatedly perform k-fold cross validation. Most of them did not consider the correlation among the replications of k-fold cross validation, and hence the variance could be underestimated. The purpose of this study is to explore whether k-fold cross validation should be repeatedly performed for obtaining reliable accuracy estimates. The dependency relationships between the predictions of the same instance in two replications of k-fold cross validation are first analyzed for k-nearest neighbors with k= 1k=1. Then, statistical methods are proposed to test the strength of the dependency level between the accuracy estimates resulting from two replications of k-fold cross validation. The experimental results on 20 data sets show that the accuracy estimates obtained from various replications of k-fold cross validation are generally highly correlated, and the correlation will be higher as the number of folds increases. The k-fold cross validation with a large number of folds and a small number of replications should be adopted for performance evaluation of classification algorithms.

原文English
文章編號8698831
頁(從 - 到)1586-1594
頁數9
期刊IEEE Transactions on Knowledge and Data Engineering
32
發行號8
DOIs
出版狀態Published - 2020 八月 1

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics

指紋 深入研究「Reliable Accuracy Estimates from k-Fold Cross Validation」主題。共同形成了獨特的指紋。

引用此