TY - GEN
T1 - Improved kNN rule for small training sets
AU - Cheamanunkul, Sunsern
AU - Freund, Yoav
N1 - Publisher Copyright:
© 2014 IEEE.
PY - 2014/2/5
Y1 - 2014/2/5
N2 - The traditional k-NN classification rule predicts a label based on the most common label of the k nearest neighbors (the plurality rule). It is known that the plurality rule is optimal when the number of examples tends to infinity. In this paper we show that the plurality rule is sub-optimal when the number of labels is large and the number of examples is small. We propose a simple k-NN rule that takes into account the labels of all of the neighbors, rather than just the most common label. We present a number of experiments on both synthetic datasets and real-world datasets, including MNIST and SVHN. We show that our new rule can achieve lower error rates compared to the majority rule in many cases.
AB - The traditional k-NN classification rule predicts a label based on the most common label of the k nearest neighbors (the plurality rule). It is known that the plurality rule is optimal when the number of examples tends to infinity. In this paper we show that the plurality rule is sub-optimal when the number of labels is large and the number of examples is small. We propose a simple k-NN rule that takes into account the labels of all of the neighbors, rather than just the most common label. We present a number of experiments on both synthetic datasets and real-world datasets, including MNIST and SVHN. We show that our new rule can achieve lower error rates compared to the majority rule in many cases.
UR - https://www.scopus.com/pages/publications/84946691480
U2 - 10.1109/ICMLA.2014.37
DO - 10.1109/ICMLA.2014.37
M3 - Conference contribution
AN - SCOPUS:84946691480
T3 - Proceedings - 2014 13th International Conference on Machine Learning and Applications, ICMLA 2014
SP - 201
EP - 206
BT - Proceedings - 2014 13th International Conference on Machine Learning and Applications, ICMLA 2014
A2 - Chen, Xue-wen
A2 - Qu, Guangzhi
A2 - Angelov, Plamen
A2 - Ferri, Cesar
A2 - Lai, Jian-Huang
A2 - Wani, M. Arif
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 13th International Conference on Machine Learning and Applications, ICMLA 2014
Y2 - 3 December 2014 through 6 December 2014
ER -