TY - GEN
T1 - Minimum risk neural networks and weight decay technique
AU - Yeh, I. Cheng
AU - Tseng, Pei Yen
AU - Huang, Kuan Chieh
AU - Kuo, Yau Hwang
PY - 2012/8/20
Y1 - 2012/8/20
N2 - To enhance the generalization of neural network model, we proposed a novel neural network, Minimum Risk Neural Networks (MRNN), whose principle is the combination of minimizing the sum of squares of error and maximizing the classification margin, based on the principle of structural risk minimization. Therefore, the objective function of MRNN is the combination of the sum of squared error and the sum of squares of the slopes of the classification function. Besides, we derived a more sophisticated formula similar to the traditional weight decay technique from the MRNN, establishing a more rigorous theoretical basis for the technique. This study employed several real application examples to test the MRNN. The results led to the following conclusions. (1) As long as the penalty coefficient was in the appropriate range, MRNN performed better than pure MLP. (2) MRNN may perform better in difficult classification problems than MLP using weight decay technique.
AB - To enhance the generalization of neural network model, we proposed a novel neural network, Minimum Risk Neural Networks (MRNN), whose principle is the combination of minimizing the sum of squares of error and maximizing the classification margin, based on the principle of structural risk minimization. Therefore, the objective function of MRNN is the combination of the sum of squared error and the sum of squares of the slopes of the classification function. Besides, we derived a more sophisticated formula similar to the traditional weight decay technique from the MRNN, establishing a more rigorous theoretical basis for the technique. This study employed several real application examples to test the MRNN. The results led to the following conclusions. (1) As long as the penalty coefficient was in the appropriate range, MRNN performed better than pure MLP. (2) MRNN may perform better in difficult classification problems than MLP using weight decay technique.
UR - http://www.scopus.com/inward/record.url?scp=84865032384&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84865032384&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-31837-5_2
DO - 10.1007/978-3-642-31837-5_2
M3 - Conference contribution
AN - SCOPUS:84865032384
SN - 9783642318368
T3 - Communications in Computer and Information Science
SP - 10
EP - 16
BT - Emerging Intelligent Computing Technology and Applications - 8th International Conference, ICIC 2012, Proceedings
T2 - 8th International Conference on Emerging Intelligent Computing Technology and Applications, ICIC 2012
Y2 - 25 July 2012 through 29 July 2012
ER -