Minimum risk neural networks and weight decay technique

I. Cheng Yeh, Pei Yen Tseng, Kuan Chieh Huang, Yau Hwang Kuo

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

To enhance the generalization of neural network model, we proposed a novel neural network, Minimum Risk Neural Networks (MRNN), whose principle is the combination of minimizing the sum of squares of error and maximizing the classification margin, based on the principle of structural risk minimization. Therefore, the objective function of MRNN is the combination of the sum of squared error and the sum of squares of the slopes of the classification function. Besides, we derived a more sophisticated formula similar to the traditional weight decay technique from the MRNN, establishing a more rigorous theoretical basis for the technique. This study employed several real application examples to test the MRNN. The results led to the following conclusions. (1) As long as the penalty coefficient was in the appropriate range, MRNN performed better than pure MLP. (2) MRNN may perform better in difficult classification problems than MLP using weight decay technique.

Original languageEnglish
Title of host publicationEmerging Intelligent Computing Technology and Applications - 8th International Conference, ICIC 2012, Proceedings
Pages10-16
Number of pages7
DOIs
Publication statusPublished - 2012 Aug 20
Event8th International Conference on Emerging Intelligent Computing Technology and Applications, ICIC 2012 - Huangshan, China
Duration: 2012 Jul 252012 Jul 29

Publication series

NameCommunications in Computer and Information Science
Volume304 CCIS
ISSN (Print)1865-0929

Other

Other8th International Conference on Emerging Intelligent Computing Technology and Applications, ICIC 2012
CountryChina
CityHuangshan
Period12-07-2512-07-29

All Science Journal Classification (ASJC) codes

  • Computer Science(all)
  • Mathematics(all)

Fingerprint Dive into the research topics of 'Minimum risk neural networks and weight decay technique'. Together they form a unique fingerprint.

Cite this