Multiple Convolutional Neural Networks fusion using improved fuzzy integral for facial emotion recognition

Cheng Jian Lin, Chun Hui Lin, Shyh-Hau Wang, Chen Hsien Wu

Research output: Contribution to journalArticle

Abstract

Facial expressions are indispensable in human cognitive behaviors since it can instantly reveal human emotions. Therefore, in this study, Multiple Convolutional Neural Networks using Improved Fuzzy Integral (MCNNs-IFI) were proposed for recognizing facial emotions. Since effective facial expression features are difficult to design; deep learning CNN is used in the study. Each CNN has its own advantages and disadvantages, thus combining multiple CNNs can yield superior results. Moreover, multiple CNNs combined with improved fuzzy integral, in which its fuzzy density value is optimized through particle swarm optimization (PSO), overcomes the majority decision drawback in the traditional voting method. Two Multi-PIE and CK+ databases and three main CNN structures, namely AlexNet, GoogLeNet, and LeNet, were used in the experiments. To verify the results, a cross-validation method was used, and experimental results indicated that the proposed MCNNs-IFI exhibited 12.84% higher accuracy than that of the three CNNs.

Original languageEnglish
Article number2593
JournalApplied Sciences (Switzerland)
Volume9
Issue number13
DOIs
Publication statusPublished - 2019 Jul 1

Fingerprint

emotions
Fusion reactions
fusion
Neural networks
Particle swarm optimization (PSO)
voting
learning
optimization
Experiments
Deep learning

All Science Journal Classification (ASJC) codes

  • Materials Science(all)
  • Instrumentation
  • Engineering(all)
  • Process Chemistry and Technology
  • Computer Science Applications
  • Fluid Flow and Transfer Processes

Cite this

@article{cb399fa0ad8e480dbbfcb7d8649dc4ca,
title = "Multiple Convolutional Neural Networks fusion using improved fuzzy integral for facial emotion recognition",
abstract = "Facial expressions are indispensable in human cognitive behaviors since it can instantly reveal human emotions. Therefore, in this study, Multiple Convolutional Neural Networks using Improved Fuzzy Integral (MCNNs-IFI) were proposed for recognizing facial emotions. Since effective facial expression features are difficult to design; deep learning CNN is used in the study. Each CNN has its own advantages and disadvantages, thus combining multiple CNNs can yield superior results. Moreover, multiple CNNs combined with improved fuzzy integral, in which its fuzzy density value is optimized through particle swarm optimization (PSO), overcomes the majority decision drawback in the traditional voting method. Two Multi-PIE and CK+ databases and three main CNN structures, namely AlexNet, GoogLeNet, and LeNet, were used in the experiments. To verify the results, a cross-validation method was used, and experimental results indicated that the proposed MCNNs-IFI exhibited 12.84{\%} higher accuracy than that of the three CNNs.",
author = "Lin, {Cheng Jian} and Lin, {Chun Hui} and Shyh-Hau Wang and Wu, {Chen Hsien}",
year = "2019",
month = "7",
day = "1",
doi = "10.3390/app9132593",
language = "English",
volume = "9",
journal = "Applied Sciences (Switzerland)",
issn = "2076-3417",
publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",
number = "13",

}

Multiple Convolutional Neural Networks fusion using improved fuzzy integral for facial emotion recognition. / Lin, Cheng Jian; Lin, Chun Hui; Wang, Shyh-Hau; Wu, Chen Hsien.

In: Applied Sciences (Switzerland), Vol. 9, No. 13, 2593, 01.07.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Multiple Convolutional Neural Networks fusion using improved fuzzy integral for facial emotion recognition

AU - Lin, Cheng Jian

AU - Lin, Chun Hui

AU - Wang, Shyh-Hau

AU - Wu, Chen Hsien

PY - 2019/7/1

Y1 - 2019/7/1

N2 - Facial expressions are indispensable in human cognitive behaviors since it can instantly reveal human emotions. Therefore, in this study, Multiple Convolutional Neural Networks using Improved Fuzzy Integral (MCNNs-IFI) were proposed for recognizing facial emotions. Since effective facial expression features are difficult to design; deep learning CNN is used in the study. Each CNN has its own advantages and disadvantages, thus combining multiple CNNs can yield superior results. Moreover, multiple CNNs combined with improved fuzzy integral, in which its fuzzy density value is optimized through particle swarm optimization (PSO), overcomes the majority decision drawback in the traditional voting method. Two Multi-PIE and CK+ databases and three main CNN structures, namely AlexNet, GoogLeNet, and LeNet, were used in the experiments. To verify the results, a cross-validation method was used, and experimental results indicated that the proposed MCNNs-IFI exhibited 12.84% higher accuracy than that of the three CNNs.

AB - Facial expressions are indispensable in human cognitive behaviors since it can instantly reveal human emotions. Therefore, in this study, Multiple Convolutional Neural Networks using Improved Fuzzy Integral (MCNNs-IFI) were proposed for recognizing facial emotions. Since effective facial expression features are difficult to design; deep learning CNN is used in the study. Each CNN has its own advantages and disadvantages, thus combining multiple CNNs can yield superior results. Moreover, multiple CNNs combined with improved fuzzy integral, in which its fuzzy density value is optimized through particle swarm optimization (PSO), overcomes the majority decision drawback in the traditional voting method. Two Multi-PIE and CK+ databases and three main CNN structures, namely AlexNet, GoogLeNet, and LeNet, were used in the experiments. To verify the results, a cross-validation method was used, and experimental results indicated that the proposed MCNNs-IFI exhibited 12.84% higher accuracy than that of the three CNNs.

UR - http://www.scopus.com/inward/record.url?scp=85068857977&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85068857977&partnerID=8YFLogxK

U2 - 10.3390/app9132593

DO - 10.3390/app9132593

M3 - Article

AN - SCOPUS:85068857977

VL - 9

JO - Applied Sciences (Switzerland)

JF - Applied Sciences (Switzerland)

SN - 2076-3417

IS - 13

M1 - 2593

ER -