Exploring Macroscopic Fluctuation of Facial Expression for Mood Disorder Classification

Qian Bei Hong, Chung-Hsien Wu, Ming Hsiang Su, Kun Yi Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In clinical diagnosis of mood disorder, a large portion of bipolar disorder patients (BDs) are misdiagnosed as unipolar depression (UDs). Clinicians have confirmed that BDs generally show ''reduced affect'' during clinical treatment. Thus, it is expected to build an objective and one-time diagnosis system for diagnosis assistance by using machine-learning techniques. In this study, facial expressions of BD, UD and control group (C) elicited by emotional video clips are collected for exploring temporal fluctuation characteristics of intensities of facial muscles expression among the three groups. The differences of facial expressions among mood disorders are investigated by observing macroscopic fluctuations. To deal with these problems, the corresponding methods for feature extraction and modeling are proposed. From the viewpoint of macroscopic facial expression, action unit (AU) is applied for describing the temporal transformation of muscles. Then, modulation spectrum is used for extracting short-term variation of AU. The multilayer perceptron (MLP)-based disorder prediction model is then applied to obtain the prediction results. For evaluation of the proposed method, 12 subjects for three group are included in the K-fold (K=12) cross validation experiments. The experiment results reached 61.1\% classification accuracy, and outperformed the other baseline methods.

Original languageEnglish
Title of host publication2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781538653111
DOIs
Publication statusPublished - 2018 Sep 21
Event1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018 - Beijing, China
Duration: 2018 May 202018 May 22

Publication series

Name2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018

Other

Other1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018
CountryChina
CityBeijing
Period18-05-2018-05-22

Fingerprint

Muscle
Multilayer neural networks
Learning systems
Feature extraction
Experiments
Modulation

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Signal Processing
  • Artificial Intelligence

Cite this

Hong, Q. B., Wu, C-H., Su, M. H., & Huang, K. Y. (2018). Exploring Macroscopic Fluctuation of Facial Expression for Mood Disorder Classification. In 2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018 [8470337] (2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ACIIAsia.2018.8470337
Hong, Qian Bei ; Wu, Chung-Hsien ; Su, Ming Hsiang ; Huang, Kun Yi. / Exploring Macroscopic Fluctuation of Facial Expression for Mood Disorder Classification. 2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018. Institute of Electrical and Electronics Engineers Inc., 2018. (2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018).
@inproceedings{01dc37c775a14fe5a67252182840ea49,
title = "Exploring Macroscopic Fluctuation of Facial Expression for Mood Disorder Classification",
abstract = "In clinical diagnosis of mood disorder, a large portion of bipolar disorder patients (BDs) are misdiagnosed as unipolar depression (UDs). Clinicians have confirmed that BDs generally show ''reduced affect'' during clinical treatment. Thus, it is expected to build an objective and one-time diagnosis system for diagnosis assistance by using machine-learning techniques. In this study, facial expressions of BD, UD and control group (C) elicited by emotional video clips are collected for exploring temporal fluctuation characteristics of intensities of facial muscles expression among the three groups. The differences of facial expressions among mood disorders are investigated by observing macroscopic fluctuations. To deal with these problems, the corresponding methods for feature extraction and modeling are proposed. From the viewpoint of macroscopic facial expression, action unit (AU) is applied for describing the temporal transformation of muscles. Then, modulation spectrum is used for extracting short-term variation of AU. The multilayer perceptron (MLP)-based disorder prediction model is then applied to obtain the prediction results. For evaluation of the proposed method, 12 subjects for three group are included in the K-fold (K=12) cross validation experiments. The experiment results reached 61.1\{\%} classification accuracy, and outperformed the other baseline methods.",
author = "Hong, {Qian Bei} and Chung-Hsien Wu and Su, {Ming Hsiang} and Huang, {Kun Yi}",
year = "2018",
month = "9",
day = "21",
doi = "10.1109/ACIIAsia.2018.8470337",
language = "English",
series = "2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018",
address = "United States",

}

Hong, QB, Wu, C-H, Su, MH & Huang, KY 2018, Exploring Macroscopic Fluctuation of Facial Expression for Mood Disorder Classification. in 2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018., 8470337, 2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018, Institute of Electrical and Electronics Engineers Inc., 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018, Beijing, China, 18-05-20. https://doi.org/10.1109/ACIIAsia.2018.8470337

Exploring Macroscopic Fluctuation of Facial Expression for Mood Disorder Classification. / Hong, Qian Bei; Wu, Chung-Hsien; Su, Ming Hsiang; Huang, Kun Yi.

2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018. Institute of Electrical and Electronics Engineers Inc., 2018. 8470337 (2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Exploring Macroscopic Fluctuation of Facial Expression for Mood Disorder Classification

AU - Hong, Qian Bei

AU - Wu, Chung-Hsien

AU - Su, Ming Hsiang

AU - Huang, Kun Yi

PY - 2018/9/21

Y1 - 2018/9/21

N2 - In clinical diagnosis of mood disorder, a large portion of bipolar disorder patients (BDs) are misdiagnosed as unipolar depression (UDs). Clinicians have confirmed that BDs generally show ''reduced affect'' during clinical treatment. Thus, it is expected to build an objective and one-time diagnosis system for diagnosis assistance by using machine-learning techniques. In this study, facial expressions of BD, UD and control group (C) elicited by emotional video clips are collected for exploring temporal fluctuation characteristics of intensities of facial muscles expression among the three groups. The differences of facial expressions among mood disorders are investigated by observing macroscopic fluctuations. To deal with these problems, the corresponding methods for feature extraction and modeling are proposed. From the viewpoint of macroscopic facial expression, action unit (AU) is applied for describing the temporal transformation of muscles. Then, modulation spectrum is used for extracting short-term variation of AU. The multilayer perceptron (MLP)-based disorder prediction model is then applied to obtain the prediction results. For evaluation of the proposed method, 12 subjects for three group are included in the K-fold (K=12) cross validation experiments. The experiment results reached 61.1\% classification accuracy, and outperformed the other baseline methods.

AB - In clinical diagnosis of mood disorder, a large portion of bipolar disorder patients (BDs) are misdiagnosed as unipolar depression (UDs). Clinicians have confirmed that BDs generally show ''reduced affect'' during clinical treatment. Thus, it is expected to build an objective and one-time diagnosis system for diagnosis assistance by using machine-learning techniques. In this study, facial expressions of BD, UD and control group (C) elicited by emotional video clips are collected for exploring temporal fluctuation characteristics of intensities of facial muscles expression among the three groups. The differences of facial expressions among mood disorders are investigated by observing macroscopic fluctuations. To deal with these problems, the corresponding methods for feature extraction and modeling are proposed. From the viewpoint of macroscopic facial expression, action unit (AU) is applied for describing the temporal transformation of muscles. Then, modulation spectrum is used for extracting short-term variation of AU. The multilayer perceptron (MLP)-based disorder prediction model is then applied to obtain the prediction results. For evaluation of the proposed method, 12 subjects for three group are included in the K-fold (K=12) cross validation experiments. The experiment results reached 61.1\% classification accuracy, and outperformed the other baseline methods.

UR - http://www.scopus.com/inward/record.url?scp=85055491921&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055491921&partnerID=8YFLogxK

U2 - 10.1109/ACIIAsia.2018.8470337

DO - 10.1109/ACIIAsia.2018.8470337

M3 - Conference contribution

T3 - 2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018

BT - 2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Hong QB, Wu C-H, Su MH, Huang KY. Exploring Macroscopic Fluctuation of Facial Expression for Mood Disorder Classification. In 2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018. Institute of Electrical and Electronics Engineers Inc. 2018. 8470337. (2018 1st Asian Conference on Affective Computing and Intelligent Interaction, ACII Asia 2018). https://doi.org/10.1109/ACIIAsia.2018.8470337