Emotion recognition of conversational affective speech using temporal course modeling

In a natural conversation, a complete emotional expression is typically composed of a complex temporal course representing temporal phases of onset, apex, and offset. In this study, subemotional states are defined to model the temporal course of an emotional expression in natural conversation. Hidden Markov Models (HMMs) are adopted to characterize the subemotional states; each represents one temporal phase. A subemotion language model, which considers the temporal transition between sub-emotional states (HMMs), is further constructed to provide a constraint on allowable temporal structures to determine an optimal emotional state. Experimental results show that the proposed approach yielded satisfactory results on the MHMC conversation-based affective speech corpus, and confirmed that considering the complex temporal structure in natural conversation is useful for improving the emotion recognition performance from speech.