An interaction-embedded HMM framework for human behavior understanding: With nursing environments as examples

Chin De Liu, Yi Nung Chung, Pau Choo Chung

Research output: Contribution to journalArticlepeer-review

36 Citations (Scopus)


This paper presents an interaction-embedded hidden Markov model (IE-HMM) framework for automatically detecting and classifying individual human behaviors and group interactions. The proposed framework comprises a switch control (SC) module, an individual duration HMM (IDHMM) module, and an interaction-coupled duration HMM (ICDHMM) module. By analyzing the relative distances between the various participants in each scene, and monitoring the duration for which these distances are maintained, the SC module assigns each participant to an individual behavior unit (comprising a single participant) or an interaction behavior unit (comprising two or more participants). The individual behavior units are passed to the IDHMM module, which classifies the corresponding human behavior in accordance with the pose, motion, and duration information using duration HMM (DHMM). Similarly, the interaction behavior units are dispatched to the ICDHMM module, where the corresponding interaction mode is classified using an integrated scheme comprising multiple coupled-duration HMM (CDHMM), in which each state has an embedded coupled HMM (CHMM). The validity of the IE-HMM framework is confirmed by analyzing the human actions and interactions observed in a nursing home environment. The results confirm that the atomic behavior unit concept embedded in the SC module enables the IE-HMM framework to recognize multiple concurrent actions and interactions within a single scene. Overall, it is shown that the proposed framework has a recognition performance of 100% when applied to the analysis of individual human actions and 95% when applied to that of group interactions.

Original languageEnglish
Article number5482142
Pages (from-to)1236-1246
Number of pages11
JournalIEEE Transactions on Information Technology in Biomedicine
Issue number5
Publication statusPublished - 2010 Sep

All Science Journal Classification (ASJC) codes

  • Biotechnology
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'An interaction-embedded HMM framework for human behavior understanding: With nursing environments as examples'. Together they form a unique fingerprint.

Cite this