Rhythm of motion extraction and rhythm-based cross-media alignment for dance videos

Wei Ta Chu, Shang Yin Tsai

Research output: Contribution to journalArticlepeer-review

15 Citations (Scopus)


We present how to extract rhythm information in dance videos and music, and accordingly correlate them based on rhythmic representation. From dancer's movement, we construct motion trajectories, detect turnings, and stops of trajectories, and then estimate rhythm of motion (ROM). For music, beats are detected to describe rhythm of music. Two modalities are therefore represented as sequences of rhythm information to facilitate finding cross-media correspondence. Two applications, i.e., background music replacement and music video generation, are developed to demonstrate the practicality of cross-media correspondence. We evaluate performance of ROM extraction, and conduct subjective/objective evaluation to show that rich browsing experience can be provided by the proposed applications.

Original languageEnglish
Article number6051493
Pages (from-to)129-141
Number of pages13
JournalIEEE Transactions on Multimedia
Issue number1
Publication statusPublished - 2012 Feb

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Media Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Rhythm of motion extraction and rhythm-based cross-media alignment for dance videos'. Together they form a unique fingerprint.

Cite this