Diffusion operators for multimodal data analysis

Tal Shnitzer, Roy R. Lederman, Gi-Ren Liu, Ronen Talmon, Hau Tieng Wu

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

In this chapter, we present a Manifold Learning viewpoint on the analysis of data arising from multiple modalities. We assume that the high-dimensional multimodal data lie on underlying low-dimensional manifolds and devise a new data-driven representation that accommodates this inherent structure. Based on diffusion geometry, we present three composite operators, facilitating different aspects of fusion of information from different modalities in different settings. These operators are shown to recover the common structures and the differences between modalities in terms of their intrinsic geometry and allow for the construction of data-driven representations which capture these characteristics. The properties of these operators are demonstrated in four applications: recovery of the common variable in two camera views, shape analysis, foetal heart rate identification and sleep dynamics assessment.

Original languageEnglish
Title of host publicationHandbook of Numerical Analysis
EditorsRon Kimmel, Xue-Cheng Tai
PublisherElsevier B.V.
Pages1-39
Number of pages39
ISBN (Print)9780444641403
DOIs
Publication statusPublished - 2019 Jan 1

Publication series

NameHandbook of Numerical Analysis
Volume20
ISSN (Print)1570-8659

Fingerprint

Modality
Data analysis
Data-driven
Geometry
Operator
Mathematical operators
Manifold Learning
Shape Analysis
Fusion reactions
Sleep
Cameras
Heart Rate
Recovery
Fusion
Composite materials
High-dimensional
Camera
Composite

All Science Journal Classification (ASJC) codes

  • Numerical Analysis
  • Modelling and Simulation
  • Computational Mathematics
  • Applied Mathematics

Cite this

Shnitzer, T., Lederman, R. R., Liu, G-R., Talmon, R., & Wu, H. T. (2019). Diffusion operators for multimodal data analysis. In R. Kimmel, & X-C. Tai (Eds.), Handbook of Numerical Analysis (pp. 1-39). (Handbook of Numerical Analysis; Vol. 20). Elsevier B.V.. https://doi.org/10.1016/bs.hna.2019.07.008
Shnitzer, Tal ; Lederman, Roy R. ; Liu, Gi-Ren ; Talmon, Ronen ; Wu, Hau Tieng. / Diffusion operators for multimodal data analysis. Handbook of Numerical Analysis. editor / Ron Kimmel ; Xue-Cheng Tai. Elsevier B.V., 2019. pp. 1-39 (Handbook of Numerical Analysis).
@inbook{9bd39ef4074d4e65bf7ee2564397a0fe,
title = "Diffusion operators for multimodal data analysis",
abstract = "In this chapter, we present a Manifold Learning viewpoint on the analysis of data arising from multiple modalities. We assume that the high-dimensional multimodal data lie on underlying low-dimensional manifolds and devise a new data-driven representation that accommodates this inherent structure. Based on diffusion geometry, we present three composite operators, facilitating different aspects of fusion of information from different modalities in different settings. These operators are shown to recover the common structures and the differences between modalities in terms of their intrinsic geometry and allow for the construction of data-driven representations which capture these characteristics. The properties of these operators are demonstrated in four applications: recovery of the common variable in two camera views, shape analysis, foetal heart rate identification and sleep dynamics assessment.",
author = "Tal Shnitzer and Lederman, {Roy R.} and Gi-Ren Liu and Ronen Talmon and Wu, {Hau Tieng}",
year = "2019",
month = "1",
day = "1",
doi = "10.1016/bs.hna.2019.07.008",
language = "English",
isbn = "9780444641403",
series = "Handbook of Numerical Analysis",
publisher = "Elsevier B.V.",
pages = "1--39",
editor = "Ron Kimmel and Xue-Cheng Tai",
booktitle = "Handbook of Numerical Analysis",

}

Shnitzer, T, Lederman, RR, Liu, G-R, Talmon, R & Wu, HT 2019, Diffusion operators for multimodal data analysis. in R Kimmel & X-C Tai (eds), Handbook of Numerical Analysis. Handbook of Numerical Analysis, vol. 20, Elsevier B.V., pp. 1-39. https://doi.org/10.1016/bs.hna.2019.07.008

Diffusion operators for multimodal data analysis. / Shnitzer, Tal; Lederman, Roy R.; Liu, Gi-Ren; Talmon, Ronen; Wu, Hau Tieng.

Handbook of Numerical Analysis. ed. / Ron Kimmel; Xue-Cheng Tai. Elsevier B.V., 2019. p. 1-39 (Handbook of Numerical Analysis; Vol. 20).

Research output: Chapter in Book/Report/Conference proceedingChapter

TY - CHAP

T1 - Diffusion operators for multimodal data analysis

AU - Shnitzer, Tal

AU - Lederman, Roy R.

AU - Liu, Gi-Ren

AU - Talmon, Ronen

AU - Wu, Hau Tieng

PY - 2019/1/1

Y1 - 2019/1/1

N2 - In this chapter, we present a Manifold Learning viewpoint on the analysis of data arising from multiple modalities. We assume that the high-dimensional multimodal data lie on underlying low-dimensional manifolds and devise a new data-driven representation that accommodates this inherent structure. Based on diffusion geometry, we present three composite operators, facilitating different aspects of fusion of information from different modalities in different settings. These operators are shown to recover the common structures and the differences between modalities in terms of their intrinsic geometry and allow for the construction of data-driven representations which capture these characteristics. The properties of these operators are demonstrated in four applications: recovery of the common variable in two camera views, shape analysis, foetal heart rate identification and sleep dynamics assessment.

AB - In this chapter, we present a Manifold Learning viewpoint on the analysis of data arising from multiple modalities. We assume that the high-dimensional multimodal data lie on underlying low-dimensional manifolds and devise a new data-driven representation that accommodates this inherent structure. Based on diffusion geometry, we present three composite operators, facilitating different aspects of fusion of information from different modalities in different settings. These operators are shown to recover the common structures and the differences between modalities in terms of their intrinsic geometry and allow for the construction of data-driven representations which capture these characteristics. The properties of these operators are demonstrated in four applications: recovery of the common variable in two camera views, shape analysis, foetal heart rate identification and sleep dynamics assessment.

UR - http://www.scopus.com/inward/record.url?scp=85072055481&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072055481&partnerID=8YFLogxK

U2 - 10.1016/bs.hna.2019.07.008

DO - 10.1016/bs.hna.2019.07.008

M3 - Chapter

AN - SCOPUS:85072055481

SN - 9780444641403

T3 - Handbook of Numerical Analysis

SP - 1

EP - 39

BT - Handbook of Numerical Analysis

A2 - Kimmel, Ron

A2 - Tai, Xue-Cheng

PB - Elsevier B.V.

ER -

Shnitzer T, Lederman RR, Liu G-R, Talmon R, Wu HT. Diffusion operators for multimodal data analysis. In Kimmel R, Tai X-C, editors, Handbook of Numerical Analysis. Elsevier B.V. 2019. p. 1-39. (Handbook of Numerical Analysis). https://doi.org/10.1016/bs.hna.2019.07.008