Seeing Sounds: The Role of Vowels and Consonants in Crossmodal Correspondences

Yang Chen Shen, Yi Chuan Chen, Pi Chun Huang

研究成果: Article同行評審

1 引文 斯高帕斯(Scopus)

摘要

Crossmodal correspondences refer to the fact that certain domains of features in different sensory modalities are associated with each other. Here, we investigated the crossmodal correspondences between speech sounds and visual shapes. Specifically, we tested whether the classification dimensions of English vowels (front–central–back) and consonants (voiced–voiceless, sonorant–obstruent, and stop–continuant) correspond to visual shapes along a bipolar rounded–angular dimension. We adapted eighteen meaningless pseudowords from a previous study that corresponded to either the round or the sharp concept. On each trial, the participants heard one of the pseudowords and saw a rounded shape and an angular shape presented side-by-side on the monitor. Participants judged which shape provided a better match to the spoken pseudoword. A logistic regression was conducted in order to elucidate the effectiveness of classification dimensions of phonemes when predicting variations in the sound–shape matchings. The results demonstrated that the sound–shape matchings were predictable using front–central–back dimensions of vowels, and voiced–voiceless and stop–continuant dimensions of consonants. Hence, we verified that sound–shape matching is underpinned by contrasting dimensions in both vowels and consonants, therefore demonstrating crossmodal correspondences at the phonetic level.

原文English
期刊i-Perception
13
發行號2
DOIs
出版狀態Published - 2022 3月

All Science Journal Classification (ASJC) codes

  • 實驗與認知心理學
  • 眼科
  • 感覺系統
  • 人工智慧

指紋

深入研究「Seeing Sounds: The Role of Vowels and Consonants in Crossmodal Correspondences」主題。共同形成了獨特的指紋。

引用此