Sensation and Perception in HCI

Interactions of humans with computers, whether directly or indirectly, invariably involve physical information being provided to the humans that must be sensed and perceived. Consequently, humans’ perceptions of their interactions with computerized systems and their ability to see, hear, feel, and understand the information provided are essential to proper functioning of the systems. This chapter provides an overview of topics in sensation and perception, with an emphasis on relevance to Human–Computer Interaction. Unlike most overviews of this type, this chapter includes a discussion of fundamental issues concerning various uses of the terms, theoretical perspectives, and methodological concerns about how to measure different aspects. Critical aspects of visual and auditory perception are examined, including topics relevant to everyday life, such as face perception and music perception, that are not often included in overviews. Finally, the topic of multimodal perception is examined in some detail because interaction in most physical and cyber environments involves multiple sensory modalities. This involvement of multiple modalities is especially evident in the multimodal and XR interfaces that are becoming ubiquitous nowadays, as well as in the robots, automatic vehicles, and other artificial intelligence implementations with which people must interact on an increasing basis.