Immersive virtual reality with multimodal interaction and streaming technology

Wan Lun Tsai, You Lun Hsu, Chi Po Lin, Chen Yu Zhu, Yu Cheng Chen, Min Chun Hu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

In this demo, we present an immersive virtual reality (VR) system which integrates multimodal interaction sensors (i.e., smartphone, Kinect v2, and Myo armband) and streaming technology to improve the VR experience. The integrated system solves the common problems in most VR systems: (1) the very limited playing area due to transmission cable between computer and display/interaction devices, and (2) non-intuitive way of controlling virtual objects. We use Unreal Engine 4 to develop an immersive VR game with 6 interactive levels to demonstrate the feasibility of our system. In the game, the user not only can freely walk within a large playing area surrounded by multiple Kinect sensors but also select the virtual objects to grab and throw with the Myo armband. The experiment shows that our idea is workable for VR experience.

Original languageEnglish
Title of host publicationICMI 2016 - Proceedings of the 18th ACM International Conference on Multimodal Interaction
EditorsCatherine Pelachaud, Yukiko I. Nakano, Toyoaki Nishida, Carlos Busso, Louis-Philippe Morency, Elisabeth Andre
PublisherAssociation for Computing Machinery, Inc
Number of pages1
ISBN (Electronic)9781450345569
DOIs
Publication statusPublished - 2016 Oct 31
Event18th ACM International Conference on Multimodal Interaction, ICMI 2016 - Tokyo, Japan
Duration: 2016 Nov 122016 Nov 16

Publication series

NameICMI 2016 - Proceedings of the 18th ACM International Conference on Multimodal Interaction

Other

Other18th ACM International Conference on Multimodal Interaction, ICMI 2016
CountryJapan
CityTokyo
Period16-11-1216-11-16

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Human-Computer Interaction
  • Hardware and Architecture
  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of 'Immersive virtual reality with multimodal interaction and streaming technology'. Together they form a unique fingerprint.

Cite this