An efficient augmented reality (AR) system for enhanced visual inspection

Shaohan Wang, Sakib Ashraf Zargar, Cheryl Xu, Fuh Gwo Yuan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Citations (Scopus)

Abstract

While manual visual inspection of structures has the advantage of being relatively simple and low cost, it is usually time consuming, labor intensive and highly subjective. Augmented reality (AR), because of its ability to provide the user with additional information about the working environment in real-time, has been used in the past to address some of the limitations of manual visual inspection by supporting human workers during the inspection process. The paper presents the development of an efficient deep learning (DL) based augmented reality (AR) system for identifying critical departures from the pristine state of the structure with focus on two anomaly categories- corrosion and fatigue cracks. Most of the common AR devices usually come with a built-in camera for capturing image/video data, a storage and a microprocessor. However, due to the limited processing power, the underlying deep learning (DL) model has to be first trained externally and a suitable version of the trained model is then deployed locally on the device. The model then outputs information for identifying critical departures from the pristine state of the structure e.g., highlighting corroded regions, fatigue cracks and/or combination of both. This information is overlaid real time over the current field of view through either a head-mounted or a hand-held AR device in order to augment the human vision. The worker can then focus on the highlighted region for a more detailed inspection. The feasibility of the proposed AR system is demonstrated using laboratory inspection of common mechanical components likes pipes, plates etc. In order to enable the model to keep learning based on the inputs from the AR glasses, a strategy for federated learning is introduced towards the end of the paper.

Original languageEnglish
Title of host publicationStructural Health Monitoring 2019
Subtitle of host publicationEnabling Intelligent Life-Cycle Health Management for Industry Internet of Things (IIOT) - Proceedings of the 12th International Workshop on Structural Health Monitoring
EditorsFu-Kuo Chang, Alfredo Guemes, Fotis Kopsaftopoulos
PublisherDEStech Publications Inc.
Pages1543-1550
Number of pages8
ISBN (Electronic)9781605956015
DOIs
Publication statusPublished - 2019
Event12th International Workshop on Structural Health Monitoring: Enabling Intelligent Life-Cycle Health Management for Industry Internet of Things (IIOT), IWSHM 2019 - Stanford, United States
Duration: 2019 Sept 102019 Sept 12

Publication series

NameStructural Health Monitoring 2019: Enabling Intelligent Life-Cycle Health Management for Industry Internet of Things (IIOT) - Proceedings of the 12th International Workshop on Structural Health Monitoring
Volume1

Conference

Conference12th International Workshop on Structural Health Monitoring: Enabling Intelligent Life-Cycle Health Management for Industry Internet of Things (IIOT), IWSHM 2019
Country/TerritoryUnited States
CityStanford
Period19-09-1019-09-12

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Health Information Management

Fingerprint

Dive into the research topics of 'An efficient augmented reality (AR) system for enhanced visual inspection'. Together they form a unique fingerprint.

Cite this