An efficient augmented reality (AR) system for enhanced visual inspection

While manual visual inspection of structures has the advantage of being relatively simple and low cost, it is usually time consuming, labor intensive and highly subjective. Augmented reality (AR), because of its ability to provide the user with additional information about the working environment in real-time, has been used in the past to address some of the limitations of manual visual inspection by supporting human workers during the inspection process. The paper presents the development of an efficient deep learning (DL) based augmented reality (AR) system for identifying critical departures from the pristine state of the structure with focus on two anomaly categories- corrosion and fatigue cracks. Most of the common AR devices usually come with a built-in camera for capturing image/video data, a storage and a microprocessor. However, due to the limited processing power, the underlying deep learning (DL) model has to be first trained externally and a suitable version of the trained model is then deployed locally on the device. The model then outputs information for identifying critical departures from the pristine state of the structure e.g., highlighting corroded regions, fatigue cracks and/or combination of both. This information is overlaid real time over the current field of view through either a head-mounted or a hand-held AR device in order to augment the human vision. The worker can then focus on the highlighted region for a more detailed inspection. The feasibility of the proposed AR system is demonstrated using laboratory inspection of common mechanical components likes pipes, plates etc. In order to enable the model to keep learning based on the inputs from the AR glasses, a strategy for federated learning is introduced towards the end of the paper.