A CT-ultrasound-coregistered augmented reality enhanced image-guided surgery system and its preliminary study on brain-shift estimation

H. Huang, H. Hsieh, D. Lee, C. Huang, T. Lee, T. Wu, Yung-Nien Sun, T. Wu

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

With the combined view on the physical space and the medical imaging data, augmented reality (AR) visualization can provide perceptive advantages during image-guided surgery (IGS). However, the imaging data are usually captured before surgery and might be different from the up-to-date one due to natural shift of soft tissues. This study presents an AR-enhanced IGS system which is capable to correct the movement of soft tissues from the pre-operative CT images by using intra-operative ultrasound images. First, with reconstructing 2-D free-hand ultrasound images to 3-D volume data, the system applies a Mutual-Information based registration algorithm to estimate the deformation between pre-operative and intra-operative ultrasound images. The estimated deformation transform describes the movement of soft tissues and is then applied to the pre-operative CT images which provide high-resolution anatomical information. As a result, the system thus displays the fusion of the corrected CT images or the real-time 2-D ultrasound images with the patient in the physical space through a head mounted display device, providing an immersive augmented-reality environment. For the performance validation of the proposed system, a brain phantom was utilized to simulate brain-shift scenario. Experimental results reveal that when the shift of an artificial tumor is from 5mm ∼ 12mm, the correction rates can be improved from 32% ∼ 45% to 87% ∼ 95% by using the proposed system.

Original languageEnglish
Article numberP08016
JournalJournal of Instrumentation
Volume7
Issue number8
DOIs
Publication statusPublished - 2012 Aug 1

Fingerprint

Augmented reality
Augmented Reality
Ultrasound
Ultrasound Image
surgery
Surgery
brain
Brain
Ultrasonics
CT Image
Soft Tissue
shift
Tissue
Medical imaging
display devices
Medical Imaging
Tumors
Phantom
Mutual Information
Fusion reactions

All Science Journal Classification (ASJC) codes

  • Instrumentation
  • Mathematical Physics

Cite this

@article{3b145a85136c4acb90fb8afcbf82299b,
title = "A CT-ultrasound-coregistered augmented reality enhanced image-guided surgery system and its preliminary study on brain-shift estimation",
abstract = "With the combined view on the physical space and the medical imaging data, augmented reality (AR) visualization can provide perceptive advantages during image-guided surgery (IGS). However, the imaging data are usually captured before surgery and might be different from the up-to-date one due to natural shift of soft tissues. This study presents an AR-enhanced IGS system which is capable to correct the movement of soft tissues from the pre-operative CT images by using intra-operative ultrasound images. First, with reconstructing 2-D free-hand ultrasound images to 3-D volume data, the system applies a Mutual-Information based registration algorithm to estimate the deformation between pre-operative and intra-operative ultrasound images. The estimated deformation transform describes the movement of soft tissues and is then applied to the pre-operative CT images which provide high-resolution anatomical information. As a result, the system thus displays the fusion of the corrected CT images or the real-time 2-D ultrasound images with the patient in the physical space through a head mounted display device, providing an immersive augmented-reality environment. For the performance validation of the proposed system, a brain phantom was utilized to simulate brain-shift scenario. Experimental results reveal that when the shift of an artificial tumor is from 5mm ∼ 12mm, the correction rates can be improved from 32{\%} ∼ 45{\%} to 87{\%} ∼ 95{\%} by using the proposed system.",
author = "H. Huang and H. Hsieh and D. Lee and C. Huang and T. Lee and T. Wu and Yung-Nien Sun and T. Wu",
year = "2012",
month = "8",
day = "1",
doi = "10.1088/1748-0221/7/08/P08016",
language = "English",
volume = "7",
journal = "Journal of Instrumentation",
issn = "1748-0221",
publisher = "IOP Publishing Ltd.",
number = "8",

}

A CT-ultrasound-coregistered augmented reality enhanced image-guided surgery system and its preliminary study on brain-shift estimation. / Huang, H.; Hsieh, H.; Lee, D.; Huang, C.; Lee, T.; Wu, T.; Sun, Yung-Nien; Wu, T.

In: Journal of Instrumentation, Vol. 7, No. 8, P08016, 01.08.2012.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A CT-ultrasound-coregistered augmented reality enhanced image-guided surgery system and its preliminary study on brain-shift estimation

AU - Huang, H.

AU - Hsieh, H.

AU - Lee, D.

AU - Huang, C.

AU - Lee, T.

AU - Wu, T.

AU - Sun, Yung-Nien

AU - Wu, T.

PY - 2012/8/1

Y1 - 2012/8/1

N2 - With the combined view on the physical space and the medical imaging data, augmented reality (AR) visualization can provide perceptive advantages during image-guided surgery (IGS). However, the imaging data are usually captured before surgery and might be different from the up-to-date one due to natural shift of soft tissues. This study presents an AR-enhanced IGS system which is capable to correct the movement of soft tissues from the pre-operative CT images by using intra-operative ultrasound images. First, with reconstructing 2-D free-hand ultrasound images to 3-D volume data, the system applies a Mutual-Information based registration algorithm to estimate the deformation between pre-operative and intra-operative ultrasound images. The estimated deformation transform describes the movement of soft tissues and is then applied to the pre-operative CT images which provide high-resolution anatomical information. As a result, the system thus displays the fusion of the corrected CT images or the real-time 2-D ultrasound images with the patient in the physical space through a head mounted display device, providing an immersive augmented-reality environment. For the performance validation of the proposed system, a brain phantom was utilized to simulate brain-shift scenario. Experimental results reveal that when the shift of an artificial tumor is from 5mm ∼ 12mm, the correction rates can be improved from 32% ∼ 45% to 87% ∼ 95% by using the proposed system.

AB - With the combined view on the physical space and the medical imaging data, augmented reality (AR) visualization can provide perceptive advantages during image-guided surgery (IGS). However, the imaging data are usually captured before surgery and might be different from the up-to-date one due to natural shift of soft tissues. This study presents an AR-enhanced IGS system which is capable to correct the movement of soft tissues from the pre-operative CT images by using intra-operative ultrasound images. First, with reconstructing 2-D free-hand ultrasound images to 3-D volume data, the system applies a Mutual-Information based registration algorithm to estimate the deformation between pre-operative and intra-operative ultrasound images. The estimated deformation transform describes the movement of soft tissues and is then applied to the pre-operative CT images which provide high-resolution anatomical information. As a result, the system thus displays the fusion of the corrected CT images or the real-time 2-D ultrasound images with the patient in the physical space through a head mounted display device, providing an immersive augmented-reality environment. For the performance validation of the proposed system, a brain phantom was utilized to simulate brain-shift scenario. Experimental results reveal that when the shift of an artificial tumor is from 5mm ∼ 12mm, the correction rates can be improved from 32% ∼ 45% to 87% ∼ 95% by using the proposed system.

UR - http://www.scopus.com/inward/record.url?scp=84865248399&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84865248399&partnerID=8YFLogxK

U2 - 10.1088/1748-0221/7/08/P08016

DO - 10.1088/1748-0221/7/08/P08016

M3 - Article

AN - SCOPUS:84865248399

VL - 7

JO - Journal of Instrumentation

JF - Journal of Instrumentation

SN - 1748-0221

IS - 8

M1 - P08016

ER -