To Improve In-Vivo Bio Images of Fast Temporal Focusing Multiphoton Microscopy by Multi-Stage U-Net Image Restoration

Yu Hao Tseng, Yvonne Yuling Hu, Chia Wei Hsu, Chun Yu Lin, Hsueh Cheng Chiang, Shean Jen Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution


The imaging speed of Temporal focusing multiphoton excitation microscopy (TFMPEM) is up to hundreds frames rate. However, the plane illumination manner suffers from the sever scattering of biotissue and signal crosstalk that blurs the image. And the deeper the worse. Nevertheless, the high acquisition rate decreases the effective excited fluorescent, which reduces the signal-to-noise ratio (SNR) of the image. In order to solve the scattering and low SNR issues, the deep learning method is proposed to restore the TFMPEM image. In this work, we construct a powerful neuron network which called multi-stage 3D U-Net. Different from the cascade method, it becomes more connection between each U-Net. The previous stage information can share with the next stage, instead of seeing as independent. Thus, we try to restore the TFMPEM via this network with Point scanning multiphoton excitation microscopy (PSMPEM) image as the ground truth. But before that way, our two systems are not sharing the same optical path architecture, it needs to do the registration first. For cross modality registration, we utilize Voxelmorph which is also a 3D U-Net architecture. And it can do the not only global but also local deformation, is flexible than classical algorithm. Hence, we do the registration and restoration via all deep learning method. Therefore, the peak signal-to-noise ratio (PSNR) of the image can be improved around 20 to 30 dB and, and structural similarity (SSIM) is close to 0.9

Original languageEnglish
Title of host publicationBiomedical Spectroscopy, Microscopy, and Imaging II
EditorsJurgen Popp, Csilla Gergely
ISBN (Electronic)9781510651647
Publication statusPublished - 2022
EventBiomedical Spectroscopy, Microscopy, and Imaging II 2022 - Virtual, Online
Duration: 2022 May 92022 May 20

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X


ConferenceBiomedical Spectroscopy, Microscopy, and Imaging II 2022
CityVirtual, Online

All Science Journal Classification (ASJC) codes

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering


Dive into the research topics of 'To Improve In-Vivo Bio Images of Fast Temporal Focusing Multiphoton Microscopy by Multi-Stage U-Net Image Restoration'. Together they form a unique fingerprint.

Cite this