Learned smartphone ISP on mobile NPUs with deep learning, mobile AI 2021 challenge: Report

Andrey Ignatov, Cheng Ming Chiang, Hsien Kai Kuo, Anastasia Sycheva, Radu Timofte, Min Hung Chen, Man Yu Lee, Yu Syuan Xu, Yu Tseng, Shusong Xu, Jin Guo, Chao Hung Chen, Ming Chun Hsyu, Wen Chia Tsai, Chao Wei Chen, Grigory Malivenko, Minsu Kwon, Myungje Lee, Jaeyoon Yoo, Changbeom KangShinjo Wang, Zheng Shaolong, Hao Dejun, Xie Fen, Feng Zhuang, Yipeng Ma, Jingyang Peng, Tao Wang, Fenglong Song, Chih Chung Hsu, Kwan Lin Chen, Mei Hsuang Wu, Vishal Chudasama, Kalpesh Prajapati, Heena Patel, Anjali Sarvaiya, Kishor Upla, Kiran Raja, Raghavendra Ramachandra, Christoph Busch, Etienne De Stoutz

研究成果: Conference contribution

30 引文 斯高帕斯(Scopus)

摘要

As the quality of mobile cameras starts to play a crucial role in modern smartphones, more and more attention is now being paid to ISP algorithms used to improve various perceptual aspects of mobile photos. In this Mobile AI challenge, the target was to develop an end-to-end deep learning-based image signal processing (ISP) pipeline that can replace classical hand-crafted ISPs and achieve nearly real-time performance on smartphone NPUs. For this, the participants were provided with a novel learned ISP dataset consisting of RAW-RGB image pairs captured with the Sony IMX586 Quad Bayer mobile sensor and a professional 102-megapixel medium format camera. The runtime of all models was evaluated on the MediaTek Dimensity 1000+ platform with a dedicated AI processing unit capable of accelerating both floating-point and quantized neural networks. The proposed solutions are fully compatible with the above NPU and are capable of processing Full HD photos under 60-100 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.

原文English
主出版物標題Proceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021
發行者IEEE Computer Society
頁面2503-2514
頁數12
ISBN(電子)9781665448994
DOIs
出版狀態Published - 2021 6月
事件2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021 - Virtual, Online, United States
持續時間: 2021 6月 192021 6月 25

出版系列

名字IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
ISSN(列印)2160-7508
ISSN(電子)2160-7516

Conference

Conference2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021
國家/地區United States
城市Virtual, Online
期間21-06-1921-06-25

All Science Journal Classification (ASJC) codes

  • 電腦視覺和模式識別
  • 電氣與電子工程

指紋

深入研究「Learned smartphone ISP on mobile NPUs with deep learning, mobile AI 2021 challenge: Report」主題。共同形成了獨特的指紋。

引用此