SiGAN: Siamese Generative Adversarial Network for Identity-Preserving Face Hallucination

Chih Chung Hsu, Chia Wen Lin, Weng Tai Su, Gene Cheung

Research output: Contribution to journalArticlepeer-review

48 Citations (Scopus)

Abstract

Though generative adversarial networks (GANs) can hallucinate high-quality high-resolution (HR) faces from low-resolution (LR) faces, they cannot ensure identity preservation during face hallucination, making the HR faces difficult to recognize. To address this problem, we propose a Siamese GAN (SiGAN) to reconstruct HR faces that visually resemble their corresponding identities. On top of a Siamese network, the proposed SiGAN consists of a pair of two identical generators and one discriminator. We incorporate reconstruction error and identity label information in the loss function of SiGAN in a pairwise manner. By iteratively optimizing the loss functions of the generator pair and the discriminator of SiGAN, we not only achieve visually-pleasing face reconstruction but also ensure that the reconstructed information is useful for identity recognition. Experimental results demonstrate that SiGAN significantly outperforms existing face hallucination GANs in objective face verification performance while achieving promising visual-quality reconstruction. Moreover, for input LR faces with unseen identities that are not part of the training dataset, SiGAN can still achieve reasonable performance.

Original languageEnglish
Article number8751141
Pages (from-to)6225-6236
Number of pages12
JournalIEEE Transactions on Image Processing
Volume28
Issue number12
DOIs
Publication statusPublished - 2019 Dec

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design

Fingerprint

Dive into the research topics of 'SiGAN: Siamese Generative Adversarial Network for Identity-Preserving Face Hallucination'. Together they form a unique fingerprint.

Cite this