Syncgan: Synchronize the Latent Spaces of Cross-Modal Generative Adversarial Networks

Wen Cheng Chen, Chien Wen Chen, Min Chun Hu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Generative adversarial network (GAN) has achieved impressive success on cross-domain generation, but it faces difficulty in cross-modal generation due to the lack of a common distribution between heterogeneous data. Most existing methods of conditional based cross-modal GANs adopt the strategy of one-directional transfer and have achieved preliminary success on text-to-image transfer. Instead of learning the transfer between different modalities, we aim to learn a synchronous latent space representing the cross-modal common concept. A novel network component named synchronizer is proposed in this work to judge whether the paired data is synchronous/corresponding or not, which can constrain the latent space of generators in the GANs. Our GAN model, named as SyncGAN, can successfully generate synchronous data (e.g., a pair of image and sound) from identical random noise. For transforming data from one modality to another, we recover the latent code by inverting the mappings of a generator and use it to generate data of different modality. In addition, the proposed model can achieve semi-supervised learning, which makes our model more flexible for practical applications.

Original languageEnglish
Title of host publication2018 IEEE International Conference on Multimedia and Expo, ICME 2018
PublisherIEEE Computer Society
ISBN (Electronic)9781538617373
DOIs
Publication statusPublished - 2018 Oct 8
Event2018 IEEE International Conference on Multimedia and Expo, ICME 2018 - San Diego, United States
Duration: 2018 Jul 232018 Jul 27

Publication series

NameProceedings - IEEE International Conference on Multimedia and Expo
Volume2018-July
ISSN (Print)1945-7871
ISSN (Electronic)1945-788X

Conference

Conference2018 IEEE International Conference on Multimedia and Expo, ICME 2018
CountryUnited States
CitySan Diego
Period18-07-2318-07-27

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Syncgan: Synchronize the Latent Spaces of Cross-Modal Generative Adversarial Networks'. Together they form a unique fingerprint.

Cite this