Generative Adversarial Networks : Basic architecture and variants
- 1. Generative Adversarial Networks Palacode Narayana Iyer Anantharaman 29 Oct 2018
- 2. References • https://github.com/lukedeo/keras-acgan/blob/master/acgan-analysis.ipynb • https://github.com/keras-team/keras/blob/master/examples/mnist_acgan.py • https://skymind.ai/wiki/generative-adversarial-network-gan • https://junyanz.github.io/CycleGAN/ • Self Attention Generative Adversarial Networks: Zhang et al
- 3. Why GAN? • GANs can learn to mimic any distribution and generate data • The data may be images, speech or music • The outputs from GANs are found to be quite realistic and impressive • Thus, GANs have a number of applications: From being a feature in products like Photoshop to generating synthetic datasets for image augmentation
- 6. GAN Workflow
- 7. Generator • Generates synthetic images given the input noise z • G is differentiable • Typically a Gaussian distribution
- 8. Training • Train on 2 mini batches simultaneously • Training samples • Generated samples • Cost
- 10. Different Variants of GAN Ref: https://github.com/lukedeo/keras-acgan/blob/master/acgan-analysis.ipynb
- 11. Cycle GAN (2017) • Original Paper: “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”, Zhu et al
- 12. Image to Image Translation • Image to image translation is aimed at finding a mapping between an input image (X) and its corresponding output image (Y), where the pair X, Y are provided in the dataset • This assumes that we are provided with such a labelled dataset with pairings • CycleGAN attempts to find a mapping between images from source and target domains in the absence of paired examples Learn G: X → Y such that the distribution of images from G(X) is indistinguishable from the distribution Y using an adversarial loss. Couple this with an inverse mapping F: Y → X and enforce a cycle consistency loss to enforce F(G(X)) ≈ X
- 14. Cycle GAN: Objective Function • Two discriminators: Dx and Dy where Dx aims to distinguish between images {x} and translated images {F(y)}. In the same way Dy aims to discriminate between {y} and {G(x)} • The objective function has 2 parts representing the losses: • adversarial losses for matching the distribution of generated images to the data distribution in the target domain • Cycle consistency losses that prevent the learned mappings G and F from contradicting each other
- 15. Losses
- 16. Exercises • Go through the original paper and answer the following: • How is the model evaluated? What are the metrics? • What are the main applications discussed in the paper? • What are the limitations and future work?
- 17. SAGAN (2018) Zhang et al Abstract • GANs often use a CNN as a generator • CNNs capture short range dependencies very well (local receptive fields) but not effective to capture long distance correlations • Self Attention Generative Adversarial Networks (SAGAN) is aimed at generating images that take in to account both short and long distance dependencies in the source images
- 18. SAGAN