InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
- 1. InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, Pieter Abbeel (UC Berkeley, Open AI) Presenter: Shuhei M. Yoshida (Dept. of Physics, UTokyo) Unsupervised learning of disentangled representations Goal GANs + Maximizing Mutual Information between generated images and input codes Approach Benefit Interpretable representation obtained without supervision and substantial additional costs Reference https://arxiv.org/abs/1606.03657 (with Appendix sections) Implementations https://github.com/openai/InfoGAN (by the authors, with TensorFlow) https://github.com/yoshum/InfoGAN (by the presenter, with Chainer) NIPS2016読み会
- 2. Motivation How can we achieve unsupervised learning of disentangled representation? In general, learned representation is entangled, i.e. encoded in a data space in a complicated manner When a representation is disentangled, it would be more interpretable and easier to apply to tasks
- 3. Related works • Unsupervised learning of representation (no mechanism to force disentanglement) Stacked (often denoising) autoencoder, RBM Many others, including semi-supervised approach • Supervised learning of disentangled representation Bilinear models, multi-view perceptron VAEs, adversarial autoencoders • Weakly supervised learning of disentangled representation disBM, DC-IGN • Unsupervised learning of disentangled representation hossRBM, applicable only to discrete latent factors which the presenter has almost no knowledge about. This work: Unsupervised learning of disentangled representation applicable to both continuous and discrete latent factors
- 4. Generative Adversarial Nets(GANs) Generative model trained by competition between two neural nets: Generator 𝑥 = 𝐺 𝑧 , 𝑧 ∼ 𝑝 𝑧 𝑍 𝑝 𝑧 𝑍 : an arbitrary noise distribution Discriminator 𝐷 𝑥 ∈ 0,1 : probability that 𝑥 is sampled from the data dist. 𝑝data(𝑋) rather than generated by the generator 𝐺 𝑧 min 𝐺 max 𝐷 𝑉GAN 𝐺, 𝐷 , where 𝑉GAN 𝐺, 𝐷 ≡ 𝐸 𝑥∼𝑝data 𝑋 ln 𝐷 𝑥 + 𝐸 𝑧∼𝑝 𝑧 𝑍 ln 1 − 𝐷 𝐺 𝑧 Optimization problem to solve:
- 5. Problems with GANs From the perspective of representation learning: No restrictions on how 𝐺 𝑧 uses 𝑧 • 𝑧 can be used in a highly entangled way • Each dimension of 𝑧 does not represent any salient feature of the training data 𝑧1 𝑧2
- 6. Proposed Resolution: InfoGAN -Maximizing Mutual Information - Observation in conventional GANs: a generated date 𝑥 does not have much information on the noise 𝑧 from which 𝑥 is generated because of heavily entangled use of 𝑧 Proposed resolution = InfoGAN: the generator 𝐺 𝑧, 𝑐 trained so that it maximize the mutual information 𝐼 𝐶 𝑋 between the latent code 𝐶 and the generated data 𝑋 min 𝐺 max 𝐷 𝑉GAN 𝐺, 𝐷 − 𝜆𝐼 𝐶 𝑋 = 𝐺 𝑍, 𝐶
- 7. Mutual Information 𝐼 𝑋; 𝑌 = 𝐻 𝑋 − 𝐻 𝑋 𝑌 , where • 𝐻 𝑋 = 𝐸 𝑥∼𝑝 𝑋 − ln 𝑝 𝑋 = 𝑥 : Entropy of the prior distribution • 𝐻 𝑋 𝑌 = 𝐸 𝑦∼𝑝 𝑌 ,𝑥∼𝑝 𝑋|𝑌=𝑦 − ln 𝑝 𝑋 = 𝑥 𝑌 = 𝑦 : Entropy of the posterior distribution 𝑝 𝑋 = 𝑥 𝑥 𝑝 𝑋 = 𝑥|𝑌 = 𝑦 𝑥 𝑝 𝑋 = 𝑥|𝑌 = 𝑦 𝑥 𝐼 𝑋; 𝑌 = 0 𝐼 𝑋; 𝑌 > 0 Sampling 𝑦 ∼ 𝑝 𝑌
- 8. Avoiding increase of calculation costs Major difficulty: Evaluation of 𝐼 𝐶 𝑋 based on evaluation and sampling from the posterior 𝑝 𝐶 𝑋 Two strategies: Variational maximization of mutual information Use an approximate function 𝑄 𝑐 𝑥 = 𝑝 𝐶 = 𝑐 𝑋 = 𝑥 Sharing the neural net between 𝑄 𝑐 𝑥 and the discriminator 𝐷 𝑥
- 9. Variational Maximization of MI For an arbitrary function 𝑄 𝑐, 𝑥 , 𝐸 𝑥∼𝑝 𝐺 𝑋 ,𝑐∼𝑝 𝐶|𝑋=𝑥 ln 𝑝 𝐶 = 𝑐 𝑋 = 𝑥 (∵ positivity of KL divergence) = 𝐸 𝑥∼𝑝 𝐺 𝑋 ,𝑐∼𝑝 𝐶|𝑋=𝑥 ln 𝑄(𝑐, 𝑥) + 𝐸 𝑥∼𝑝 𝐺 𝑋 ,𝑐∼𝑝 𝐶|𝑋=𝑥 ln 𝑝 𝐶 = 𝑐 𝑋 = 𝑥 𝑄 𝑐, 𝑥 = 𝐸 𝑥∼𝑝 𝐺 𝑋 ,𝑐∼𝑝 𝐶|𝑋=𝑥 ln 𝑄(𝑐, 𝑥) + 𝐸 𝑥∼𝑝 𝐺 𝑋 𝐷KL 𝑝 𝐶 𝑋 = 𝑥 ||𝑄 𝐶, 𝑥 ≥ 𝐸 𝑥∼𝑝 𝐺 𝑋 ,𝑐∼𝑝 𝐶|𝑋=𝑥 ln 𝑄(𝑐, 𝑥)
- 10. Variational Maximization of MI Maximizing 𝐿𝐼 𝐺, 𝑄 w.r.t. 𝐺 and 𝑄 With 𝑄(𝑐, 𝑥) approximating 𝑝 𝐶 = 𝑐 𝑋 = 𝑥 , we obtain an variational estimate of the mutual information: 𝐿𝐼 𝐺, 𝑄 ≡ 𝐸 𝑥∼𝑝 𝐺 𝑋 ,𝑐∼𝑝 𝐶|𝑋=𝑥 ln 𝑄 𝑐, 𝑥 + 𝐻 𝐶 ≲ 𝐼 𝐶 𝑋 = 𝐺 𝑍, 𝐶 ⇔ • Achieving the equality by setting 𝑄 𝑐, 𝑥 = 𝑝 𝐶 = 𝑐 𝑋 = 𝑥 • Maximizing the mutual information min 𝐺,𝑄 max 𝐷 𝑉GAN 𝐺, 𝐷 − 𝜆𝐿𝐼 𝐺, 𝑄 Optimization problem to solve in InfoGAN:
- 11. Eliminate sampling from posterior Lemma 𝐸 𝑥∼𝑝 𝑋 ,𝑦∼𝑝 𝑌 𝑋=𝑥) 𝑓 𝑥, 𝑦 = 𝐸 𝑥∼𝑝 𝑋 ,𝑦∼𝑝 𝑌 𝑋=𝑥),𝑥′∼𝑝 𝑋′ 𝑌=𝑦) 𝑓 𝑥′ , 𝑦 . 𝐸 𝑥∼𝑝 𝐺 𝑋 ,𝑐∼𝑝 𝐶|𝑋=𝑥 ln 𝑄 𝑐, 𝑥 = 𝐸 𝑐∼𝑝 𝐶 ,𝑧∼𝑝 𝑧 𝑍 ,𝑥=𝐺 𝑧,𝑐 ,𝑐∼𝑝 𝐶|𝑋=𝑥 ln 𝑄 𝑐, 𝑥 , By using this lemma and noting that 𝐸 𝑥∼𝑝 𝐺 𝑋 ,𝑐∼𝑝 𝐶|𝑋=𝑥 ln 𝑄 𝑐, 𝑥 = 𝑬 𝒄∼𝒑 𝑪 ,𝒛∼𝒑 𝒛 𝒁 ,𝒙=𝑮 𝒛,𝒄 𝐥𝐧 𝑸 𝒄, 𝒙 we can eliminate the sampling from 𝑝 𝐶|𝑋 = 𝑥 : Easy to estimate!
- 12. Proof of lemma Lemma 𝐸 𝑥∼𝑝 𝑋 ,𝑦∼𝑝 𝑌 𝑋=𝑥) 𝑓 𝑥, 𝑦 = 𝐸 𝑥∼𝑝 𝑋 ,𝑦∼𝑝 𝑌 𝑋=𝑥),𝑥′∼𝑝 𝑋′ 𝑌=𝑦) 𝑓 𝑥′ , 𝑦 . ∵ l. h. s. = 𝑥 𝑦 𝑝 𝑋 = 𝑥 𝑝 𝑌 = 𝑦 𝑋 = 𝑥 𝑓 𝑥, 𝑦 = 𝑥 𝑦 𝑝 𝑌 = 𝑦 𝑝 𝑋 = 𝑥 𝑌 = 𝑦 𝑓 𝑥, 𝑦 = 𝑥 𝑦 𝑥′ 𝑝 𝑋 = 𝑥′, 𝑌 = 𝑦 𝑝 𝑋 = 𝑥 𝑌 = 𝑦 𝑓 𝑥, 𝑦 = 𝑥 𝑦 𝑥′ 𝑝 𝑋 = 𝑥′ 𝑝 𝑌 = 𝑦 𝑋 = 𝑥′ 𝑝 𝑋 = 𝑥 𝑌 = 𝑦 𝑓 𝑥, 𝑦 = r. h. s. ∵ Bayes’ theorem
- 13. Sharing layers between 𝐷 and 𝑄 Model 𝑄 𝑐, 𝑥 using neural network Reduce the calculation costs by sharing all the convolution layers with 𝐷 Image from Odena, et al., arXiv:1610.09585. Convolution layers of the discriminator 𝐷 𝑄 Given DCGANs, InfoGAN comes for negligible additional costs!
- 14. Experiment – MI Maximization • InfoGAN on MNIST dataset • Latent code 𝑐 = 10-class categorical code 𝐿𝐼 quickly saturates to 𝐻 𝑐 = ln 10 ∼ 2.3 in InfoGAN Figure 1 in the original paper
- 15. Experiment – Disentangled Representation – Figure 2 in the original paper • InfoGAN on MNIST dataset • Latent codes 𝑐1: 10-class categorical code 𝑐2, 𝑐3: continuous code 𝑐1 can be used as a classifier with 5% error rate. 𝑐2 and 𝑐3 captured the rotation and width, respectively
- 16. Experiment – Disentangled Representation – Dataset: P. Paysan, et al., AVSS, 2009, pp. 296–301. Figure 3 in the original paper
- 17. Experiment – Disentangled Representation – Dataset: M. Aubry, et al., CVPR, 2014, pp. 3762–3769. InfoGAN learned salient features without supervision Figure 4 in the original paper
- 18. Experiment – Disentangled Representation – Dataset: Street View House Number Figure 5 in the original paper
- 19. Experiment – Disentangled Representation – Dataset: CelebA Figure 6 in the original paper
- 20. Future Prospect and Conclusion Mutual information maximization can be applied to other methods, e.g. VAE Learning hierarchical latent representation Improving semi-supervised learning High-dimentional data discovery Unsupervised learning of disentangled representations Goal GANs + Maximizing Mutual Information between generated images and input codes Approach Benefit Interpretable representation obtained without supervision and substantial additional costs