[DL輪読会]Disentangling by Factorising
7 likes3,541 views
The document discusses FactorVAE, a method for disentangling latent representations in variational autoencoders (VAEs). It introduces Total Correlation (TC) as a penalty term that encourages independence between latent variables. TC is added to the standard VAE objective function to guide the model to learn disentangled representations. The document provides details on how TC is defined and computed based on the density-ratio trick from generative adversarial networks. It also discusses how FactorVAE uses TC to learn disentangled representations and can be evaluated using a disentanglement metric.
![DEEP LEARNING JP
[DL Papers]
Disentangling by Factorising
Hirono Okamoto, Matsuo Lab
http://deeplearning.jp/
1](https://image.slidesharecdn.com/20180720disentanglingbyfactorising-180720000930/85/DL-Disentangling-by-Factorising-1-320.jpg)
[DL輪読会]Disentangling by Factorising
- 1. DEEP LEARNING JP [DL Papers] Disentangling by Factorising Hirono Okamoto, Matsuo Lab http://deeplearning.jp/ 1
- 2. : Disentangling by Factorising n 2018 ICML accepted n : Hyunjik Kim, Andriy Mnih n β-VAE disentanglement metric n disentangle n rotation position x scale position y Shape ( ) gif https://github.com/1Konny/FactorVAE
- 3. : beta-VAE n : β-VAE: LEARNING BASIC VISUAL CONCEPTS WITH A CONSTRAINED VARIATIONAL FRAMEWORK n ICLR 2017(poster) n : n VAE disentangle n disentanglement VAE … azimuth entangle ??
- 4. β-VAE : n : disentangle n ⇒ n p(z) N(0, I) n β 1 βVAE VAE n β z disentangle n β trade off …
- 5. β-VAE : disentangle n accuracy disentangle n 1. k ( scale) n 2. L n n encode n encode n 3. 2 z ( ) y (y=Wz ) n disentangle z y n = metric score n K-1 disentangle 100% … Disentangle Metric Scale Scale z 0
- 6. FactorVAE β-VAE ( ) n β-VAE disentanglement n disentanglement n ⇒ Total Correlation Penalty n β-VAE disentanglement metric n ( : L iteration ) n ( … ) n (K K-1 disentangle 100% ) n ⇒ a new metric for disentanglement new metric old metric
- 7. FactorVAE : Total Correlation Penalty n FactorVAE n Total Correlation (TC): n GAN z density-ratio trick q(z) ¯q(z) = p(z|y = 1) p(z|y = 0) = p(y = 1|z) p(y = 0|z) ⇡ D(z) 1 D(z) density-ratio trick q(z) = Z pdata(x)q(z|x)dx <latexit sha1_base64="PEMDYUO5OQ+UOtLO81MQvQtmmIU=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIF2pUaSrYKsRk5pUoFMRXpySWJNZqVGgChWsqNFMq4gWUDfQMwEABk2EIZSgzQEFAvsByhhiGFIZ8hmSGUoZchlSGPIYSIDuHIZGhGAijGQwZDBgKgGKxDNVAsSIgKxMsn8pQy8AF1FsKVJUKVJEIFM0GkulAXjRUNA/IB5lZDNadDLQlB4iLgDoVGFQNrhqsNPhscMJgtcFLgz84zaoGmwFySyWQToLoTS2I5++SCP5OUFcukC5hyEDowuvmEoY0BguwWzOBbi8Ai4B8kQzRX1Y1/XOwVZBqtZrBIoPXQPcvNLhpcBjog7yyL8lLA1ODZjNwASPAED24MRlhRnqGBnqGgSbKDk7QqOBgkGZQYtAAhrc5gwODB0MAQyjQ3laG1QxbGLYysTHpMBkzmUKUMjFC9QgzoAAmOwDSDpSU</latexit><latexit sha1_base64="PEMDYUO5OQ+UOtLO81MQvQtmmIU=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIF2pUaSrYKsRk5pUoFMRXpySWJNZqVGgChWsqNFMq4gWUDfQMwEABk2EIZSgzQEFAvsByhhiGFIZ8hmSGUoZchlSGPIYSIDuHIZGhGAijGQwZDBgKgGKxDNVAsSIgKxMsn8pQy8AF1FsKVJUKVJEIFM0GkulAXjRUNA/IB5lZDNadDLQlB4iLgDoVGFQNrhqsNPhscMJgtcFLgz84zaoGmwFySyWQToLoTS2I5++SCP5OUFcukC5hyEDowuvmEoY0BguwWzOBbi8Ai4B8kQzRX1Y1/XOwVZBqtZrBIoPXQPcvNLhpcBjog7yyL8lLA1ODZjNwASPAED24MRlhRnqGBnqGgSbKDk7QqOBgkGZQYtAAhrc5gwODB0MAQyjQ3laG1QxbGLYysTHpMBkzmUKUMjFC9QgzoAAmOwDSDpSU</latexit><latexit sha1_base64="PEMDYUO5OQ+UOtLO81MQvQtmmIU=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIF2pUaSrYKsRk5pUoFMRXpySWJNZqVGgChWsqNFMq4gWUDfQMwEABk2EIZSgzQEFAvsByhhiGFIZ8hmSGUoZchlSGPIYSIDuHIZGhGAijGQwZDBgKgGKxDNVAsSIgKxMsn8pQy8AF1FsKVJUKVJEIFM0GkulAXjRUNA/IB5lZDNadDLQlB4iLgDoVGFQNrhqsNPhscMJgtcFLgz84zaoGmwFySyWQToLoTS2I5++SCP5OUFcukC5hyEDowuvmEoY0BguwWzOBbi8Ai4B8kQzRX1Y1/XOwVZBqtZrBIoPXQPcvNLhpcBjog7yyL8lLA1ODZjNwASPAED24MRlhRnqGBnqGgSbKDk7QqOBgkGZQYtAAhrc5gwODB0MAQyjQ3laG1QxbGLYysTHpMBkzmUKUMjFC9QgzoAAmOwDSDpSU</latexit><latexit sha1_base64="PEMDYUO5OQ+UOtLO81MQvQtmmIU=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIF2pUaSrYKsRk5pUoFMRXpySWJNZqVGgChWsqNFMq4gWUDfQMwEABk2EIZSgzQEFAvsByhhiGFIZ8hmSGUoZchlSGPIYSIDuHIZGhGAijGQwZDBgKgGKxDNVAsSIgKxMsn8pQy8AF1FsKVJUKVJEIFM0GkulAXjRUNA/IB5lZDNadDLQlB4iLgDoVGFQNrhqsNPhscMJgtcFLgz84zaoGmwFySyWQToLoTS2I5++SCP5OUFcukC5hyEDowuvmEoY0BguwWzOBbi8Ai4B8kQzRX1Y1/XOwVZBqtZrBIoPXQPcvNLhpcBjog7yyL8lLA1ODZjNwASPAED24MRlhRnqGBnqGgSbKDk7QqOBgkGZQYtAAhrc5gwODB0MAQyjQ3laG1QxbGLYysTHpMBkzmUKUMjFC9QgzoAAmOwDSDpSU</latexit> Total Correlation Penalty VAE objective
- 8. FactorVAE : Total Correlation Penalty n FactorVAE n Total Correlation (TC): n GAN z density-ratio trick q(z) = Z pdata(x)q(z|x)dx <latexit sha1_base64="PEMDYUO5OQ+UOtLO81MQvQtmmIU=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIF2pUaSrYKsRk5pUoFMRXpySWJNZqVGgChWsqNFMq4gWUDfQMwEABk2EIZSgzQEFAvsByhhiGFIZ8hmSGUoZchlSGPIYSIDuHIZGhGAijGQwZDBgKgGKxDNVAsSIgKxMsn8pQy8AF1FsKVJUKVJEIFM0GkulAXjRUNA/IB5lZDNadDLQlB4iLgDoVGFQNrhqsNPhscMJgtcFLgz84zaoGmwFySyWQToLoTS2I5++SCP5OUFcukC5hyEDowuvmEoY0BguwWzOBbi8Ai4B8kQzRX1Y1/XOwVZBqtZrBIoPXQPcvNLhpcBjog7yyL8lLA1ODZjNwASPAED24MRlhRnqGBnqGgSbKDk7QqOBgkGZQYtAAhrc5gwODB0MAQyjQ3laG1QxbGLYysTHpMBkzmUKUMjFC9QgzoAAmOwDSDpSU</latexit><latexit sha1_base64="PEMDYUO5OQ+UOtLO81MQvQtmmIU=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIF2pUaSrYKsRk5pUoFMRXpySWJNZqVGgChWsqNFMq4gWUDfQMwEABk2EIZSgzQEFAvsByhhiGFIZ8hmSGUoZchlSGPIYSIDuHIZGhGAijGQwZDBgKgGKxDNVAsSIgKxMsn8pQy8AF1FsKVJUKVJEIFM0GkulAXjRUNA/IB5lZDNadDLQlB4iLgDoVGFQNrhqsNPhscMJgtcFLgz84zaoGmwFySyWQToLoTS2I5++SCP5OUFcukC5hyEDowuvmEoY0BguwWzOBbi8Ai4B8kQzRX1Y1/XOwVZBqtZrBIoPXQPcvNLhpcBjog7yyL8lLA1ODZjNwASPAED24MRlhRnqGBnqGgSbKDk7QqOBgkGZQYtAAhrc5gwODB0MAQyjQ3laG1QxbGLYysTHpMBkzmUKUMjFC9QgzoAAmOwDSDpSU</latexit><latexit sha1_base64="PEMDYUO5OQ+UOtLO81MQvQtmmIU=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIF2pUaSrYKsRk5pUoFMRXpySWJNZqVGgChWsqNFMq4gWUDfQMwEABk2EIZSgzQEFAvsByhhiGFIZ8hmSGUoZchlSGPIYSIDuHIZGhGAijGQwZDBgKgGKxDNVAsSIgKxMsn8pQy8AF1FsKVJUKVJEIFM0GkulAXjRUNA/IB5lZDNadDLQlB4iLgDoVGFQNrhqsNPhscMJgtcFLgz84zaoGmwFySyWQToLoTS2I5++SCP5OUFcukC5hyEDowuvmEoY0BguwWzOBbi8Ai4B8kQzRX1Y1/XOwVZBqtZrBIoPXQPcvNLhpcBjog7yyL8lLA1ODZjNwASPAED24MRlhRnqGBnqGgSbKDk7QqOBgkGZQYtAAhrc5gwODB0MAQyjQ3laG1QxbGLYysTHpMBkzmUKUMjFC9QgzoAAmOwDSDpSU</latexit><latexit sha1_base64="PEMDYUO5OQ+UOtLO81MQvQtmmIU=">AAAChHicSyrIySwuMTC4ycjEzMLKxs7BycXNw8vHLyAoFFacX1qUnBqanJ+TXxSRlFicmpOZlxpaklmSkxpRUJSamJuUkxqelO0Mkg8vSy0qzszPCympLEiNzU1Mz8tMy0xOLAEKxQvIF2pUaSrYKsRk5pUoFMRXpySWJNZqVGgChWsqNFMq4gWUDfQMwEABk2EIZSgzQEFAvsByhhiGFIZ8hmSGUoZchlSGPIYSIDuHIZGhGAijGQwZDBgKgGKxDNVAsSIgKxMsn8pQy8AF1FsKVJUKVJEIFM0GkulAXjRUNA/IB5lZDNadDLQlB4iLgDoVGFQNrhqsNPhscMJgtcFLgz84zaoGmwFySyWQToLoTS2I5++SCP5OUFcukC5hyEDowuvmEoY0BguwWzOBbi8Ai4B8kQzRX1Y1/XOwVZBqtZrBIoPXQPcvNLhpcBjog7yyL8lLA1ODZjNwASPAED24MRlhRnqGBnqGgSbKDk7QqOBgkGZQYtAAhrc5gwODB0MAQyjQ3laG1QxbGLYysTHpMBkzmUKUMjFC9QgzoAAmOwDSDpSU</latexit> z GAN z
- 9. FactorVAE : A New Metric for Disentanglement n 1. k x L n 2. x encode z n 3. d k Old metric New metric z 0
- 10. : β-VAE vs factorVAE, 2D Shapes n β-VAE FactorVAE disentanglement metric ( ) n ( ) ( ) n y x size shape shape entangle …
- 11. : InfoWGAN-GP, 2D Shapes n Info-GAN + WGAN-GP n n infoGAN ( ) n (infoGAN …( )) InfoWGAN-GP β-VAE Factor-VAE better
- 12. : β-VAE vs factorVAE, 3D Shapes n disentangle factorVAE ( ) n shape scale disentangle ( )
- 13. : β-VAE vs factorVAE, 3D Chairs βVAE …? leg style ??
- 14. : β-VAE vs factorVAE, 3D Faces βVAE …? azimuth ??
- 15. : β-VAE vs factorVAE, CelebA FactorVAE
- 16. n contribution n 2D Shapes 3D Shapes factorVAE betaVAE disentanglement scores n β-VAE disentanglement metric metric n failure mode n VAE GAN n limitation n Total Correlation disentangling n ex) q(z|x) p(z)=N(z|0,I) TC=0 x n n future work n n
- 17. n Adversarial Autoencoder n infoGAN
- 18. : adversarial autoencoder n n AAE VAE n ICLR 2016 workshop
- 19. : InfoGAN n InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets n NIPS 2016 n disentangle n disentangle n ex) mnist
- 20. : n : n I(X; Y) = H(X) - H(X|Y) n H(X) H(X|Y) Y X n x, y n KL n p(x) p(x|y) ( ) n wikipedia
- 21. : n : c n : n : I(c; G(c, z)) n GAN c c n z n disentangle …? n L
- 22. : n p(c|x) Q(c|x) n lemma A.1 P(c|x) n Q D
- 23. : MNIST n condition n c ~ unif(-1, 1) n 10 y ~ cat(10) n D c c’ n https://github.com/znxlwm/pytorch-generative-model- collections/blob/master/infoGAN.py n n c c’ disentangle n c c’
- 24. : yz c ( ) yz c ( ) n s