Download - [PR12] intro. to gans jaejun yoo
![Page 1: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/1.jpg)
GANsPR12와 함께 이해하는
* Generative Adversarial Nets Ian Goodfellow et al. 2014를 바탕으로 작성한 리뷰
Jaejun YooPh.D. Candidate @KAIST
PR12
16th Apr, 2017
![Page 2: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/2.jpg)
안녕하세요 저는
유재준
- Ph.D. Candidate
- Medical Image Reconstruction,
- http://jaejunyoo.blogspot.com/
Topological Data Analysis, EEG
![Page 3: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/3.jpg)
Generative Adversarial Network
![Page 4: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/4.jpg)
Generative Adversarial Network
![Page 5: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/5.jpg)
PREREQUISITESGenerative Models
“FACE IMAGES”
![Page 6: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/6.jpg)
PREREQUISITESGenerative Models
* Figure adopted from BEGAN paper released at 31. Mar. 2017 David Berthelot et al. Google (link)
Generated Images by Neural Network
![Page 7: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/7.jpg)
PREREQUISITESGenerative Models
“What I cannot create, I do not understand”
![Page 8: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/8.jpg)
PREREQUISITESGenerative Models
“What I cannot create, I do not understand”
If the network can learn how to draw cat and dog separately, it must be able to classify them, i.e. feature learning follows naturally.
![Page 9: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/9.jpg)
PREREQUISITESTaxonomy of Machine Learning
From Yann Lecun, (NIPS 2016)From David silver, Reinforcement learning (UCL course on RL, 2015)
![Page 10: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/10.jpg)
PREREQUISITES
Slide adopted from Namju Kim, Kakao brain (SlideShare, AI Forum, 2017)
y = f(x)
![Page 11: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/11.jpg)
PREREQUISITES
Slide adopted from Namju Kim, Kakao brain (SlideShare, AI Forum, 2017)
![Page 12: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/12.jpg)
PREREQUISITESTaxonomy of Machine Learning
From Yann Lecun, (NIPS 2016)From David silver, Reinforcement learning (UCL course on RL, 2015)
![Page 13: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/13.jpg)
PREREQUISITES
Slide adopted from Namju Kim, Kakao brain (SlideShare, AI Forum, 2017)
![Page 14: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/14.jpg)
PREREQUISITES
Slide adopted from Namju Kim, Kakao brain (SlideShare, AI Forum, 2017)
![Page 15: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/15.jpg)
PREREQUISITES
Slide adopted from Namju Kim, Kakao brain (SlideShare, AI Forum, 2017)
![Page 16: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/16.jpg)
PREREQUISITES
Slide adopted from Namju Kim, Kakao brain (SlideShare, AI Forum, 2017)
![Page 17: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/17.jpg)
PREREQUISITES
Slide adopted from Namju Kim, Kakao brain (SlideShare, AI Forum, 2017)
![Page 18: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/18.jpg)
PREREQUISITES
Slide adopted from Namju Kim, Kakao brain (SlideShare, AI Forum, 2017)
![Page 19: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/19.jpg)
PREREQUISITES
Slide adopted from Namju Kim, Kakao brain (SlideShare, AI Forum, 2017)
![Page 20: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/20.jpg)
PREREQUISITES
Slide adopted from Namju Kim, Kakao brain (SlideShare, AI Forum, 2017)
![Page 21: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/21.jpg)
PREREQUISITES
Slide adopted from Namju Kim, Kakao brain (SlideShare, AI Forum, 2017)
![Page 22: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/22.jpg)
PREREQUISITES
Slide adopted from Namju Kim, Kakao brain (SlideShare, AI Forum, 2017)
* Figure adopted from NIPS 2016 Tutorial: GAN paper, Ian Goodfellow 2016
![Page 23: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/23.jpg)
Generative Adversarial Network
![Page 24: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/24.jpg)
Generative Adversarial Network
![Page 25: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/25.jpg)
SCHEMATIC OVERVIEW
z
G
D
x
Real or Fake?
Diagram ofStandard GAN
Gaussian noise as an input for G
![Page 26: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/26.jpg)
z
G
D
x
Real or Fake?
Diagram ofStandard GAN
지폐위조범
경찰
SCHEMATIC OVERVIEW
![Page 27: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/27.jpg)
z
G
D
x
Real or Fake?
Diagram ofStandard GAN
지폐위조범
경찰
QP
SCHEMATIC OVERVIEW
![Page 28: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/28.jpg)
Diagram ofStandard GAN
Data distribution Model distributionDiscriminator
SCHEMATIC OVERVIEW
* Figure adopted from Generative Adversarial Nets, Ian Goodfellow et al. 2014
![Page 29: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/29.jpg)
Minimax problem of GAN
THEORETICAL RESULTS
Show that…
1. The minimax problem of GAN has a global optimum at 𝒑𝒑𝒈𝒈 = 𝒑𝒑𝒅𝒅𝒅𝒅𝒅𝒅𝒅𝒅
2. The proposed algorithm can find that global optimum
TWO STEP APPROACH
![Page 30: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/30.jpg)
THEORETICAL RESULTSProposition 1.
![Page 31: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/31.jpg)
THEORETICAL RESULTSProposition 1.
![Page 32: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/32.jpg)
THEORETICAL RESULTSMain Theorem
![Page 33: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/33.jpg)
THEORETICAL RESULTSConvergence of the proposed algorithm
![Page 34: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/34.jpg)
THEORETICAL RESULTSConvergence of the proposed algorithm
"The subderivatives of a supremum of convex functions include the derivative of the function at the point where the maximum is attained."
![Page 35: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/35.jpg)
RESULTS
* Figure adopted from DCGAN, Alec Radford et al. 2016 (link)
What can GAN do?
![Page 36: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/36.jpg)
RESULTS
What can GAN do?
Vector arithmetic(e.g. word2vec)
![Page 37: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/37.jpg)
RESULTS
What can GAN do?
Vector arithmetic(e.g. word2vec)
![Page 38: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/38.jpg)
RESULTS
What can GAN do?
Vector arithmetic(e.g. word2vec)
* Figure adopted from DCGAN, Alec Radford et al. 2016 (link)
![Page 39: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/39.jpg)
RESULTS
“We want to get a disentangled representation space EXPLICITLY.”
Neural network understanding “Rotation”
* Figure adopted from DCGAN, Alec Radford et al. 2016 (link)
![Page 40: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/40.jpg)
DIFFICULTIES
![Page 41: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/41.jpg)
DIFFICULTIES
![Page 42: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/42.jpg)
DIFFICULTIES CONVERGENCE OF THE MODEL
![Page 43: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/43.jpg)
DIFFICULTIES CONVERGENCE OF THE MODEL
![Page 44: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/44.jpg)
DIFFICULTIES HOW TO EVALUATE THE QUALITY?
![Page 45: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/45.jpg)
DIFFICULTIES HOW TO EVALUATE THE QUALITY?
![Page 46: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/46.jpg)
DIFFICULTIES MODE COLLAPSE (SAMPLE DIVERSITY)
* Slide adopted from NIPS 2016 Tutorial, Ian Goodfellow
![Page 47: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/47.jpg)
RELATED WORKS
* Unrolled GAN Luke Metz et al. 2016
![Page 48: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/48.jpg)
RELATED WORKS
* Unrolled GAN Luke Metz et al. 2016
![Page 49: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/49.jpg)
RELATED WORKS
* CycleGAN Jun-Yan Zhu et al. 2017
* SRGAN Christian Ledwig et al. 2017
Super-resolution
Img2Img Translation
![Page 50: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/50.jpg)
RELATED WORKS
* infoGAN Xi Chen et al. 2016
Find a CODE
![Page 51: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/51.jpg)
RELATED WORKS
Find a CODE
* infoGAN Xi Chen et al. 2016
![Page 52: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/52.jpg)
RELATED WORKS
“The information in the latent code c should not be lost in the generation process.”
c
z
G
D
x
I
Real or Fake?
Mutual Info. infoGAN: maximize I(c,G(z,c))
Diagram ofinfoGAN Impose an extra constraint to learn disentangled feature space
![Page 53: [PR12] intro. to gans jaejun yoo](https://reader034.vdocuments.pub/reader034/viewer/2022050613/5a650d637f8b9aa2548b6545/html5/thumbnails/53.jpg)
THANK YOU [email protected]