Exploring and Exploiting Interpretable Semantics in GANs - CVPR 2020 Tutorial

Overview

Explore the interpretable semantics in Generative Adversarial Networks (GANs) through this comprehensive tutorial presented at CVPR'20 iMLCV. Delve into deep generative representation, GAN dissection for interpreting latent units, and the identification of causality in latent space. Learn about aligning latent space with attribute space, layer-wise stochasticity, and semantic hierarchy in GANs. Discover techniques for face synthesis, unsupervised attribute discovery, and GAN inversion for real faces. Gain insights into challenges and solutions in GAN inversion, extended latent codes, and image manipulation applications. Understand the concept of semantic diffusion and image processing with GAN prior, providing a comprehensive overview of cutting-edge research in interpretable vision and generative models.

Syllabus

Intro
GANs for Synthesizing Images
Generative Adversarial Training
Tutorial Outline
Deep Generative Representation
GAN Dissection for Interpreting Latent Units
Random Walk in Latent Space of Bedroom
Multiple Levels of Scene Descriptors
Identifying the Causality in Latent Space
Aligning Latent Space with Attribute Space
Pushing Latent Code through Boundary
Result on turning up the lights
Ageing the scenes
Layer-wise Stochasity
Semantic hierarchy emerges
Changing layout at Layer0-1
Varying category (Bedroom to Dining Room) at layers 3-6
Varying category (Bedroom to Living Room) at layers 3-6
Latent Semantics in Face Synthesis GANS
Interpolation in the Latent Space
InterfaceGAN: Bridging Latent Space to Attribute Space
GANalyze for changing image memoriability
All the approaches below need supervision
Unsupervised Attribute Discovery in GANS
Issues for unsupervised approaches
Make me cooler
GAN Inversion: Inverting Real Faces to Latent Code Synthesized Image x = G(Z)
GAN Inversion for Faces
GAN inversion is challenging!
Extended latent codes
Image2StyleGAN inversion: it kind of works!
But it seems overfitting the given image
Issue: resulting code might be out of the original latent domain
Comparison with Image2StyleGAN
Demo of image manipulation
Demo of image interpolation
Fun Application: Semantic Diffusion
Image Processing with GAN Prior