Patched-Face-Regeneration-GAN

Introduction

In this project, my aim was to develop a model that could regenerate patched/covered parts of human faces, and achieve believable results. I used the Celeb-A dataset, and created a Generative Adversarial Network with a Denoising Autoencoder as the Generator and a Deep Convolutional Network as the Discriminator. I chose this architecture based on Avery Allen and Wenchen Li's Generative Adversarial Denoising Autoencoder for Face Completion.

The Denoising Autoencoder has 'relu' activations in the middle layers while the output layer had a 'tanh' activation. Each Convolution layer was followed by a BatchNormalization layer. The Discriminator has 'LeakyReLU' activations for the Convolution part, with a BatchNormalization layer following every Conv layer. At the end, the output from the CNN segment was flattened and connected to a Dense layer with 1 node, having 'sigmoid' as the activation function. This would enable the discrimator to predict the probability that the input image is real.

I added distortions to the images in two ways:-

• Added random Gaussian noise.
• Added random sized Black Patches.

The entire training was done on a GTX 1080 GPU, and took about 12days.

A few sample generated images are present in saved_imgs.

Setting Up

1. Create a new Python/Anaconda environment (optional but recommended). You might use the environment.yml file for this purpose (Skip Step-2 in that case).

2. Install the necessary packages. Refer to the packages mentioned in environment.yml.

3. The training checkpoints and saved generator and discriminator models are located in /training_checkpoints, /saved_generator and /saved_discriminator respectively.

4. Download the Celeb-A dataset, and place it in the directory in the following manner:-

├─── Patched-Face-Regeneration-GAN
     ├─── ..
     ├─── saved_imgs     
     ├─── training_checkpoints
     ├─── saved_discriminator
     ├─── saved_generator
     ├─── discriminator.png
     ├─── generator.png
     ├─── inference_output.png
     ├─── environment.yml
     ├─── Face_Generation.ipynb    
     └─── images 
           └─── img_align_celeba
               ├─── 000001.jpg
               ├─── 000002.jpg
               ├─── 000003.jpg
               ├─── 000004.jpg
               ├─── 000005.jpg
               ├─── ..
               ├─── ..
               ├─── 202597.jpg
               ├─── 202598.jpg
               └─── 202599.jpg

Output

In the following figure, the first row shows the Actual images, the second row shows the Patched/Distorted Images and the last row shows the images produced by the generator.

Some more images produced by the generator:-

References

• https://www.cc.gatech.edu/~hays/7476/projects/Avery_Wenchen/
• https://www.tensorflow.org/tutorials/generative/dcgan
• https://arxiv.org/abs/1406.2661