CycleGAN for Image Transformation

Introduction

Welcome to the CycleGAN project for generating images in the style of Monet from target photos. This repository implements the CycleGAN architecture, which learns mappings between two domains (images) using deep neural networks. In this case, we aim to translate photos into paintings resembling the style of Monet's artworks.

Overview

CycleGAN is unique because it employs two generators (G and F) and two discriminators (D_X and D_Y). Unlike traditional GANs, CycleGAN focuses on not only generating realistic images (GAN loss) but also ensuring that the translated images can be reversed to their original forms (cycle consistency loss). This dual-loss mechanism helps preserve key characteristics of the original images during translation.

The primary components of this implementation include:

• Generator (G): Transforms photos (X) to Monet-style paintings (Y).
• Generator (F): Transforms Monet-style paintings (Y) back to photos (X).
• Discriminator (D_X): Distinguishes real photos from generated Monet-style paintings.
• Discriminator (D_Y): Distinguishes real Monet-style paintings from generated photos.

Architecture

The model architecture consists of:

• Downsample and Upsample layers to extract and reconstruct features at multiple scales.
• Instance Normalization to stabilize and accelerate training.
• Group Normalization to handle different batch sizes effectively.
• Convolutional and Transpose Convolutional layers for image transformation.

Dataset Preparation

Data is prepared using:

• JPEG and TFRecord files for both Monet paintings and photos.
• TensorFlow's data processing utilities (tf.data) for efficient dataset management.
• Custom functions (decode, read, load_dataset, gan_dataset) to load, decode, and preprocess images.

Model Training

• Initialization: Models are initialized within a TensorFlow strategy (MirroredStrategy or TPUStrategy) based on available hardware (CPU, GPU, or TPU).
• CycleGAN Class: A custom Keras model (CycleGAN) encapsulates the training loop, loss functions, and optimization steps.
• Loss Functions: Defined for generators (gen_loss_fn), discriminators (disc_loss_fn), cycle consistency (cycle_loss_fn), and identity mapping (identity_loss_fn).

Resources

This implementation references various resources:

• junyanz/CycleGAN on GitHub
• "Hands-On Image Generation with TensorFlow" by Packt Publishing
• "Cycle Generative Adversarial Network (CycleGAN)" on GeeksforGeeks
• "What is CycleGAN and How to Use It?" on Medium
• "Building Autoencoders in Keras" on Keras Blog
• Instance Normalization: The Missing Ingredient for Fast Stylization (arXiv)