Colorize

Generative adversarial neural networks for realistic image colorization.

Yosemite Valley with El Capitan and Bridalveil Fall, colorized by the model

Encoder

The grayscale image goes through a ResNet-101 encoder, which extracts features at several resolutions.

Global Features

A second branch classifies the image as a whole. Its output is fused with the encoder’s features, so the type of scene informs the colors.

Decoder

The decoder predicts the two color channels of the Lab color space. Together with the original lightness channel, they form the colorized image.

Discriminator

During training, a PatchGAN discriminator rates small regions of the image as real or generated. Its rating is part of the generator’s loss.

5M+
training images from Open Images
48 h
per full training run
~6,000
photographs colorized

Tech Specs

Model
Generative adversarial network with a ResNet-101 encoder
Discriminator
PatchGAN
Upscaling
SRGAN, optional
Training data
More than 5 million images from Open Images
Training hardware
IBM Power (PPC64LE), 2 × NVIDIA Tesla
Frameworks
PyTorch, TensorFlow, OpenCV
Inference
CUDA, cuDNN, TensorRT
API
Python and Flask on x86, PPC64LE and ARM
Deployment
Docker Compose behind Nginx
Colorize icon

About the Project

I built Colorize from February to November 2020 at the T-Systems InnovationCenter, as part of my apprenticeship at Deutsche Telekom.