Colorize
Generative adversarial neural networks for realistic image colorization.
Encoder
The grayscale image goes through a ResNet-101 encoder, which extracts features at several resolutions.
Global Features
A second branch classifies the image as a whole. Its output is fused with the encoder’s features, so the type of scene informs the colors.
Decoder
The decoder predicts the two color channels of the Lab color space. Together with the original lightness channel, they form the colorized image.
Discriminator
During training, a PatchGAN discriminator rates small regions of the image as real or generated. Its rating is part of the generator’s loss.
- 5M+
- training images from Open Images
- 48 h
- per full training run
- ~6,000
- photographs colorized
Tech Specs
- Model
- Generative adversarial network with a ResNet-101 encoder
- Discriminator
- PatchGAN
- Upscaling
- SRGAN, optional
- Training data
- More than 5 million images from Open Images
- Training hardware
- IBM Power (PPC64LE), 2 × NVIDIA Tesla
- Frameworks
- PyTorch, TensorFlow, OpenCV
- Inference
- CUDA, cuDNN, TensorRT
- API
- Python and Flask on x86, PPC64LE and ARM
- Deployment
- Docker Compose behind Nginx
About the Project
I built Colorize from February to November 2020 at the T-Systems InnovationCenter, as part of my apprenticeship at Deutsche Telekom.