CS 194-26: Image Colorization

Jacob Hanson-Regalado

Description

The goal of this project is to align glass plate negatives sourced from the Prokudin-Gorskii image collection to generate RGB images. For each photograph, we are given 3 negatives each one corresponding to red, green, or blue. My task was to align these negatives such that when stacked together they produce a coherent RGB image with little discrepancies. To do so, I employed an exhaustive search on compressed images such that I tested out shifting the R/G negatives so that they match the B negative closest. For larger TIF images, I used a pyramid method where I first aligned the channels at a very low resolution then gradually increased the resolution to its original value while performing small updates on the alignment. Each channel might not match perfectly since they represent different colors in a scene, so I instead scored different alignments based on the edges in each negative by using a Sobel filter. To score different alignments, I used the L2 distance between one negative's feature vector and the other's. Besides using a Sobel filter, I also utilized adaptive histogram equalization to adjust the contrast correctly in each aligned image, and used white balancing to adjust the colors as well.

Sample Inputs

cathedral

monastery

tobolsk

Results

cathedral

R alignment: 12 pixels down, 3 pixels right
G alignment: 5 pixels down, 2 pixels right

monastery

R alignment: 3 pixels down, 2 pixels right
G alignment: 3 pixels up, 2 pixels right

tobolsk

R alignment: 6 pixels down, 3 pixels right
G alignment: 3 pixels down, 3 pixels right

church

R alignment: 58 pixels down, 4 pixels left
G alignment: 24 pixels down, 4 pixels right

emir

R alignment: 107 pixels down, 41 pixels right
G alignment: 49 pixels down, 23 pixels right

harvesters

R alignment: 123 pixels down, 15 pixels right
G alignment: 59 pixels down, 18 pixels right

icon

R alignment: 90 pixels down, 23 pixels right
G alignment: 41 pixels down, 18 pixels right

lady

R alignment: 114 pixels down, 11 pixels right
G alignment: 53 pixels down, 7 pixels right

onion_church

R alignment: 108 pixels down, 37 pixels right
G alignment: 50 pixels down, 26 pixels right

self_portrait

R alignment: 175 pixels down, 37 pixels right
G alignment: 77 pixels down, 29 pixels right

three_generations

R alignment: 110 pixels down, 14 pixels right
G alignment: 50 pixels down, 17 pixels right

train

R alignment: 86 pixels down, 32 pixels right
G alignment: 42 pixels down, 6 pixels right

workshop

R alignment: 106 pixels down, 12 pixels left
G alignment: 53 pixels down, 1 pixels left

Extra Results

village

R alignment: 4 pixels up, 1 pixels right
G alignment: 3 pixels up, 1 pixels right

park

R alignment: 14 pixels down1 pixels left
G alignment: 3 pixels down, 0 pixels left

Bells & Whistles

Better Features

In pre-processing, I applied a sobel edge filter to each negative so that instead of aligning based on pixel values we aligned on the edges in each negative. This greatly improved alignments, especially for the Emir image.

Before

After

Automatic Contrasting

To accomplish automatic contrasting, I utilized adaptive histogram equalization implemented within scikit-image under the function name 'equalize_adapthist'. This function was applied after the proper displacements were found and the channels combined into a single image.

Before

After

Automatic Color Balancing

I implemented 2 forms of color balancing. First, I did white balancing which found the highest intensity value in each channel and normalized it such that the highest value was 1 (with an image scale of [0, 1]). Second, I did gray balancing which found the average value of each channel and normalized them such that the average was 0.5. This was also done post-processing This was also done in post-processing. The example below does gray balancing and improves the red hues in the bottom right.

Before

After