CS 194-26 Project 1

Colorizing the Prokudin-Gorskii Photo Collection


Overview

In this project, we implement a program that automatically colorizes digital Prokudin-Gorskii glass plate images. The result is achieved by extracting the three color channel images and properly aligning them through image-processing techniques.

Non-aligned colorized workshop image, with all three color channels naively stacked on top of each other.

Exhaustive search approach

In this preliminary approach, we remove a margin of 15% around the edges of each color channel. We align the channels by searching over a window of displacements (-15, 15) along both axes, scoring each one with an image matching metric. The best score corresponds to the displacement that most accurately colorizes the image. In our program, we run the L2 norm (SSD) and normalized cross-correlation (NCC) metrics with similar results.

monastery, R: (2, 3), G: (2,-3)
cathedral, R: (3, 12), G: (2,5)
tobolsk, R: (3, 6), G: (3,3)

Image pyramid

For larger images, we implement a more efficient procedure-- the image pyramid. The recursive approach rescales the image down to the coarsest scale, where a single-scale search is run. Once this estimate is returned, we utilize this information to narrow our search in larger images further down the pyramid.

emir, R: (0, 6), G: (24, 48)
icon, R: (23, 89), G: (17, 41)
lady, R: (11, 113), G: (8, 52)
self_portrait, R: (37, 176), G: (29, 78)
harvesters, R: (13, 123), G: (17, 59)
melons, R: (13, 178), G: (10, 81)
three_generations, R: (11, 111), G: (13, 52)
train, R: (32, 87), G: (6, 42)
village, R: (22, 137), G: (12, 64)
workshop, R: (-12, 105), G: (0, 53)
onion_church, R: (38, 105), G: (27, 50)

Additional images

Here are some additional images I picked out from the Library of Congress collection.

barracks, R: (0, 7), G: (0, 1)
cabin, R: (-5, 10), G: (-3, 2)
fence, R: (1, 12), G: (1, 3)
railroad, R: (-2, 8), G: (-1, 1)
reflection, R: (1, -4), G: (1, -3)

Bells and whistles

1) Sobel filters for better alignment

For some images, such as emir.tif, the color channels do not align properly. This can be fixed by preprocessing the images with a sobel filter. When we process our color channels based on their sobel filters, we are able to align the channels on their edges and produce an accurate colorization.

emir, R: (0, 6), G: (24, 48)
emir aligned using sobel filter, R: (41, 105), G: (24, 49)
melons, R: (13, 178), G: (10, 81)
melons aligned using sobel filter, R: (13, 177), G: (10, 80)

2) Automatic cropping

For automatic cropping, I utilized the Sobel edge filter to determine where the horizontal and vertical edges are. Then, I determined which lines take up more than a given percentage of the total width/height of the image (this is a parameter which can be adjusted for more aggressive or conservative cropping). The final crop is then decided based on the location of these lines across all three channels. We can see that this results in generally good crops, although there is some color bleeding in a few of the pictures.

self_portrait
self_portrait cropped
melons
melons cropped
icon
icon cropped
emir
emir cropped

3) White balancing

For white balancing, I took the middle 92% of pixel values and rescaled the intensity of the pixel values based on these values. We can see that certain white features in the photos below appear brighter, such as the emir's headpiece or the clothes of the harvesters. The shadows also appear darker, as can be seen with the photo of the man sitting in the shadows with the melons.

emir cropped
emir cropped with white balance
harvesters cropped
harvesters cropped with white balance
melons cropped
melons cropped with white balance

4) Automatic contrasting

To automatically increase the contrast of the images, I used adaptive histogram equalization. Adaptive histogram equalization works by looking at a pixel's neighbors in order to set its intensity values. I found it to work better than global histogram equalization, which sometimes resulted in strange looking photos since we are looking at all pixel values across the photo.

icon
icon with automatic contrast
self_portrait
self_portrait with automatic contrast