CS 194-26 Project 1 - Colorizing the Prokudin-Gorskii Photo Collection

Andy Wu, Fall 2020

Overview

Sergei Mikhailovich Prokudin-Gorskii, a pioneer in color photography, traveled throughout the Russian Empire, documenting what he saw. He photographed a variety of scenes using 3 lenses, with red, blue, and green filters. The glass negatives can then be combined to reconstruct a colored photograph.

Approach

In this project, automate the reconstruction of the colored images captured by Prokudin-Gorskii. we split the glass negatives into their respective color channels and then align the red and green color channels to the blue channel, producing a sharp color image.

To align the three color channels, I implemented two metrics, Sum of Squared Differences (SSD) and Normalized Cross Correlation (NCC). We compare the pixel brightness values, and pick the alignment with the best SSD or NCC score.

This method requires looping through all possible alignment offsets to find the one with the best score. Through testing, I found that the performance of SSD and NCC were about the same, so I used NCC for aligning the images. I reduced the amount of computation by limiting the offset to within +/-10% of the size of the image, since the different channels are pretty close to aligned to start with.

Another major optimization was to implement an image pyramid. Instead of searching through every pixel of a 4K image, we can recursively downscale the image, and find the best offset for the smaller image, and then use that as a starting search point in the original image. This optimization reduces the amount of computation drastically, allowing the program to find the optimal offset within a few seconds.

Provided Images

Castle

green offset: [35, 3]
red offset: [98, 4]

Cathedral

green offset: [5, 2]
red offset: [12, 3]

Emir

Due to the different brightness values for the red color channel, the alignment process does not work very well. The blue and green channels are aligned properly.
green offset: [49, 24]
red offset: [ 309, -333]

Harvesters

green offset: [60, 17]
red offset: [124, 13]

Icon

green offset: [41, 17]
red offset: [89, 23]

Lady

green offset: [55, 8]
red offset: [117, 11]

Melons

green offset: [82, 11]
red offset: [178, 13]

Monastery

green offset: [-3, 2]
red offset: [3, 2]

Onion Church

green offset: [51, 27]
red offset: [108, 36]

Self Portrait

green offset: [79, 29]
red offset: [176, 37]

Three Generations

green offset: [53, 14]
red offset: [112, 11]

Tobolsk

green offset: [53, 14]
red offset: [112, 11]

Train

green offset: [43, 6]
red offset: [87, 32]

Workshop

green offset: [53, 0]
red offset: [105, -12]

Other Examples

Boat

green offset: [21, 16]
red offset: [146, 32]

Bridge

green offset: [26, -17]
red offset: [77, -37]

House

green offset: [26, 9]
red offset: [63, 10]