CS 194-26 Project 1

Colorizing the Prokudin-Gorskii photo collection

Sahil Upadhyay

Overview

In the early 20th century, Sergey Mikhaylovich Prokudin-Gorskii traveled across the Russian empire and took color photos of everything he saw. He captured three exposures of each scene onto a glass plate using separate red, green, and blue filters. In this project, we're attempting to align each set of three negatives to reproduce the images captured by Prokudin-Gorskii.

Approach

My approach toward aligning the images utilizes the sum of squared distances (SSD) between each pair of negatives. Theoretically, after attempting different combinations of horizontal and vertical translations between the negatives, the optimal alignment is the one that produces the smallest SSD.

I had also implemented normalized cross-correlation (NCC) as a metric, which represents two images as normalized vectors and looks at their dot product. Ideally, the vectors with the highest dot product would indicate the best alignment. This approach ends up giving me basically the same results as SSD, so I stick with SSD throughout the project.

The basic algorithm involves using the green color plate as the base, then aligning the red and blue plates with it as best as possible. For small images, this shifts the non-base plate up to 15 pixels in either direction along the x and y axes for a total of 900 possible alignments. Over all of those alignments, I keep track of the shift that minimizes the SSD. Once this is done for both the red and blue plates, the three layers are combined into one image for the final result.

For larger images, I uses an image pyramid. This scales the input images down 50% repeatedly until the largest dimension is not greater then 300 pixels. Here, I run the basic algorithm on the small image, which gives me a general neighborhood to refine the alignment within. From here, I double the current image size and realign the image within a radius of 3 pixels around the previous coarse alignment; this process is repeated until I reach the original image size.

These approaches give me satisfactory results. There are still some chromatic aberrations, but they seem to arise from the data in the plates rather than from the alignment.

Small Images

Note: all listed shift amounts are relative to the green color plate.

red shift (1, 7); blue shift (-2, -5)

red shift (-1, 4); blue shift (-1, -3)

Large Images

red shift (17, 57); blue shift (-24, -49)

Extra Images

red shift (13, 56); blue shift (9, -12)

Bells and Whistles

Automatic Contrasting

I scale the intensities of the images such that the darkest pixel reached 0 and the brightest pixel reached 1, if they aren't already. Though the results are not drastic, this process increases the dynamic range of the images and makes them appear less flat.

Automatic White Balancing

The first step in automatic white balancing is estimating the illuminant, which I assume is the average color of the image. After calculating this color, I manipulate each channel to shift the illuminant to gray.

In this first example, automatic white balancing removes the bluish cast from the image and makes it seem more natural.

Original

However, in this second example, automatic white balancing adds a blue tone to the image that I find less appealing.

Original

Possible Improvements

In many of my outputs, I don't see noticeable improvements or changes after implementing my bells and whistles. This might be in part due to the algorithms I use for both automatic contrasting and white balancing, which take into consideration the data from the image borders. This border data is irrelevant to the image itself, which I assume throws off the results. However, I feel like I would see better results by cropping out the borders and then running the automatic contrasting and white balancing procedures.