Colorizing the Prokudin-Gorskii Photo Collection

Background

Sergei Mikhailovich Prokudin-Gorskii travelled around the Russian Empire taking "color" photos by recording each scene three times, using a different red, blue, or green filter. Here, we attempt to produce color images from his negatives using image processing techniques.

Approach

We split the original image into three parts, the blue, red and green channels simply by taking the overall height of the image and dividing by three. We then attempt to align the red and green images to the blue image.

To accomplish this alignment we calculate value difference of the different color channels for possible displacements within a $[-n, n]$ window. We tried using both the L2 norm and the NCC (normalized cross-correlation), however the L2 norm was much faster and there was no visible difference.

This works fine for relatively small images, but in order to speed up processing for larger images, we utilize an image pyramid. For the most downscaled images, we use a larger window of displacement, then translate that to the next level of scaling, etc. Thus, when we align the original images, we only need to use a relatively small window to get a good result.

Because of the nature of the negatives, there tended to be a lot more noise around the borders, so we also performed alignment by cropping each side by about 5% to produce better results.

Some images with very colorful red, blue, and green portions were difficult to align using just by computing RGB differences of the pixels. This is because the values of red, green, and blue for the same pixel varied widely. Instead, we computed the $0.5|(x-1,y) - (x+1, y)|$ of every point in attempt to mimick the gradient of the image and then performed alignment using the filter image.

We also attempt to automatically crop out the border of the image, with varying degrees of success. We start at the edge rows and columns for each color channel and checking how much each pixel row or column differs from each other using the standard deviation. If the standard deviation is greater than a certain threshold, we stop cropping. This method worked well for the white edges, but did not work very well for the colored edges.