Images of the Russian Empire

Overview

Sergei Mikhailovich Prokudin-Gorskii (1863-1944) [Сергей Михайлович Прокудин-Горский, to his Russian friends] was a man well ahead of his time and was especially intrigued with color photography. With the support of the Tzar, he came up with this idea: record three exposures of every scene onto a glass plate using a red, a green, and a blue filter in each exposure.

In this project, we are tasked to automatically output a color image given grayscale glass plates images from the Prokudin-Gorskii Collection. To do so, we need to extract three color channels, place them on top of each other, and align them to form an RGB image.

Primitive Approach: Image Alignment

In our primitive approach, we exhaustively search over an image grid of possible displacements (I chose [-15, 15] pixels) and score each displacement with the normalized cross correlation metric (NCC). Another is normalized cross-correlation (NCC), which is simply a dot product between two normalized vectors: (image1./||image1|| and image2./||image2||). 

This works well for images that are relatively small (less than 400 x 400 pixels), but exhaustive search becomes prohibitively expensive if pixel displacement is too large. Thus, I need to implement a faster, optimized search procedure - an image pyramid to solve the problem.

Optimized Approach: Adding an Image Pyramid

In our optimized version, we are essentially still using the same approach as the naive version - testing different displacements for each channel against a reference channel (which I set as blue). However, I recursively downscale my image by a scaling factor of 2, using skimage.transform.rescale until I hit a base case. The base case occurs when the image has been downscaled to be less than 200 x 200 pixels (an arbitrary size I chose). Once we hit this case, I use search grid of [-15, 15] to search for the optimal (x, y) displacement for the downscaled image. At each level as we recurse back up to the original size image, I multiply the displacement returned at the previous pyramid layer (downscaled image displacement) by the scaling factor of 2 and add that to the optimal displacement of the current sized image within a [-1, 1] pixel search grid. Conceptually, we are just tweaking images more and more as the quality of the image becomes higher and higher at each level.

Problems I Encountered & Fixes

To decide on the base image size of 200 x 200 pixels was pretty challenging. I tested all the images with different based case image sizes (400 x 400, 100 x 100, 200x200) and overall 200 x 200 was the best one. I originally used 400 x 400 for my base image size but I found that for self_portrait, a smaller base case image size was required for the image to be aligned properly. .

The second issue I had was with emir.tif. Because of the saturation of the emir’s coat - which is blue, I had to change the reference channel I am shifting the other images to from blue to green to fix this problem. Once I did that, the image came out aligned with just the image pyramid optimization.

Final Images

My Own Colorized Photos the Prokudin-Gorskii Glass Plates