Images of the Russian Empire:
Colorizing the Prokudin-Gorskii photo collection

Roma Desai | CS-194 Project 1

 

BACKGROUND

 

Sergei Mikhailovich Prokudin-Gorskii (1863 – 1944) was a Russian chemist and photographer known for his pioneering work in the field of color photography. He toured the Russian Empire photographing the country using three different exposures for every scene with red, green and blue filters. Although there was no way to translate these images into color photographs at the time Prokudin-Gorskii was alive, his work was later recognized and digitized a few years after. In this project, I will be taking Prokudin-Gorskii’s glass plate negatives and aligning them to produce the color versions of his original photographs. I have used a variety of methods such as NCC, image pyramids, and more to accurately reconstruct the images.

 

 

PROCESS

 

Each image file was a copy of the same image with three different filters stacked vertically. The top, middle, and bottom images corresponded to the blue, green, and red filters respectively. To begin, I separated these three images and employed various techniques to align the three-color channels properly. At the basic level, these techniques involved searching over all possible displacements of the green and red filtered images and aligning them to the blue filtered image.

 

 

 NORMALIZED CROSS-CORRELATION (NCC)

 

NCC involved searching over the range of all possible displacements and taking the dot product to find the best displacement vector. This displacement vector was then used to align the green and red images to the blue image. To find the displacement vector, I searched over a range of [-15, 15] pixel displacement and used the displacement that maximized the following equation:

 

I1/I1 * I2/I2 where I1 and I2 are image vectors.

 

While this method worked well, the black and white borders of the images threw the algorithm off course for certain images. To remedy this issue, I cropped the image edges by about 12% of the total image size before taking the dot product to find the displacement vector. 

 

Overall, this technique worked well for smaller images (< 400 x 400 pixels) and a few results are shown below:

 

 

 

IMAGE PYRAMIDS

 

 

While the NCC method worked well for smaller images, the runtime for exhaustive searching on larger images was too long.  For larger images, I used image pyramids to search lower resolution versions of the image first and slowly move up to finer and finer alignment. I scaled each image down until it was manageable by the NCC algorithm and then recursively called the NCC function while scaling the image back up to normal to get the exact displacement vector. This method worked well for larger images and a few results are shown below.

 

 

 

RESULTS

 

 

Here is the result of my algorithm on all of the images (+ 3 of my own choosing)! While the above algorithms work well for most images, not all of the images fared as well.

 

 

 

 

 

 

IMPROVEMENTS

 

 As shown above, the emir did not do as well due to having different brightness values across the three filtered images. For images such as the emir, analyzing raw pixels will not be enough to accurately recreate the image. 

 

 

Improvement 1: Canny Edge Detection

 

Because raw pixels cannot be used to align the emir, we must look at the edges in the picture and align the red and green filtered images on top of the blue filtered image based on edges. I used the canny edge detector function from the skimage python library which significantly improved the image. I first generated an image showing the edges of the emir image as shown below. I then used that image to calculate the displacement. Here is the before and after result on the emir image:

 

 

 

 

Improvement 2: Auto Contrasting

 

The NCC and image pyramid algorithms also caused many images to have low contrast. To remedy this, I implemented a technique known as histogram equalization. Histogram equalization spreads out the most frequent intensity values of an image. Here are few results: