Images of the Russian Empire:

Colorizing the Prokudin-Gorskii photo collection

David Tseng

Overview

The goal of this project is to colorize the photos in the Prokudin-Gorskii photo collection. The photos in this collection are basically triplets of grayscale photos, each representing the R, G, and B channel of the colorized image. In order to provide the colorized version of the image, we can align the three channels, stack them as an RGB image, and then display it.

Methods

The main method I used to align two channels together was to iteratively try to shift one channel in the x and y directions until the normalized cross correlation (NCC) between the pixels was maximized. In order to remove the effects of the image border on the NCC, I only looked at a window centered on the image. For an image of size (m,n), I used a window of size (m/2, n/2). However, the main issue with this method is that it is computationally expensive. For large images (such as the .tif files), this brute-force method takes quite awhile. To address this problem, I used the suggested image pyramid scheme to search for the optimal alignment. The image pyramid scheme (described in detail here) is essentially a divide-and-conquer method where I recursively downsample the original image and find the alignment on the smaller image. This method speeds up the computation time significantly, but it still not able to align the emir.tif file properly.

Extra credit

For extra credit, I applied the Canny edge detector to the images beforehand, and then aligned the resulting images using NCC. With this method, I was able to successfully align all of the images properly.

Results

Below are the results from applying the naive alignment method with normalized cross-correlation to the smaller example .jpg files.

nativity.jpg | Green: (3, 1), Red: (8, 0) monastery.jpg | Green: (-3, 2), Red: (3, 2)
cathedral.jpg | Green: (4, 2), Red: (11, 3) settlers.jpg | Green: (7, 0), Red: (14, -1)

Below are the results from applying the image pyramid alignment method with normalized cross-correlation to the larger example .tif files.

harvesters.tif | Green: (59, 16), Red: (123, 13) icon.tif | Green: (41, 17), Red: (90, 23)
emif.tif | Green: (48, 24), Red: (0, 0) village.tif | Green: (64, 12), Red: (137, 22)
turkmen.tif | Green: (56, 21), Red: (116, 28) self_portrait.tif | Green: (78, 29), Red: (175, 37)
train.tif | Green: (43, 6), Red: (87, 32) three_generations.tif | Green: (50, 14), Red: (110, 11)
lady.tif | Green: (53, 8), Red: (116, 11)

Notice that the emir.tif above is not aligned correctly. This is likely due to the fact that the three channels do not have the same brightness values. To fix this, we can first process the image with the Canny edge detection method before using the image alignment method. Since the other images have already been aligned successfully, only emir.tif is shown below for comparison.

Without edge detection With edge detection
Green: (48, 24), Red: (0, 0) Green: (49, 23), Red: (103, 41)

Below are several additional images from the Prokudin-Gorskii collection, aligned with the edge detection + image pyramid method.

00912v.jpg | Green: (1, 1), Red: (2, 1) 00876v.jpg | Green: (4, 0), Red: (8, 0)
00335a.tif | Green: (13, -1), Red: (82, -2) 01087a.tif | Green: (77, 28), Red: (168, 43)