Colorizing the Prokudin-Gorskii photo collection

by Pauline Hidalgo

Overview

The goal in this project was to programmatically align the red and green color channels to the blue color channels of Gorskii's glass plate images, creating a sharp RGB color image.

Details

To get each color channel, each input image is split into thirds. To prevent incorrect alignment based on the black and white borders of the image, I crop each color channel by 7.5% of pixels on each side.

To measure how similar 2 color channels were to each other, I implemented sum of squared differences and normalized cross correlation. NCC generally gave a cleaner output, so that is what was maximized in the search (as opposed to minimizing SSD).

For smaller .jpeg images, it was sufficient to do an exhaustive search over [-15, 15] displacements in both x and y directions. For larger .tif images, I used an image pyramid with a scale factor of .5 and 6 levels. At the top-most level, I look at the image at 1/32nd the resolution of the original and initially do an exhaustive search over [-15, 15] displacements in both x and y directions. For subsequent levels, I recursively rescale the image and the chosen displacements by a factor of 2 and search over [-1, 1] displacements in both x and y directions until the bottommost level.

The large difference in brightness between the R and B channels causes the R channel in emir.tif to look misaligned. Since the NCC uses raw pixel values, it will not be large even for good displacements due to this inconsistency in brightness. I noticed the G and B channels aligned well and were more consistent in brightness, so I tested out aligning to G instead of B and the result was much better.

Images

All example images, plus a few of my own choosing. Offsets are displayed as color: (x offset, y offset)

red: (12, 3), green: (5, 2)

red: (3, 2), green: (-3, 2)

red: (6, 3), green: (3, 3)

red: (108, 36), green: (52, 26)

red: (176, 36), green: (78, 28)

red: (124, 14), green: (60, 16)

red: (178, 12), green: (82, 10)

red: (112, 10), green: (54, 14)

red: (86, 30), green: (42, 4)

red: (104, -12), green: (52, 0)

red: (90, 22), green: (40, 16)

red: (98, 4), green: (34, 2)

red: (150, -494), green: (48, 24)

red: (56, 16), blue: (-48, -24)

red: (112, 12), green: (46, 8)

red: (114, -24), green: (52, 0)

red: (114, -16), green: (54, 0)

red: (116, -38), green: (26, -18)


Bells and Whistles

I noticed some images had strong overtones of one color and low contrast, so I wanted to try automatically adjusting the contrast using histogram equalization from scikit-image, which I applied to each of the color channels before aligning them as before. By spreading out the most frequent intensity values, the images look much more vibrant, but the noise in the background becomes more prominent as well.

original
red: (114, -16), green: (54, 0)

equalized
red: (114, -16), green: (54, 0)

original
red: (104, -12), green: (52, 0)

equalized
red: (106, -10), green: (52, 2)