Colorizing the Prokudin-Gorskii photo collection

Kaushal Partani

Overview

Gorsky was a Russian chemist and photographer who decided to document 20th century Russia prior to color photography. By taking pictures with red, green, and blue filters, Gorsky was able to create color images of landscape, scenery, and lifestyles of early 20th century Russia. This project tackles recreating the color image via image processing.

Process

My code first splits the 3 layer images into 3 separate parts that are not aligned. For smaller images, such as the jpg files I proceed with the following process:

1. Grab center 50 by 50 pixels square
2. For displacements in the range of [-15 to 15], calculate the best shift by minimizing the SSD between the following pairs: (green, blue) and (red, blue). Note that I used the blue filter as my base image.
3. Roll the green and red images, and then stack them atop each other to get the resultant color image.

However, this method does not work on the larger tif files that reach close to 100 Mb per image. For these images, I used a pyramid speed up, which works as follows:

1. Start by cropping the edges from all 3 filter images.
2. For each pair (green, blue) and (red, blue) create two image stacks that contain the image at various resolutions.
3. Line up the lowest resolution pair (one from the top of both stacks) with a significant displacement, around [-15 to 15] for ~500 X 500 px images.
4. Process the rest of the stack in pairs, slightly adjusting the scaled result from the previous iteration by small amounts of displacement such as [-5 to 5].

The reason this works is because now, at higher resolutions, we do not need to run heavy computations such as rolling for each displacement (x,y) pair.

Failures

This process only really failed on emir.tif, the reason being that the brightness values across all channels are not consistent. Unfortunately, with emir.tif, the brigtness of the blue channel is brighter compared to the red and green channels, meaning that the SSD scores would be skewed. This led to the incorrect alignment that results for emir.tif in the following images.

Images, Shifts, and Execution Times

monastery

1. green shift: (2, -3)
2. red shift: (2, 3)
3. execution took: 0.08601951599121094 seconds

tobolsk

1. green shift: (3, 3)
2. red shift: (3, 7)
3. execution took: 0.08802437782287598 seconds

cathedral

1. green shift: (2, 5)
2. red shift: (3, 12)
3. execution took: 0.08701920509338379 seconds

flowers

1. green shift: [-3, 17]
2. red shift: [-14, 118]
3. execution took: 22.1284658908844 seconds

lady

1. green shift: [9, 52]
2. red shift: [12, 113]
3. execution took: 21.513184309005737 seconds

coast

1. green shift: [-17, 25]
2. red shift: [-32, 100]
3. execution took: 22.308473110198975 seconds

workshop

1. green shift: [0, 53]
2. red shift: [-12, 105]
3. execution took: 21.57487416267395 seconds

onion_church

1. green shift: [27, 51]
2. red shift: [36, 108]
3. execution took: 21.945960760116577 seconds

strawberries

1. green shift: [20, 23]
2. red shift: [25, 58]
3. execution took: 21.091282844543457 seconds

three_generations

1. green shift: [14, 52]
2. red shift: [11, 111]
3. execution took: 21.351343154907227 seconds

self_portrait

1. green shift: [29, 79]
2. red shift: [37, 176]
3. execution took: 22.551103115081787 seconds

icon

1. green shift: [17, 41]
2. red shift: [23, 89]
3. execution took: 21.89848017692566 seconds

train

1. green shift: [5, 42]
2. red shift: [32, 87]
3. execution took: 21.877399682998657 seconds

harvesters

1. green shift: [17, 59]
2. red shift: [13, 123]
3. execution took: 21.714906215667725 seconds

castle

1. green shift: [3, 34]
2. red shift: [4, 98]
3. execution took: 22.040982484817505 seconds

emir

1. green shift: [24, 49]
2. red shift: [-165, 2]
3. execution took: 21.29481339454651 seconds

melons

1. green shift: [10, 81]
2. red shift: [13, 178]
3. execution took: 22.131022453308105 seconds

sheksna

1. green shift: [18, 21]
2. red shift: [30, 56]
3. execution took: 21.127774000167847 seconds