# Images of the Russian Empire: Colorizing the Prokudin-Gorskii photo collection

## Overview

In the early 1900s, Prokudin-Gorskii won Tzar's special permission to travel across the vast Russian Empire and take color photographs of everything he saw. He recorded three exposures of every scene onto a glass plate using a red, a green, and a blue filter. In this project, we take the digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce color images.

The general approach I took in this project is that, given a 3-channel image, I first divided it into three equal parts and aligned the second and the third parts (G and R) to the first (B). And then I implemented an alignment algorithm to find the best (x,y) displacement vector for each image and output one sharp color image.

## Naive Alignment

For smaller .jpg images, I implemented a simple single-scale approach: I first choose Sum of Squared Differences (SSD) to be my metric, and then use for loops to search exhaustively over a specified window of displacements. Finally, the algorithm returns the displacement that gives the lowest SSD. Note that before condusting this algorithm, I first cropped the border a little bit from all sides of the R, G, B images.

cathedral.jpg / Red: (12, 3) / Green: (5, 2)

monastery.jpg / Red: (3, 2) / Green: (-3, 2)

tobolsk.jpg / Red: (6, 3) / Green: (3, 3)

## Pyramid Alignment

For larger .tif images, exhaustive search will become prohibitively expensive due to the large pixel displacement. Thus, I had to implement a pyramid speedup for the alignment algorithm. The pyramid alignment basically means that we maintain a multi-level image stack, where each level contains the same image but at different scales. The top-level is the image with the smallest size, while the bottom level stores the image with the original size. In order to find the best displacement, instead of exhaustively searching for it on the origin image, I first downscaled the image to smaller size; after obtaining the displacement on this small image with the help of proper metrics, I calculated the range of possible displacements for next level. Repeat this process for several times until I reached the bottom level - in this way, I could find the best displacement from a much smaller region.

castle.tif/ Red: (98, 4) / Green: (35, 3)

emir.tif / Red: (132, -283) / Green: (49, 24)

harvesters.tif / Red: (124, 13) / Green: (59, 16)

icon.tif / Red: (89, 23) / Green: (41, 17)

lady.tif / Red: (111, 11) / Green: (49, 9)

melons.tif / Red: (179, 13) / Green: (82, 10)

onion_church.tif / Red: (108, 36) / Green: (51, 26)

self_portrait.tif / Red: (176, 37) / Green: (78, 29)

three_generations.tif / Red: (112, 11) / Green: (53, 14)

train.tif / Red: (87, 32) / Green: (42, 5)

workshop.tif / Red: (104, -12) / Green: (52, 0)

### Extra Images

antique.jpg / Red: (14, -1) / Green: (6, 0)

apple_trees.jpg / Red: (11, 9) / Green: (5, 5)

arc.jpg / Red: (9, 1) / Green: (4, 1)

harvest.jpg / Red: (14, 4) / Green: (2, 3)

## Bells & Whistles

Before

emir.tif / Red: (132, -283) / Green: (49, 24)

After

emir.tif / Red: (107, 40) / Green: (49, 23)

Due to the brightness inconsistency across three channels, the previous algorithm was not able to find the proper displacement for emir.tif. In order to get rid of the influence of the brightness, I used the the Sobel filter from the skimage library to find edges in the R, G, B image, and find the displacement with these edge images instead of the original images. And it turns out that this approach gives me pretty good result.