Project 1: Images of the Russian Empire: Colorizing the Prokudin-Gorskii photo collection

Benjamin Wu, cs194-26-afu

Background

Sergei Mikhailovich Prokudin-Gorskii revolutionized a concept of color photography in recording images onto a glass plate. He hoped he would portray photos and history books for the youth in a completely new light, educating them in a different and diverse culture and show the modernization of Russia. Many noticed his promise and innovation, including the Tsar, as he traveled throughout Russia with his red, blue, and green filters. His work still resounds today as this project allows us to use the full-size glass plate images to produce the spectacles that Sergei had hoped for.

Small Scale JPEG implementation

To first gather intuition, I proceeded to think about the small scale JPEGs. Even before this, I attempted just putting the pieces on top of each other to achieve this:



Then I proceeded to implement the mentioned SSD by shifting my red and green filters by a maximum of 15 in any direction across my blue filter as reference (blue always used as reference throughout). By calculating the L2 Norm across every one of these shifts and proceeding with the minimum, I achieved a naive SSD alignment, as shown below:



While this worked reasonably well, I looked at Piazza and was curious that NCC performed even better, so I implemented with values based on the normalized dot product and used this throughout the rest of the project (albeit for the JPEGs, it didn't change anything).

Cropping

I only ended up cropping a hardcoded 10% (5 on either edge) of the image to take away artifacts near its border. The result of which can be seen here:

Preprocessing with a filter

After the subtle hint given in lecture, I preprocessed each of the three filters with a Sobel filter in order to accentuate edges and therefore, more clearly align. See below for results:

Results

Pyramid Scheme

Now grasping the general idea, I can now contemplate things like runtime efficiency and much much larger images. Using the image pyramid as mentioned in the spec, I recursively halved the size to an allowed base where my naive NCC could process it and then built back up. By building back, we scale the offset and then check a small area around that calculated offset. I determined to go with a small 6x6 box from (-2, 3) in both directions and a base case of at most 10000 of width*height. This worked out quite smoothly as seen in the results.

Challenges

I had to stay in a lab for all of today because I did not want to figure out my Windows problems on my laptop. Other issues were my own bugs, such as forgetting to stack the updated Red and Green filters and tracing my loops/recursion.

Results

Bonus Gallery from the Prokudin-Gorskii Collection

Picture	Green Alignment	Red Alignment
Cathedral	(2,5)	(3,12)
Castle	(-18,-10)	(-53,5)
Emir	(23,49)	(107,40)
Flowerbush	(20,76)	(31,155)
Harvesters	(17,60)	(13,123)
House	(18,11)	(37,35)
Icon	(17,42)	(23,90)
Lady	(9,56)	(13,120)
Monastery	(2,-3)	(2,3)
Nativity	(1,3)	(1,7)
Self Portrait	(29,78)	(37,176)
Settlers	(-1,14)	(0,7)
Three Generations	(12,54)	(8,111)
Train	(1,41)	(29,85)
Turkmen	(22,57)	(28,117)
Village	(12,64)	(137,21)