CS194-26
Project 1 - Images of the Russian Empire: Colorizing the Prokudin-Gorskii photo collection

Kevin Chen (cs194-26-aag)

Project Background







Sergei Mikhailovich Prokudin-Gorskii was able to capture color images by taking three exposures of the same scene, through red, green, and blue filters. In this project, I created an algorithm to create a color image out of the three different exposures by cropping the original glass panes into the three color channels, aligning, and finally stacking the different exposures. The method of evaluation was the sum of squared differences between pixels, and for larger images, I implemented a recursive image pyramid algorithm to speed up processing time.

Cathedral





Processing Small Image Files: Basic SSD Evaluation

As suggested by the project spec, I started off by using SSD evaluation as my primary metric in comparing two images. By scanning over a [-15, 15] window of displacements and keeping track of the minimum SSD and its respective displacement, we were able to align and reconstruct smaller image files including all of the jpg's which are displayed below. These images all turned out pretty good!

CathedralCathedral: No Shift

MonasteryMonastery: No Shift

TobolskTobolsk: No Shift




























CathedralCathedral: Red[3, 12], Green[2, 5]

MonasteryMonastery: Red[2, 3], Green[2, -3]

TobolskTobolsk: Red[3, 6], Green[3, 3]




























Processing Large Image Files: SSD Evaluation with Image Pyramids

For larger files like the tif's, we needed to figure out a way to process larger files without taking an exorbitant amount of time. The solution was image pyramids which essentially recursively shrunk our image by 50% until the height and width (in pixels) were less than 250. Then, we would run our original SSD processing over the [-15, 15] window and get the displacement. Finally, we would rescale up while running SSD on a [-1, 1] displacement window to further refine the final displacement values. This sped the processing time up significantly as we only scan the 900 pixel window once, at the base case of recursion and on the smallest image.

The results for the larger image files were a bit more mixed. In particular, Emir, Harvesters, and Village could have some improvement. As noted in the project spec, the exposures of Emir did not have the same brightness level which attributed the most to the flawed reconstruction. As for Harvesters and Village, I believe the over saturation of greens may have skewed the displacement results.

EmirEmir: No Shift

HarvestersHarvesters: No Shift

IconIcon: No Shift




























EmirEmir: Red[34, 52], Green[24, 49]

HarvestersHarvesters: Red[13, 95], Green[16, 60]

IconIcon: Red[22, 89], Green[17, 40]




























LadyLady: No Shift

MelonsMelons: No Shift

Onion ChurchOnion Church: No Shift




























LadyLady: Red[-4, 136], Green[0, 55]

MelonsMelons: Red[12, 178], Green[4, 82]

Onion ChurchOnion Church: Red[36,108], Green[25, 52]




























Self PortraitSelf Portrait: No Shift

Three GenerationsThree Generations: No Shift

TrainTrain: No Shift




























Self PortraitSelf Portrait: Red[-3, 170], Green[-1, 80]

Three GenerationsThree Generations: Red[9, 111], Green[12, 54]

TrainTrain: Red[29, 87], Green[2, 43]




























VillageVillage: No Shift

WorkshopWorkshop: No Shift




























VillageVillage: Red[0, 90], Green[0, 64]

WorkshopWorkshop: Red[-13, 101], Green[-1, 53]




























Misc Notes

Looking at the original images, one will notice black borders. After a little trial and error, I chose values of 20 and 100 for jpg and tif respectively, to crop in from each side. This helped with our SSD evaluation's accuracy as the borders were throwing off our calculations. These parameters could be played with to further optimize results.

Extra Images from the Prokudin-Gorskii collection

3 extra images showing the before and after

CastleCastle: No Shift

LandscapeLandscape: No Shift

Still LifeStill Life: No Shift




























CastleCastle: Red[0, 5], Green[0, 2]

LandscapeLandscape: Red[-13, 102], Green[-7, 26]

Still LifeStill Life: Red[0, 11], Green[0, 2]