Sergei Mikhailovich Prokudin-Gorskii was able to capture color images by taking three exposures of the same scene, through red, green, and blue filters. In this project, I created an algorithm to create a color image out of the three different exposures by cropping the original glass panes into the three color channels, aligning, and finally stacking the different exposures. The method of evaluation was the sum of squared differences between pixels, and for larger images, I implemented a recursive image pyramid algorithm to speed up processing time.
As suggested by the project spec, I started off by using SSD evaluation as my primary metric in comparing two images. By scanning over a [-15, 15] window of displacements and keeping track of the minimum SSD and its respective displacement, we were able to align and reconstruct smaller image files including all of the jpg's which are displayed below. These images all turned out pretty good!
Cathedral: No Shift
Monastery: No Shift
Tobolsk: No Shift
Cathedral: Red[3, 12], Green[2, 5]
Monastery: Red[2, 3], Green[2, -3]
Tobolsk: Red[3, 6], Green[3, 3]
For larger files like the tif's, we needed to figure out a way to process larger files without taking an exorbitant amount of time. The solution was image pyramids which essentially recursively shrunk our image by 50% until the height and width (in pixels) were less than 250. Then, we would run our original SSD processing over the [-15, 15] window and get the displacement. Finally, we would rescale up while running SSD on a [-1, 1] displacement window to further refine the final displacement values. This sped the processing time up significantly as we only scan the 900 pixel window once, at the base case of recursion and on the smallest image.
The results for the larger image files were a bit more mixed. In particular, Emir, Harvesters, and Village could have some improvement. As noted in the project spec, the exposures of Emir did not have the same brightness level which attributed the most to the flawed reconstruction. As for Harvesters and Village, I believe the over saturation of greens may have skewed the displacement results.
Emir: No Shift
Harvesters: No Shift
Icon: No Shift
Emir: Red[34, 52], Green[24, 49]
Harvesters: Red[13, 95], Green[16, 60]
Icon: Red[22, 89], Green[17, 40]
Lady: No Shift
Melons: No Shift
Onion Church: No Shift
Lady: Red[-4, 136], Green[0, 55]
Melons: Red[12, 178], Green[4, 82]
Onion Church: Red[36,108], Green[25, 52]
Self Portrait: No Shift
Three Generations: No Shift
Train: No Shift
Self Portrait: Red[-3, 170], Green[-1, 80]
Three Generations: Red[9, 111], Green[12, 54]
Train: Red[29, 87], Green[2, 43]
Village: No Shift
Workshop: No Shift
Village: Red[0, 90], Green[0, 64]
Workshop: Red[-13, 101], Green[-1, 53]
Looking at the original images, one will notice black borders. After a little trial and error, I chose values of 20 and 100 for jpg and tif respectively, to crop in from each side. This helped with our SSD evaluation's accuracy as the borders were throwing off our calculations. These parameters could be played with to further optimize results.
3 extra images showing the before and after
Castle: No Shift
Landscape: No Shift
Still Life: No Shift
Castle: Red[0, 5], Green[0, 2]
Landscape: Red[-13, 102], Green[-7, 26]
Still Life: Red[0, 11], Green[0, 2]