For this project we had to take three color channels (R,G,B) and properly align, minimizing artifacts and noise. The images come from Sergei Mikhailovich Prokudin-Gorskii, who travelled Russia taking photographs (three "exposures of every scene onto a glass plate using a red, a green, and a blue filter" to get each color channel. The main challenge for this assignment was automatically aligning all three color channels.

For the naive solution, we exhaustively search a small space of translations (circularly shifting the image) and measuring the score (via Sum of Squared Differences) for each alignment (using the blue channel as a reference). This works on the smaller images (in the range of hundreds of pixels in each dimension). However for larger images, it is too slow to simply search through a sufficiently large window of translations. Here are the results on images (after these images were cropped by 30 pixels on each side):

R:[2, 3] G:[2, -3]

R:[-1, 14] G:[0, 7]

R:[0, 7] G:[1, 3]

R:[3, 12] G:[2, 5]

Because the naive approach does not work on the larger glass plate images, we have to use a coarse-to-fine approach, representing the image as a pyramid of images (with lower resolution at the top and successively higher resolutions at each level). We first search through the first level image for a translation that minimizes the SSD score. The resulting translation is just an approximate solution which is used as a starting point for our search in the next level in the pyramid, an image with twice that resolution. We do this incrementally better and better search until we’ve reached the highest (original) resolution image. This allows us to reduce the amount of space we’d have to search for an optimal translation by only using the high resolution photo.

Initially, the results of using this approach were not very good, some images appeared to be slightly misaligned. However, after cropping a fixed amount of the edges off, the results improved. Other fixes were required too; the 'emir' photo aligned much better when using the green channel as a reference point, not the blue channel (like the rest of the photos). This one is shown in comparison below:

R: [-286,1] G: [8,16]

R: [6, 18] G: [-8, -16]

Other Results Using Image Pyramids:

The following pictures are results of using the pyramid approach on images after cropping their borders:

R:[11, 29] G:[2, 14]

R:[9,37] G:[7,18]

R:[7,46] G:[4,22]

R:[4,40] G:[5,19]t

R:[8,29] G:[6,13]

R:[4,36] G:[3,16]

R:[12,57] G:[10,26]

R:[4,36] G:[4,17]

R:[-4, 43] G:[-2, 4]]