Overview

The overall project seeks to align images taken in the R,G,B planes in order to create a colorized image as a result. For my approach, I first naively aligned the images which led to very misaligned images, before implementing a single-scale solution that would compute the SSD over the image over a window of 10% the size of the original image. In order to improve the speed in which I performed alignments, I implemented an image pyramid, along with doing it over a smaller window to improve the overall image alignment accuracy. Some images I ran into problems with still with the image pyramid were `emir.tif` and `boat.tif`, although reducing the cropped image to check windows did seem to help `boat.tif`. After that I applied a few Bells and Whistles to see if I could improve my results

Naive vs. Sum of Square Differences

I started off with naive alignments just to see how things looked with a naive alignment. I then applied SSD as a metric for aligning the glass plates, and limited the displacement to [-min(width, height)/10, min(width,height)/10]. I also implemented NCC, although it seemed to take a lot longer and seemed to have similar results. The displacement window didn't work well with the .tif images, so I noted that and made that change for the image pyramid.

Pyramid

To improve efficiency, I used an image pyramid. I calculated scale to log2 of the minimum dimension to calculate how many times to resize the pyramid, and scaled by 0.5 every time. This way you could work with a smaller displacement (I chose log10), at different scales to reduce the window that you need to calculate over.

Examples of Alignment Going Wrong

SSD and NCC are both functions that calculate over the intensity values, and because of that, it can be hard to score across the RGB planes, as one plane might have very different intensities from another (especially in the case that it is more Red, Green, or Blue). This happened for me on Emir's image and the village.