I ended up with two functions: align_naive and align. The former is the basic approach that was outlined in the problem description: an exhaustive search over a window of displacements for the R/G channels to find the optimal displacement for alignment onto the B channel. I found that a [-10, 10] window in both horizontal and vertical directions worked well. I implemented both L2 norm (SSD) and normalized cross-correlation (NCC) as a metric, and arbitrarily chose NCC for my final code. I ran both and found no visible difference in output. An extra step I took was to crop both images (image-to-align and image-being-aligned-to) by 10% on each side. This is to crop out the borders during alignment. I did not crop the final image, however, because I thought the high constrast, unclean edges looked cool.
To process larger images (the .tifs), I implemented an image pyramid in the align function. This function scales both input images by a user-determined alpha value (default 0.5) via skimage's rescale function, which performs downsampling, and recursively aligns the downsampled images. I then prematurely shift the image-to-align, such that it is now roughly aligned, and then adjust its alignment with greater percision but at a faster speed than naively aligning by cropping both images by a little less than the alpha value, and perform align_naive on these cropped images. Finally, I return the sum of the two displacements as the final displacement. A zoomed out view of this algorithm is making incremental shifts per level of recursion, each closer to the optimal alignment and each with increasingly smaller input image sizes to run NCC on. The result is a very fast (less than 20s) alignment process on large images (70MB).
I toyed around with the stopping conditions for recursion. I was thinking either a maximum depth or a minimum image size. The former was to prevent large images from processing too long, but this turned out to not matter, as the image size case was hit first. The base case I ended up using is when either height or width is less than 100 pixels. I eyeballed this value and like many other of the guessed values--I was lucky to have chosen a number that produced good results.
One tricky thing: in the second-last row of the results below, there are two versions of emir.tif--the first used the same image processing technique as the others. With a nudge from a friend (shoutout Amy Hung), I learned that this is due to the prominence of blue in Emir's photo. This means that instead of stacking R/G onto B (which had a significantly brighter-looking raw image), it was better to stack R/B onto G. Looking at the raw images, this makes intuitive sense as the middle G channel looked like an "in between" of the top and bottom R/B channels respectively. This was the only issue I ran into.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|