CS 194-26: Image Manipulation and Computational Photography

Project Overview:

In this project, we are given three images, each of which correspond to an exposure of an image with a blue, green and red filter respectively. We then had to auto-align the pictures to create a new color image, optimizing to have as fiew visual artifacts as possible, to find the proper (x,y) alignment of the images.

My Approach:

Before I started, I realized that artifacts in the image meant that I first needed to crop off the outermost eighth of the image to ensure correct alignment. However, this wouldnt affect the final alignment, as it would be the same no matter what was cropped off. I approached finding the shift by constructing an image pyramid, with each next level having half the resolution of the previous level until the total image size was less than 50 x 50. For the lowest level, I then exhaustively searching over a window of possible displacements within 15 pixels of the inital displacement for the Sum of the Square Differences, aka the L2 Norm. Once I found that for the lowest resolution, I was able to take the previous resolution's shift and search over the space of [2 * previous_shift_x - 1: 2 * previous_shift_x + 1, 2 * previous_shift_y - 1: 2 * previous_shift_y + 1 ]. By the time the algorithm got to the full image, the total optimal shift was found, which could then be applied.

Challenges:

My initial approach was to get the brute force algorithm to work and then port it over to an image pyramid. However, this was not successful, and led me down a wrong path. In the end, I had to scrap the work I had done and start over. Another problem was run time. While the small images ran quickly, on the order of 5 seconds per image, as the image got larger, the run time grew exponentially, eventually reaching around 4 minutes and 30 seconds per image. At this point, I realized that my SSD function was not vectorized, and by vectorizing it, i was able to get from 4 minutes 30 seconds down to about 8 seconds.