Overview: In this project, we take images from the Prokudin-Gorskii photo collection and process them. Each pre-processed image consists of three color channel images (Blue, Green, and Red) which we align and combine to create a single color image.

Approach: Each image is split into three (BGR) and offsets are calculated for the G and R images by comparing them to B. For the small .jpg images, a simple exhaustive search is done over a small window of offsets to find the offset with the least Sum of Squared Differences in relation to B. Initially, the split images were used as is when calculating offsets. However, this did not work well for all images due to the black and white borders around each image, so the borders were simply cropped so that offset calculation would focus on the image content in the center. This produced good results for all images, and since the .jpg images were very small, the running time was almost instantaneous.

For the large .tif images, exhaustive search over an even larger window would take too long, so we instead use an image pyramid. Starting with the coarsest scale (based on factors of 2), offsets are calculated for very downscaled images and then used recursively to calculate offsets for less coarse scales until the original scale image is reached. This essentially allows for logarithmic runtime as opposed to exhaustively searching over thousands of potential offsets and calculating SSDs for massive images every time. Each .tif image only took about 5 seconds to process.

This produced very nice results with a few exceptions (emir.tif, village.tif, as well as a few additional downloaded images). The offsets on these images were off by a lot due to the different color channels not having the same brightness values. This was remedied by converting the BGR images into edge maps, essentially allowing images to be matched by their edges rather than based on pixel values. However, this also resulted in slightly lower quality images for other files, so edge map conversion was only used for specific files that needed it.

Finally, I found that starting with coarser scales in the image pyramid procedure made for slightly better image quality.