For this project we are working with images from the Prokudin-Gorskii collection, available online from the Library of Congress. These photographs of the Russian Empire come in sets of three glass plate negatives, with red, green, and blue color filters respectively. By stacking them together with the correct alignments, we can reconstruct color photographs. Even with simple methods, the results are surprisingly decent.
First, each set is split into three images of equal height.
For smaller images, a brute force approach where we take the green and red images and slide them over the blue image pixel-by-pixel to find the best offset for each was sufficient.
For the purposes of the algorithm, the "best" offset is one that results in an image whose overlap with the blue image is most "similar", where similarity is measured with the sum of squared differences (SSD) metric. The intensities of the pixels in the areas to be compared are subtracted, squared, and them summed together. The smaller the sum, the more similar the images.
This is too slow for large images, so a modified alignment function was used. This function creates an image pyramid by recursively generating subimages, each scaled down from the previous layer. Alignment is done with the brute force approach at each layer starting at the smallest, and the results are incorporated into the overall displacement.
Alignment generally didn't work well until cropping was implemented. For simplicity, a fixed percentage of the images were cropped. Removing a larger percentage for the .tif files gave better results.
Even after cropping, certain images aligned incorrectly, such as emir.tif, village.tif, and self_portrait.tif. In some cases the error was substantial, which could potentially be attributed to an incorrect displacement in small subimages being propagated and magnified up through larger subimages. Fiddling with cropping and changing the window size to be a function of layer number such that it gave middle layers the largest windows seemed to help.
Emir.tif is still imperfect, likely due to the differences in brightness between its color channels. Since SSD only goes off of intensities, it may have been thrown off.
Two of the additional images I downloaded, s_fresco.tif and s_fresco3.tif, failed to align at all. I speculate that the reasons are similar to emir.tif. Or perhaps specific complex patterns/being more symmetrical can cause mismatch in the small subimage alignment. Decreasing window size helped s_fresco.tif, but degraded other images unacceptably. Maybe the pyramid alignment estimate veers off and is unable to correct itself if the optimal displacement is close to the origin?
Additionally SSD and more generally similarity between color intensities may not be the ideal way to align images as opposed to others, such as edge detection algorithms.
For the rest of the images, the quality appears acceptable, marred only by slight fuzziness or other artifacts in some cases.
