The goal of the project was to take 3 color channel images and align them on top of each other to create an RGB image. The pictures used were from the Prokudin-Gorskii Photo Collection.
First I had a naive approach which used exhaustive search along with SSD to get the displacement with the best score and align the images at that displacement. The search was across [-15,15] on both axes. The blue channel is kept constant and the red and green ones are aligned to it. This however only worked for the smaller jpeg files. For the larger .tiff files I used a gaussian pyramid with a scale starting 1/64 the resolution of the original picture. I then worked upwards recursively each time finding the best displacement and using that as the starting point of the search at the next level. I used the rescale method to help scale images.
One of the problems I ran into was that the edges of the image would come out a bit misaligned and blurry. To fix that problem, I cropped out 10% of the picture on all the sides. This improved the picture considerably and could be done because the centers of the pictures are more likely to align properly.
cathedral.jpg
Red channel offset : (3,12)
Green channel offset: (2,5)
tobolsk.jpg
Red channel offset : (3,6)
Green channel offset: (3,3)
monastery.jpg
Red channel offset : (2,3)
Green channel offset: (2,-3)
church.tif
Red channel offset : (-4,58)
Green channel offset: (4,25)
melons.tif
Red channel offset : (13,178)
Green channel offset: (10,81)
three_generations.tif
Red channel offset : (11,112)
Green channel offset: (14,53)
lady.tif
Red channel offset : (11,112)
Green channel offset: (9,51)
train.tif
Red channel offset : (32,87)
Green channel offset: (5,42)
self_portrait.tif
Red channel offset : (37,176)
Green channel offset: (29,78)
icon.tif
Red channel offset : (23,89)
Green channel offset: (17,41)
workshop.tif
Red channel offset : (-12,105)
Green channel offset: (0,53)
onion_church.tif
Red channel offset : (36,108)
Green channel offset: (26,51)
harvesters.tif
Red channel offset : (13,123)
Green channel offset: (16,59)
emir.tif
Red channel offset : (-635,93)
Green channel offset: (24,49)
For Emir, both the Red and Blue channels have very different values for the dress he is wearing. The reason why all the traditional measurements SSD, NCC failed is because it looks for absolute similarities. The brightness values are also different which also affect the ability of this being perfectly aligned. I tried making changing the order of the merge by keeping the red channel constant instead of the blue but it did not result in any significant change. I think the autocropping functionality would help in aligning this picture perfectly.
