|
|
|
|
|
|
|
|
|
|
|
|
|
|
The objective of this project was to develop an algorithm that could align the three different photos for the different colors for a given scene in the Prokudin-Gorskii Photo Collection. To do this, for smaller jpg images, I performed an exhaustive search over a range of possible offsets and chose the offset with the best similarity score (L2, NCC, etc). For larger images, this method becomes infeasible because the space of possible offsets becomes too large to search. So instead, I used the pyramid approach: first computing the offset on a down-scaled, lower resolution version of the image, using this offset as the center of the search space for a higher resolution version, and repeating until I arrive back at the original resolution.
The exhaustive search portion of the project went quite smoothly for me, however, I ran into numerous problems at the pyramid search portion. Initially, the images weren't matching up at all, and the reason turned out to be that I wasn't computing my offsets correctly at each level. After that, the code was taking too long to run. To address this, I tried changing my original iterative solution to a recursive one, but it didn't make much of a difference. Instead, I found that using L2 instead of NCC resulted in significantly lower runtimes for me (~2/3rds the time). However, it didn't work as well on a couple of the images. To fix this, I tried changing the window size and pyramid depth, and eventually came to a combination where only two of the input images were misaligned.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Two images were misaligned: Emir and church. Emir is only misaligned slightly, and I speculate that this is due to the different channels having different brightness values, as mentioned in the instructions.
For church, the red channel is drastically misaligned. My code aligns them correctly if I decrease the window size or use NCC. This leads me to believe that for a certain large offset, the red channel's L2 score is actually lower than the the aligned L2 score. However, if I decrease the window size, several of the other images do not align correctly because the search space is not large enough. If I use NCC, the code runs too slowly.
|
|
|
|
The only misaligned image is Zakat na Mori e (Sunset at the sea). The red channel seems to be vertically misaligned from the other two. I believe the reason for this is the presence of many horizontal lines in the image, like the horizon and the crest of the waves. The L2 metric will be the smallest when these horizontal lines line up with each other. However, for the red channel it appears that the crest of the wave has been lined up with the horizon of the other channels. This would indeed result in a small L2 score, but it is not the correct alignment.