The goal of this project is to produce color photographs using images taken by Sergei Mikhailovich Prokudin-Gorskii. This man traveled around the Russian Empire and photographed objects at three exposures while using different colored filters (red, green and blue). These images are found online (on the Library of Congress website).
For the image pyramid, I decreased the image to a quarter of its previous size (halved in each dimension) until the image is at most 500x500 pixels. At that point, I proceed to find the best alignment as noted above. After a best-alignment is found, I then recurse back up, multiplying the displacement values by 2 for each recursion so that the alignment matches for the much larger original input images.
The goal of this portion is to improve intensity distribution for each channel and thus get a better distribution of intensities for the channel. For this part, I attempted to equalize channel histograms. For each individual channel, I used numpy's histogram function to calculate the channel's histogram to get the input intensity distribution. Then, I normalized the histogram. Finally, I interpolated the original image with the histogram to get the scaled channel. (Alternatively, I can also use OpenCV's equalizeHist method which is faster than my program's current method.)
The goal of this section was to detect and crop off colored borders in order to improve the image. To do this, I used OpenCV's sobel method to detect the edges vertically and horizontally for each color channel (after alignment has occurred). I threshold each channel's sobel and conduct bitwise operations to AND the binary thresholds together. Next, I sum up the values in columns and in rows and take the largest sum as the border boundaries and crop everything outside those boundaries. Finally, I crop each image using the borders I determined.
The goal of this section is to use features other than the RGB channel pixel intensity to align the images. I used OpenCV's sobel method to find the image edges and calculated the edge displacement. The displacement was then made to the images themselves and stacked as mentioned above.