At the turn of the century, Sergei Mikhailovich Proudin-Gorskii traveled across the Russian Empire to capture history. For each scene, he recorded three exposures using red, green, and blue filters. By aligning these three RGB components, I was able to colorize these images.
My approach was to align the red channel against the blue channel and then align the green channel against the blue channel. I would shift the red channel horizontally and vertically within a (-15, 15) pixel window. I took the minimum squared sum difference between the shifted version of the red channel and the blue channel. The equation I used for the squared sum difference is shown below:
Overall, each image took about 2-3 seconds to render.
My initial approach did not work for these larger images. There was a combined issue of the process being too slow since the .tif files were very large and the (-15, 15) displacement window being too small. A solution to this was to use an image pyramid. Given the large .tif images, I would resize them to be half of the original size. I stopped once the shrunken images had dimensions less than 250x250. Once I reached that point, I calculated the displacements within a (-15, 15) window using the same algorithm that I did previously. I used the shifts calculated in the smallest image and doubled them to shift the next largest image which is double in size. Afterwards, I calculated the displacements of this larger shifted image within a (-2, 2) window using the same algorithm. I proprogated these displacements up to the original image and applied the accumulated shifts to the large .tif files.
I also cropped off 25% of the original red, blue, green channels. The reason for this is because the borders were affecting how the horizontal and vertical displacements were calculated. By using the inner portion of the image, the horizontal and vertical displacements were more true to the pixels of the image rather than being significantly affected by the borders. I used the crop function in the skimage.util module.
Overall, each large .tif image took about 10 seconds.
Most of the images aligned properly, with the exception of the photo of Emir. The reason for this is that the costume has very high intensity in the blue channel, but very low intensity in the red channel. As a result, when trying to better align the two channels, there is a higher value when computing the difference between (Ired - Iblue)2 (the sum squared difference). This violates the underlying concept that the different channels will correlate in intensities.
A solution to the misaligned Emir image involves aligning on better features such as on edges. I used skimage.filter.sobel function as my edge detection algorithm. My before and after aligning on edges is shown below.
I used skimage.filter's sobel function for edge detection. The results are shown above in the Alignment Issues section above.
