Sergey Mikhaylovich Prokudin-Gorskii was a pioneer in the field of color photography. He had the idea of using red, blue, and green filters on a "triple-lensed" film camera, essentially capturing each color channel in one shot. As a result, Prokudin-Gorskii was given approval by Tsar Nicholas II to travel around early 20th-century Russia and capture its beauty in color.
Although Prokudin-Gorskii had the ability to capture images in color, his camera was not advanced enough to produce a single full-color image. The camera produced three frames for the same image -- a red channel, a green channel, and a blue channel. The goal now is to split each color channel with their respective color intensities and align the channels on top of each other.
Taking the three-channel input from above, first we can split the image into the three frames you see above. It turns out that the order of the frames from top to bottom is Blue channel, Green channel, and Red channel. With each of these channels split up, we must now choose a reference frame to align on top of. The following images used the Green channel as the reference. Once we choose our reference, we test different (x,y) coordinate displacements on our image to be aligned and see which displacement gives us the highest similarity to our reference. The metric used for the following images is the Sum of Squared Differences and the displacement window is 30x30 pixels. After aligning each channel to our reference, we stack all of the channels on top of each other and obtain our color image.
As it turns out, our result here is not well-aligned. This is because the outer edges vary greatly across each color channel. This is most likely because Prokudin-Gorskii's camera lenses were stacked on top of each other, leading to slight differences on the edges of the frame. To fix this, we first crop each channel to only focus on the "inside"/center of each channel instead of the whole frame. Doing, so we obtrain the following result --
We are doing great so far! Truth be told, the monastery we were working with above is a small image (.jpg), thus the alignment displacement was not that large. However, once we test larger images (.tif) we will find that the alignment displacements can be much larger than what we can see in our 30x30 pixel window. This is what would result after running our current alignment on a large image.
Not so good, however we are not quite done yet. There is a way we can align large images using a pyramid scheme. No, not multi-level marketing, but decomposing our large image into a series of smaller-resolution images. Imagine we had a 1000x1000 pixel image. This would represent the base of our pyramid. The next level would be the same image downsized to 500x500 and so forth. At the top of the pyramid, we have an incredibly small resolution image that is easy to operate upon. We use this idea to implement pyramid alignment, where we find the best displacements at the top of the pyramid and pass that information to each of the lower levels of the pyramid. Once we reach the base of the pyramid, we no longer have to search the whole image, only a small section of it since the higher levels already told us approximately where to start searching. After implementing pyramid alignment, we can achieve the following results--
|
|
|
|
If you look closely, the last image of Emir did not come out properly aligned. This is because Emir's robe varies drastically in color intensities. There is a great amount of intensity in the blue channel while the red channel is not nearly as active. This problem becomes even worse when we consider that we only care about the alignment at the center of the image.
To solve this issue and produce better quality images in general, we can implement various other features that can smooth out these imperfections. This includes feature-based alignment, auto white-balancing, and even alignment using edge detection.These implementations are encouraged to achieve better quality photos, but for now the current results suffice.
With (x,y) displacement values for Green and Red channels aligned on top of Blue channel.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|