Before color photography, Sergei Prokudin-Gorskii was able to capture color information in his images. He did this by recording the same scene or object three times, each through a different color filter. Though there was no way to print color photographs or to view these images in color at the time, the recent digitization of these images allows us to automatically create a single color image from the three filtered images.
The original image, shown above, is made up of the same image captured three times, each with a different color filter. The most basic way to create a color image from the original image is to split the image into 3 sections, each representing the use of a different filter, and then layering these in the order RGB. However we run into a problem if these individual images do not align perfectly. My naive approach to account for this is to find the best alignment by searching over a small window ([-15,15] pixels) of possible displacements. To align two images I first select a window of interest on the base image, ignoring larger borders. I then calculate the normalized cross-correlation between this window of the base image and each possible shift in x and y ([-15,15] pixels) of this window on the second image. The shift resulting in the greatest normalized cross-correlation is then used as my offset vector. From there I can shift this second image by the offset vector for a better alignment to the first image. Note that I chose to align the red and blue channels both to green. I inially started by aligning the red and green layers to blue, but because blue is a fairly dominant color in many of the images, the intensities of the blue values were much larger. This causes an incorrectly calculated offset, as we look for an alignment based on similarity of pixel intensity between the blue channel and the others.
Additionally, this initial alginment method is not feasible for larger images, so I needed to incorporate an image pyramid in the process of calculating the offset vector. I take the images I want to align and iteratively halve their resolution until the image is of a certain smaller size. From there I calculate the offset vector using the same naive alignment described previously, but now on a scaled down image. I then scale this vector back up by a factor of 2, depending on the number of iterations of resolution decrease.
Offsets: R [5 16], B [-48 -24]
Offsets: R [64 0], B [-56 -16]
Offsets: R [48 8], B [-40 -16]
Offsets: R [56 0], B [-48 -8]
Offsets: R [6 1], B [3 -2]
Offsets: R [ 4 -1], B [-3 -1]
Offsets: R [96 8], B [-80 -24]
Offsets: R [56 0], B [-48 -8]
Offsets: R [40 24], B [-40 0]
Offsets: R [64 8], B [-56 -16]
Offsets: R [72 8], B [-64 -8]
Offsets: R [96 0], B [-80 -8]
Offsets: R [80 24], B [-72 -32]
Offsets: R [ 56 -32], B [-48 16]
As a very basic attempt to increase contrast of the photos, I rescaled each color channel. I first cropped each edge of the image by 10% of the size of the image to avoid incorporating any incorrect edge values. Then I found the minimum value of each color and subtracted these values from their corresponding color value at every pixel across the image. Now the minimum pixel has become zero and all other values have lowered. I next find the max value of each color channel across the whole image and divide the other values for that color by the maximum. The maximum value for any color is now equal to 1 and all other values are between 0 and 1 based on their original pixel intensities. I then multiply all the values for each color channel by 255, the desired maximum pixel value to stretch the range of pixels for each color across all intensities 0 to 255. Unfortunately this method did not create a very noticeable increase in contrast of the images.
Examples:
Since my initial attempt showed little improvement, I tried adjusting the contrast differently. I followed the same pattern of subtracting out a low value, dividing by a large value, and then scaling back up to the maximum range of 0 to 255. However this time I chose the low value as 2 standard deviations below the mean for each given color and the large value as 2 standard deviations above the mean. I then followed the same pattern and finished by clipping each pixel value for each color so that if it was below 0 it would become 0 and if it were above 255 it would get set to 255. This way there were no values outside of the allowed pixel intensities. This greatly increased the contrast for many images, while it had unintended changes to the color balance for others. This issue is likely due to the fact that we are manipulating each color channel completely separately from the other two.
Examples: