CS194-26 Project 1

Sergei Mikhailovich Prokudin-Gorskii (1863-1944) traveled across the Russian Empire, creating a collection of colored photographs by taking black and white photographs through a red filter, a green filter, and a blue filter, the three of which can then be projected through color filters and aligned on top of each other to produce a colored photograph.

In this project, I used image processing techniques in order to transform the raw images into colored ones. In order to align the three images, I naively searched for the displacement between a range of pixels that would minimize the sum of squared distance between the pixel values of pairs of images. Since this is computationally inefficient, I added to this process an image pyramid, which rescales the image multiple times by a factor of one-half until the image is deemed small enough to be processed naively. From there, I applied the naive search starting from the coarsest image to the most detailed image, searching over a smaller set of displacements between images, and instead of minimizing sum of squared distances, I maximized the normalized cross-correlation.

Although there were initial problems regarding properly aligning the images in a few of the examples, these were solved by tuning a parameter that represented an artificial border around the image to ignore for calculations, thus making the sum of squared distances and normalized cross-correlation more accurate for the parts of the image that should be aligned.

In addition, I implemented automatic border cropping to get rid of various colored borders that are formed by the original borders of the images be misaligned after calculating and applying the displacement for the images. This was done by detecting approximate vertical and horizontal lines by starting at lines within the image and working outwards, searching for two parallel arrays that are close to each other and exceed are beneath a certain threshold (tuned parameter) when calculating the normalized cross-correlation of the two. Below are the example images after going through automatic border cropping. One thing to note is that this method has a hard time when used on images that have line like features near the border, such as in icon and in nativity, which may lead to excess cropping.

Furthermore, I implemented automatic contrasting by readjusting all pixel values across each triplet of images such that the maximum pixel from any of the three images is scaled to 1, while the minimum is scaled to 0. This did not make much of a difference for the majority of the images, but the difference can be seen in the icon image below.

Automatic white balnce was implemented by taking the average color of each channel of the image (the assumed illuminant) and rescaling each channel's pixel values so that the average would be at 0.5. These results were more noticeable than those of automatic contrasting. See results below under "Example Images"