CS194-26 Project 1

Isabel Zhang


Project Overview

The goal of this project is to produce color photographs using images taken by Sergei Mikhailovich Prokudin-Gorskii. This man traveled around the Russian Empire and photographed objects at three exposures while using different colored filters (red, green and blue). These images are found online (on the Library of Congress website).


Best Images

Emir with edge alignment, automatic cropping, automatic contrasting
Turkmen with edge alignment, automatic cropping, automatic contrasting
Mountain with RGB alignment, automatic cropping
Catholic women with RGB alignment, automatic cropping


Methodology


  1. First, I read in the input image and divide the input image into thirds (vertically) to represent the blue, green, and red channels.
  2. Next, I determine the image dimension. If the image is greater than 500 px by 500 px in dimension, then I use image pyramiding to speed up the computational process. Otherwise, I proceed normally and do not use image pyramiding (see below for more details) to speed up the computation and alignment process.
  3. Now that I have 3 images, each corresponding to their own channels, I align the red and green images (now referred to as the 'rolled image') to the blue image.
  4. I displace the rolled image (up to 15px left or right and up to 15px up down) to find the maximum score of alignment. I then use normalized cross correlation where I look for the highest score (which will provide images that are closely aligned). An alternative method is to use the sum of squared differences algorithm where the smaller the difference between the two image scores, the better the alignment.
  5. After finding the best displacement, I alter the rolled image and adjust it so that it best aligns with the blue channel image.
  6. Finally, I stack the arrays to create a color image and write that to file.

For the image pyramid, I decreased the image to a quarter of its previous size (halved in each dimension) until the image is at most 500x500 pixels. At that point, I proceed to find the best alignment as noted above. After a best-alignment is found, I then recurse back up, multiplying the displacement values by 2 for each recursion so that the alignment matches for the much larger original input images.



Basic Colorized Images and Selected Colorized Images


Bells and Whistles


Automatic Contrasting

The goal of this portion is to improve intensity distribution for each channel and thus get a better distribution of intensities for the channel. For this part, I attempted to equalize channel histograms. For each individual channel, I used numpy's histogram function to calculate the channel's histogram to get the input intensity distribution. Then, I normalized the histogram. Finally, I interpolated the original image with the histogram to get the scaled channel. (Alternatively, I can also use OpenCV's equalizeHist method which is faster than my program's current method.)

icon.tif without auto-contrast
icon.tif with auto-contrast
lady.tif without auto-contrast
lady.tif with auto-contrast
chinese_man.jpg without auto-contrast
chinese_man.jpg with auto-contrast

Automatic Cropping

The goal of this section was to detect and crop off colored borders in order to improve the image. To do this, I used OpenCV's sobel method to detect the edges vertically and horizontally for each color channel (after alignment has occurred). I threshold each channel's sobel and conduct bitwise operations to AND the binary thresholds together. Next, I sum up the values in columns and in rows and take the largest sum as the border boundaries and crop everything outside those boundaries. Finally, I crop each image using the borders I determined.

turkmen.tif without automatic cropping
turkmen.tif with automatic cropping
lady.tif without automatic cropping
lady.tif with automatic cropping
emir.tif without automatic cropping
emir.tif with automatic cropping and edge alignment

Better Features: Edge Detection

The goal of this section is to use features other than the RGB channel pixel intensity to align the images. I used OpenCV's sobel method to find the image edges and calculated the edge displacement. The displacement was then made to the images themselves and stacked as mentioned above.

emir.jpg with RGB channel alignment G[-16, -56], R[112, 0]; (5 seconds)
emir.jpg with edge alignment; G[-16, -56], R[-40, -104]; (6 seconds)