Jeffrey Shen Project 1

This is my submission for CS 194-26 Project 1: Colorizing the Prokudin-Gorskii Photo Collection. We start off with a glass plate with 3 images filtered through a blue, green and red lenses and want to output a colored image, combining the 3 color channel images together. Using a combination of the image pyramid, L2 norm, and image rolling to align then applying the color channels and color grading the image, we are able to produce a full vibrant color image.

I first start off by dividing the image into 3 parts for each blue, green, and red image.

On each of these separate sections, I clean the image by first implementing a automatic border cropping for the outer white border (B&W1). This is done through thresholding for the outer white border and having a BINARY_INV in order to turn the outside border black so that the drawContour function can find the outer border. I also tested out using erode on the thresholded image to reduce any noise in the border, however it did not end up helping too much.

Then, after cropping according to the dimensions from the drawContour function, there is a set percent to be cropped for border between the black and white border to clean up the rest of the constant black border, so that the final red image looks like this. By removing the borders cleanly, it greatly improved the process during alignment so that the borders did not influence the score calculation.

These 3 color channel images then enter the image_pyramid function. This recursive function exits when the image is smaller than 250 px and has a reduction factor of 2. These values were calculated through trial and error to minimize the computation time while preserving image alignment. The images are aligned by comparing over a radius of 40. The image overlaps are kept and the rest are cropped out when comparing so that they are the same size. I tested out using edge detection in the image to align them using the Canny Edge algorithm (B&W5). However this edge detection increased the computational time signficiantly (from 2 to 30 mins), and minimally improved the alignment procedure so I decided it was not worth the computation time. Then using the L2 Norm, we find the offset that is the most similar. Here is the first iteration of the image pyramid (size 202 w, 176 h):

Here is the last iteration of the image pyramid (size 3182 w, 2763 h):

After aligning the green to blue and red to blue, we resize all the images to make sure they are the same size. I tested out aligning to the green color channel however it didn't seem to make a big difference. After resizing and traversing up and down the pyramid, we want to stack all the colors by assigning each image to the correct color channel and adding all the images together to create the color image. This is done by setting each grayscale image to the specific color channel and merging them together. Our final step is to colorgrade the image and make it look even better. I implemented a histogram normalization on the Value channel of the image in HSV to increase the contrast. Then, I also increased the saturation by multiplying each value by a saturation adjustment of 1.3(B&W4).
This is before the color adjustment:

This is after the color adjustment:

As you can see the color grading helped bring out the contrast in the image. This was especially helpful for when images were whitewashed from too much exposure and the saturation makes the colors pop out a bit more.

Here are the rest of the images. Mouse over the image for the Red and Green image offsets from the Blue image in the format (Red offset y, Red offset x), (Green offset y, Green offset x)