CS194-26: Intro to Computer Vision and Computational Photography, Fall 2021
Project 1: Colorizing the Prokudin-Gorskii Photo Collection
Angela Chen
Overview
The Prokudin-Gorskii photo collection contains glass plate images separated into three color channel images.
In this project, the goal is to use image processing techniques to create a single color image with as few
visual artifiacts as possible. The general idea to achieve this is to separate each image into three color channel (R, G, B color channels) images,
align them, and then stack them on top of each other so that a single RGB color image is created.
In this report, I will first go over a simple single-scale exhaustive search to align smaller jpg images.
Then I will go over implementing a multi-scale pyramid search for larger tiff images.
Part 1: Exhaustive Search
Each glass plate image contains three color channel images.
I want to separate this single glass plate image and stack the three color channel images on top of each other to create a single RGB color image.
However, if I just separate and stack without aligning, the resulting image does not look very good with visual artifacts and you can see the separate color channels where they don't align so well.
To align the images, I searched over a window of displacements in both the x and y directions.
I chose a displacement window of [-25, 25] pixels. I shifted one image over by each displacement in the
window and calculated the Sum of Squared Differences (SSD, defined as sum(sum((image1-image2).^2))) between the two images.
The aligned image with the lowest SSD score was the best aligned image.
I aligned both the R and G channel images to the B channel image. After aligning the R and G channel images
to the B channel image and stacking all three on top of each other,
I just cropped out the border by taking the central 90% region of the colorized image so that the image looked nicer.
While the image above is not as bad as no alignment, I think it can look a little better.
Since the most important part of the image is the central region, I decided to only calculate SSD on
the central 65% region of both the aligned image (R or G channel image) and the base image (B channel immage).
The result looked much better.
Then I just did the same thing with the other two jpg given example images.
Part 2: Pyramid Search
Since tiff images are so large, running exhaustive search on them would take too much time. This is where pyramid searching comes in.
For each pyramid, I created an image pyramid and did exhaustive search on each level, starting from the smallest resolution to the largest (size of the original tiff image).
To create an image pyramid, I stored the original image first. Then using that original image, I scaled it down by some scale factor (a double from 0.0 to 1.0).
For example, if the scale factor is 0.5, then the rescaled image is half the size of the previous, etc.
Then I stored this new rescaled image.
I kept rescaling down the image using the previous rescaled image until the desired number of total rescaled images (aka levels).
The following images are all very compressed jpg images of the example tiff images.
The most challenging part of pyramid search for me was getting a fast runtime.
The above two images took around 3 minutes each, and this was after varying the scale factor
and number of pyramid levels.
I decreased the window displacement size and central cropping region for the following images,
and pyramid search become much faster, taking around a minute or less,
while still maintaining good colorized image results.
Colorizing more images from the Prokudin-Gorskii photo collection
Here's some more images I downloaded and colorized from the Prokudin-Gorskii photo collection.
I downloaded all the images as jpg and ran my exhaustive search algorithm from part 1 on them.
Attempting Some Bells and Whistles
Some images were kind of dull, so I played around a bit with some color contrast.
For the following images, I split each image into the three B, G, and R channels.
I then equalized the histogram for each color channel then merged them back together to create
some interesting color contrast.
For tree_stream.jpg, I really wanted to bring out the warm tones and make the image more saturated,
kind of make the image have a more autumn feel for the season. I again split tree_stream.jpg into
B, G, and R color channels and increased the values in each channel.
After merging the color channels back together, I converted the merged image to HSV so I could
increase the saturation and decrease the value a little since I found the merged image to be a little
too bright. I then converted the HSV back to BGR to display and save the image.