CS194-26 Proj 1
(Project Web Page)
: Colorizing the Prokudin-Gorskii photo collection;
Project Overview
In this project, we were given a set of images taken by Sergei Mikhailovich Prokudin-Gorskii in the early 1900s. Ahead of his time, he took pictures of countless landmarks using three expores of every scene using a red, blue, and green filter. While there was no way to integrate the three photographs into a singular color image at the time, our project today is to align the three images taken, layer them on top of each other, and then use the in-built Python libraries to display the images.
My approach
My approach to this project was to develop a script on Jupyter notebook that could successfully process a single image, and use the algorithm and techniques to develop abstractions that could generalize to all of the images in the data set. I used the boilerplate code provided and noticed all I needed to implement was the `align` function. I then shifted the first image 15 pixels vertically and horizontally and then overlayed the two images. For each of the displacements, I calculate a score that corresponded to the difference of the images (lower the score the closer the two images are aligned) and then use the displacement with the minimum score. This score function are based on those provided: SSD (sum of squared differences) and NCC (normalizd cross correlation).
Challenges
Once I completed the basic implementation, it worked well for the smaller images, but I underestimated how long it would actually take for the larger images. Then I proceeded to implement the pyramiding function.
I implemented pyramid aligning with an iterative approach as follows:
First I down sampled the two images by 50% until they were less than 300 pixels tall. I have a preset max_dispacement used to find the best vertical and horizontal dislacements for this blurriest image. Then I use the images one scale above, align the two images based on the displacements I found for the previous down sampled images, and then fine tune the adjustments. I continue this process until I've fine tuned the alignment for the original images.
Here's how it works:
Most downsampled (300px x 300px) ~ adjust by +- MAX_DISP (ex. 8).
Then, one level higher (600 x 600) ~ adjust by +- MAX_DISP / 2 (ex. 4)
Then, one level higher (1200 x 1200) ~ adjust by +- MAX_DISP / 4 (ex. 2)
.. until we fine tune the original image by 1 pixel
Another challenge I had was when I was calculating the score between two images, I originally used the whole images. But the black borders from the image, and the colorful borders from shifting the image added unncessary noise and made some images seem incorrectly more similar or different than they actually were. To solve this issue, I added another parameter called frame_size; I only compared a frame within each of the images a small subsction of the center of each image.
Results
Small Images
Large Images
Bonus Pictures
Bells and Whistles
Edge Detection
Notice how the Green and Blue images were already aligned perfectly
originally. The Red and Blue filtered images are the problem. Once we
take an image with everything filtered except its edges, it works. Let's
take a look at what the edges look like
The Canny edge detector uses a variance parameter which you can use
to specify how sensitive you want the edge detector to be. If you set
a high standard deviation it returns fewer pixels indicated to be part
of an edge.
Let's look at the difference in image quality with a sigma of 5 instead of 1
The differences are minimal with this change.
Histogram Equalization
Note this was implemented using the skimage libraries provided