CS194-26 Project 1: Colorizing the Prokudin-Gorskii photo collection

Part 1: Exhaustive Search

For this project, I found the starter code extremely helpful. I first defined the l2_norm loss function, which just calculates the Sum of Squared Differences. I then used the starter code to divide the cathedral image into 3 parts and break it down into its B, G, and R components. I stacked them together using np.dstack just to see how the whole thing worked, and then I got to work writing the align function. I knew that I had to iterate through each pixel in a [-15, 15] displacement, so I knew that I would need a double for loop: one for the x displacement and one for the y displacement. I checked the l2_loss value for each combination of pixel displacements, and then returned the displacement pair that gave the smallest loss value. I also shifted image1 by that displacement using np.roll, and returned that in my align function. Initially, this worked fine for cathedral.jpg, but horribly for monastery.jpg, so I had the idea to crop the borders off the individual colour component photos so that there was no border and less noise by the edges. I wrote the crop function and called it on B, G, and R. Afterward, monastery.jpg started looking a lot better, so I kept this implementation. This exhaustive search method worked great for the 3 .jpg images, but obviously I had to make changes when I moved onto the .tif images.

Part 2: Pyramid Search

For the pyramid search, I analyzed a lot of Piazza instructor comments and really tried to break down what they were saying to understand how to implement it. Piazza said that the bottom of the pyramid should be the original resolution, so I got the idea that the top should start out as a fraction of the original resolution and increase as it goes down the pyramid. I recognized that I would need to multiply the shift values as I multiplied the scaling factor as well, because the images were getting in terms of resolution and the shift values should reflect that. I implemented this and called my previous align function (with some adjustments, like passing in a shift parameter and using the pre-existing shift values in the for loops) within the pyramid search, but a lot of the .tif photos were super blurry when I stacked them together. I realized that I was multiplying the scale factor in the wrong place, and that I needed to do it before passing in the shift values to my align_shift function. This made the photos a lot less blurry and a lot less improperly displaced.

At this point, all of the example photos were aligning great except emir.tif. This was because the images of that photo did not have the same brightness values, as mentioned in the spec. I decided to try using a green base instead of blue, because on Piazza a few people were saying that switching bases helped them. After doing this, it worked! So I wrote a separate function just for emir.tif, called produce_tif_image_emir.

Trying it out on a few images from the Prokudin-Gorskii collection:

I downloaded a few colour separation negatives from the collection as .tif files and applied the pyramid search to them. These are the results.