CS 194-26: HW1

This project uses image processing techniques to produce color image outputs from Prokudin-Gorskii glass plate images.

Method

For small jpeg images I had a "naive" algorithm. This algorithm would compute a center patch of one image, and slide a smaller patch of the other image and compute the NCC value. Our goal was to maximize the NCC value which shows us that the two patches have the highest correlation. I originally used SSD and aimed to minimize the error although this was not an ideal approach because it doesn't take into account the large difference in rgb values.

For larger images we don't want to follow the "naive" algorithm as it would take forever to compute for lare patches, and wouldn't be effective if we had smaller patches. Therefore I downsampled the image until it was similiar in size to our small iamges (~300x300) and found the best patch. From there we scaled back up the image, and using the "best" patch we found, realigned to find the new best patch within the previous patch. That way on each upsampled patch, we were looking at a much smaller patch than we would have if we had implemented it using the naive method.

Challenges

A big challenge I had was being able to correctly upscale the images and keeping track of the coordinates to relocate. When we were finding the amount to roll, it was much easier since we only had to keep track of the offset compared to a relative patch size. Although when we are scaling up the images we want to keep track of the coordinates relative to the entire image. I found this to be a bit more difficult. My solution was to keep track of the coordinates of the top left patch, and once finding the offset, add the offset to the top left coordinates such that I could always keep track of the "best" x, y values of a patch as coordinates relative to the entire image. Then everytime I scaled an image up by 2x, I was able to just scale my best x, y coordinates by 2 and know exactly where my new "best patch" started.