CS 194-26 Project 1 by Siddharth Karia (sidkaria@berkeley.edu)

Overview & Approach

This project required us to align three different greyscale pictures representing the color spectrums and fit them exactly on top of each other to form the color image.

My approach was similar to the one described in the spec. I created a function (bestShiftAtDepth) that would find the best shift [x,y] for an image on top of another image that would give the smallest squared distance between the two image's matrices. Aka, the most alignment for the two images. To achieve this, it would search through all shifts within [-dimension, dimension] for both x and y, giving dimension^2 SSD calculations.

This above function was run only once for the smaller images, with a larger dimensions size, 20. For the larger .tif images, I used the image pyramid described in the spec. This recursive function called bestShiftAtDepth at multiple levels in recursive calls (decreasing the image scale by a factor of 2, and increasing dimensions searched by 1 each level down), shifted the image accordingly with the result, found its own better fit on its own level, and updated the best shift accordingly, before returning back to upper levels with higher scaled images. This offered a speedup over searching through a very necessarily large number of dimension shifts one time (due to the images larger size).

Essentially, to describe it better, we are refining the alignment smaller and smaller after each recursive return/callback by closing the window or area for the algorithm to search the best shift. We don't need to do the extra computation on the higher levels with more precise images because the coarser images already did the large shifts for us.

There were many images that initially had issues in alignment, but I figured that the least SSD should definitely detect these stark shifts. After I realized that borders could be causing these inherent issues, I hardcoded a couple of values to crop from all sides of jpg and tif files, and all images started to align perfectly.

Parameters Used

Crop Value: the amount to crop per image. For smaller .jpg files, I used 20 pixels on every side, and for larger .tif files I used 150 pixels. For Emir, I found that a smaller crop value worked better so I used that condition.

Pyramid Depth (.tif only): This determines how many recursive calls / depths we should go in the image pyramid. I kept this at 5 for all .tif files.

Pyramid Base Dimensions (.tif only): This determines the 2*dimension x 2*dimension array that needs to be searched when shifting the bottom level (most coarse) image in the pyramid to find the least SSD. Used 10.

Single Scale Dimensions (.jpg only): Same as above but the hardcoded dimensions used for the single-scale implementation, with only one call to bestShiftAtDepth. Used 20.

Results

Included are final aligned results of each image after the algorithm was run on them. Also included are the values of the [x, y] shift for the green layer and the red layer with respect to the anchored blue layer.

At the bottom are a few other random images taken from the Prokudin-Gorskii Collection website.

Cathedral, Monastery, and Tobolsk were smaller jpg images that used the purely single-scale algorithm, while the rest were larger tif images that utilized image pyramids.

Castle

Green Shift: [35, 2]

Red Shift: [98, 3]

Cathedral

Green Shift: [5, 2]

Red Shift: [12, 3]

Emir

Green Shift:

Red Shift:

This Emir image was the only one that seemed to have an issue aligning, potentially due to the colors each having a different effect on the exposure / brightness of the image. For example, the difference in the attire between the blue image and the red image is almost opposite ends of the greyscale spectrum.

A better solution than using least SSD for this would be to use an edge detection algorithm that aligns sharp jumps in intensity / value.

Lady

Green Shift: [56, 0]

Red Shift: [118, 10]

Melons

Green Shift: [82, 9]

Red Shift: [178, 12]

Monastery

Green Shift: [-3, 2]

Red Shift: [3, 2]

Onion Church

Green Shift: [52, 25]

Red Shift: [108, 36]

Self Portrait

Green Shift: [78, 28]

Red Shift: [176 36]

Three Generations

Green Shift: [54, 12]

Red Shift: [111, 9]

Tobolsk

Green Shift: [3, 3]

Red Shift: [6, 3]

Train

Green Shift: [42, 1]

Red Shift: [86, 30]

Workshop

Green Shift: [53, -1]

Red Shift: [105, -13]

Harvesters

Green Shift: [60, 16]

Red Shift: [124, 13]

Icon

Green Shift: [40, 17]

Red Shift: [89, 23]

Alignment Run on Other Images from Prokudin-Gorskii Collection

Onion Church 2

Green Shift: [20, 17]

Red Shift: [48, 28]

Boat

Green Shift: [14, -9]

Red Shift: [133, -13]

Mansion

Green Shift: [27, 9]

Red Shift: [63, 9]