Project 1 - Brian Agustino

Overview

For this first project, given an image consisting of 3 colorized images from the Prokudin-Gorskii Photo Collection, I made a program that could find the best alignment between these 3 colorized images. By using either Sum of Squared Differences (SSD) or Normalized Cross-Correlation (NCC), we can find the optimal height and width displacement to best align these images.

Approaches to Best Align the Colorized Images

There are 2 approaches that I implemented to find the best alignment for the images.

• Exhaustive Search:

  1. Splitting the whole image into 3 colorized images (Red, Green, and Blue)
  2. Choose a specific colorized image as the base of the alignment (example: Blue) which would not be shifted in the program
  3. Crop out the images to have the alignment score to be focused on the center of the image and lower the processing speed
  4. Shift the images such as Red on Blue and Green on Blue on a predetermined range (ex: [-15,15] pixels) vertically and horizontally. And take the SSD/NCC and find the width and height displacement that gives the best score of SSD/NCC.
  5. Apply the height and width displacement on the Red and Green images and stack them together with the Blue image. This would produce a nice aligned image

• Pyramid Approach: Applying the Exhaustive approach but more efficiently

  1. Splitting the whole image into 3 colorized images (Red, Green, and Blue)
  2. Choose a specific colorized image as the base of the alignment (example: Blue) which would not be shifted in the program
  3. Since the dimension of the image is too large to process, we rescale the colorized images into smaller dimensions (by a factor of 2) and work down.
  4. We apply the Exhaustive search on the rescaled image and get the width and height displacement that gives the best SSD/NCC score
  5. We then rescale the width and height displacement based on the original image dimension by multiplying the displacement by (2^rescale_factor).
  6. We would then rescale the image into double the dimension of the current image.
  7. Repeat steps 4-6 until we reach the image with dimensions of original_image_dimension/2
  8. Sum all the width and height displacement that has been rescaled based on the original image and apply the displacement on the Red and Green image
  9. Stack the aligned red, aligned green, and blue image (Note: This is the case if aligning on Blue)

Result from Exhaustive Search

cathedral_output.jpg Red: (3, 12) Green: (2, 5)

monastery_output.jpg Red: (2, 3) Green: (2, -3)

tobolsk_output.jpg Red: (3, 7) Green: (3, 3)

Result from Pyramid Search

church_output.jpg Red: (-6, 58) Green: (4, 24)

harvesters_output.jpg Red: (16, 124) Green: (18, 58)

icon_output.jpg Red: (22,90 ) Green: (18, 40)

lady_output.jpg Red: (10, 112) Green: (8, 48)

melons_output.jpg Red: (14, 178) Green: (10, 82)

onion_church_output.jpg Red: (38, 108) Green: (28, 50)

self_portrait_output.jpg Red: (36, 174) Green: (28, 78)

three_generations_output.jpg Red: (12, 112) Green: (16, 48)

train_output.jpg Red: (32, 88) Green: (6, 40)

workshop_output.jpg Red: (-12, 104) Green: (0, 56)

emir_output.jpg Red: (-6, 0) Green: (24, 48)

Note: Aligning the emir image on the blue image does not output a clear image as the brightness on the colorized images are not consistent.

A solution for the emir image

A solution to counter the blurry image of emir is to change the orientation where we would align the colorized images on the GREEN image, not the BLUE. By using the pyramid search on the GREEN image we get the following result:

emir_green_output.jpg Red: (16, 56) Blue: (-24, -48)