g: [5 2], r: [12 3]
g: [-3 2], r: [3 2]
g: [3 2], r: [6 3]
g: [48 24], r: [64 42]
The project was to take digitized glass plate images from Prokudin-Gorskii and produce a color image. The glass plate image contains 3 color channel images, so the goal was to align these three images.
First, I extracted the three color channel images by taking the overall height and dividing by 3 and then slicing the image into 3 separated color channel images. Stacking the three color channel images directly after being separated did not product a good color image because they were not aligned. You must align the 3 color channels in 2 steps: align green to blue, then red to blue. When green and red are both aligned to blue, I stacked the three channels to produce a colored image.
I exhaustively searched all displacements of the first image within 15 pixels. Out of all displacements, I chose the best one by computing the sum of squared differences (SSD) of the pixel values between the the displaced first image and second image and choosing the displacement with the lowest SSD. This alignment was super slow for high quality images or large files.
To speed up the naive alignment and align large image files more quickly, I used the pyramid representation in image processing from course to fine. I rescaled the two images down by a factor of 2. Then, I used my naive SSD alignment to get a displacement vector. Since my displacement vector was with respect to the rescaled down images, I had to scale my displacement vector to match the true scale of the original image. Using my scaled-up displacement vector, I displaced the first image and proceeded to keep recursively aligning from the coarsest scale to the finest scale until the scale matched the original scale of the image. I chose to start at the course scale of 1/16, which allowed for 4 recursive calls until the scale reached the original scale (1).
Because the borders would affect the SSD greatly, I only computed the SSD on the middle 5/6ths of the two images. The use of SSD between two color channels generally seemed to work well,but it assumes that the brightness or darkness of a color channel implies a similar brightness or darkness of the other color channels, which is not necessarily true when there is a lot of bright blue, red, or green in the picture. Therefore, this method especially fails in alignment for the emir.jpeg. For the man's clothing, there are high values in B, but not R, so an actual good alignment would yield a high SSD.
