In this project, our goal was to take the digitized Prokudin-Gorskii glass plate images and use image processing techniques to automatically produce a color image. To create these glass plate images, Sergei Prokudin-Gorskii recorded three exposures of the same scene onto a glass plate using a red, a green, and a blue filter. By aligning these three different colored images, one RGB color image could be produced.
When figuring out how to align the three images, the green channel image was used as the source of comparison. I calculated the alignment of the red channel image with the green channel image, and then the alignment of the blue channel image with the green channel image. Then, I overlayed these images on top of each other to form the one color image.
To calculate the alignment, I first narrowed down the area of comparison between two of the images by only comparing the middle 50% of the images. This helped get rid of the problem with borders around the images, which would sometimes detract from a proper alignment. It also sped up the calculation process because there were fewer pixels to compare.
Next, I searched over a window of size [-15, 15] pixels of possible displacements for the red and blue channel images compared against the green channel image. I calculated the SSD of each displacement against the green channel, choosing the displacement that had the smallest SSD.
Now that I had obtained the displacements for the red and blue channels, I overlayed these channels' images onto the green channel image to get the final result.
If the image is very large, the naive approach is very slow because it has to shift a large number of pixels when calculating the SSD for the displacemnts. In this situation, the image pyramid approach is preferable. This approach also starts off with the same cropping of the middle 50% of the image. Then, in a recursive loop, the image is scaled down by a factor of 2 until it hits a small enough threshold image size. At this point, I use the naive approach with a displacement window of [-25, 25]. Now, as we go back up the recursive calls, I still use the naive approach, but on a smaller window size [-2, 2] that's centered around the base case displacement found. On the current level, it's important to center around the previous level's displacement multipled by 2. This is because we are now at a 2x bigger scale of the image in the previous level.
A recap of above is essentially, we scale the image down to a manageable size where we can afford to calculate a large displacement window. Once this search space is narrowed down to be centered around this first displacement, we can use a smaller window in future searches.
For some of the images, the alignment was noticeably off when I used the blue channel as the source of comparison. This is because the different color channel images didn't have the same brightness values, which skewed the SSD metric. However, when I changed this to use the green channel as the source and computed the red and blue channel displacements, this lessened the problem.
