The goal of this assignment is to take the digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce a color image with as few visual artifacts as possible.
I followed the general approach suggested in the spec, wherein I first implemented single-scale displacement search and then added recursive calls to implement multi-scale displacement. I used SSD as the distance metric throughout. This yielded good results on most of the images, but not right away.
At first, I simply split the image into thirds vertically and started aligning, as the starter code does, but this yielded blurry results. I then tried cropping the borders, a major source of noise for the SDD metric, and the results improved significantly. My cropping approach was nothing fancy; I simply cropped a fixed margin off of each image depending on whether it was a B, G, or R and whether it was JPEG or TIFF, yielding 6 possible margins in total.
In terms of my development workflow, I began using Jupyter so I could view images and see my the effects of my changes relatively quickly. At the end, I packaged the code into a plain Python script which reads and processes all the input images together.
As expected, the Emir turned out blurry using this approach. This is due to fact that the SDD metric compares the numerical values of the color channels to each other, which does not capture similarity very well when the channels vary significantly in brightness level. In particular, we can see below that the Emir's robe is the brightest part of the image in the B channel, but the darkest in the R channel.
To get around this, we would have to featurize the images somehow before applying our comparison metric; one approach that comes to mind for this is edge detection.
