Background¶
We are given individual glass panes from the Prokudin-Gorskii archives, and our task is to combine the three panels corresponding to blue, green, and red into one colored image. The challenging part of this assignment really is the alignment and processing. Each panel is not entirely centered, and they additionally contain different levels of brightness and contrast.
Naive Implementation¶
For my naive approach, I went with a straightword pyramid alignment. The aignment procedure from comparing one panel to a base panel and in our cases we used Blue panel as the base. For instance, in aligning the G to B, I preselected a reasonable displacement window of +/- 30 pixels, and analyzed the similarity between the two images. I explored using both cross-correlation and sum of squared distances, and I found the latter to provide better results. For the smaller JPG images this was sufficient, but for the larger TIFs, we had to employ the use of image pyramids.
Image pyramids were a straight forward concept, but a little tricky to implement. We effectively just downscale the image several times, and process each one from the lowest resolution to the highest. This allows us to avoid exhaustive searches on the larger resolutions, as we only need to search the level of detail missed by lower resolution (which is a much smaller dimensional search). We keep track of the displacements in each resolution, and scale accordingly when we go to higher resolution images in the pyramid, and search from there.
An important factor is that on each image and each resolution, we only do alignment on the inner ~80% of the image in order to avoid edge artifacts.
For my naive images, I additionally automatically cropped each image to the inner 80%, to avoid the artifacts on the edges. There are more sophistacted cropping algorithms possible, but this served sufficient for a naive approach. Each image is additionally presented with the red and green shifts found by my naive approach
Required Images¶
Extra Images¶
Naive Analysis¶
Interestingly, the naive approach worked really well for each image, except for the emir. To understand why that particular case failed, we can look at the individual channels of the image to get a better sense.
Here you can clearly see in each panel, the colors filtered by the glass panes result in very different colors captured for the emir. Additionally, the brightness is very different between the different images. This will require an additional feature to properly match this.
Bells & Whistles Feature 1: Edge Detection¶
To rectify the color and brightness imbalance in the emir photo, I decided to incorporate edge detection as a feature to align on. I used Canny edge detection within sklearn, and I additionally swapped back to cross-correlation for my loss function, primarily because it produced a slightly better image.
New Emir¶
