In this project, we reconstruct color images from the grayscale images of Russian photographer Prokudin-Gorskii. This is accomplished by image alignment via normalized cross correlation, and further optimized via a image pyramid.
Each image is recreated from 3 greyscale images, each containing information from the three main color channels: red, green, and blue. However, these three images have small differences in alignment (in only the x-y plane) and thus we must shift the color channel images around and then combine to create a correct image. This is accomplished by having a metric of what a "good" alignment is, and constantly shift one channel around until this metric is maximzed, indicating the images have aligned. I experimented with two different metrics: sum of squared differences and normalized cross correlation, and settled on normalized cross correlation. Before alignment, I also cropped the border of the images by a fixed amount to remove the black outline, which led to incorrect alignments. Here are my results for image alignment of all the small images.
Red: (12,3) Green: (5,2)
Red: (3,2) Green: (-3,2)
Red: (7,0) Green: (3,1)
Red: (15,-1) Green: (7,0)
For the larger images, running the naive solution of matching every possible combination in a given area is not efficient enough. Instead, we will construct an image pyramid to improve effiency. Essentially we start with a scaled down image (more coarse) and run our naive algorithm. We then use this value as a starting point in a slightly more detailed image, repeating until we eventually are running the algorithm on the full original image. Here are the alignments for the large images:
Red: (123,14) Green: (59,17)
Red: (90,23) Green: (41,17)
Red: (111,12) Green: (52,8)
Red: (175,37) Green: (78,29)
Red: (110,12) Green: (51,14)
Red: (86,32) Green: (42,6)
Red: (116,28) Green: (55,21)
Red: (127,22) Green: (64,12)
Red: (0,-4) Green: (48,24)
As you can see from the last image, our simple normalized cross correlation algorithm produces mainly aligned images, but still isn't perfect. The alignment step will be further improved below via edge detection.
To improve the image alignment, I implemented the Sobel-Feldman operator in order to run cross correlation on edges rather than on the entire image. You can see below the improvements from using edge detection. This image produced the most noticable improvement, but images across the board had minor improvements.
Red: (110,-19) Green: (48,-3)
Red: (143,72) Green: (61,50)
Red: (187,-7) Green: (88,-1)