Colorizing the Prokudin-Gorskii photo collection

Lucy Wang

Overview

In this project, I used image processing techniques to take the digitized Prokudin-Gorskii glass plate images and automatically produce a color image out of it. I first extracted the three color channel images, then place them on top of each other, and align them so that they form a single RGB color image. I also implemented image pyramid and edge detection techniques to improve the quality of color images.

Image Aligning

For each of the lower resolution images, I take the approach below to produce the final colored image:
  1. Seperate the glass plate images into three color channel images by splitting vertically by 3.
  2. Crop each color channel image to reduce alignment noise.
  3. Align red and green image to blue image by selecting a 100x100 pixel area in the middle of green/red image, translate it over a 15x15 window of possible alignments on the corresponding pixel of blue image, and find the alignment that produces the least sum of square differences.
  4. Once we have an alignment for each of the red and green image, stack the three images together to produce a single RGB color image.

Image Pyramid

As for higher resolution images, simple image aligning becomes extremely slow and inefficient. Therefore, I used image pyramid technique on top of image aligning, which significantly speeded up the process.

  1. Seperate the glass plate images into three color channel images just like before.
  2. Recursively shrink the image by a factor of 2 using OpenCV, until it's smaller than 500x500 pixels. Then use image aligning techniques discussed above to find the best alignment.
  3. Working backward with recursion, scale up the image by 2, then realign the larger image using a smaller window size of alignment (2x2 window), add this new alignment to the best alignment we've found so far to improve alignment accuracy.
  4. When we have the original resolution image back, we align the image using the best alignment returned. Then stack the three images together to produce the final color image.

Results

Cathedral
Red Offset: (3, 12)
Green Offset: (2, 5)
Monastery
Red Offset: (2, 3)
Green Offset: (2, -3)
Tobolsk
Red Offset: (3, 7)
Green Offset: (3, 3)
Castle
Red Offset: (-11, 103)
Green Offset: (-11, 39)
Harvesters
Red Offset: (2, 116)
Green Offset: (17, 63)
Icon
Red Offset: (22, 86)
Green Offset: (14, 47)
Lady
Red Offset: (23, 119)
Green Offset: (14, 55)
Melons
Red Offset: (15, 111)
Blue Offset: (-30, 110)
Onion Church
Red Offset: (41, 93)
Green Offset: (31, 55)
Emir
Red Offset: (34, 100)
Green Offset: (20, 45)
Self Portrait
Red Offset: (15, 103)
Blue Offset: (-34, -94)
Three Generations
Red Offset: (20, 116)
Green Offset: (20, 57)
Train
Red Offset: (33, 95)
Green Offset: (-6, 47)
Workshop
Red Offset: (-18, 105)
Green Offset: (-1, 63)

Here are some self-selected images from Prokudin-Gorskii Image Collection

Camel
Red Offset: (-27, 111)
Green Offset: (-19, 55)
Lastochkino Gniezdo
Red Offset: (-18, 79)
Green Offset: (-22, 4)
V Oranzhereie
Red Offset: (-1, 71)
Green Offset: (-38, -70)

Edge Detection for Image Alignment (Bells & Whistles)

Due to the differnet brighness values of the Emir image, simply using image alignment through color channels does not work. Therefore, I used Canny Edge Detection as a new image alignment feature. Instead of passing in color channels to find the best alignment, I applied edge detection on green/red images, and compare with the edge detected blue image. As shown below, the quality of alignment is significantly improved.

Alignment using color channels
Alignment using edge detection

Challenges

When I applied the image alignment algorithm on several images (melons.tif, self_portrait.tif, V_oranzhereie.tif), the alignment effect wasn't as good as other images. This might due to the repetitive features on those images, which makes them difficult to align. I solved this problem by changing the order of the alignment, aligning red/blue channels to green channels.