Images of the Russian Empire - Colorizing the Prokudin-Gorskii Photo Collection

Sergei Mikhailovich Prokudin-Gorskii traveled across the vast Russian Empire and took color photographs of everything he saw. His idea was simple: record three exposures of every scene onto a glass plate using a red, a green, and a blue filter. The goal of this project is to take the digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce a color image with as few visual artifacts as possible.

Gallery

My Approach

1. Crop images with specific percent

First, compute the height of each part. Then, separate the jpg or tif file into three color channel images. The borders of the images will probably hurt my results. In order to compute my metric on the internal pixels only, I set the special percent to be for example 10%, which means 10% of length and width are deleted from the left, right, top and bottom of the original image. See the following, Left is the original image and right is the image after cropping.

2. Rescale and contrast enhancement in preprocessing

Rescale is doing the work of automatic contrasting, which makes the darkest pixel be zero and the brightest pixel be one. Then contrast enhancemnet is done in the green and red image based on the histogram of the blue image.

3. Image matching metric

Sum of Squared Differences(SSD): The sum of squared differences measures the difference between two color channel images. The smaller the calculated value, the better the alignment.

Normalized Cross-Correlation(NCC): The normalized cross-correlation calculates the correlation between two color channel images. The bigger the calculated values, the more correlated the two channel images are, the better the alignment.

For example, both SSD and NCC are used in processing workshop.tif with the same parameters. It takes SSD 7.296585 seconds to compute and the result is (G:[52,0], R:[104,-12]), while NCC uses 22.801829 seconds to compute and gets the same result. Accroding to the reference, NCC is more accurate compared with SSD in the most conditions.

4. Image Pyramid

In the single-scale implementation, a small range of displacement [-15, 15] is selected for searching. But for high resolution images, exhaustive search becomes pretty expensive as pixel displacement is large. For this reason, image pyramid is used. Image pyramid speeds up the process by rescaling the image and works with the smaller copies. In my process, I set the number of pyramid levels to be 5. Searching range for the coarsest scale is set to be [-15,15] and for the other scales, it is [-5,5]. Scale factor is 2. Start from the coarsest scale, go down the pyramid and update the estimate. (previous_estimate_displacement*2=present_initial_displacement)

Bells and Whistles

Better features

Edge detection: an image processing technique for finding the boundaries of objects within images. It works by detecting discontinuities in brightness.

(1) Canny Edge Detection is a multi-stage algorithm - noise reduction, finding intensity gradient of the image, non-maximum suppression and hysteresis thresholding.

(2) Prewitt Edge Detection is used for detecting vertical and horizontal edges in images.

As shown in the following images, Canny can get more details compared with Prewitt.

Image gradient: directional change in the intensity or color in an image.

The Canny edge detector uses image gradient for edge detection.

Automatic white balance (postprocessing)

(1) Estimate the scene illumination.

(2) Correct the color balance of the image.

The gray world algorithm assumes that the RGB values are linear, so i changed rgb to linear color space and excluded the top and bottom 10% of pixels.

Preprocessing part is done in the My Approch section.

Automatic contrasting (preprocessing)

Work of rescaling image intensities is done in the My Approch section. Histogram images are shown below.

Automatic cropping

First, i changed the white pixel in the image to black. Then local standard deviation filter is used to detect edge or border. Although i could figure out where edges are based on the filtered image, it's hard to let computer know. So i check the histogram of suspected parts in each dimension and set 10% of each dimension as the threshold to determine the location of border. But this approach would make the whole process not automatic. Maybe i can figure out a clever way to realize real automatic cropping in the future.

More images

References

[1] http://graphics.cs.cmu.edu/courses/15-463/2004_fall/www/Papers/MSR-TR-2004-92-Sep27.pdf

[2] https://en.wikipedia.org/wiki/Edge_detection

[3] https://www.mathworks.com/help/images/ref/imresize.html

[4] https://www.loc.gov/collections/prokudin-gorskii/

[5] https://medium.com/@nikatsanka/comparing-edge-detection-methods-638a2919476e

[6] https://www.mathworks.com/help/images/comparison-of-auto-white-balance-algorithms.html

[7] https://inst.eecs.berkeley.edu/~cs194-26/fa20/hw/proj1/