Colorizing the Prokudin-Gorskii photo collection

Tyler Zhu

Overview

Our goal this project is to colorize the Prokudin-Gorskii photo collection, which contains negatives in red, blue, and green taken in Soviet Russie in 1907, by finding the correct alignments from the strips and adding whatever necessary color corrections to make it look good.

Method

My method was quite standard. I implemented both exhaustive and pyramid search as outlined in the project spec (along with Violet and Vicky's helpful insight). I used SSD for both, as NCC was giving the same exact results for .jpeg images and running far longer (given the extent of the operations needed). For alignment, I left blue as the base negative to which the other negatives aligned to as set up by the project starter code, but I modified this for some of the images (namely emir.tiff and self_portrait.tiff) which made them work better.

I implemented pyramid search with some basic constants features. I had some issues with np.roll which plagued me most of the time, but besides that coding was smooth (aside with finding the best settings for the pyramid search that actually ran in under 30 seconds per negative alignment).

Finally, for my Bells and Whistles, I implemented the contrast smoothing technique outlined in class using the CDF of the pixel intensity distribution, which helped solve my problem aligning some of the pictures (specifically church.tiff) as well as made others with high/low saturation more even (specifically workshop.tiff). Overall, I very much enjoyed this project.

Results

Exhaustive Search on JPEG Images

All Jpeg images were aligned with exhaustive search aligning the green and red negatives to the blue negative.

Cathedral

Cathedral: Green offset by (5, 2), Red offset by (12, 3).

Monastery

Monastery: Green offset by (-3, 2), Red offset by (3, 2).

Tobolsk

Tobolsk: Green offset by (3, 2), Red offset by (6, 3).

Pyramid Search on TIFF Images

We performed pyramid search on the TIFF images to find the best alignments due to how large they were. We aligned the green and red negatives to the blue negative again, except for emir.tiff and self_portrait.tiff which had markedly increased coherence when aligned with respect to the green negative instead.

Church

Church: Green offset by (54, 582), Red offset by (82, 586).

For church, our naive results are quite bad. I suspect this is due to the large amount of blue naturally present which messes with everything (little contrast) and makes aligning noisy. I tried aligning to all three channels, and nothing helped.

Harvesters

Harvesters: Green offset by (58, 10), Red offset by (126, 13).

Lady

Lady: Green offset by (54, -6), Red offset by (113, -5).

Onion Church

Onion Church: Green offset by (52, 22), Red offset by (108, 36).

Three Generations

Three Generations: Green offset by (52, 4), Red offset by (111, 9).

Workshop

Workshop: Green offset by (52, -4), Red offset by (102, -12).

Emir

Workshop: Blue offset by (-46, -10), Red offset by (56, 12).

Icon

Workshop: Green offset by (40, 16), Red offset by (90, 22).

Melons

Workshop: Green offset by (82, 4), Red offset by (178, 12).

Self Portrait

Workshop: Blue offset by (-80, 1), Red offset by (98, 6).

Train

Workshop: Green offset by (41, -2), Red offset by (92, 27).

I also found my own TIFF images to run my algorithm on, named guardhouse.tiff and painting.tiff.

Guardhouse

Workshop: Green offset by (14, 0), Red offset by (30, -4).

Painting

Workshop: Green offset by (54, -6), Red offset by (674, -11).

As one can see from the large red numbers, the alignment here does not work very well.

Bells and Whistles

For my bells and whistles, I implemented the histogram equalization mentioned in lecture to even out some of the low-contrast images in the dataset. I ran the pyramid search on purely the contrasted images, then for church used the offsets obtained on the original pictures, while for workshop I used the new images for visual improvement.

Histogram Equalization from class. Source: Alyosha's Slides.

This worked miraculously for church.tiff, which finally looks aligned now.

Church Original. Green offset by (54, 582), Red offset by (82, 586).

Church, equalized. Green offset by (22, -4), Red offset by (54, -14). Images combined are the originals, not the balanced ones.

It also helped reduce over saturation in some pictures like the workshop picture.

Workshop Original: Green offset by (52, -4), Red offset by (102, -12).

Workshop, equalized: Green offset by (52, -4), Red offset by (102, -12) (still the same). Images combined are the balanced ones.