CS 194-26 PROJECT 1 - Colorizing the Prokudin-Gorskii photo collection

Shreyas Patankar

I. Overview

The goal for this project was to accurately align 3 versions (red, green, blue) of the same photo to create a cohesive color image. This was done by first separating the 3 vertically stacked images (see sample below), then finding the best alignment for them using various algorithms. Here's a sample input: Monastery And a sample output: Monastery Out

II. Method

  • Load the image into the program.
  • Partition the image into its individual color channels.
  • Pre-crop the edges of the image to get rid of most black-space on the outside edges. I have cropped the height by 1/24 of the original height on both sides and 1/12 of the width on both sides.
  • Align the images since they were taken at slightly different heights. For alignment, I have implemented a recursive image pyramid. The depth varies - I empirically found that aligning with a starting image of ~100x100 works best for the base case, then work my way up from there. The depth I go to is therefore log2(height / 100). Once the best alignment at the max depth is found, I simply have to align within the 4 pixels the one pixel turns into at the next level.
  • Crop the edges to remove minor shifting. I remove 1/100th of the original width and height.
  • Stack the aligned channels and save the resulting image.

    • Final Results

    workshop.tif 
Red Offset: ( -12 , 104 )
Green Offset: ( 0 , 52 )

    emir.tif 
Red Offset: ( 40 , 24 )
Green Offset: ( 24 , 48 )

    monastery.jpg 
Red Offset: ( 2 , 2 )
Green Offset: ( 2 , -3 )

    three_generations.tif 
Red Offset: ( 8 , 111 )
Green Offset: ( 12 , 52 )

    castle.tif 
Red Offset: ( 0 , 96 )
Green Offset: ( 0 , 32 )

    melons.tif 
Red Offset: ( 0 , 160 )
Green Offset: ( 0 , 80 )

    onion_church.tif 
Red Offset: ( 36 , 104 )
Green Offset: ( 24 , 48 )

    train.tif 
Red Offset: ( 32 , 86 )
Green Offset: ( 4 , 43 )

    tobolsk.jpg 
Red Offset: ( 2 , 6 )
Green Offset: ( 2 , 2 )

    icon.tif 
Red Offset: ( 22 , 88 )
Green Offset: ( 16 , 40 )

    cathedral.jpg 
Red Offset: ( -975 , -960 )
Green Offset: ( -975 , -960 )

    self_portrait.tif 
Red Offset: ( 48 , 52 )
Green Offset: ( 28 , 78 )

    harvesters.tif 
Red Offset: ( 12 , 112 )
Green Offset: ( 16 , 56 )

    lady.tif 
Red Offset: ( 8 , 112 )
Green Offset: ( 8 , 55 )

    Issues

    I found that I had issues with images that had redundant patterns as opposed to a clear subject. These images included melons.tif, cathedral.jpg, and self_portrait.tif. The SSD for such images, presumably, was quite similar in many locations due to the redundancy. I attempted going to a greater & shallower image pyramid height, as well as additional "jitter" for the base case. Although there were minor improvements in some images (like melons, for example), they did not fully solve the problem.

    Other Examples

    Here are a few more examples from the Prokudin-Gorskii collection.

    man.tif 
Red Offset: ( 0 , 104 )
Green Offset: ( 4 , 40 )

    vase.tif 
Red Offset: ( -16 , 112 )
Green Offset: ( -4 , 16 )

    boat.tif 
Red Offset: ( 92 , 100 )
Green Offset: ( 44 , 47 )