Project 1 : Colorize Me!


In this project, we have have photo plates that uses a novel camera design. This camera took three photos of the same object in three different light channels. This was work done in the the late-1800s and early 1900s by Sergei Mikhailovich Prokudin-Gorskii. Here are going to combine these by aligning the different photos to create a color photo.

Approach Outline

  1. Split photos into thirds by splitting along the y-axis
  2. Calulate the offset for the green and yellow channels using an exhastive search or an image pyramid
  3. Combine the images using the offset
  4. Save and display the image


Before the offsets were calulated, I replaced the borders of the photos with the average value of the pixels not covered by the new borders. This was done to reduce noise at the edges of the photos and so that when an image is rolled, it does not split the image. This was done because if an image has a repeating pattern across the image, it would not cause a miss-alignment.

Exhastive Search

To do exhastive search for aligning the images. All alighments are simple vertical and horizational shifts. For the small images, I did search over offsets over [-30,30]. It use the sum of square differences to determine how correlated two layers are.

Image Pyramid

I implemented the image pyramid recusively. The base case is having an image no larger than 200 pixels on each dimension and doing an exhastive search on that. Ofterwise, it would calulate the offset for an image whose dimensions are half of the imputted image. Then the offset are doubled to scale back to the original image. Then, an small exhastive search is on done on original image using offsets that do not differ more than by one for any of the scaled offset's coordinates.

The image pyramid works is pretty fast. It can find the offsets for any of the large images in about 6 seconds. On the small images, it can find the offsets in about half a second, if I change the base case to have an imputted image to be no larger than 100 pixels in any dimensions.

Small Photos

--------------- tobolsk.jpg ---------------

green offset: 2 3

red offset : 3 6

Missing Picture

-------------- cathedral.jpg --------------

green offset: 2 5

red offset : 3 12

Missing Picture

-------------- monastery.jpg --------------

green offset: 2 -3

red offset : 2 3

Missing Picture

Big Photos

---------- three_generations.tif ----------

The old man and lady have colored highlights. I think it is due to the parallex from the three cameras not being in the excat same place. So some of the potiental solutions is morphing the image with some non-linear function in order to fit the other parts of the image.

green offset: 13 52

red offset : 11 111

Missing Picture

------------ onion_church.tif ------------

green offset: 26 51

red offset : 36 108

Missing Picture

--------------- melons.tif ---------------

green offset: 9 81

red offset : 13 178

Missing Picture

-------------- workshop.tif --------------

green offset: 0 53

red offset : -12 105

Missing Picture

---------------- emir.tif ----------------

This image did not produce a lot of good results because the blue channel was very bright in comparision to the other channels. So the bright and darks spots are not will correlated. So to fix this, it would involve canny edge detection.

green offset: 24 49

red offset : -1 -161

Missing Picture

---------------- icon.tif ----------------

green offset: 17 41

red offset : 23 89

Missing Picture

---------------- lady.tif ----------------

green offset: 8 53

red offset : 11 115

Missing Picture

------------ self_portrait.tif ------------

green offset: 29 79

red offset : 37 176

Missing Picture

---------------- train.tif ----------------

green offset: 5 42

red offset : 32 87

Missing Picture

------------- harvesters.tif -------------

green offset: 16 59

red offset : 13 123

Missing Picture

--------------- castle.tif ---------------

green offset: 3 34

red offset : 4 98

Missing Picture

My Photos

--------------- studio.tif ---------------

green offset: 20 78

red offset : 25 151

Missing Picture

--------------- church.tif ---------------

green offset: -19 -10

red offset : -56 -2

Missing Picture

-------------- painting.tif --------------

green offset: 2 28

red offset : 7 68

Missing Picture