CS 194-26 PROJECT 1 
Kenan Jiang

  1. Overview
    In the project, I aligned digitized RGB negatives to form a color image. These negatives are from the photo collection of Russian artist Prokudin-Gorskii, who had token pictures of every scene during the time Russian Empire.
  2. Approach
    For .jpg images, I first cropped the image from shape (341, 391) to (301, 301). By cropping the image before aligning, I hope to get rid of some borders that may have large effects on my SSD score. Then I define an "align" function. I apply "align" to make red negative to blue negative, and green negative to blue negative. In align function, I shifted the red/green negative by a possible displacement pair (x, y) from all possible window. Then I calculated the Sum of Squared Differences score for this displacement pair. Then I compared all the score and picked the lowest one. The displacement pair for this score is the displacement I need. Then I just moved my red and green negatives according to their displacements and stack g, r, b channel together. 
    For .tif images, I used pyramid align because .tif image is large. I first decided to use a pyramid with level of 5 and each level we downscale with a factor of 0.5. For example, at a certain layer X of our pyramid (the bottom layer being our original image, top being the smallest), I call my "align" function recursively to get the best displcaement for X-1 layer. Then I multiple this displacement by 2 to get new_displacement, and shift my layer X image by this new_displacement, so now my layer X is aligned based on the advice from all the layers above it. To pass the alignement information down, I added layer X's displacement by calling "align" function on only a small center of layer X image to get current_displacement. Then I add new_displacement and current_displacement and pass it down to layer X+1...
  3. Results
    4. cathedral
    cathedral g:[5, 2], r:[12, 3]
    monastery
    monastery g:[-3, 2], r:[3, 2]
    tobolsk
    tobolsk g:[3, 2], r:[6, 3]
    three_generations
    three_generations g:[52, 17], r:[107, 14]
    emir
    emir g:[48, 24], r:[294, -552]
    workshop
    tobolsk g:[54, 0], r:[105, -12]
    zakat
    Own choosing: zakat g:[74, -41], r:[113, -67]
    train
    train.jpg g:[42, 7], r:[85, 32]
    icon
    icon g:[41, 18], r:[90, 23]
    harvesters
    harvesters g:[59, 18], r:[123, 15]
    lady
    lady g:[57, 6], r:[118, 10]
    self_portrait
    self_portrait g:[77, 29], r:[175, 37]
    castle
    castle g:[33, 2], r:[98, 5]
    melons
    melons g:[84, 10], r:[180, 13]
    onion_church
    onion_church g:[50, 26], r:[108, 37]
    siren
    Own choosing: siren g:[49, -5], r:[96, -23]
  4. Failure
    As you can see, my emir image is not clearly aligned. I think this is because they do not have the same brightness value on different color channels at same location. So my SSD score is not a good metric becasue the closest brightness value displacement cannot represent the true location of alignment.
  5. Bells & Whistles
    I did auto cropping. Fronm top, left, right, bottom, I check standard deviration of each row/column. For example, when checking border from top, I iterate through a few rows, find the one with max standard deviration and crop everything above this line. This is because borders usually have smaller std because they are the same color.
    cathedral
    before cropping
    extra_cathedral
    after auto cropping
    The improvement is obvious.