CS 194-26 Fall 2020

Project 1

ChristianMurray

Overview

In this project the goal is to create colored images by overlaying three plates, R, G, and B, that are photographs taken of the same image by Gorskii. By figuring out how much to displace each plate, we can overlay the three optimally to create the sharpest and clearest color image from them.

Approach

Image mapping algorithms must be created in order to help more intelligently determine how to overlay the plates. These algorithms have a hard tradeoff for accuracy and runtime, so a balance must be found where the image will be overlaid correctly while completing the task in a reasonable timeframe.

Tradeoffs for how exhaustive searches are, and how deep image pyramids are, have to be determined. By exhaustively searching over ranges of pixels to find the displacement based on a heuristic, a naive solution was found. Applying this to larger files (such as tif files) was not ideal, so a smarter implementation had to be devised.

This is where recursive exhaustive searches through an image pyramid was implemented.

Naive Search

Exhaustively search over the range of [-15, 15] pixels and find the largest value of the normalized cross correlation to determine the most appropriate offset to create the colored image.

cathedral Green offset: x[2] y[-6] Red offset: x[3] y[-8]
monastery Green offset: x[2] y[-12] Red offset: x[3] y[-15]
tobolsk Green offset: x[2] y[-6] Red offset: x[3] y[-11]

Image-Pyramid

For larger images such as the provided Tiff files, naive searches were not optimal for finding the correct overlay. By using an image pyramid, we can continually scale the image by a factor of 2 until we can find the optimal displacement.

These recursive calls on an exhaustive search could be very slow, so care was taken to find the average minimum number of recursive calls (4) and smallest displacement window (-14, 14) to exhaustively search for the best results paired with decent runtime.

Some results, such as Emir, were not nearly as accurate. This is due to a large descrepancy in the RGB plates brightness in conjunction with a lack of a function to intelligently crop borders. Implementation of such a function and/or layering the plates in a different order would likely provide a far sharper image.

Results

[harvesters] Red offset: vert(56) horiz (4) Green offset: vert(28) horiz (8)
[emir] Red offset: vert(44) horiz (16) Green offset: vert(-4) horiz (8)
[castle] Red offset: vert(4) horiz (104) Green offset: vert(4) horiz (36)
[icon] Red offset: vert(20) horiz (20) Green offset: vert(8) horiz (16)
[train] Red offset: vert(40) horiz (8) Green offset: vert(20) horiz (4)
[melons] Red offset: vert(112) horiz (0) Green offset: vert(48) horiz (4)
[onion_church] Red offset: vert(-8) horiz (4) Green offset: vert(-28) horiz (24)
[self_portrait] Red offset: vert(-48) horiz (36) Green offset: vert(-24) horiz (0)
[three_generations] Red offset: vert(-48) horiz (8) Green offset: vert(-28) horiz (12)
[lady] Red offset: vert(36) horiz (16) Green offset: vert(24) horiz (-4)
[workshop] Blue offset: (40, -20) Red offset: (20, -4)

More examples

A series of other images I found interesting to test the alogrithm on. View of Voznesene was slightly less well aligned likely due to the large variance in brightness of each plate in the clouds in the sky as well as the border on the picturre

Results

[View of Voznesene] Red offset: vert(96) horiz (4) Green offset: vert(-4) horiz (4)
[View of Voznesene] Red offset: vert(112) horiz (-4) Green offset: vert(52) horiz (4)
[Site where the relics of Grand Duchesses Evfrosiniia and Evdokiia are intered] Red offset: vert(-4) horiz (4) Green offset: vert(-24) horiz (4)