CS194 Project 1

Overview

In this project, I took a set of images with blue, red, green channels and colorize them. This involves separating each image into three (for blue, red, green channels), aligning those three images and stacking them together. The starter code took care of splitting the image. My work mostly focus on aligning the images.

Approach

There are three pieces in this project. One is to define a cost function that calculate the mismatchment between two images. For this, I used the Sum of Squared Differences (SSD)as suggested on the spec.

Second is the function to align two images. Here, I use two for loops to a range of possible shift and find the best alignment according to the cost function.

Third is image alignment using image pyramid. This is to speed up image alignment for large images. This is pretty easy to implement using recursion and the align function defined in second part. Here, I use skimage.transform.rescale to scale down the images, and when the images are small enough (base case) I call align(image1, image2) on them, otherwise, I scale them down by 0.5 and call align_pyramid(image1, image2) on them. At each level, I use the displacement calculated on the previous recurse to find a new displacement at the current level.

Problem

At first, I get blurry images after alignment. I solves this by altering the cost function to not take into accountthe boundaries of two images when calculating the misalignment.

Currently, the result of emir is worse than those of others. This is because the brightness between different channels of emir is quite different. Therefore, SSD cost function cannot capture the different very well.

Result

all displacement is represented as (height, width), i.e (35, 3) means the image is moved down 35 pixels and moved right 3 pixels.

castle

castle_dg(35, 3)_dr(98, 4).jpg

Displacement for green channel: (35, 3). Displacement for red channel: (98, 4).

cathedral_dg(5, 2)_dr(12, 3).jpg

cathedral_dg(5, 2)_dr(12, 3).jpg

Displacement for green channel: (5, 2). Displacement for red channel: (12, 3).

emir_dg(49, 24)_dr(88, 43).jpg

emir_dg(49, 24)_dr(88, 43).jpg

Displacement for green channel: (49, 24). Displacement for red channel: (88, 43).

harvesters_dg(60, 16)_dr(124, 13).jpg

harvesters_dg(60, 16)_dr(124, 13).jpg

Displacement for green channel: (60, 16). Displacement for red channel: (124, 13).

icon_dg(41, 17)_dr(89, 23).jpg

icon_dg(41, 17)_dr(89, 23).jpg

Displacement for green channel: (41, 17). Displacement for red channel: (89, 23).

lady_dg(55, 8)_dr(111, 12).jpg

lady_dg(55, 8)_dr(111, 12).jpg

Displacement for green channel: (55, 8). Displacement for red channel: (111, 12).

melons_dg(82, 9)_dr(177, 11).jpg

melons_dg(82, 9)_dr(177, 11).jpg

Displacement for green channel: (82, 9). Displacement for red channel: (177, 11).

monastery_dg(-3, 2)_dr(3, 2).jpg

monastery_dg(-3, 2)_dr(3, 2).jpg

Displacement for green channel: (-3, 2). Displacement for red channel: (3, 2).

onion_church_dg(51, 26)_dr(108, 36).jpg

onion_church_dg(51, 26)_dr(108, 36).jpg

Displacement for green channel: (51, 26). Displacement for red channel: (108, 36).

self_portrait_dg(79, 29)_dr(175, 34).jpg

self_portrait_dg(79, 29)_dr(175, 34).jpg

Displacement for green channel: (79, 29). Displacement for red channel: (175, 34).

three_generations_dg(55, 13)_dr(112, 10).jpg

three_generations_dg(55, 13)_dr(112, 10).jpg

Displacement for green channel: (55, 13). Displacement for red channel: (112, 10).

tobolsk_dg(3, 2)_dr(6, 3).jpg

tobolsk_dg(3, 2)_dr(6, 3).jpg

Displacement for green channel: (3, 2). Displacement for red channel: (6, 3).

train_dg(42, 6)_dr(87, 32).jpg

train_dg(42, 6)_dr(87, 32).jpg

Displacement for green channel: (42, 6). Displacement for red channel: (87, 32).

workshop_dg(53, -1)_dr(102, -13).jpg

workshop_dg(53, -1)_dr(102, -13).jpg

Displacement for green channel: (53, -1). Displacement for red channel: (102, -13).