CS294 Project1: Colorization

Zeyang Bao^*

University of California, Berkeley


Abstract: In this project, I'm asked to implement some algorithms to align the three channels of a image. Then stack them together to the a colorized image. There are severl methods to do that:Minimize Sum of Squared Differences (SSD) distance and maximize normalized cross-correlation (NCC). I have implement two methods. However, the algorithms are very expensive when working on high-resolution images, so I also impleme the multiscale pyramid version.

Single-Scale Version

SSD

For image1 and images, to align image1 images, find two shifts dx and dy to minize the two images' "distance".
That is Argmin sum(sum(image1+dx+dy - image12).^)

NCC
Intermediate results. Top row: input images. 2nd row: results from individually optimizing hand and object. 3rd row: results from joint optimization (two viewpoints per example). Bottom row: results after the refinement.

Multi-Scale Version

Logic

The above image shows how image pyramid works. The original image is down-sample / resize by a constant ratio and I use 2x here. After resized by 4 times, the origal size will be 1/16 of the original size. Then it is much easier to do the previous SSD / NCC search method.

Another thing I noticed that is the colored margins which are black or white. It will effect the result. Since when I roll the image, the left margin's black comes to the right size, top margin's black goes to the bottom which means in this area, the distance are zero. To eliminate the impact for some extend, I decided to crop the margin of a constant value. The results are also shown below. There are some better way, I will discuss in the later section.

Result

Questions Address

Brightness are different on each channel image for a certain image, and it will cause some erros when apply the above SSD or NCC methods.

It is shown that there the pixel distribution of each channel is extremely different, which could lead to huge calculation different is we directly apply SSD or NCC; To solve it, I applied the

To Ace the Questions

Brightness Equivalent

Method. In order to align two channel images, I apply a method that adjust the brightness of channel b before calculate the distance or similarity of two images.

Auto Crop

In order to have better alignment, we can have a better crop of the boundary for example Canny boundary detection. I choose several bad cases in normal alignment method to show the effectivity.