CS194-26: Image Manipulation, Computer Vision and Computational Photography Project 5

By Alex Kassil (cs194-26-aca)

Part A: MANUAL MANUAL MANUAL

Step 1 Shoot the pictures

I took photos of a painting on my wall, inside a room, and the stairwell

Already noticle is a weird green tint from my photographs, this will cause issues later one

Step 2 Recover homographies

Lets label some points! Just like project 3

Step 3 Warp and rectify the images

Once I computed the points and created the homography function, I can now compute the delaunay triangula... I mean rectify an image and recover the whole painting by finding the homography between it and the unit square, which I did for the two paintings, because who doesn't love art?

I used inverse warping to create images that look quite clean

Step 4 Mosaics!

Now I have everything I need to warp to create a mosaic. Lets try to warp the painting of a sunset photos to have a mosaic with a rectified painting

First lets warp the images, and find a nice border for both of them to fit

Now add together the photos!

Oh no, that looks not write, super bright in the overlap region.

What happends if we overlap painting2 over painting1 or the other way around?

These look much better, but you can see some lines streaking

Let us try feathering! Apply some masks, then combine!

Compared to featherine, I think overwriting actually looks nicer

More mosaics!

I wish I could go outside and not just take pictures inside my house :/

What did I learn?

SO MUCH! Woah writing a vectorized warp function took SO LONG. And it was sooo fast, and quite elegant in the end. Adding points manually isn't too bad, and can lead to some really nice results. Another thing that made my warp waaay faster way using np.where to get index that wasn't out of bounds, that was a key insight that allowed me to vectorize my code

Part B: AUTOMATIC AUTOMATIC AUTOMAIC

Step 1: Corners

I used the provided harris corner detection code to find some corners

Step 2 ANMS

Then after reading the ANMS section many times, as well as looking at other webpages, I finally understood it. The point is to compute pairwise distances between each point, and then find the nearest point that is greater harris value than you, that is your score. Score of global max is infinity, and then sort by score and return top n. Returning top 500 worked for me.

Here you can see it finding really intersting points like corner of paintings and other things like that

Step 2 Feature Descriptor Extraction

Using feature detection I could find pair up points between the two images

Step 4 RANSAC

And finally RANSAC provides with robust pairs and removes the outliers

Below are the original manual warps on the left, ransac on the right

Difference in quality is quite similar, but the difference in work is no manual labeling!

What I learned?

It was amazing in the end to have one ransac function that just took images and produced the final mosaic. That felt like something that could be a polished part of a camera app on a phone. Testing your code on small examples is important, and turning snippets into functions into bigger functions is crucial. I learned how one computer and one programmer together with a few photos can do quite some cool stuff.