Project 6B Submission

Name: Rohan Murthy(cs194-26-adk)

Part A

Overview: Image Warping and Mosaicing

This project involved taking pictures and creating mosaics out of them by calculating the homography matrix from one photo towards another, warping them, and combining them.

Shoot and Digitalize Pictures

Here are the images I took!

Recover Homographies

Before warping an image to be used in a mosaic, you must first find the homography matrix that will be used to do this warping. Since I use inverse warping and fix my second image I take second image correspondance points as the p points (xi,yi) and the first image correspondance points as the p' points(x'i,y'i). This is a least squares problem, Ax=b where x is the homography matrix in vector form, b is vector([0 0 ... 0 1]) where the number of zeros is 2*(number of correspondance points). The A matrix is [[-x1 -y1 -1 0 0 0 x1*x'1 y1*x'1 x'1][0 0 0 -x1 -y1 -1 x1*y'1 y1*y'1 y'1] ... [-xn -yn -1 0 0 0 xn*x'n yn*x'n x'n][0 0 0 -xn -yn -1 xn*y'n yn*y'n y'n] [0 0 0 0 0 0 0 0 1]] where n is the number of correspondance points. I solved this least squares problem to recover the homography matrix.

Image Rectification

This part of the project involved taking a photo and looking at it from a different perspective. The way I did this was first finding the homography matrix. For both of my cases here, I found the points on each corner of the keyboard/rug and found the homography matrix from these points to four points which formed a rectangle. Once I got this matrix, I warped the original photo using this matrix. My warping algorithm is very similar to the one I used in project 4, which involves inverse warping and using interpolation.

Here is my keyboard, with a top perspective!

Here is my rug, with a top perspective!

Mosaics

This part of the project involved taking multiple pictures from different perspectives and combining them to make a mosaic. I made two picture mosaics in which I kept the second image the same, but warped the first image towards it. To do this, I first padded my images so that when I warped the first image, the full warped image could show. Then, using points of objects/things that were in both images, I created a homography matrix . I then used this matrix to warp the first image using the technique described above in the image rectification section. Once this first image was warped, I combined this with the second image to create the mosaic.

Please note that for my mosaics I scaled down the images so that the warping didn't take too long.

Here are the original images on campus and the mosaic!

Here are the original images of the sunset from my apartment and the mosaic!

Here are the original images of my apartments hallway and the mosaic!

Coolest Thing I Learned

The coolest thing I learned from this part is the power of homographies. It is amazing how you can look at things with a different perspective that you might not have been able to see before. For example, using my rug picture above, it is amazing how you can use this, with interpolation, to see it in a completely new way; like you are standing above it without having to actually do that.

Part B

Feature Matching For Autostitching

This part of the project involved referencing this paper to auto stitch images and warp to create mosaics like in the previous part.

Detecting Corner Features

Using the given harris.py file, I was able to get all of the harris corners for the images. I used a min_distance of 20 for the peak_local_max function call so that I wouldn't get an overwhelming amount of corners. For the images I got around ~1.2k points.
Here are the images for the first mosaic and their harris corners.

Adaptive Non-Maximal Suppression

This part involved using section 3 of the paper to make sure that the harris corners that are used further down the line are uniformally sampled. For each point, I found the smallest distance(radius) to another point which has a higher harris intensity value. Then, I took the 500 points that had the highest radii of all the points. After using this suppression, here are the points for the first images:

Feature Descriptor Extraction and Matching

Using section 4 and 5 of the paper, I used the MOPS method to essentially match the corner points in the first image of the mosaic to the second image in the mosaic. The descriptor of a corner is essentially an 8x8 matrix of pixels that surround a corner. For each corner I found the corresponding corner in the other image that had the smallest distance between descriptors, but only actually used it if it satisfied the ratio value, described in the paper, of 0.7 or smaller.
After doing feature matching, here are the points for the first images:

RANSAC and Producing Mosaic

I used the RANSAC method as described in class to pare down the corners(points) that will be used in the mosaic creation. For creating the mosaic, I used the same method as what I did in part A (see my description) above in the Mosaic section.
After doing RANSAC, here are the points for the first images:


Here are the original images on campus, the mosaic from part A and the mosaic for this part B!

Here are the original images of the sunset from my apartment, the mosaic from part A and the mosaic for this part B!

Here are the original images of my apartments hallway, the mosaic from part A and the mosaic for this part B!

Overall, I think that it turned out pretty good. Though I think that doing the manual process from part A had results that were a bit better than those in part B.

Coolest Thing I Learned

The coolest thing I learned from this part was that you can find corners that are closely identical in photos taken from different perspectives. This is amazing and offers a really fast way of creating mosaics and manipulating multiple images in general.