Project 6B Submission
Name: Rohan Murthy(cs194-26-adk)
Part A
Overview: Image Warping and Mosaicing
This project involved taking pictures and creating mosaics out of them
by calculating the homography matrix from one photo towards another, warping them,
and combining them.
Shoot and Digitalize Pictures
Here are the images I took!
Recover Homographies
Before warping an image to be used in a mosaic, you must first find the homography matrix
that will be used to do this warping. Since I use inverse warping and fix my second image
I take second image correspondance points as the p points (xi,yi) and the first image
correspondance points as the p' points(x'i,y'i). This is a least squares problem, Ax=b
where x is the homography matrix in vector form, b is vector([0 0 ... 0 1]) where the number
of zeros is 2*(number of correspondance points). The A matrix is
[[-x1 -y1 -1 0 0 0 x1*x'1 y1*x'1 x'1][0 0 0 -x1 -y1 -1 x1*y'1 y1*y'1 y'1] ...
[-xn -yn -1 0 0 0 xn*x'n yn*x'n x'n][0 0 0 -xn -yn -1 xn*y'n yn*y'n y'n]
[0 0 0 0 0 0 0 0 1]] where n is the number of correspondance points. I solved this least
squares problem to recover the homography matrix.
Image Rectification
This part of the project involved taking a photo and looking at it from a
different perspective. The way I did this was first finding the homography matrix.
For both of my cases here, I found the points on each corner of the keyboard/rug
and found the homography matrix from these points to four points which formed a
rectangle. Once I got this matrix, I warped the original photo using this matrix.
My warping algorithm is very similar to the one I used in project 4, which involves
inverse warping and using interpolation.
Here is my keyboard, with a top perspective!
Here is my rug, with a top perspective!
Mosaics
This part of the project involved taking multiple pictures from different perspectives and
combining them to make a mosaic. I made two picture mosaics in which I kept the second
image the same, but warped the first image towards it.
To do this, I first padded my images so that
when I warped the first image, the full warped image could show.
Then, using points of objects/things that were in both images, I created a homography matrix
. I then used this matrix to warp the first image using the technique described above in the
image rectification section. Once this first image was warped, I combined this with the second
image to create the mosaic.
Please note that for my mosaics I scaled down the images so that the warping didn't take too long.
Here are the original images on campus and the mosaic!
Here are the original images of the sunset from my apartment and the mosaic!
Here are the original images of my apartments hallway and the mosaic!
Coolest Thing I Learned
The coolest thing I learned from this part is the power of homographies. It is amazing how you
can look at things with a different perspective that you might not have been able to see before.
For example, using my rug picture above, it is amazing how you can use this, with interpolation,
to see it in a completely new way; like you are standing above it without having to
actually do that.
Part B
Feature Matching For Autostitching
This part of the project involved
referencing this paper
to auto stitch images and warp to create mosaics like in the previous part.
Detecting Corner Features
Using the given harris.py file, I was able to get all of the harris corners for the images. I used a
min_distance of 20 for the peak_local_max function call so that I wouldn't get an overwhelming amount of
corners. For the images I got around ~1.2k points.
Here are the images for the first mosaic and their harris corners.
Adaptive Non-Maximal Suppression
This part involved using section 3 of the paper to make sure that the harris corners that are used further
down the line are uniformally sampled. For each point, I found the smallest distance(radius)
to another point which has
a higher harris intensity value. Then, I took the 500 points that had the highest radii of all the points.
After using this suppression, here are the points for the first images:
Feature Descriptor Extraction and Matching
Using section 4 and 5 of the paper, I used the MOPS method to essentially match the corner points in the first image
of the mosaic to the second image in the mosaic. The descriptor of a corner is essentially an 8x8 matrix
of pixels that surround a corner. For each corner I found the corresponding corner in the other image
that had the smallest distance between descriptors, but only actually used it if it satisfied the ratio value,
described in the paper, of 0.7 or smaller.
After doing feature matching, here are the points for the first images:
RANSAC and Producing Mosaic
I used the RANSAC method as described in class to pare down the corners(points) that will be used in
the mosaic creation. For creating the mosaic, I used the same method as what I did in part A (see my description)
above in the Mosaic section.
After doing RANSAC, here are the points for the first images:
Here are the original images on campus, the mosaic from part A and the mosaic for this part B!
Here are the original images of the sunset from my apartment, the mosaic
from part A and the mosaic for this part B!
Here are the original images of my apartments hallway,
the mosaic from part A and the mosaic for this part B!
Overall, I think that it turned out pretty good. Though I think that doing the manual process from part A had results that were a bit better than those in part B.
Coolest Thing I Learned
The coolest thing I learned from this part was that you can find corners that are closely identical
in photos taken from different perspectives. This is amazing and offers a really fast way of
creating mosaics and manipulating multiple images in general.