Soham Kale CS194 Project 5A

Part 1: Image Warping and Mosaicing

Image Rectification

I used a picture I took of the British parliament a little while ago. I wanted to make the side of the building be front-parallel so it looked like I had taken it from the front.

Parliament
An image of the British parliament

I created my homography matrix H by solving the following overdetermined system of linear equations using the correspondances (x, y) to (x', y'). I solved the system using linear least squares.

H matrix
Homography Matrix
Solve for H matrix
System of linear equations to solve

I then warped my image by creating an inverse warp function H^-1, and then creating a polygonal coordinate mask that covered the entire image. I applied H^-1 to the mask, and applied to my rectified image the pixel values of where each coordinates of the transformed mask lay.

I got the following results:

Rectified British Parliament
Rectified British Parliament

Blend the images into a mosaic

I moved onto making a mosaic out of the following three images from my backyard:

I manually defined the following correspondances for them:

I then made a panorama piece by piece, starting with the left and the middle images, then adding the right image:

Full Backyard Panorama

Because I used a basic rectilinear homography, and not more complicated variants like ones that use elliptical geometries, the edges of my pano are highly warped and grainy.


What I learned

I think the most interesting I learned was how different types of linear transformations can have different distortion capabilities on images, and why the homography was the best choice for this type of distortion.




Part 2: Feature Matching for Autostitching

I decided to test my code for these points on these two images of my desk:

Harris Interest Point Detector

I applied a harris interest point detector on the first image to get the following harris function on my image:


Adaptive Non-Maximal Suppression

I followed the algorithm given in the paper to implement ANMS, and compared my results to generating coordinates instead by sampling randomly from my harris points, and choosing the highest valued harris points:

AMNS coordinates
AMNS coordinates
Random coordinates
Random coordinates
Maximal harris value coordinates
Maximal harris value coordinates

Feature Descriptors

I followed the algorithm in the paper to generate feature descriptors at every point given by the AMNS algorithm. My descriptor had a width of 30 and I used a stride of 4 when sampling from my image; this was an enlargened version of what I used later on (I had a width of 8 for later parts), but it's to visualize my features for now:

Feature Matching

I followed the given algorithm to use SSD to compare each feature with every other feature to generate correspondances, and used the method by Lowe to only use correspondances that are far better than a second choice. The following is the correspondances I found on images 1 and 2:


RANSAC

Next I used RANSAC to only keep the correspondances that gave the best homography. After I computed a homography using the points left over, the following gives the result of the homography on the points on image 1 to image 2, compared to the actual points on image 2:

Actual vs. homography approximate feature points on image 2

I used the approximated homography to calculate a panorama between the two images:

MOPS panorama

I also manually defined points and computed a panorama that way as well:

Manual panorama

I would say that the MOPS algorithm actually gave a better result in this case.


Examples

I tried both the MOPS and manual technique on a few other examples.

Desk

MOPS panorama
Manual panorama

Backyard

MOPS panorama
Manual panorama

Office

MOPS panorama
Manual panorama

What I learned

I thought it was really cool to see the feature descriptors be really good for finding correspondances for being so small + sparse (when compared to the size of the actual image). It seemed so unlikely that correspondances points this good would be made considering that there was no gaurantee that the points that AMNS returned in one image would have analogous points in the other image.