Auto Alignment

Due to time constraint, I could not implement RANSAC and automatic alighment with panorama. The parts I did do are the Harris Corner, ANMS, and Feature Descriptors.

Harris Corner Detection and ANMS

By using the staff provided code which finds the Corner Response Function (R = det(M) - alpha trace(M)^2). However, to find a diverse and high impact points, we use ANMS to spread the distribution of points without losing importance. Here are examples of the key points that were detected after being processed by Harris Corner Detection and ANMS. The before and after is shown side-by-side.

Feature Descriptors

Afterwards, I find a feature descriptor for each keypoint by getting an 8 x 8 descriptor from a 40 x 40 pixel, then normalizing the 8 x 8 matrix to have a mean of 0 and sd of 1. I then use nearest neighbors with the dist2 error fuction (calculates squared distance between two sets of points), and threshold points by the ratio 1NN / 2NN. Ideally, the threshold should be around 0.5 so that we can eliminate points with marginal alignment.

RANSAC and Panorama

Did not do.

Pictures

The input pictures are mapped along with the corresponding rectified/mosaic pictures.

Homographies

Homography1 Homography2

Warp (Rectified)

Rectified

Panorama

Panorama

Lessons

I learned how to set up a proper workflow for an image process project. It was a bit messy before, and it's still a bit messy now but I wrapped the autonomous functiosn in a class that I'm working on for part 2. Otherwise, I've utilized most the knowledge I've learned in the past weeks about homography and projects along with linear algebra to synthesize a pretty amazing application. Also taking pictures on my iPhone, I realized the axis directions were flipped for Preview but not for streaming the data with skio/cv2 so I resorted to using pictures from a friend for computing (except for the rectified picture 3, which is the picture I found aforementioned directional anomoly).