CS 194-26

Project 6: [Auto]Stitching Photo Mosaics

Minjoo Sur

Part A : IMAGE WARPING and MOSAICING

Rectified Pictures

By using homography matrix, we can find the original pattern of an object/poster with an angle. Below are examples of original and rectified images.

Warping

Using Homography matrix, we can warp one picture onto another. First, we multiply H with [x,y,1] to check the size. After we check what size our warped picture will have, we now fill in pixels of the the warped pictures with the original mapped points using inverse H. After this, we blend two pictures together to get panaramic view of the combined picture. I used linear blending for this using 0.5 as weight for both pictures. Below are the original pictures and the results.

Part B : FEATURE MATCHING for AUTOSTITCHING

Detecting corner features in an image

In partB, we now make the image stitching automatically. First step to acheive this goal is using Harris corners mathod. After run Harris edge detection, we get the points which are considered as corners (potentially good features) by the algorithm. Below is examples for Harris Edge detection points.

Adaptive Non-Maximal Supression

Now, with the points we got from the previous step with Harris Corners, we calculate ANMS(Adaptive Non-Maximal Supression). ANMS allows us to filter out some less important corners since many corner points are too close to each other and we can select the strongest corner in the radius. Thus, we sort the points by radius(from small to large) for each point and pick the strongest points. Below are the pointes after running ANMS.

Matching Features

Now we have significant features on each image. To stitch left and right image, we need to match the feature points. Here, we leave features which exits both on the left and right pictures. We extract axis-aligned 8x8 patches. We sample these from the larger 40x40 window to have a good descriptor. Don’t forget to bias/gain-normalize the descriptors Below are the examples.

RANSAC Features

For RANSAC, we pick random 4 points out of all good feature points and calcaulte H matrix with these 4 points. After this, with calculated value, we checked the original picture b and H*A = b'. We compare every single feature point's SSD to check how feature points are accurate, and count how many points have its SSD less than the threshold. We repeat picking 4 points and calculating above for 500 times and choose the H matrix where it has the most corresponding points under the threshold. Now, as we did in partA, we warp pictures using this H matrix.

Automatic vs Manual Image stitching

This are comparison picture from partA manual stitching and partB automatic stitching. Automatic stitched ones are slightly better, but not significantly better in my opinion - not because automatic is bad, but because manual ones are actually pretty good (except the Encore Hotel picture).

]

Summary & What I learned

Through this project, I learned how to warp images and make it into panoramic view iamge using the the homography matrix and blending. It was interesting to learn Homography matrix could be used to get the size of the warped image and each point(pixel) in the warped picture can be mapped back to the original image using inverse of H. For the automatic stitching part, when we run RANSAC, choosing 4 random points repleatedly and pick the best Homography matrix derived by those 4 points using SSD was brilliant! I thought it was cool because this reminded me of a lot of things I learned in other classes and I could realize how some linear algebra theory can be applied in the real life example. I enjoyed this project a lot!