CS194-26: CS194-26: Image Manipulation and Computational Photography

Project 5 – [Auto]Stitching Photo Mosaics

St. ID: 3034311700

Part A - Image warping and mosaicing

Shoot the Pictures

Due to the quarantine photo shooting was limited to living room and balcony of my apartment.

Recover Homographies

Homography is a transformation P=Hp, where H is a 3x3 matrix with 8 degrees of freedom (lower right corner is a scaling factor and is set to 1).

The homography transformation matrix was computed with the four correspondence points (p1,P1),(p2,P2),(p3,P3),(p4,P4) written as 2×9 matrices concatenated together such as:

Warp the Images and Rectification

Having established the homography matrix, the transformation matrix is applied to every pixel of the first image. To make sure the wrapping works correctly, image rectification was performed. I applied warping to a square table top view and used as points for the second image the corners of a square of suitable size defined by hand.

Original Image:                                                                                     Warped Image:

Blend the images into a mosaic

Image 1:                                                        Image 2:                                                        Blended Image:

Image 1:                                                        Image 2:                                                        Blended Image:

Part B - Feature Matching for Autostitching

Two images close to each other were selected in an attempt to help the algorithm more easily match the corners:

Image 1:                                                                            Image 2:

Detecting corner features in an image

The Harris corners framework was used for feature detection, identifying the changes in intensity by shifting a window in all directions. The amount of change will be high for a corner.

Image 1:                                                                            Image 2:

Adaptive Non-Maximal Suppression

To reduce the number of candidate points selected in an image by the Harris corners framework, we implement Adaptive Non-Maximal Suppression, which allows keeping the Harris points with the highest scores but who are concurrently well separated. To that end, for each candidate point, a suppression radius is defined:

Image 1:                                                                  Image 2:

Feature Descriptor

We will define a feature description for each point and then use the L2 distance between their feature descriptors and the nearest neighbors will be considered matched points. A 40x40 patch around the point is selected, blurred by resizing this patch to 8x8 and normalized.

Feature Matching

To identify features that look similar and thus likely to be good matches, we compute the sum of squared differences between the descriptors, while using the approach due to Lowe of thresholding on the ratio between the first and the second nearest neighbors.

Unfortunately, my feature matching didn’t work as you can see from the selected points:

Image 1:                                                        Image 2:

RANSAC for Homography calculation

The procedure above is expected to return false matches that work as outliers in homography calculation. To discard those, RANSAC is used. Of course, given my feature matching returned wrong results my homography turned out wrong as well (and due to that I didn’t proceed to compute more homographies through RANSAC).