Project 6: Image Warping and Mosaicing

The goal of this project is to warp images to create image mosaics.

Procedure

The first thing I did was take photos on my Android and defined at least 4 correspondences between the images with ginpoint in python.

Then the next step would be to recover the 3x3 homography transformation matrix H. By setting the scaling factor i to 1, then we have 8 unknown variables to solve. As there are two equations per point, we need a minimum of 4 pairs of corresponding points to either directly solve for H or estimate H using least squares.

Using the homography matrix, I did an inverse warp to create the transformed image.

Results

Procedure

Finally for image mosaics, we find the homography matrix from corresponding points of the two images, and then warp one of the images to the perspective of the other and then overlap them.

Results

We use the Harris Corner Detector algorithm to generate potential points of interest in the images by finding corners, or points whose local neighborhoods stand in two different edge directions. Because the algroithm returns too many values, we use Adaptive Non-Maximal Supression to keep the key points for the image.

We now use our interest points as feature descriptors. These feature descriptors will be 8x8 patches that are downsampled from 40x40 windows centered on the interest points to eliminate high frequency signals. We additionally subtract out the mean and normalize to unit variance.

Next we find feature matches between the two images. First we compute the squared distance from every feature in the first image to every other feature in the second image (essentially a nearest-neighbor search). We use the Lowe ratio test to ensure that the best match should be much better than the next best match when it comes to matching interest points.

Next we use RANSAC to compute a homography that only uses accurate pairs. In the algorithm, we randomly choose a set of 4 pairs to compute a homography, and then apply that homography to the matched points in the first image and compute the SSD error between them and their corresponding match points in the second image. By repeating this with many randomly generated sets of pairs and keeping track of the largest subset of inliers (points whose error is less than some threshold), we can then compute the final homography using that largest subset.

After getting a good homography, we can now proceed to stich the images together as we did in Part 6a.