Shoot the Pictures

I took the pictures with my iPhone using the AE/AF feature to lock the exposure and focus. I shot a total of three scenes around campus:

Recover Homographies

We want to recover the homography matrix H such that p'=Hp, where p is the point in the source image and p' is the warped location of p. We know that the equation satisfies

So H has 8 unknowns. We can solve for the values of H by providing at least 4 correspondences and using linear least squares. By expanding out the matrix product and substituding w in the first two lines, and rearranging, we get the following form of a LLS problem Ax=b for n>=4:

I used matplotlib's ginput to define the following correspondences for each scene:

Warp the Images

For warping the image, I use the inverse mapping method. First, I calculate H_inv. Then I define the target image as an all black image, and for every single point on the target, transform it using H_inv and bilinear sample from the source image. For the homography transformation, after multiplying the homogeneous coordinate by H_inv, I have to divide every element by the last element to get the actual coordinate. In the code, I used OpenCV's remap function to do this sampling.

Image Rectification

To test the warp, I took two images of an art gallery, and rectified one painting in each, by selecting the 4 corners of the painting and projecting it onto simply a square in the middle of an 800 by 800 image. We see that my homography and warping are both correct.

Blend the images into a mosaic

For creating the mosaic, I selected the middle image as the base, and warped every other image to that base image. Here is the step by step procedure for Scene 1. Take the second image as the base. First I padded the base image to the final size of the mosaic:

Then calculate the homography from image 1 to image 2 (offsetting the points on the base image by the padding), and warped image 1

Then, I first naively blended both image with a weighted average using the following mask which has 0.5 at overlapping regions:

To get:

However, we see that the overlapping region has this "ghosting" effect, and we can see the seams very clearly. Instead, use a mask that smoothly blends between one image to the other in the overlapping region, which can be calculated as the ratio of the distance (Manhatten distance is faster to calculate than Euclidean) to image 1 and image 2. As follows:

This will give us a much better looking blending without any visible seams (on the right):

Using the same smooth blending procedure, the following are the results for Scene 2 and Scene 3.

Tell us what you've learned

I loved looking at the final images, they look so nice. I also learned just how sensitive homographies are to the point selection, as a small change in points greatly affects the warping.