Project 6: Panoramas (part 1)
Joyce Luong (acs)

To create image mosaics, we need to do this:

  1. Shoot and digitize pictures to put into a panorama.
  2. Recover the homography (transformation) H from all images to the center image with linear algebra.
  3. Use the homography H to warp non-center images.
  4. Blend images into a mosaic.
  5. ???
  6. Profit.

Part 1: Shooting Pictures It is important to take pictures such that there is a decent amount (40% to 70%) of overlap.

Part 2: Recover Homographies Given the following format for homographies:

[w*x']   [a b c]   [x]
[w*y'] = [d e f] * [y]
[ w  ]   [g h 1]   [1]
With a little manipulation, we can get these equations:
x' = a*x + b*y + c - g*x*x' - h*y*x'
y' = d*x + e*y + f - g*x*y' - h*y*y'
And thus have the resulting Ah = b matrix:

We solve for h in Ah = b with least squares, tack on a 1 at the end, and reshape into a 3 x 3 matrix. This is our homography matrix, H.

After defining the correspondences by hand, we wrote a function to fine-tune them automatically with SSD over various displacements in a 10 x 10 box.

Part 3: Image Warping and Rectification To warp images, we found the dot product between the homography H and the point we wanted to warp away from. Then we solved for w and divided the resulting x and y element-wise by w to account for it. This is an example of an inverse warp.

At this time, we were able to rectify images--taking a section of an image, declaring it should be a certain shape (say, a perfect rectangle, as in the window and the paper below), and warping the image to that shape. As a result, the plane is "frontal-parallel".

Part 4: Mosaics Quite similar to above; this time, we took the images we shot from Part 1 that weren't in the center and warped them to the shape of the middle image. To not accidentally lose data (since scikit-image is fond of cropping off edges) we pad around the middle image with some 500 pixels each time.

Finally, we used alpha-blending with a mask to seamlessly blend images together into a mosaic.

Tell us what you've learned! Creating rectified view of single images is surprisingly simple, and is pretty cool!