CS 194-26 Fall 2021 - Project 4A

Image Warping and Mosaicing

George Gikas


Shoot the pictures

For the mosaic images, I chose to shoot pictures of a painting in my hall stairwell, the lounge in the hall, and the street and playing fields outside below. Below are the sets of images used for each mosaic.

Middle image of stairwell
Right image of stairwell
Middle image of outside
Right image of outside
Left image of lounge
Middle image of lounge
Right image of lounge

Recover Homographies

In order to perform the warping of an image to another, we need to recover the homography matrix between two sets of correspondence points. The transformation matrix is the matrix H in the equation p'=Hp, where p' and p are the correspondence points between the two images. This equation can be rewritten as below, with i set to 1 in our case:

Following this, we can expand the equations out and rewrite them as the following below, which can be solved with least squares to retrieve a - h for the matrix H:

Using the homography, we can then warp an image to another by matrix multiplying the homography matrix with the image.


Image Rectification

For image rectification, I selected a box region within the original image, then select an arbitrary box that we want to warp said points to. Following this, I then computed the inverse of the homography and multiplied all of the points in the output image by it to receive their corresponding locations in the original image, which is then used to get the pixel values. For any of the points that mapped to locations outside of the boundaries of the original image, it is filled in with black instead:

Original picture of hallway
Hallway rectified based on side
Original picture of kitchen
Kitchen rectified based on floor

Blend the images into a mosaic

To blend a series of images into a mosaic, I computed the homography of all images in a set against a middle image. Below are the correspondence point sets for each image that I am trying to warp into a mosaic:

Middle image of stairwell
Right image of stairwell
Middle image of outside
Right image of outside
Left image of lounge
Middle image of lounge
Middle image of lounge
Right image of lounge

By then warping the corners of the image, I can determine the maximum and minimum x and y coordinates, which tell me the size of the overall mosaic that I am creating. I then warp each image into their final locations and blend each resulting image together into one, creating the mosaic:

Mosaic of stairwell
Mosaic of outside
Mosaic of lounge

What I learned

I found the image rectification portion to be incredible interesting. I found it incredible that using the information available within one perspective, it would be possible to compute a homography then warp said image to appear as if from a new perspective.