Overview

In this project, we capture images that only differ by the rotation angle the image was taken from. We then compute homographies between the images to "stitch" them together. The images used are taken from outside my house in Southern California. additionally, we manually extract 31 correspodning features from each image. These features will be used to compute a homography.

Homography

We now need to compute a transformation between the images. Specifically, there is translation and perspective, thus we require a homography.

As setup, we set $$\begin{bmatrix}a & b & c \\ d & e & f \\ g & h & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ y_1 \\ 1 \end{bmatrix} = \begin{bmatrix} wx_2 \\ w y_2 \\ w \end{bmatrix}$$ From this, we can solve for each row $$wx_2 = ax_1 + by_1 + c$$ $$wy_2 = dx_1 + ey_1 + f$$ $$w = gx_1 + h y_1 + 1$$ Expanding \(w\) in the first two equations and then subtracing all but the \(1\) to the other side, we get $$x_2 = ax_1 + by_1 + c - gx_1x_2 - h y_1x_2$$ $$y_2 = dx_1 + ey_1 + f - gx_1y_2 - h y_1 y_2$$ which is linear in our homography parameters. We can then use least squares to solve for the homogrpahy parameters.

Warping the Image

With the homography computed, we have everything we need to warp the image. We warp the right image to align with the left image. This gives us the following result

Image Rectification

To double check our code, we now test rectification. That is, we take a single image and transform it to changed the persepctive of it. We use a random page from one of my notebooks for this. The features are 4 points chosen from the image, and the transform maps them to a hand picked square.

Blending

Before trying anything complicated to stich the images, I first simply overlap them. The warped image is placed on top of the original image. While initially seeming fine, there is an edge artifact on the boundary of the warped image.

Now we use a Laplacian Stack instead. The first image is a the left image overlayed ontop of the warped image. Then the second image is warped right image overlayed on the left image. The mask is initially set to be one in the warped image, and 0 elsewhere. Then teh boundary in the mask is set to linearly fade off. This gives a much more seamless mosaic, but leaves artifacts where the images disagree, such as on the white car.

What I Learned

It took me a long time to figure out how to transform the homogrpahy map to form a least squares linear in the parameters of the homography, but when I figured it out, I felt enlightened. Additionally, it was cool to see a bunch of concepts such as transforms and Laplacian Stacks being used again, but in a completely new setting.