Project 5 - [Auto]Stitching Photo Mosaics

5A Part 1 - Image Warping and Mosaics

By Bernard Zhao for CS194-26 Spring 2020

All code can be found in P5.ipynb

Shooting the Pictures

These photos were shot from the same spot, just a short walk from my house. I shot enough for a whole 180 degree view, but I'll use all of those in part 2.

im2 im1

Recovering Homographies

I handpicked 6 correspondences in Photoshop, and then wrote them down in the notebook. I then used the below formulation and solved using least squares to obtain the homography matrix (after reshaping and adding 1).

[x1,1y1,11000x1,1x2,1y1,1x2,1000x1,1y1,11x1,1y2,1y1,1y2,1x1,ny1,n1000x1,nx2,ny1,nx2,n000x1,ny1,n1x1,ny2,ny1,ny2,n][abcdefgh]=[x2,1y2,1x2,ny2,n]\begin{bmatrix} x_{1,1} & y_{1,1} & 1 & 0 & 0 & 0 & -x_{1,1} x_{2,1} & -y_{1,1} x_{2,1} \\ 0 & 0 & 0 & x_{1,1} & y_{1,1} & 1 & -x_{1,1} y_{2,1} & -y_{1,1} y_{2,1} \\ & & & & \vdots & & & \\ x_{1,n} & y_{1,n} & 1 & 0 & 0 & 0 & -x_{1,n} x_{2,n} & -y_{1,n} x_{2,n} \\ 0 & 0 & 0 & x_{1,n} & y_{1,n} & 1 & -x_{1,n} y_{2,n} & -y_{1,n} y_{2,n} \\ \end{bmatrix} \begin{bmatrix} a \\ b \\ c \\ d \\ e \\ f \\ g \\ h \end{bmatrix} = \begin{bmatrix} x_{2,1} \\ y_{2,1} \\ \vdots \\ x_{2,n} \\ y_{2,n} \end{bmatrix}

Check out computeH to see the implementation.

Warping the Images

I used an inverse warp to project im1 onto im2:

im1_warped

Check out warpImage to see the implementation.

Image Rectification

The warping works perfectly for rectification as well. Check out my quarantine setup where I was doing this project:

before after

Now you can (sorta) see my screen from my original perspective!

Mosaic Blending

Now using an alpha channel, we can put im2 and im1_warped together:

combined

In the next part I will introduce some smarter blending.

What did I learn?

This was a surprisingly tough project because of how tedious offset calculation became. Hopefully this makes the second part easier. I've gotten very familiar with numpy array indexing, which is honestly some magic as well. It was fun to play with my camera manually tweaking the settings. It also was interesting to see the effects of my lens hood in the images, as the imperfect blending makes the difference in lens flare very obvious.