Image Warping and Mosaicing

Approach

The following equation represents the project mapping between two points, that share the same center of projection. We use the homography matrix, H, to warp points, p, to the transformed set of points, p'.

How do recover the homography matrix, given two images A and B? If we look closer at the homography matrix, we notice that the value of i is actually 1. That leaves us with 8 unknowns for the other values in the matrix. Therefore, we can set up a different system of 8 equations to solve for those variables, which requires 4 (x, y) points. We can solve for the homography matrix as follows:

Note: we can actually utilize the data from more than 4 points. This results in additional equations, which causes the matrix to be over-determined, but that's okay. We can use the least squares solver to estimate the best values for the 8 unknowns, regardless of how many equations we have.

Results

I was able to successfully generate the homography matrix between two images. It's hard to produce tangible results of this part in isolation. However, the impact of this step being implemented successfully can definitely be seen in the parts to come.

Approach

Image rectification involves a kind of image transformation. Popular photo editing mobile apps have actually recently integrated image rectification as one of their editing tools (e.g. vsco, Instagram). It takes a photo, taken at a certain perspective or angle, and warps the image so that it appears to have been taken from a different perspective.

In order to rectify an image, we use the homography matrix that we compute for the two images. Then we inverse warp image A to image B using the inverse of the homography matrix.

I found it most useful to transform images that had a clear rectangular subject. Image A would be this rectangular subject in a different perspective, and image B would be the result of rectifying the subject to the perspective we would see if we were viewing it straight on.

Results

Data Rules Everything Around Me

This first image was a sign I saw in SF. I thought it was hilarious and my friend took it (at an angle) from the passenger seat of a car. It's a great example of something we can transform with image rectification in order to see what the sign would look like face one (a perspective we probably rarely get for large billboards in the city).

Color Factory

This last image recitification I performed was a bit of a challenge for myself. I decided to try to transform something without a clear rectangular subject, meaning I did not have a good guide to picking points. This is a photo of some text on a wall, that I took at an angle. I wanted to try to transform the image so we would be reading the text straight on.

Approach

We can use the image rectification results to create mosaics of our images. There are a few steps involved for creating a mosaic between to images A and B:

1. Pick one of the two photos to be your "base" image. In this case, let's choose image A
2. Warp image B to image A, using the image rectification method developed in part 2
3. Blend image A with the rectified version of image B

The process for creating the mask for the blending step was quite difficult. It involved a lot of tedious work in figuring out where the boundaries of the overlap were. We want those edges to be blended evenly. After determining the x positions of the boundaries, I created a mask and utilized alpha blending.

If we want to stitch together more than two images, that is also easily doable. The resulting mosaic will just become the "base" image for the next mosaic process.

Results

Eshelman

This was the first mosaic construction I attempted. I chose this one because there was a clear rectangular subject that I could use as a warping reference between the two images. Although those two are pretty well aligned, I think that the resulting mosaic still has some ghosting/misalignment because I wasn't able to match the points exactly. Also, I was not able to acquire a tripod, so I think that there's definitely a bit of shifting going on.

Doe