by: Gavin Fure
We'll be using projective warping to create image mosaics. This is pretty similar to what we've done in previous projects, but here we get to use 8 degrees of freedom in our warps!
I took a few pictures of the campenile:
Upper Image |
Upper Image with points |
---|---|
Lower Image |
Lower Image with points |
In this section we will write a function to compute a homography, a transformation matrix from one set of correspondance points to another. This homography matrix will have 8 unknowns, allowing us to perform perspective warps. The original problem formulation looks like this:
p' = H*p
Where p is the first set of correspondances (an nx2 matrix) and p' is the second (also nx2). H is a 3x3 matrix. We can reformulate this as a system of equations, specifying for each point:
For each point, we can substiture for wi to remove it:
We can formulate this system of equations as a least-squares problem:
As long as we have more than 4 points, we can solve this overdetermined least squares problem!
Now that we have a function that can produce a homography matrix, we can use it to transform some images. Here is the result of warping the 'upper' campenile picture to the 'lower' one's points:
Warped Upper Image |
Warped Upper Image with pts |
---|---|
Lower Image |
Lower Image with points |
Now that our warp is working, we can warp some flat surfaces to directly face us! This is one my favorite features in phone scanner apps (like camscanner). It's really helpful, and can produce some really cool effects when applied in the right way. Here are my results:
Original | Rectified |
---|---|
I used the corners of each object as correspondance points, and warped them to the corners of the image. This method was pretty quick and dirty, and doesn't preserve the original aspect ratios. That's why Link looks squished in that second warp. However, I think that this is proof our warp is working as intended!
At this point, we have everything we need to make some panoramams! We can calculate the homography matrix between the points on two images, then warp them onto the same canvase. Of course, we'll have to create a canvas that can fit all the pictures. To do this, we can send the corners of each image through its homography matrix, then use the max x and y values to create a blank canvas using np.zeros. If we have negative minimum values (which we often do!), then we can subtract the max values by the negative min values. Then, when we are warping, we can translate the results by the negative min values. This allows us to line everything up. For blending, I used a basic average mask. Each pixel value will be divided by the number of images that use that pixel. For example, if three images all include some arbitrary pixel, then the mosaic will add all of their corresponding pixel values to the output canvas. Since there are 3 of them, we divide the resulting value by 3. This is done by creating a mask (adding 1 to an empty array in the location of used pixels for each image). Then, we divide the raw mosaic by this mask.
Here are some of my results!
Upper Campenile |
Lower Campenile |
---|---|
Campenile Blending Mask |
Campenile Mosaic |
This one turned out pretty good! The trees on the right are unfortunately a bit blurry, but it was windy that day so they probably moved around between pictures. There is a fairyly visible seam in the sky, because of the way I blended images. There wasn't nearly enough time for me to create a better blending option, but I think it turned out okay anyways. Let's look at another mosaic.
Left Gate |
Right Gate |
---|---|
Left Gate Points |
Right Gate Points |
Gate Mosaic
The trees on this one are very blurry, probably also because of wind. Also a strange eyeball monster was walking through the gate while I was taking pictures, so they ended up moving. It's kind of a fun effect, they look like a ghost. The gate itself isn't as clear as it could be. This is probably due to point selection. I was only able to just click on them with my mouse (not do any sort of cross-correlation procedure around clicked points, which would definitely improve the results). This one is less successful than the others due to those factors. Let's look at one final mosaic:
Left Flag |
Left Flag |
Right Flag |
---|---|---|
Left Flag Points |
Center Flag Points |
Right Flag Points |
Flag Mosaic
This one turned out pretty well! It has a decent amount of edge artifacts (again due to the simplistic blending technique I am using), but besides that it looks pretty good, and it uses 3 images!!
We wrote functions to compute homographies, perspective-warp images according to those homographies, and compose warped images together to create a mosaic/panorama type image.