In this project, we create panoramics by finding transforms between multiple images and blending them together.
I took pictures of three different areas: my kitchen, towards the bay from the roof of my building, and in the Chemistry plaza. It was important to have a significant amount of overlap between the pictures so that matching points could be found for computing the transforms.
|
|
|
|
|
|
Next, I computed the transform between the images. This was done by assuming that the lower right corner of the transform matrix was 1, and then using the fact that p' = Hp, where p' = [wx', wy', w], H = [[a, b, c], [d, e, f], [g, h, 1], and p = [x, y, 1].
This gives us the equations ax + by + c - gxx' - hyx' = x' and dx + ey + f - gxy' - hyy' = y'.
So, I selected 10 corresponding points in each image (4 are necessary for there to be exactly one solution, since we have 8 unkowns), applied those equations, and used least squares to solve for H.
Next, to warp the images, I used inverse warping by using cv2.remap to map the coordinates of the second image to the corresponding locations in the original image and take those pixel values.
To make sure that the warping was working correctly, I tried rectifying a straight wall so that it was close to frontal parallel in the image.
|
 |
Then I warped the rest of my images as well. These are all the second image being warped to match the perspective of the first image.
|
|
 |
|
|
 |
So, I warped all of the second images to match the first one's perspective, and then I overlaid them on each other. I do not show any examples here, but for more than 2 images, I first create a mosaic of 2 images, then ask the user for more correspondence points between the newly created mosaic and a third image, and so on. To blend them together, I found the coordinates of pixels for which both images were not 0, and added 0.5 of each image in that region. For the rest of the images, I simply added the images together, since one of the images was 0, so its contribution would not increase the brightness of the result. The lines between the images in the roof and Chemistry plaza mozaics are pretty visible; I think that is because the brightnesses differ between the images; I should have made sure that my camera used the same brightness setting for the different images.
|
|
 |
|
|
 |
|
|
 |
The most interesting thing that I learned was how relatively simple it was to warp the images into each others' perspectives. It is just a simple system of equations to solve for the transformation, and then that transformation can be applied to every point in an image. I would have expected there to be multiple different transformations, or some nonlinear transformation that would be required.