Our goal is to create a photo mosaic by taking multiple pictures that have some overlap, warping one image towards the other, and blending them together to form a mosaic.
For each image, I chose the corner points of the sign, determined correspondence points of my desired output image (which was a square), and warped the image towards those correspondence points. This was done by determining the H matrix and applying an inverse warp to each pixel in the output image, using interpolation to determine pixel values from the original image.
Original Image
Rectified Image
Original Image
Rectified Image
For each image, I first chose 8 points from the left image and 8 points from the right image to use in determining the H matrix. Once I had done so, I warped the left image towards the right. I determined the shape of the warped image by seeing how the H matrix transformed the corners of my original image, and went through each pixel coordinate in the output image. For each of these coordinates, I applied an inverse warp and used interpolation to find the pixel values from the original image. One thing to note are the parts of the warped image where the inverse warp resulted in a pixel coordinate out of bounds of the original image. You can see that interpolation mapped this coordinate to the nearest coordinate of the original image, giving us streaking lines.
Left Image
Left Warped towards Right
Right Image
Left Image
Left Warped towards Right
Right Image
Left Image
Left Warped towards Right
Right Image
Now that I had a warped left image, I needed to first align it with the right image for it to be blended. To align the 2 images, I chose the same point in each image (for example, the same corner of the same building) and found the column and row offset from this point in each image. This gave me thre desired effect of aligning the 2 images on top of each other, albeit with a line where the 2 images met. Once the 2 images were aligned, I used alpha blending (with alpha = the distance transform of each image) to blend the 2 images together.
Warped Left Image
Right Image
Blended Image
Warped Left Image
Right Image
Blended Image
Warped Left Image
Right Image
Blended Image
Changing the POV of an image through warping is pretty cool, and you can change the shape of objects in images. One thing I had to account for was that either of the 2 shapes could be irregular - we are able to take a sign that, given the perspective, is a trapezoid, and warp it into a square. Likewise, our transformation could turn a square image into an irregular one, which required some extra processing to determine the boundaries of the output shape.