In this project, I manage to create a panoramic photo with image warping and mosaicing.
The first step is to shoot the pictures in the way where transformation between the pictures is projective and there is overlap to find enough correspondances.
I try to get the photos by standing still and simply rotating to take multiple pictures at first, but it does not work pretty well.
So I come up with the idea to build a scene in Unreal Engine 4 with some official assets and set up a fixed camera to take pictures in different rotation angles, and it seems to work even when I only have the overlap about half of the images
The next step is to recover the homography between two images. We need to find the homography matrix H that calculates the point(x',y') in the target image that corresponds to the point(x,y) in the source image as the following
We need at least 4 points to recover H since it is a 3x3 matrix with 8 degrees of freedom (lower right corner is for scaling, and we set it to 1)
However, to blend more precisely, I set more than 4 points(generally 8 points) for each image and use least-squares to get matrix H as follow
I also try to rectify some images to front viewpoint in order to ensure that I have the correct function to find optimal transformation matrix
Here is a sample:
Now I can blend together multiple images into a mosaic. To simplify, I use 2 images and warp left image and right image together.
Blending is done with alpha blending and a cos function evaluated from 0 to pi/2 over half the image
The result is as below:
The result of the mosaic images taken by my own camera is acceptable but not that perfect
Although it may look wired(my images are too long and the overlap is not that much), the final mosaic result for the UE4 ones is still satisfying