Projective transformation models the image capturing system and lights as light rays converging to a centero of projection, i.e. our camera. The image is obtained by placing a projection plane in front of our camera, i.e the focal plane of our camera. However, there is nothing stopping us from changing the focal plane and re-project the light rays on a different plane, thereby achieving the effect of image rectification and photo mosaicing.
For the same lighting condition, projecting the light rays on different projection planes produces images from different perspective. These images are related through the relationship of projective transformation. The projective transformation is captured by a 3 by 3 isomorhpic matrix called homography. Hence, to re-project an image, the first task is to recover the homography. To recover the homography, we have to find at least 4 corresponding points in the source image (the image to re-project) and target image (the image whose projection plane we will project on). Four pairs of corresponding points give us 8 sample points, which allows us to recover the homography which has a degree-of-freedom of 8. Once we obtain the corresponding points, we use the least square method described here to recover the homography
One application of homography is image rectification. The idea is that by the semantic content of the image, we know that something should have right angle, e.g tiles, frames, bricks, walls, which are projected into different angles due to projection. To rectify such content, we first obtain the homography. The source sample points are the corners of the right-angled content, the target sample points can be just the four corners of the result image (as such, the right angles are recovered in re-projection). Here is one example.
We can also apply the concept of homography to obtain panoramic image. If we take a few images that have the same center of projection, e.g. images taken from a tripod and only rotating the camera, we can find the homography between each pair of images. Then, we can use one image as the target plane of project and re-project the other image onto this reference image. Once re-projected, we can use simple blending (e.g. alpha blending) methods to put together these images.
First, we calculated the result image dimension and warp the two images onto the result image correspondingly
Notice that we only translate the center image to the final position of itself in the result image since we are re-projecting all other images on the the projection plane of the center image.
Once we have the warped images, we just blend them together. A little more on the details of blending: to avoid obvious ghosting, I chose to blending only the portions in either images that containt the feature points. I picked the leftmost feature points and rightmost feature points to form central band of blending areas. Then I used alpha blending in this band and use the original pixel values from each image outside of this band.
Finally, the stitched image is clipped to remove all the blank areas leftover by projection. And we repeat the above process with the stitched two images and one new image on the right, we can obtain a mosaic.
source:
A few more examples:
The previous application assumes a planar projection plane. However, if we assumes a cylindrical projection plane, our transformation matrix is reduced to mere translation from projection. This is because when we rotate the camera, we are essentially projecting onto a rotated projection plane. By putting these projection planes together, we can approximate a cylinder. By translating based on feature points, we can produce rather amazing results. However, due the the lack of consideration for projection, we can observe that straight lines in this image is no longer straight.
Cylindrical projection works really well if the object is really far away, since if the object is far, there is less arc in its light right projection plane. Hence a mere translation of images could give really good results. It's even easier than homography since less feature points are required.
On the other hand, when there are many lines in the scene and the objects are closer, the lack of consideration for projection can have more obvious artifacts.
I really enjoyed combining artistic creation with the computer science knowledge we learned in class. I also found applying linear algebra in homography very powerful and amazing.