To create image mosaics, we need to do this:
H
from all images to the center image with linear algebra.H
to warp non-center images.
Part 1: Shooting Pictures It is important to take pictures such that there is a decent amount (40% to 70%) of overlap.
Part 2: Recover Homographies
Given the following format for homographies:
[w*x'] [a b c] [x]
[w*y'] = [d e f] * [y]
[ w ] [g h 1] [1]
With a little manipulation, we can get these equations: x' = a*x + b*y + c - g*x*x' - h*y*x'
y' = d*x + e*y + f - g*x*y' - h*y*y'
And thus have the resulting Ah = b
matrix:h
in Ah = b
with least squares, tack on a 1 at the end, and reshape into a 3 x 3 matrix. This is our homography matrix, H
.After defining the correspondences by hand, we wrote a function to fine-tune them automatically with SSD over various displacements in a 10 x 10 box.
Part 3: Image Warping and Rectification
To warp images, we found the dot product between the homography H
and the point we wanted to warp away from. Then we solved for w
and divided the resulting x
and y
element-wise by w
to account for it. This is an example of an inverse warp.
At this time, we were able to rectify images--taking a section of an image, declaring it should be a certain shape (say, a perfect rectangle, as in the window and the paper below), and warping the image to that shape. As a result, the plane is "frontal-parallel".
Part 4: Mosaics
Quite similar to above; this time, we took the images we shot from Part 1 that weren't in the center and warped them to the shape of the middle image. To not accidentally lose data (since scikit-image
is fond of cropping off edges) we pad around the middle image with some 500 pixels each time.
Finally, we used alpha-blending with a mask to seamlessly blend images together into a mosaic.
Tell us what you've learned! Creating rectified view of single images is surprisingly simple, and is pretty cool!