Overview
In this part of the project, we stitch images shoot at almost same place but with different consecutive angle into a big large picture. The idea is very simple: we project all the photos linearly into a common plane, which we compute by defining common pairs of points shared by every pair of images. Then we simply blend them together to get the big photo.
Shoot the Pictures
Thanks to Japheth Wong, I used his photos for our demo of mosaicing technique.
Scene 1 | Scene 2 | Scene 3 |
---|---|---|
Recover Homographies
We shall first recover homographies by selecting several points in the photo and assign the place that they should end up with in the “common plane”. Note by doing so we got several pairs of corresponding points. We here only consider the situdation of such transform of points is projective transformation. Mathematically we know all projective transformation can be defined by a 3x3 matirx. So we propsoe this problem as a least square problem.
We need to minimize sum(\(||Hx - x'||_2^2\)). By modifying this optimization problem a little bit, we can compute the transformation matrix and solve it using any optimization package:
# Matrix for transforming im1 into im2
def computeH(im1_pts,im2_pts):
a = np.array(im1_pts).T
b = np.array(im2_pts).T
A = None
B = None
for i in range(a.shape[0]):
p1 = a[i, :]
p2 = b[i, :]
new_stack = np.zeros((2, 8))
new_stack[0, 0:3] = np.append(p1, 1.0)
new_stack[0, 6:8] = np.array([-p1[0]*p2[0], -p1[1]*p2[0]])
new_stack[1, 3:6] = np.append(p1, 1.0)
new_stack[1, 6:8] = np.array([-p1[0]*p2[1], -p1[1]*p2[1]])
if A is None:
A = new_stack
B = p2
else:
A = np.vstack((A, new_stack))
B = np.concatenate((B, p2))
sol = np.linalg.lstsq(A, B)[0]
return np.append(sol, 1.0).reshape((3,3))
Warp the Images
Now we got the way to computing the transformation matrix, we can define a set of correspoding images between every pair of overlapped images. We first define their correspondence points as such:
Scene 1 | Scene 2 |
---|---|
Scene 2 | Scene 3 |
---|---|
Now we use the middle picture as the common place, then warp the photos on two sides into this plane:
Scene 1 | Scene 2 | Scene 3 |
---|---|---|
Image Rectification
Note once we are able to compute the projective transormation matrix, we are also able to rectify any photo by defining the any four points as the orthogonal plane (orthogonal to our sight).
Scene | Rectified scene |
---|---|
Scene | Rectified scene |
---|---|
Scene | Rectified scene |
---|---|
Blend the images into a mosaic
In the section of image warping, we have already gotten the ability to project each photo onto the right plane, what we need to do to stitch images is only posistion them correctly:
Scene 1 | Scene 2 | Scene 3 |
---|---|---|
Here are more examples from (Hugin)[http://hugin.sourceforge.net/tutorials/two-photos/en.shtml]
Scene 1 | Scene 2 |
---|---|
We project scene 2 onto scene 1 plane
And we can also project scene 1 onto scene 2 plane