Credits to Mumu Lin for photos of San Francisco
In order to obtain obtain some interesting photos for this stitching project, my friend Mumu and I took a day in San Francisco with a Canon EOS Rebel T3i camera. To take the photos, we stood at a fixed point, capturing images while rotating. The rotation was not as precise as could have been achieved with a tripod and some exposure settings were not locked, so some of the pictures came out slightly differently. However, I believe that we have obtained enough results to do some really interesting stuff.
Here are some sample photos we obtained. Most of these are part of multiple sets of redundant photos:




For simplicity, the two photos I will start working with are taken from Berkeley BART. I already defined 18 points of correspondence for making a mosaic out of them, which I will use at the end of this segment.


The goal here (similar to project 4) involves finding a 3x3 transform matrix used to translate pixels across the canvas. Such involves solving a system of linear equations to find a set of variables [h11, h12, h13, h21, h22, h23, h31, h32]
, with h33
constrained to 1.0 for normalization. That means that a correspondence needs at least 4 (x, y) points.
Each source point (x, y) will be used to create a row in matrix A
along with the destination point...
[x1, y1, 1, 0, 0, 0, x1 * px1, y1 * px1]
[0, 0, 0, x1, y1, 1, x1 * py1, y1 * py1]
... with b
being [px1, py1, ...]
.
Since some systems might be overdetermined (i.e. more than 4 points) I use numpy's least square solver for this task.
Now that the homography matrix can be retrieved, we have to define a transform that applies to every pixel of the image. To apply a homography, dot product H * [x, y, 1]
to receive w * px, w * py, w
, the operation I will define as warp(x, y) > px, py
.
warp
to the corners of the image.Here is the result on a secret door from a shabu shabu restaurant in Anaheim, warped to a square.


If you know the proportions of objects, you can also rectify into other shapes. For example, this Vanguard card has ratio 4:3.


This part is much the same as the previous, except the points will be overdetermined this time and we'll need to deal with blending multiple images.
Per some suggestions from other classmates, for intersecting portions of images, I used the max
function to decide which pixel to use and it turned out pretty well.






You'll notice the mosaic isn't quite perfect for the candy shop, due to the level of detail here. Better precision would require more correspondence points. Additionally, the guy in the background moved slightly, causing him to ruin part of the illusion. As you probably guessed, these mosaics are merely an approximation of the same scene, as there is a variable of time involved.
Again, defining correspondences that precisely transform the picture is hard:



The BART one is probably the most convincing due to the simplicity of the geometry. In the next portion of this project, we'll see how to automatically (and hopefully better) define the correspondence points to make a mosaic.