Panoramas - Part I

In this project, I explore using homographic transformations to stitch images taken with the same center of projection into a panorama. This is similar to simulating a single image capture with a larger field of view (FOV). A large part of the exploration included automatic feature detection and matching. Corners are used as features and they are matched across adjacent images.

Another neat trick of homographic transformations is the ability to "rectify" an image to make it appear as if it were being looked at from another angle. This is commonly used to take surfaces captured at an oblique angle and warp them to look as if captured orthogonally.

These are the original images taken from the same position, rotating the camera.

Next the side images are warped to match the perspective of the middle image.

Finally, the images are blended into one image.

One drawback to the flat projection that is used is that the closer the field of view approaches 180 degrees, the larger the size of the image, due to the perspective transformation. This makes taking more than 3 photos with a normal wide angle lens not as useful.

One interesting thing I tried was to take vertical photos to take advantage of the wide field of view to expand vertical view and using the panaramo stitching to still maintain a wide horizontal field of view.

Note how close to the house this picture was taken. A normal phone camera would not be able to capture this much of the house standing from the sidewalk beside it in one shot without panorama functionality. Even wide angle lenses will warp straight lines, which is not noticeable in this stitch.

Original images were taken at an oblique angle. A homograhphic transformation was applied, using the corners of a rectanlge as the target coordinates.

A Harris corner detector is used to find candidate features in two adjacent images. Then 8x8 patches are sampled from the 40x40 region around the corner. This is the feature descriptor that is used to compare corners from an image. In the images below, they are marked by small red crosses.

However, there are many corners in each image that don't have a corresponding corner in the adjacent image. Each corner feature descriptor's SSD distance to each other corner's descriptor is calculated. The corners whose first nearest neighbor was 0.5 or less than the next best match was kept as a good feature. After this candidate decimation, the remaining features are generally in regions where the two images overlap.

A final filtering of outliers is done using RANSAC, randomly sampling 4 features and calculating the homography based on those corners to the best matching feature's corner in the adjacent image. The homography is applied to the rest of the corners whether or not the corner is an inlier is based on a threshold of Euclidean distance between the transformed point and the actual point in the second image. The homography with the highest number of inliers at the end of a fixed number of iterations was considered the estimated homograhy.

One thing to note is that this method did not work well with the second set of images as the large number of redundant features due to the tiles confused the algorithm. So only two images were able to be stitched out of the 5. The results with the first set of images is the same as that shown above.