Project 6 part A, Mosaics

Rectified Images

---

Methodology

---

Below are pictures of three Mosiacs

1. East Asian Library, Berkeley
2. McCone Hall, Berkeley
3. Pictures in my church building, Alameda

Corresponding Points

I first plotted corresponding points between images I wanted to blend into mosiacs. For each mosiac I blended two images.

Homography

Next, with the images between each point, I computed a 3x3 homography between them. With this matrix, I can take one of my corresponding points in one image and then apply it to another

Warping

I applied the homography to one of the images using scikit's warp function.

Blending

I tried three blends

1. adding two images together and having one overlap another (overlap method)
2. Adding half of each image in an overlap area between the two images, then adding the remaining non-intersecting parts (Linear Blending
3. Masking from the middle of the intersection section to the end of one image to apply Project 3's laplacian stack blend. (Laplacian Blending)

East Asian Library

---

McCone Hall

---

Church Pictures

---

Part B, find matching corners automatically

---

Below are the results of auto-stiching, where we find the points of interest automatically, and use those points to stitch images together. The process is divided into two parts

Find the corners: Feature detection

I used a function called get_harris_corners, given by the staff, to find all the x,y coords that might be a corner, according to the definition of a harris corner. A harris corner is a patch where if we move it, the image vector changes dramatically. In the middle of the corner is the x,y coord. I used Adaptive Non-Maximal Suppression described in the Brown paper (given in the project) to filter out a lot of the harris corner points (so we do not have to process too many points). ANMS takes every corner, finds its Fhm value (the value on the matrix given in get_harris_corners, the coord's corner strength), and assigns each corner a value r that is the shortest distance between it and another corner that is a constant stronger than it.

Match corners of two images together: Feature Matching

We do this by

1. 'Russian Granny' Algorithm

   I take one patch p1 in one image, find the closest patch p2 in the other image based on lowest ssd. Record error as 1-NN. Then find second closest patch, call difference 2-NN. if 1-NN / 2-NN is below a certain threshold I set, then I add p1 and p2 as a pair.

2. RANSAC

   1. Take any four pairs of points (p1, p2) randomly
   2. make a homography out of them, H
   3. find inliers using this homography, where SSD of p2 and Hp1 is less than a threshold epsilon
   4. Keep number of inliers, repeat
   Repeat this loop for a certain amount of time. We can do all unique combinations of points. We do this loop and choose the four points that give us the most inliers. Then we refind the inliers, make another homography H_new, out of all the inliers.