Cheng Cao
This part of this project is about auto selecting harris features and performe feature matching in order to performe auto image stitching.
Features points of the images are first extracted from harris corner detection. This process will produce all the "corner features" in the image. Then we use adaptive non-maximal suppression to select a few "key" features from the set of corners we extracted earlier from harris corner deteciton.
We can compute the minimum radius for each feature corners, and then we take the top 250 points with largest radius as the features.
Here're some examples from three sets of data. The first set contains architectual building:
And the harris corners are labeled blue, while the adaptive non maximal suppresion gives a selected 250 features from all the corners. These features are labeled yellow:
Here's the second set of data.
Here's the third set of data:
Feature matching are achived first by extracting a 8x8 feature vector for each feature point. This 8x8 feature vector is sampled from a 40x40 region. Then all the features from one image is compared to the features in the target image. For sanity check and for more consistent code, the orignal image is also compared to itself. We can see that the feature extraction is repeatable and the original image always match.
The homography is computed through 4 point RANSAC. As the feature points are matched automatically it may contain noisy and faulty information. Instead of using least square, which will bias the result towards the noise, RANSAC is used. RANSAC is a process where some random data points are taken and all other points are tested to see how many of them matches the model. The model with highest amount of matching data points is determined as the best model.
After the homography is computed, the image is morphed using the same method as part 5a.
A function is constructed to warp the images. If first takes in the image and constructs an interpolation function out of the image so that the image will be continous and reduces aliasing. Then the bounding box of the images are warpped using the computed homography to get the bounding box of the warpped image in the target plane. Then a buffer of the size of the bounding box is constructed and for every pixel on this buffer the color value is evaluted through the interpolation function, with their corresponding coordinates computed through homography.
This can be summarized as:
buffer = [...]
interpolated = Bilinear Interpolation (image)
Foreach pixel P in buffer
- Coordinate in Image = Homography * Coordinate of Pixel
- P = interpolated(Coordinate in Image)
Instead of mapping image pixels to the target plane, we do it other way around so that we are sampling a color value for each pixel in target plane. By doing this we won't have any black pixels or gaps in the warpped image (a kind of undersampling artifact). This also has better performance as it does not over-evalute pixels in over-sampled areas.
As stated before all the images are warpped onto one image (the first image in the sequence). That means that image is served as the target image plane. Here's the result of warping the three images from set 1 to the first image's plane.
Then the images are blended together using the bounding box information we get eariler when warpping the image. The images are multiplied by weight and summed into the final image. Then the final image are divdided by the summed weight. Without weighted averaging, the ghosting are very apparent and the edges of the image is pretty jarring.
Set 1, automatically selected control points:
Manually selected control points:
We can see manually selected version matches better on some part of the image as the images are not taken from the exact same spot. Human is able to select better fine tuned control points while the automatic version is trying to match everything.
Set 2, automatically selected control points:
Manually selected control points:
Set 3, automatically selected control points:
