Project 6A: Image Warping and Mosaicing

Vivien Nguyen || cs194-26-afd


Part 1 & 2: Shooting Photos and Recovering Homographies

To begin creating our image warps and mosaics, we need to acquire some images! I took some pictures around Soda Hall, as well as outside of Algorithm Coffee on University Ave. For the pictures inside VCL and looking out at Jacobs, I forgot to heed the warning about locking AF/AE on iOS -- hence, you can see the lighting changes between the left image and the right image when they become mosaicked together.

However, I did make sure to not translate the camera while taking the photos -- since my scenes were not necessarily planar nor at distance infinity, a translation would have revealed too much depth in the scene, thus ruining the mosaic.

To recover homographies, we follow a similar method to in Project 4. Given an Image A, Pts A, Image B, and Pts B, we would like to compute a warp that allows us to (in my case) warp Image B "into" Image A. To do this, we can solve linear least squares between points A and B. I used the equations shown below:

Part 3: Warping the Images

Once we have the homography, we can it to warp the images in various ways. Here, we demonstrate examples of "rectifying" our images. Rather than taking correspondance points from another photo, we take something that we know should be planar in our scene -- the wall, floor, etc -- but that has been tilted due to the angle the photo was taken at. Then, we can rectify it such that we are looking at the indicated plane head one. If we chose the floor, for example, we would then warp our image into one where we are looking directly down at the floor, rather than at some grazing angle.

I picked correspondance points of what should be a square/rectangle on the following original photos, then hand-defined rectangular coordinates to warp into.

Original photo of VCL
Carpet of VCL rectified
Original photo of Jacobs Hall
Facade of Jacobs Hall rectified

Part 4: Image Mosaics

Now that we know we can successful recover homographies and warp images, we can start putting together some mosaics! For the following three results, I did not implement any kind of blending; aside from the edge seen due to differences in lighting, I did not feel like I had any significant artifacts (though they are there if you look closely!). (Also, I ran out of time for this project. However, I will complete some kind of feathering or blending for the second part!)







Overall

I had a lot of difficulty managing the invese warp this time. Although ultimately, it wasn't very difficult, I think I got too caught up in the details about how to initially warp the corners of the image to create a "canvas", and then inverse warp points from this canvas into the original image, to fill in the colors.

Considering I did not do any intellient blending, or even interpolating colors, I feel like the results are already quite interesting and compelling! I look forward to continuing this in the second part of the project.