CS 194-26: Project 4

Image Warping and Mosaicing

Emily Ma

Summary, What I Learned: In this project, I used image warping to create mosaics of scenes from multiple images. I also used image warping to rectify some images to appear frontal-parallel. This was a very interesting project as I got to implement panaorama photos essentially and I learned how to find the homographic transformation between points using least-squares. It was also cool to see how image warping can have applications to information retrieval as I was able to rectify my notebook at an angle, and be able to read the notes after rectifying. Overall, I learned a lot about homographic transformations and how to interpolate warped images.


Part 1: Shoot and Digitize Pictures

I captured series of images with identifiable geometric shapes. I also made sure to overlap about half of the image between consecutive shots. Below are the three sets of images that I plan to turn into a mosaic.



Part 2: Recover Homographies

To create the mosaic, I first needed to define correspondence points between the images based on key features. I used 8 correspondence points for each image as labeled in the images.



After finding these points, I then determined the homography matrix that maps one set of points to the other. Since this was an overdetermined system, I solved for H using least-squares that would give the solution that minimizes the error. I wrote out the original equations for the homography and then substituted w in. The final result was the following matrix AH = b where b has all the x, y points of the destination correspondences in x0, y0, x1, y1... order.

Part 3: Warp the Images

Now that I had a homography matrix defining the homographic transformation between the images, I used the inverse of H to interpolate and map the original pixel values to the new pixel locations using cv2.remap. This transforms the start image so that its shape aligns with the points defined in the end image and the pixel values are interpolated to avoid aliasing. For each set of images, I warped one image to the other image. I made sure to leave room for the second image to be blended in the next step.

Part 4: Rectify Images

The process of rectifying an image warps a rectangular surface so that it looks frontal-parallel even if it is a different perspective in the image. I did the same process as warping an image by first computing the homography from least squares. I defined four correspondence points at the four corners of the blue face on the Rubix Cube and the four points on the notebook. For the other set of points to warp the image too, I created points of a square/rectangle based on how the ratios I estimated of the sizes and how large they would be frontal-parallel. For example, the cube I used square points of [[50, 50], [200, 50], [200,200], [50,200]]. My final results turned out very good as I was able to rectify the images. It was interesting to note that this also magnified any nonuniformity. For example, the cube edge originally was not totally straight since the face has two multiple surfaces, so you can really see that when rectified.

Part 5: Blend Images into a Mosaic

Now that I had my warped images, I blended the two images together in each set to create a mosaic. I translated the both images to leave room for the other image to be added in for my final canvas size. I found the just a weighted average was too abrupt. Instead, I added an alpha channel that would be transformed with the original image where the edges are more transparent. The final pixel values would be weighted by the alpha channel so the edges are more blended.