International student participating in the Brazilian Scientific Mobility Program (BSMP) at University of California, Berkeley, for the 2015-2016 academic year. Enrolled as a Computer Science Extension student. Born and raised in Rio de Janeiro, Brazil. Undergrad student at Pontifical Catholic University of Rio de Janeiro, studying Computer Engineering and Mathematics. Traditional and Digital art lover and currently interested in Computer Graphics and Artificial Intelligence - but who knows what new field of study I might fall in love with! Trying to learn the most I can during this one year here in Berkeley.
The goal of this assignment is to get your hands dirty in different aspects of image warping with a “cool” application -- image mosaicing. You will take two or more photographs and create an image mosaic by registering, projective warping, resampling, and compositing them. Along the way, you will learn how to compute homographies, and how to use them to warp images. The steps of the assignment are:
For the first part of the assignment, we will rectify one part of an image. For that, we need to calculate the homography between two sets of given points. Four correspondences are necessary because of the degree of freedom of a homography matrix. Below, you can see some of the formula derivations used. In the formulas, we assume that many points will be given. To use that extra information, we apply the method of the least squares so we can get a better result with less errors.
For the rectification itself, we ask the user to choose exactly four points. We define four points in the resulting image that those original points will be mapped to and then we calculate the homogeneous matrix that takes one image into another. To fill the pixels, we calculate the inverse transformation and then, for each pixel in the resulting image, we get the corresponding pixel in the original picture.
The results were pretty good. It is possible to see patterns and words that were not possible to read in the original image.
For the panoramas, we can use the same homography procedure. However, the target points now will be defined by the user. One important part here is to determine what is the size of the final panorama. For that, we get the borders of the image that will be transformed and calculate the coordinates on the new projection. After that, we calculate the bounds of the resulting image.
We do this procedure image by image. At the end, we have a panorama.
There are some important things to notice here: lines continue straight in this panorama. There are no curvy deformations. This creates this unique effect that only a planar projecion can achieve. However, objects far from the center start to get distorted compared to the rest of the image.
The image below was my first result with the mosaic. The images are just overlaped on the top of each other using a 50% alpha for each pixel, which resulted in the artifact that can be seen. The ligth in the two pictures was different, creating a difference in color. We will fix that later.
Instead of projecting the images on a plane, we may choose another surface. Here I projected each image on the surface of a cylinder. On a cylinder, the transformations that we need to search for are just translations.
One important part of this projection is that we need to know the focal length of our camera. This heavily changes the result for the transformations. Below, you can see different examples of focal lengths. If you choose this parameter wrong, the matching between adjacent images will not be good, creating artifacts in the panorama.
By translating and matching each of the transformed images we can build the following panorama. Here, straight lines are not lines anymore. However, the whole picture fits in a reasonable size and there are not many distortions near the edges. The blending artifacts are still present.
For blending the images, lets use weighted averaging for the pixels. However, lets do that according to a mask. We want to make sure that pixels near the border of the image are blended more easily than the pixels near the center. I have tried different masks. The first one that I tested was the gaussian (circle-looking image), but it did not work very well because of the shape of the image (a rectangle). Then, I have changed to a rectangular mask that decreases whenever you get close to one the borders.
In our transformation algorithm, we include the masks. The transformed masks will give the information about the weight of each pixel in the resulting image for the linear blending.
The impact this method has is pretty big. You can still notice some small artifacts in the image, but not it looks much more natural. Success!
The same procedure can be done on the cylindrical projection images. However, we must also transform the mask into the cylindrical projection, exactly as we do with the image itself. The rest is exactly the same as in the planar projection. The results are great.
This is another example. Here, I have taken four pictures from the outside of the International House on Berkeley. The algorithm worked pretty well for this case with only some minor artifacts.
This example did not work. Well, many things could explain the failure of this example. First, the images have very different lightnings. Second, the focal length is not necessarily right. Third, the user input may have inserted some errors, specially because of the error propagation resulting from the number of pictures and their layout. Those effects created this weird looking images with tons of artifacts. It looks almost like a cubism picture!
Here, we will try to take one step further: we are going 360! Lets try to create a panorama with a set of pictures that completes a full turn. For this, it was important to make sure that the camera was standing still and rotated only on its vertical axe. I have used a tripod for pictures.
Note: Here, you can see that the tripod resulted in very neat matchings!
The difference from the normal panorama is that whenever we get to the end, we need to do one last iteration. Here, we try to align the result so far with itself. The result is an image with duplicated content. However, once we have that image, we just need to crop it so it has the right size and BAM - a 360 panorama!
Okay. One thing failed for this. The assumption that the tripod was tottaly horizontal on a flat surface was not true. The resulting images do not complete a 360 panorama perfectly because it is sightly tilted. The solution? Before cropping, we should also rotate to correct this this. This fix is still needed.
I learned that homographies are a pretty useful tool for many applications. I also realized that the key to get good results are the little details and corrections. For each step I had to worry about the resulting size of the images and discretizations - things that could lead into bugs. I also learned that the world would be a very beautiful place with more me's.
Website built using bootstrap.