CS 194: Image Blending

Caroline Moore - cs194-26-aew


Project Description

The purpose of this probject was to understand various techniques to combine or blend pairs of images. By experimenting with and comparing these methods, we gained a better understanding of the strenths and weaknesses of each method, as well as an understanding of the situations each method works best in.


Part 1.1: Image Sharpening

To extract the high frequency edges of an image, we convolve it with a Laplacian kernel, which is a Gaussian subtracted from an impulse. In effect, this removes the low frequency blurred part of the image that is the result of convolving it with the Gaussian. We can then add these edges back to the original image to get a sharppened image. Below is the effect of this process on a picture of a waterfall.


Part 1.2: Hybrid Images

Humans primarily see high frequencies. However, these frequences are not visible at a distance so only low frequencies can be seen. We can use this phenomenon to create hybrid pictures that appear to be one image up close but a different image far away. By combining the high frequency part of one image with the low frequency part of another image, the high frequency image is visible up close and the low frequency image is available at a distance. We use a Laplacian image to extract the high frequency part of the image and a Gaussian to extract the low frequency part of the image.

To start with, we combine the high frequency part of an orchid and the low frequency part of a butterfly in black and white.

We can improve this image by adding color. However, this still doesn't look good because the butterfly is too clear close up. This is because the white background is too bright so the black of the butterfly stands out, no matter how blury it is. We can improve the hybrid image even more by removing the white background of the butterfly.

I also used this method to combine images of Harry Potter and Voldemort and images of a peacock and a rose.

We can also take the Fourier Transform of the images, the images after they have been filtered, and the final image to see how this process changes the frequencies in the image. The Fourier Transforms of the peacock and rose images are below.


Part 1.3: Gaussian and Laplacian Stacks

Sometimes it is useful to manipulate the frequencies of an image independently. To extract these frequencies, we create a Gaussian or Laplacian stack of an image. A Gausian stack of a pictuere is a sequence of images where the first image is the original picture and every subsequent image is the previous image after it has been convolved with a Gaussian kernel. A Laplacian stack of a picture is a sequence of images where each is the image in the same position of Gaussian stack minus the image in the next position of the Gaussian stack. The last image in the Laplacian stack is last image in the Gaussian stack. This means that the sum of all images in the Laplacian stack is the original picture. Below are the Gaussian and Laplacian stacks of Gala Contemplating the Mediterranean Sea which at a distance of 20 meters is transformed into the portrait of Abraham Lincoln by Salvador DalĂ­, the butterfly and orchid hybrid image from Part 1.2, and the rose and peacock hybrid image from Part 1.2. In each of these stacks, the low frequency image is much clearer on the top and the high frequency image is much clearer on the bottom.


Part 1.4: Multiresolution Blending

One use of Laplacian stacks is image blending. When combining two images by fading all frequencies equally, the divide will either be too sharp from the low frequencies not being faded gradually enough or there will be ailasing from the high frequencies being faded too slowly. One way to compensate for this is to merge each frequency, or each level of a Laplacian stack, individually and then combine the frequencies back together.

Below are the Laplacian stacks of an apple, an orange, and the mask used to combine the images.

Next, we multiply the apple by the mask and the orange by one minus the mask at each stage and then add them together. The result is below.

Finally, we add all the images in the bottom row together to get an oraple. The result is an apple blended with an orange.

We can use this method to blend other images.

We can even use it to combine images using a mask with an irregular shape. However, in this case, the second image was too small so the fire became blurred.


Part 2.1: Image Recration from Gradient

An image can be represented by a single pixel value and an array of gradients. Then, the image can be reconstructed by using the gradients and the pixel value as constraints and solving for the optimal solution. I did this to reconstruct the image below.


Part 2.2: Gradient Domain Fusion

Humans primarily see differences between colors, not the true colors of the pixels. We take advantage of this in antoher method to combine images, called gradient domain gusion. This method takes differences between neighboring pixels in a source image and tries to duplicate those differences in a target image without leaving sharp edges on either side. The colors in the final will be different than the colors in the original source image because the gradients from the source will be applied to a different starting color. However the overall appearence will be the same.

Gradient domain fusion can be implemented as a least squares problem. However, the size of the problem to solve grows quadratically with the number of pixels in the mask. This means that gradient domain fusion is much slower than multiresolution blending, especially as the image grows in size.

We can see that this is clearly better than just copying over pixels because this leaves a ring of black around the dragon that is distinct from the sky. We can also see that the dragon gets lighter, but it is still very clearly a dragon.

Gradient Domain Fusion does not work well when the backgrounds are very different colors, such as in the example below. When the algorithm attempts to map the gradients from the phoenix into the forest, the phoenix becomes too washed out. There is also a blury darkness around the phoenix from the black background and the trees.

We can take the images of the eye and fire from Part 1.4 and use Gradient Domain Fusion to combine them. When the images are blended using Gradient Domain Fusion, there is no bluring of the edges so the fire does not appear blurred. We can also compare these images to the image if the fire is simply copied on top of the eye. In this case, there is a clear line betweeen the black around the fire and the black inside of the pupil, though only if you look closly.