George Lee cs194-26-adq
For this exercise, I worked with manipulating images through the following exercises: unsharp mask filtering, creation of hybrid images, multiresolution blending, and poisson blending.
The unsharp mask is created through the following formula:
unsharp_mask = image - gaussianblur(image)
To sharpen an image using the unsharp mask, we can apply the following formula:
sharpened_image = image + alpha * unsharp_mask
Below are my results of this process. From left to right, we have the original image, the unsharp mask, and the sharpened image.
We continue by taking a look at hybrid images. With reference to this paper, we were able to generate the images.
A high level description of the process is to have two images, called a and b. We pick one image, image a, and take the high frequencies that make up the image, and merge them with the low frequencies of the other image, image b.
We can see some results below:
Here is my favorite hybrid:
Taking a look at the 2d fourier transform of this blended result, we get:
One image that didn't make it through the testing phase is this one below:
The issue here is that important features, such as the eyes, nose, and lips are not mapped directly to each other; only the ears are correctly mapped.
In a future exercise I will work to create this mapping.
We now proceed by doing multiresolution blending. The detailed description of how this was done can be found by reading this paper. A high level description of what is done, is that we separate an image to layers, where each layer contains different frequency ranges of the image. The paper defines these layers as a laplacian stack. We then blend the image with another, by applying a mask to the images we want to blend, and then blending each individual layer using something like the gaussian filter. We blend the lower frequencies more, while blending the higher frequencies less.
We can see some results below:
Here is the laplacian stack for the generated image:
In the past half of this exercise, we have been working exclusively on the raw pixel values of an image, and blending accordingly. This works for a good number of cases, but is not always what we want, especially if we want to make what we insert into the image look as if it were always part of it.
To do this, we manipulate the gradients of images. Given a source image s, and target image t, we can blend a new resulting image, v, using the following formula:
Let's start off with a small image, applying the formula above. Here, we use a mask equivalent to the full image. Thus, we can expect the final result to be equivalent to the original image. On the left, is the original, and on the right is the reconstructed image from the calculated gradients.
Now that our proof of concept has been realized, let us now work on larger images. Starting with:
We can get the following. On the left is a naive copy. On the right we have the blended image.
To make this image, I applied a mask onto both the source image. From this, I calculated a new pixel value for each pixel that was captured within the mask of the image, using the above formula. Finally, I copied the resulting masked source image into the target image. To speed up the computation significantly, I used the scipy.sparse library, to speed up the calculations of the new pixels.
Here are some additional images that I created:
Here is one failure case. Here, the intensity of the dark sky almost completely overpowers the picture of the eye. As a result, it is hard to see the blended result.
Let us now regroup and compare our results between poisson blending and multiresolution blending.
Recall our earth and sun images.
Here are the results of blending using the same masks. On the left is the result of multiresolution blending, and on the right is the result of poisson blending.
In this case, multiresolution blending looks like what I intended. This result is a failure case for poisson blending, as the picture of the sun greatly overpowers the Earth. I would use poisson blending when blending in two regions that are similar to each other in tone.