Project

Overview

In this project, we used various techniques involving frequency and gradient blending to acheive varying image effects. these include image sharpening, hybrid images, and blending.

Image Sharpening

In smoothing images, we often apply a Gaussian filter. A Gaussian filter is a type of low-pass filter, which eliminates high frequencies. As a result, edges and other fine details are removed in favor of the broader features of the image. By subtracting a smoothed image from the original image, then, we get the effect of a high-pass filter: this is called the Laplacian of the Gaussian. A laplacian highlights edges and other sharp features. Thus, by adding the Laplacian of the Gaussian of the image to the original image, we can amplify the edges of the picture. This is called image unsharpening.
The effect of the unsharpening depends on "how much" of the Laplacian we add - this is controlled by a paramater alpha. Below is an image without unsharpening, followed by one with an alpha of .75 (meaning we add the laplacian at 75% intensity) followed by one with an alpha greater than one.

The unsharpening had a clear effect on the crispness of the image. However, with too high of an alpha, the image can become too sharp.
Here are some other examples of unsharpened images. The original image is first.

We can see that the original image's quality affect how much it can be improved.

Hybrid images

In the previous part, we showed how we can construct a low-pass filter from a Gaussian and a high-pass filter from the Laplacian of the Gaussian. If we use a high-pass filter on one image and a low-pass on the other, we can create a "hybrid" image where the high frequency component is dominated by one image and the low frequency part is dominated by another. This has the effect of showing a "different" image depending on the angle and distance from which you view the image.
Here are some examples:

Out of these, the last image was least successful because so much of the image was similar, so trying to blend them led to a lot of fuzziness.
Here is a frequency spectrum of the two images that created the first hybrid image, as well as the result of applying the filters:

Laplacian Stacks

In order to create laplacians at multiple frequences, we can employ a gaussian/laplacian stack. A gaussian stack repeatedly applies a Gaussian to produce images with a different cutoff frequency. We can then subtract two of these images and get a band-pass filter which contains the frequences between two of the Gaussian images. If the last image in the Laplacian (second) stack is the same as in the Gaussian, we can add all the images in the Laplacian stack to retreive the original image.
Here are the Gaussian and Laplacian stacks for the first hybrid image before.

In the Gaussian stack we can more clearly see the low frequencies of the dog, in the Laplacian, we can see the high frequencies of the racoon.

Frequency Blending

In order to seamlessly merge two images, it can help to blend them at multiple frequencies. To get them at each frequency, we use a Laplacian. We can then use image feathering by smoothing a binary mask. Then we can use this smoothed filter to get a smooth transition at the border.
Here are some examples, with the source images and the blended image.

We can see that in the second example, the blending "fails" when the two halves are sufficiently different.
Finally, here is an example using an irregular mask:

The blending isn't optimal because the colors/textures of the two images are off.

Gradient Blending

When merging, instead of doing feathering, often times it is better to merge such that the changes in the images (the gradient) is preserved but the absolute values of each pixel can change. This can help with the color mismatch we saw previously.
In order to solve this, we want to minimize the change in the difference between two pixels in our merged image (that is the gradient should not change). There are many ways to satisfy this constraint, by constructing images with different hues. We additionally specify the constraint that images on the border of the image we are merging in should be as close to the background as possible. This ensures that the change in hue is as seamless as possible.
In this way, we can exactly reconstruct an image just by minimizing gradients:

Here are some examples of images created in this way:

The last example shows how that if the two images are sufficiently different, trying to match their hues can cause things to get very messy.
As an example of the power of this gradient blending (which we call Poisson blending), here's what the original image looks like if we don't try to minimize gradients across:

This is the result of taking the background image below, masking out the mouth, and simply copying it over to another position.

If instead, you minimize the gradients before moving over, and repeated this once more, you'd get the nice image above

Bells and Whistles

The effects acheived in black-and-white can be accomplished in color by applying the effect to each color channel separately. Here's some examples:

In the hybrid example, only the low-frequency image has color. I found there was too much "tearing" otherwise.
In addition to Poisson blending, we can also blend images using "mixed blending". Instead of minimizing the gradient in the source image, if the gradient in the background is larger, we choose to minimize that instead. This allows us to preserve features in the background image.
As an example, here's the same image we blended with frequency blending using Poisson blending:

And here it is using mixed blending:

The latter image helps blend the background a little better. The mask should be more tightly cropped, however - if the textures are too different, gradient blending can only do so much.
Here's another example of how mixed blending helps preserve background gradients