Frequencies and Gradients with Kevin

Image Sharpening

Did you know Jar Jar Binks is secretly a Sith Lord? If you don't know what he looks like, this is him below.

That is the face of pure evil. So evil you can't even tell. Let's take a sharper look at him so we can recognize him better. We'll take a high pass filter on the image below to get the edges and then add this result to the original image, accentuating the edges.

Now let's add this image to the original. Note the sharpness of Jar Jar Bink's expression, contemplating the demise of the Jedi.

Hybrid Images

Speaking of Sith Lords, I'm sure you remember Darth Vader. Spoiler alert: he used to be the Chosen One, Anakin Skywalker. Well, now he's trapped behind a black mask. But I heard if you look at him in the distance, you can still see Anakin.

Hmm, why is that? Anakin is actually at a very low frequency and Vader is at a very high frequency, and since they are combined, you can see Vader up close, but Anakin starts showing up in the distance. Let's take a look at their frequencies.

Spectral Analysis

Let us look at the spectograms for the original images

Now let us look at the spectograms for the original images with a high pass filter applied to Vader and a low pass filter applied to Anakin


And to finish, let us look at the combined image and spectograph.

More Examples

Let's look at a couple more examples. We have a new robot that changes shape when you look at it close up and far away called R2-B8. However, this isn't as good of a blend. Some of the darks in BB-8 are overpowering and you can see BB-8 pretty well and its not a very clean blend.

Did you know the Death Star is actually a donut? From far away when you see it from planetside, it looks like the Death Star we all know. But close up, it's clearly just a donut.

Stacks on Stacks: Gaussian and Laplacian

Back to Vader and Anakin, what if we do some decomposition of the image. Let's make a gaussian and laplacian of this image.

Now let's try this with color as well. Let's look at these wonderful pieces of art

Lincoln

Mona Lisa

Multiresolution Blending

Ok, now back to some serious things. Is Jar Jar Binks actually a Sith Lord? Wait look what I found.

That looks like Jar Jar Binks the Sith Lord! Not very realistic though... I'm not convinced. Anyways it appears the Death Star is visible from crater lake tonight.

How did the Death Star get there? Let's take a look at the Laplacian Stacks for the masked Death Star and the original image.

Image Recreation with Gradients

We start by implementing simple example recreating an image using gradients. We first compute the x-gradient and y-gradient. This is our target matrix. We solve a system of linear equations where we look to minimize ||Ax - b|| where A is our set of constraints, x is the resulting image, and b is our source image. Each row of A represents computing a gradient, x_i - x_j, where we will pick an x vector that minimizes the difference between the computed gradient and the actual gradient. Our last constraint is to match intensities, so we add a constraint that the top left pixel should match the original image.

Poisson Blending

On to more exciting things. Let us use gradients to blend some cool images together. Well we had the death star earlier. It didn't look that good. Well let's try to blend it better. We again have an A matrix, which is our constraint matrix. Each row of this matrix is a constraint that corresponds to a pixel. If the pixel is outside the designated mask, then we just use the background image. If the pixel is inside the designated mask, then we apply a gradient. Basically it is the difference between the current pixel and each pixel around it. Let's look at the set of two images and mask below.

Not doing any gradient blending at all, we get this pretty bad image.

Now doing some gradient blending, we get this wonderful work of art below. Actually it's real because you can see the Death Star from Crater Lake.

More Examples

These examples were not as good. Darth Sidious in the Oval Office was quite sub par as the image was initially very dark and we are blending it into a very light background. I think this stark contrast in colors made the blending difficult.

The same can be said for the ewok. It originally is in a dark green blend and also the presence of the grass which is very textured makes it difficult to blend into.

Anyways, fun fact, you can see X-wings in Chicago at a roof top bar called Cindy's.

Also, conclusion: Jar Jar Binks is actually a Sith Lord.

This image is much more realistic than the frequency blending one.

It seems that blending in the gradient domain, the image blended into the background matches the intensity of the surrounding pixels much more closely. As a result, the image looks more like it fits in. However, if you have a similar intensity and setting, the frequency does a really good job. Just take a look at the oraple!