Fun with Frequencies and Gradients

CS 194-26: Computational Photography, Fall 2018, Project 3

Nathan Petreaca

cs194-26-afq

This project involved a lot of frequency and gradient domain image editing. Bells and Whistles: -Used color for the hybrid images in 1.2 -Used color for the Multiresolution blending 1.4

1. Frequency Domain

1.1 Warmup (sharpen)

Here we "sharpen" an image by creating a Gaussian blurred version of the image, an image where each pixel is a Gaussian average of the pixels around, it has the low frequencies. We subtract the blurred image from the original to get a Laplacian version of the image, it has the high frequencies. We add this Laplacian back to the original image to increase the high frequencies prominence

1.2 Hybrid Images

Now we can use the above Laplacian and Gaussian filters to create a hybrid image. We get the low frequencies of one image using a Gaussian filter and the high frequencies from another image using a Laplacian filter. Combine the low frequency image with the high frequency image and you create an image that changes with viewing distance.

Example 1: Cat and Man

Here we get the low frequencies from a man and the high frequencies from a cat.

Example 2: 90's Anime Red Velvet Irene and Seulgi

Here we mix 2 Anime versions of Kpop idols


The Laplacian filtered image is the left image above and the Gaussian filtered is the one on the right

Above are the Fourier transformed versions of the images. As shown the Gaussian removes high frequencies, the Laplacian preserves only the high frequencies.

Just for consistency provided below is the stack decomposition of the above example. More on what this means in 1.3

Stack decomposition of above

1.3 Gaussian and Laplacian Stacks

Here, we create Laplacian and Gaussian stacks. Basically, this is just applying each of this filter to an image at increasing intensity (in terms of convolution kernel size).

Decomposing Lincoln and Gala

Here we decompose the image by Salvador Dali

Gaussian Stack: As you can see simply getting blurrier

Laplacian Stack: Also getting blurrier. The last one is a Gaussian

Now, if we stack all the images in the Laplacian stack down, we get the original image. Here it is (looks identical huh)
This is because each layer of this stack carries a different range of frequencies

1.4 Multiresolution Blending

Now that we can use the Laplacian stack to get different ranges of frequencies, we can blend images together in a more comprehensive way by blending at each frequency/stack level.

Below we create a Laplacian stack of an image of an apple and an image of an orange. We create a mask and create a Gaussian stack of it. Using this mask we blend the images together at each Laplacian stack level.

Apple + Orange = Oraple

Gfriend laugh over 90s anime Seulgi's face

Here we do the same thing as above but with a different shaped mask.

Blending Seulgi and Irene

Same procedure as above
This kind of didn’t work because the mask needed to be rotated and have some other things done to it. Has the cool effect of looking like a memory in my opinion though?

2. Gradient Domain

Here we focus on using the gradient domain for blending. The gradient domain is the domain of changes in pixel values as opposed to raw pixel values. Two dissimilar pixel neighbors are considered a higher value then two similar pixels with large values. By using this domain, we can blend images such that they lower their overall change in the image. We consider each pixel to be a variable to be solved for in an optimization problem that minimize some constraint

2.1 Toy Problem

Here we use gradient domain processing techniques (least squares and a series of constraints) to recreate an image.

Looks like it works

2.2 Poisson Blending

Here we now blend two images by solving for pixels in a target region by lowering the change in pixels around the edge of the region.

Dolphins

Here we have the two images to blend and our mask

Now we do the described process for each color channel

Here are the variables









Putting the layers together



Kind of a bit red, but looks cool

Compare Multiresolution blending and Poisson Blending: Gfriend laugh over 90s anime Seulgi's face

Here we do the multiresolution blending example with a Poisson blend.
The multiresolution blend is fuzzier and softer on the edges, but does not adjust color.

As shown, it does what it was supposed to do, but it may not be the best place to use Poisson blending.

Compare Multiresolution blending and Poisson Blending: Blending Seulgi and Irene

Here we do the same thing as above again for demonstration purposes.

Penguin and dog

Look at that a penguin and a dog