Fun with Frequencies and Gradients - CS194-26 FA 2018 Project 3

Neil Ambar Gupta, cs194-26-aef


Project Idea

The idea was to mess about with image frequencies and structure via gaussian and laplacian filters (ranging from stacks to sharpening).


1.1 Sharpening

How it Works

This works by taking a low-pass filter of the image (gaussian) using a 9x9 kernel, which results in the highest frequencies of the image, then adding a fraction of the original image minus these values back to the original image (determined by a variable alpha).

Original Schnauzer
Alpha = 0.5
Alpha = 1
Alpha = 2

I tried various alpha values from 0.5 - 2 and found that the range 0.5 - 1 was the softest on the eyes yet a perfect sharpening point.


1.2 Hybrid

How it Works

The way hybriding two images together works is to take the low pass filter of one image and the high pass filter of the other image and sum them together. In order to do this one must Gaussian blur the first image (low pass) and for the second image subtract the gaussian filtered version of that image in a similar way to unsharpen an image from 1.1. Had to use FFT to execute the filtering of the image. Regarding the kernels of the Gaussian I noted that sigma = 55 worked best for low filtering and sigma = 25 worked well for high pass filtering.

First Image (low pass)
Second image (high pass)
'
Oski FFT
Bruin FFT
Hybrid FFT after Finish
'
Hybrid
Done When Switched Places

These aren't the perfect backgrounds because I wanted to display a challenge but the hybrid still works despite that.



1.3 Stacks

How I did it

Recall image pyramid is a multi-scale representation of itself with smoothing and rescaling; the difference between that and the stack - it doesn't contain the scaling/smoothing. Looking in our Gaussian stack, consecutive images layers are weighted down using a Gaussian blur; Laplacian stacks are a deviation from a Gaussian stack. For each layer we track the difference between the corresponding Gaussian layer and the level above it (the first level is the difference between the original image and first level Gaussian). I will show the isolation of these stacks, just below are the Gaussian stacks and even lower Laplacian stacks of our earlier hybrid image:


1
3
4
5
6
1
2
3
4
5

Note: Going back to the lowest layer of these stacks show the original filters from before in 1.2.


1.4 Blending

How it Works

Making use of the blending techniques shown in the 1983 paper by Burt and Adelson, we make a half-and-half image between two images in such a way that the split between the images are soft Basically taking in two images and a mask (half black, half white), we make a Gaussian stack of the mask as the unifying key between two distinct images. Then we make a Laplacian stack for each of the input images. For each layer, combine the two images in the Laplacian stacks with the mask from the Gaussian stack illustrated per the formula in the paper.

Orange
Apple
Orapple
Garoppolo
Kaepernick
Gapernick
Mars
Moon
Moors
Problems

I had a tough time blending without losing brightness (dark orapple problem) that I could not fix unfortunately.


2.1 Toy Story Problem

How it Works

Moving on to gradient domains, we now have a problem of using an individual pixel and a gradient to reconstruct an image. This is known as the toy story problem because we do this with Buzz and Woody. The goal is that given a source image and a goal image v, we try to do three things: first we must minimize the x-direction gradient difference between the source and goal, then we do the same for the y-gradients, then finally ensure that the top left corner of the image are minimized in difference (the same). This minimization problem turns into a least squares problem of the following equation (Av = b). A is the sparse matrix of the source image (the gradient coordinates and the last row has the top left coordinates for both images), b is the gradient equation result and v is the vector we want to reconstruct the source. Results:

Toy Story Original
Toy Reconstructed