Filters and Frequencies

Roth Yin | rothyin@berkeley.edu

Part 1.1: Finite Difference Operator

Partial derivatives: Separately calculate the partial derivatives in x and y direction, which is to convolve the image with the given finite difference vectors.

Gradient magnitude: For each point, this is the length of the vector whose components are the two partial derivatives at this point as calculated above, which can be calculated as (∂x2 + ∂y2)1/2

cameraman
Original
D_x
Partial Derivative in x
D_y
Partial Derivative in y
gradient magnitude
Gradient Magnitude
Edge image with threshold 0.23
Edge Image with Threshold 0.23 (without losing the tripod leg)


Part 1.2: Derivative of Gaussian (DoG) Filter

The following outputs are with the same method as above, except that the original image is blurred with Gaussian first. Differences after applying Gaussian filter:

The output with the Gaussian filter:

pdx
Partial Derivative in x
pdy
Partial Derivative in y
grad_mag
Gradient Magnitude

The DoG filters (Gaussian blur of the partial derivatives):

dogx
DoG in x
dogy
DoG in y

A single convolution with the DoG filters results in the same images as two convolutions above. These are shown below:

dogx2
Partial Derivative in x
dogy2
Partial Derivative in y
grad_magg2
Gradient Magnitude


Part 2.1: Image "Sharpening"

The following images are processed with unsharp masking.

taj
Original Taj
taj_sharpened
Sharpened Taj
church
Original Church
church_shaprpened
Sharpened Church
south
Original South Hall
south_shaprpened
Sharpened South Hall

The following sharp image is blurred, and resharpened with the technique above. Though the image in small scale looks similar, upon close inspections, the fine details with high frequencies (the hair and winkles on the face) are lost, which is inevitable after Gaussian blur, and the image in general is over contrasted in the attempt to look sharp. Though unsharp masking can "sharpen" the image to some extent, it cannot revert ablurred image back to its sharp state.

monkey
Original Monkey
monkey_sharpened
Blurred and Resharpened Monkey


Part 2.2: Hybrid Images

The hybrid images are produced with the following techniques.

Example 1:

close
Bearded Mouth Closed
smile
Beardless Mouth Open
blurred
Blurred Bearded Mouth Closed
sharpened
Sharpened Beardless Mouth Open
mouth hybrid
Mouth Hybrid

close
Bearded Mouth Closed
smile
Beardless Mouth Open
blurred
Blurred Bearded Mouth Closed
sharpened
Sharpened Beardless Mouth Open
mouth hybrid
Mouth Hybrid

More examples:

Original
flower fan
Hybrid
mouth hybrid
debris prosperity
hybrid

Failed example: It fails because the shapes do not really match up, and the leaves are too different from each other. If the apple is blurred more it might be better.

apple berry
hybrid


Bells & Whistles (Extra Points) 1/4

Apply the process for gray-scaled images (described above in Part 2.2) to each channel of the color image. Combine the channels in the end.

The specific use of colors to enhance the effect depends on the specific images used. For example, the fan is by itself black and white, so the comparison below does not generate very meaningful results. But in general, the colored high frequencies and non-colored low frequencies give the best results.

No Color Colored High frequency Colored Low frequency Full Color
mono cn nc cc
mono cn nc cc
mono cn nc cc


Part 2.3: Gaussian and Laplacian Stacks

Gaussian stack:

Laplacian stack:

Mask: The following Oraple is done with the regular mask. The left half is all 1 and the right half is all 0 for the apple. Reversed for the orange.

Multiresolution blending:

The following set of images show layers of the Laplacian stacks and the final combined images.

1-0 2-0 3-0
1-1 2-1 3-1
1-2 2-2 3-2
1-c 2-c 3-c


Part 2.4: Multiresolution Blending

Process is described above in Part 2.3.

Regular masking (as in Part 2.3):

1-0 2-0 3-0
1-1 2-1 3-1
1-2 2-2 3-2
1-c 2-c 3-c

Irregular masking:

Mask: The eye region is active for the eye image. Reversed for the face.

This blending is not ideal, and has more nuance to it. See its color counterpart for more information.

1-0 2-0 3-0
1-1 2-1 3-1
1-2 2-2 3-2
1-c 2-c 3-c

Mask: The robot hand is active for the robot hand image. Reversed for the human hand.

This blending is not ideal, and has more nuance to it. See Bells & Whistles (Extra Points) 4/4 for more information.

1-0 2-0 3-0
1-1 2-1 3-1
1-2 2-2 3-2
1-c 2-c 3-c


Bells & Whistles (Extra Points) 2/4

Colored multiresolution blending: Apply the process for gray-scaled images (described above in Part 2.3) to each channel of the color image. Combine the channels in the end.

1-0 2-0 3-0
1-1 2-1 3-1
1-2 2-2 3-2
1-c 2-c 3-c

1-0 2-0 3-0
1-1 2-1 3-1
1-2 2-2 3-2
1-c 2-c 3-c

This blending is not ideal, and has more nuance to it. See Bells & Whistles (Extra Points) 4/4 for more information.

1-0 2-0 3-0
1-1 2-1 3-1
1-2 2-2 3-2
1-c 2-c 3-c


Bells & Whistles (Extra Points) 3/4

The gaze case is special. Because the eyes are taken from comics, it does not have the natural and sophisticated change of color as real images. We only want the red channel to be blended.

1-0 2-0 3-0
1-1 2-1 3-1
1-2 2-2 3-2
1-c 2-c 3-c


Bells & Whistles (Extra Points) 4/4

In the last example above (hand shaking), the regular technique results in aura around the robot hand. This is because the white background gets considered when doing Gaussian blending.

A solution I found was to fill the background with the color of the skin, so that the Gaussian blending will only blend the actual color (the skin color) as desired.

This is achieved by first recognizing the region that is not background (with color threshold), and average the rest of the active regions' colors. Then fill the background with that color.

1-0 2-0 3-0
1-1 2-1 3-1
1-2 2-2 3-2
1-c 2-c 3-c


Reflection

All these results are based on the knowledge of human perception, which is not achievable solely with Computer Science. In addition to the technical details of this project, it emphasizes the power and significance of the interdisciplinary thinking and applications that I should constantly be aware of.



All images are from open online search.