Project 2: Fun with Filters and Frequencies!

By Adam Chang

Part 1: Fun with Filters

Part 1.1: Finite Difference Operator

In this part of the project, we convolved a finite difference kernel over an image in order to calculate gradients in both the x and y directions. The finite difference operators are [[-1, 1]] for dx and [[-1],[1]] for dy. The dx operator computes changes in the image intensity in the horizontal directions. The dy operator computes changes in the image intensity in the vertical directions. Dx catches vertical lines in the image, while dy catches horizontal lines in the image. We then calculate the magnitude of the gradient at each pixel by taking squared sum of the gradient in both directions, adding them, and then taking the square root. Below are the results of performing the Finite Difference Operation on the provided cameraman.jpg.

cameraman.png
Original cameraman image
cameraman_dx.jpg
Cameraman gradients in the x direction
cameraman_dy.jpg
Cameraman gradients in the y direction
cameraman_grad_magnitude.jpg
Cameraman gradient magnitudes per pixel
cameraman_finite_diff.jpg
Edge image created by thresholding gradient magnitudes

Part 1.2: Derivative of Gaussian (DoG) Filter

In the second part of Part 1, we apply a gaussian kernel filter to the image before performing the finite difference operation. This blurring reduces the high frequencies and gets rid of the noise causing the random white dots throughout the edge image in Part 1.1. However, we also lose the already blurry background buildings from our edge image. After applying the gaussian blur, these are no longer discernable as edges, which may or may not be desirable. We also show that convolutions are associative. Instead of applying the gaussian kernel blur to the image and then performing the finite difference operation, we pre-compute a derivative of gaussian filter by applying the finite difference operators to the gaussian kernel, and then convolving the combined filter to the image. This results in the same output image.

cameraman_deriv_of_gauss.jpg
Cameraman edge image, finite difference after gaussian kernel blur
cameraman_deriv_of_gauss_one_conv.jpg
Cameraman edge image, combined finite difference and gaussian kernel blur filter

Part 2: Fun with Frequencies

Part 2.1: Image "Sharpening"

In Part 2.1, we perform an image "sharpening" operation by extracting the high frequencies of the image and increasing their intensity. As discussed in lecture, this is not actually performing sharpening, which would in its true form be adding higher frequencies to the image than currently exist. Instead, this operation provides the illusion that there are higher frequencies by intensifying the already existing high frequencies in the image. Below are results from sharpening images.

Original Sharpened
taj.jpg
Taj image original
taj_unsharpen_filter.jpg
Taj image sharpened
firework.jpg
Firework image original
firework_unsharpen_filter.jpg
Firework image sharpened
lego_truck.jpg
Lego truck image original
lego_truck_unsharpen_filter.jpg
Lego truck image sharpened
night_street.jpg
Night street image original
night_street_unsharpen_filter.jpg
Night street image sharpened

We also performed a test to see that this operation is not equivalent to actually sharpening the image. We blurred an image, then performed sharpening on the blurred version of the image and we observe the results and compare the sharpened blurred image to the original. It becomes quite obvious that the original resolution and sharpness was not recovered by the sharpening operation, and instead lines in the blurred image become more intense and contrasty. The sharpened image also contians a lot more noise than the original.

firework.jpg
Firework image original
firework_blurred.jpg
Firework image blurred
firework_blurred_unsharpen_filter.jpg
Firework image blurred then sharpened

Part 2.2: Hybrid Images

In Part 2.2, we generate hybrid images, images that are comprised of 2 different component images, only one of which is visible at any one time based on how far you are standing away from the image. We take one of the component images and run it through a low pass filter and the other component image and run it through a high pass filter. Adding the two images results in a hybrid image. If you are close the image, you can make out the high frequencies of the component image run through the high pass filter. If you are far from the image, you can only make out the low frequencies of the component image run through the low pass filter. Below are hybrid image results. We see in the last example with the tree and the sapling the important of alignment. Regardless of the high and low pass filters, if the two images don't line up for most of their subject matter, both images will be visible at the same time.

Component Image 1 Component Image 2 Hybrid Image
camry.png
Image of Camry, run through high pass filter
apolloie.jpg
Image of Apollo IE, run through low pass filter
car_hybrid.jpg
Car hybrid image
nyc_before.png
Image of NYC before development, run through high pass filter
nyc_after.png
Image of NYC after development, run through low pass filter
nyc_hybrid.jpg
NYC hybrid image
sapling.jpg
Image of sapling, run through high pass filter
tree.jpg
Image of tree, run through low pass filter
tree_hybrid.jpg
tree hybrid image

For the car hybrid image with the Camry and Apollo IE, we perform a discrete fourier transform to see the effect of our filters and how the two images are added to form our final image.

camry.png
Image of Camry
car_camry_dft.jpg
Image of Camry DFT
car_camry_high_pass.jpg
Image of Camry after high pass filter
car_camry_high_pass_dft.jpg
Image of Camry after high pass filter DFT
apolloie.jpg
Image of Apollo IE
car_apolloie_dft.jpg
Image of Apollo IE DFT
car_apolloie_low_pass.jpg
Image of Apollo IE after low pass filter
car_apolloie_low_pass_dft.jpg
Image of Apollo IE after low pass filter DFT
car_hybrid.jpg
Hybrid image
car_dft_combined.jpg
Hybrid image DFT

Part 2.3: Gaussian and Laplacian Stacks

Part 2.3 had us create Gaussian and Laplacian stacks of images. Gaussian stacks are created by convolving successively larger and higher sigma gaussian kernels against the original image to generate blurrier and blurrier images (lower frequencies). Laplacian stacks represent the difference between successive images in the gaussian stack. Below is a visualization of the laplacian stacks of the apple and orange as well as the blended oraple.

apple_0_laplac_masked.jpg
Apple laplacian level 0 masked
orange_0_laplac_masked.jpg
Orange laplacian level 0 masked
oraple_0_laplac.jpg
Oraple laplacian level 0 (combined)
apple_1_laplac_masked.jpg
Apple laplacian level 1 masked
orange_1_laplac_masked.jpg
Orange laplacian level 1 masked
oraple_1_laplac.jpg
Oraple laplacian level 1 (combined)
apple_2_laplac_masked.jpg
Apple laplacian level 2 masked
orange_2_laplac_masked.jpg
Orange laplacian level 2 masked
oraple_2_laplac.jpg
Oraple laplacian level 2 (combined)
apple_3_laplac_masked.jpg
Apple laplacian level 3 masked
orange_3_laplac_masked.jpg
Orange laplacian level 3 masked
oraple_3_laplac.jpg
Oraple laplacian level 3 (combined)
apple.jpg
Apple original image
orange.jpg
Orange original image
blended_oraple.jpg
Oraple blended image

Part 2.4: Multiresolution Blending

In addition to the oraple example, we perform blending on some custom image pairs to generate some interesting images. We also support using custom masks as opposed to the vertical divide down the center of the image.

baby.jpg
Baby
sun.jpg
Sun
baby_mask.jpg
Mask used to select baby parts of image
blended.jpg
Baby and Sun blended with the mask

Below is another example blending a cat and a tank. We see that the alignment is very critical and minor misalignments lead to only limited success in blending.

cat.jpg
Cat
tank.jpg
Tank
cat_mask.jpg
Mask used to select cat parts of the image
blended.jpg
Cat and Tank blended with the mask

We can create a laplacian stack breakdown for the cat and tank blended image just like for the oraple above.

cat_0_laplac_masked.jpg
Cat laplacian level 0 masked
tank_0_laplac_masked.jpg
Tank laplacian level 0 masked
cattank_0_laplac.jpg
Cattank laplacian level 0 (combined)
cat_1_laplac_masked.jpg
Cat laplacian level 1 masked
tank_1_laplac_masked.jpg
Tank laplacian level 1 masked
cattank_1_laplac.jpg
Cattank laplacian level 1 (combined)
cat_2_laplac_masked.jpg
Cat laplacian level 2 masked
tank_2_laplac_masked.jpg
Tank laplacian level 2 masked
cattank_2_laplac.jpg
Cattank laplacian level 2 (combined)
cat_3_laplac_masked.jpg
Cat laplacian level 3 masked
tank_3_laplac_masked.jpg
Tank laplacian level 3 masked
cattank_3_laplac.jpg
Cattank laplacian level 3 (combined)
cat.jpg
Cat original image
tank.jpg
Tank original image
blended.jpg
Cattank blended image