CS 194: Project 2

Theodora Worledge

Part 1: Fun with Filters

Part 1.1: Finite Difference Operator

To take a discrete derivative of an image, we can convolve the image with the humble finite difference filters, one to take the partial derivative with respect to x, and the other to take the partial derivative with respect to y. These filters are as follows:

finite_difference_filter

One can observe that these filters represent the discrete derivative by taking the difference between two pixels.

After calculating the partial derivatives with respect to x and y, I calculated the magnitude of the gradient by taking the L2 norm:

l2_norm

To obtain the edge image, I used a threshold of 0.1 to binarize the gradient magnitude image into values of 1 (> 0.1) and 0 (<= 0.1).

Part 1.2: Derivative of Gaussian (DoG) Filter

To reduce noise in the gradient of the magnitude, we can blur the image (convolve with a low pass, Gaussian filter) before taking the partial derivatives and taking the L2 norm. Due to the associativity of the convolution operation, we can either convolve the Gaussian filter with the image and then convolve with the finite difference filters, or we can convolve the Gaussian filter with the finite difference filters and then convolve the image with those derivatives of the Gaussian (DoG). I try both approaches below. For my edge image creation, I used the same threshold of 0.1 as in part 1.1 of the project.

Part 2: Fun with Frequencies!

Part 2.1: Image "Sharpening"

Images that were blurred and then sharpened do not return to the original level of sharpness in the original version because information (specifically, the high frequency information) was lost during the blurring step, and therefore, is not recoverable through the subsequent sharpening step. As can be seen in the examples, sharpening a blurred image will result in an image that is sharper than the blurred image, but still blurrier than the original.

Part 2.2: Hybrid Images

hybrid_man_cat_cropped
Hybrid photo with Derek and Nutmeg
original_mask
Original photo with mask
High passed photo with mask
original_no_mask
Original photo without mask
low_passed_no_mask
Low passed photo without mask
hybrid_mask_no_mask_cropped
Hybrid photo with and without mask
ft_hybrid_mask_no_mask_cropped
Hybrid photo with and without mask: log magnitude of Fourier transform
original_eggs
Original eggs benedict
original_eyes
Original flower eyes
hybrid_eye_egg
Hybrid photo with eggs benedict and flower eyes
Doesn't work very well :(

I tried introducing color to the hybrid images. On the face mask hybrid image, the high frequency image took on the colors of the low frequency image, leaving me with a face mask the same color as my skin tone. My lips are also a lot more noticeable through the face mask due to their color. Adding color to the sneaker/slipper hybrid image works nicely in that the high frequency sneaker image now has distinct color to it. Once again, however, the purple color of my socks in the low frequency image is bright and distracts from the hybrid quality of the image.

Part 2.3: Gaussian and Laplacian Stacks

Here, I multiplied the outputs of images run through a Laplacian stack by the output of a mask run through a Gaussian stack. Each row corresponds to a different frequency band of the original images, from highest frequency to lowest frequency. The left-most column is the mask applied to the apple image, the middle column is the mask applied to the orange image, and the right-most column is the average of the first two columns.

Part 2.4: Multiresolution Blending (a.k.a. the oraple!)

I'm very happy with the result of this earlier/later merged sunset photo! My friend took these photos in Spring 2020 on the Big C hike. I always thought it would be cool to merge them like this.

original_lake
Original lake photo
original_otter
Original later sunset photo
otter_final_blended
Blended lake and gigantic otter

I used an irregular mask to create this image of a gigantic otter in a lake. This result is alright - it doesn't have visible seams along the water, but you can still see me swimming through the otter's head, even though I was trying to hide myself behind the otter. It was very difficult to get the water to blend to this extent; although the colors were similar, the texture of the water in the original otter photo is a lot smoother than the texture of the water in the original lake photo.

Here are the masked images from different frequency bands I used to construct the blended sunset photo, step-by-step:

The most interesting thing I learned through this project is how crucial (and difficult) it is to choose good images for hybrid blending, multi-resolution blending, and other similar techniques. The best image pairs for hybrid blending share similar colors and brightness in corresponding parts of the image. The best image pairs for multi-resolution are those with similar backgrounds and spatial compatibility, i.e. whether the mask singles out a particular object in one image and an empty space in the other image. If images are not chosen well, then the incompatible features of the original images can detract from the hybrid or blended effect in the result image.