Part 1.1: Finite Difference Operator

To calculate the partial derivative in x and y, I convolved the image with finite difference operators Dx and Dy

For Dx, it basically gives us the results of img[y, x] - img[y, x + 1] and similarly for Dy, we get img[y, x] - img[y + 1, x] which finds differences between adjacent pixels which can approximate rate of change. To get the gradient magnitude I took sqrt((dx/dt)^2 + (dy/dt)^2). To find the real edges without noise, I binarized the gradient magnitude by choosing an appropriate threshold.

Original cameraman.png
Dx of cameraman.png
Dy of cameraman.png
Gradient magnitude of cameraman.png
Binarized gradient cameraman.png

Part 1.2: Derivative of Gaussian (DoG) Filter

First, I applied a Gaussian filter on the cameraman image before applying the same Dx, Dy operations.

Dx of cameraman.png
Dy of cameraman.png
Gradient magnitude of cameraman.png
Binarized gradient cameraman.png

These gradient images now have less noise than the ones that were not first filtered with a Gaussian. Also, the edges were now slightly blurred, which leads to them showing up as thicker. The edges also stick out more consistently.

Now, to speed things up, I can apply the Dx and Dy filters on the gaussian kernel itself (which is a much shorter operation given the sizes of kernels compared to the entire image). Then, I can use the new kernels on the image, saving us a convolution operation on the entire image.

Gaussian dx kernel
Gaussian dy kernel
Dx of cameraman.png
Dy of cameraman.png
Gradient magnitude of cameraman.png
Binarized gradient cameraman.png

This gives the same results as before.

Part 2.1: Image "Sharpening"

Image sharpening was implemented using the unsharp mask technique. This involves getting high frequencies of the image by subtracting a blurred version from the original image. Then we can add some scaled version of the high frequencies (scaled by alpha) to the original image to increase the higher frequencies in the original image and "sharpen" it.

Original taj.jpg
taj.jpg alpha = 2
taj.jpg alpha = 5
taj.jpgalpha = 10

We can also apply this to another image.

alpha = 2
alpha = 5
alpha = 10

In this final example, I will take an image, apply a Gaussian blur to it, and then try to sharpen the blurred image.

Blurred + Sharpenalpha = 5
Blurred + Sharpenalpha = 10

Part 2.2: Hybrid Images

Hybrid images involve being able to see one thing when very close up to the image and another when far away. Our eyes are best at picking up high frequency signals when close, and low frequency signals from far. To take advantage of this, we can blend two images together by taking the low frequency signal from one of them (the one we want to see from far away) and the high frequency signal from the other (the one we want to see close up). To get the low frequency signal, we can apply a Gaussian blur to the image. To get the high frequency signal, we can subtract the blurred image from the original image.

Derek and Nutmeg Hybrid

In this example, I implemented the Bells and Whistles of using color. Here, both Derek and Nutmeg were kept in color since that seems to give the best results. In general, it would be reasonable to remove the color for the higher frequency image, since it doesn't show up that clearly even when kept in. Here are all the other combinations of color.

Derek (BW) + Nutmeg (BW)
Derek (BW) + Nutmeg (Color)
Derek (Color) + Nutmeg (BW)

It is hard to see the color on Nutmeg, but if you focus on the right side arm in the photo, you can see some difference.

Here are some other examples that I thought looked pretty cool:

Messi and Maradona Hybrid

Fourier Analysis

I also did a frequency analysis of this process for the Messi + Maradona photos:
Messi Frequency
Messi Frequency after High Pass Filter
Maradona Frequency
Maradona Frequency after Low Pass Filter
Combined Frequency
Final Good Example:
Batman and Joker Hybrid
Here are some examples that did not work too well:
Jimothy Hybrid
The biggest problem with this image was that I couldn't get the close up version to work properly. I think the problem was due to a lot of key features of the face not lining up properly. From far, however, it looks fine.
Elephant and Lion Hybrid
The problem with this one as well was that the animals didn't quite line up and their features are quite different. In order to make the lion visible close up, I also had to blur the elephant quite a lot.

Part 2.3: Gaussian and Laplacian Stacks

To get a Gaussian stack, I took the original image and apply a Gaussian filter on it. Then, we can continue applying Gaussian filters over and over again on them. This is basically a stack of images with decreasing frequency ranges. To get a Laplacian stack, We take each consecutive image from the Gaussian stack and subtract the two. This gives us mid-band frequency images. The final image would be the same as the lowest frequency image in the Gaussian stack. If we add all the images in the Laplacian stack, we will get back the original image.

Gaussian Stack for Apple

Laplacian Stack for Apple

Gaussian Stack for Orange

Laplacian Stack for Orange

Part 2.4: Multiresolution Blending (a.k.a. the oraple!)

To blend these images, we can take the a mask image that represents the boundary and create a Gaussian stack using them. We also take the Laplacian stacks for each of the images and sum up mask * lapacian image 1 + (1 - mask) * laplacian image 2. This gives us a merged version at each frequency band. Then, I can sum up all the merged frequency bands.

Gaussian Mask

Apple Masked

Orange Masked

Oraple Laplacian Hybrid

Final Summed Hybrid Image

Batman + Joker

Blended with Vertical Line

Golden Gate Bridge + Lightning

Golden Gate Bridge

Gaussian Mask

Golden Gate Masked

Lightning Masked

Golden Gate + Lightning Laplacian Hybrid

Final Image

Golden Gate Bridge + Northern Lights

Golden Gate Bridge
Northern Lights

Final Image

Earth from Earth

Moon from Earth
Earth from Moon

Final Image

Earth from Earth

Bells and Whistles

I discuss the Bells and Whistles within each section directly. I used color for everything by processing the red, green, and blue channels separately. For the hybrid images, I found that using colors for both high and low frequency images looked the best. I think the coolest part/most important thing I learned from this project was the impact frequency has on images and human vision. The way hybrid images allow us to see different things up close versus far away was pretty incredible since it just uses differing high and low frequency blends.