In this part, we explore the 2D convolutions and Gaussian filtering.
First we use the finite difference filters $D_x = \begin{bmatrix} 1 & -1\end{bmatrix}$ and $D_y = \begin{bmatrix} 1 \ -1 \end{bmatrix}$. By convolving the image with the filters separately, we get the partial derivative in x and y of our image. Below is the result.
Then, to get the gradient magnitude, we use the following formula:
$\lVert \nabla f \rVert = \sqrt{(\frac{\partial f}{\partial x})^2 + (\frac{\partial f}{\partial y})^2}$ The image is below.
To supress the noise in the image, we binarize the image by choosing the threshold to be 0.08 (in grayscale). Then, we get the supressed edge image below.
Although we include some noise, we include as many edges as possible.
This time we use a smooth filter first -- Gaussian filter. We first blur the image with the Gaussian filter (using appropriate size and sigma value), then we repeat the procedure in part 1.1 to get the result shown below.
You can see that the edges in our new image are much thicker and more smooth, since there is much less noise if we first blur the image with gaussian filter.
Now we try the order of operation in a different way. We first convolve Gaussian filter with our finite difference operators. The result is shown below.
Using these derivatives of Gaussian filters, we can use only one convolution with our image. And the result is below.
Yes, the result is the same! (I also find the difference between the two matrices, and the difference is 0)
In this part, we utilize the high and low frequencies of the image and play with the frequencies to get sharp images, hybrid images, and blended images.
To sharp the image, we first subtract the low frequency (blurred image) from the original image to get the high frequency of the image, and then we add the high frequency back to our original image in order to sharpen it. Here, I combine these two steps into one! I use what is called the unsharp mask filter. It's defined in the lecture as below.
To convolve the color image with the filter, I convolve three color plates (r,g,b) with the filter separately and stack them together to get the convolved color image.
Below is the result of apply unsharp filter on the original images. I use $\alpha = 3$.
Here is the comparison between the original sharp image and the sharpened blurred image.
The sharpened blurred image is slightly more blurred and slightly less sharp than the original sharp image. This is because after the blurring, some high frequencies of the sharp image is removed, so when we try to sharpen it again, the resulting image may not be as good as the original sharp image.
The first step is to align the two images. Then, we apply lowpass filter on one image and highpass filter on the other image. Finally, we combine the two filtered images together to get our hybrid image. You can see the first image (low frequency) when you are far away and see the second image (high frequency) when you are near it.
Below are some interesting hybrid images.
Below is the Fourier transform analysis of my favorite hybrid image -- Mr. Efros.
Frequency domains of the input images:
And the rest:
Finally, below is the frequency domain for our hybrid image:
At first, I did not choose the size and sigma value of my gaussian filter carefully, resulting a bad hybrid image where high frequency image still shows up and low frequency image can not be seen at a very far distance. The bad example is below.
After experimenting and adjusting the size and sigma value parameters, I successfully get a good hybrid image shown several images above.
I use color for convolution, and I get the result below.
In my opinion, using color for both frequencies gives the best result, since colors give more information of the elements in the image. You can see that using color for low frequency gives very similar result to the one for both frequencies, and using color for high frequency gives very similar result to the one using only grayscale images.
Before blending the two images, we need to create Gaussian and Laplacian stacks for help. Because it is a stack, we don't need to downsample the image in every level.
Basically, for Gaussian stacks, I convolve the original image with a Gaussian filter first, and then convolve the resulting image with a Gaussian filter again. I keep doing this to get the five layers of my Gaussian stacks. As for Laplacian stacks, I subtract the next lower layer of Gaussian stacks from the current layer of Gaussian stacks to get every layer of Laplacian stacks. The lowest layer of the Laplacian stacks is the lowest layer of the corresponding Gaussian stacks in order to recreate the original image.
My Gaussian stacks have 5 layers, and each lower layer has size 10 greater and sigma value 2 greater than the previous layer. My starting size is 30 and sigma value is 2. Because of this property, the lower level of the image is more blurred and gives better overall result.
My Laplacian stacks also have 5 layers in total and below are the ones for the apple and the orange. Levels from high to low are in the order from left to right and top to bottom in the figure. Notice that I use min-max normalization to better display the result.
The colored ones are also included below.
Now it's time to blend the two images (apple and orange) together to get the oraple!
We reference the steps in the paper (Spline83, P230) below:
For this blended image, we use a regular half and half mask (as R in the step 1b) below.
Following the steps above, let's recreate the 12 figures of our oraple similar to the book.
And we get our oraple!
The colored ones are also included below.
And also the colored oraple!
I use irregular mask to create my own blended images. The irregular circle mask is below (two different size sof the circles for the following two blended images).
And the 12 figures:
From this project, I learn the importance of the Gaussian filter. It is useful for smoothing the edge image, sharpening the image, and you can use it to derive another important filter -- Laplacian filter!
Captain America from https://www.marvel.com/characters/captain-america-steve-rogers
Da Vinci from https://www.biography.com/artist/leonardo-da-vinci
Monalisa from https://en.wikipedia.org/wiki/Mona_Lisa
Mr Bean from https://www.mentalfloss.com/article/62455/15-things-you-might-not-know-about-mr-bean
Professor Efros from https://www2.eecs.berkeley.edu/Faculty/Homepages/efros.html
Sky from https://theconversation.com/curious-kids-what-is-the-sky-157834
Soccer 1 from https://www.latimes.com/sports/highschool/story/2021-05-08/high-school-boys-soccer-southern-section-playoff-pairings
Soccer 2 from https://www.amazon.com/Yanen-Traditional-Recreation-Performance-Different/dp/B07HVBC7PB