1 Fun with Filters

1.1 Finite Difference Operator

In this section the camera man image in Figure 1 was convolved with finite difference operators defined as:

\(D_x = \begin{bmatrix} 1 -1\end{bmatrix}\) , \(D_y = \begin{bmatrix}1 \\-1\end{bmatrix}\)

to get partial derivatives of the camera man image (Efros and Kanazawa, 2021).

Figure 1: Camera Man

Figure 2: a)Partial Derivative wrt x b)Partial Derivative wrt y

Figure 3: a)Gradient Maganitude Image b) Binarized Gradient Maganitude Image

Figure 2.a. and 2.b. represent the partial derivative with respect to (wrt) \(x\) and \(y\), respectively. The Partial derivative wrt \(x\) is given by \(\frac{\partial f(x,y)}{\partial x}\), and by definition it captures the change in the image intensity as \(x\) changes. This is why the vertical lines of the image are more prominent in Figure 2.a. Whereas the partial derivative wrt \(y\), given by \(\frac{\partial f(x,y)}{\partial y}\), captures horizontal line on an image, as observed in Figure 2.b.

Combining the partial derivatives of an image into a vector gives you the image gradient, which is defined as :

\[ \nabla f = \begin{bmatrix}\frac{\partial f(x,y)}{\partial x}, \frac{\partial f(x,y)}{\partial y} \end{bmatrix}\] (Szeliski, 2020)

The gradient of an image points in the direction that experiences rapid change in intensity. Computing \(L_2-norm\) of the image gradient captures the gradient magnitude image, shown in Figure 3.a., also known as the edge strength and is given by:

\[|| \nabla f || =\sqrt{\frac{\partial f(x,y)}{\partial x} ^2+ \frac{\partial f(x,y)}{\partial y}^2 } \] (Szeliski, 2020)

So gradient magnitude image tells us how fast the image intensity is changing and the image gradient gives us the direction in which the image is rapidly changing. The gradient magnitude image acts like an edge detector because pixels with high change( i.e. High \(\frac{\partial f(x,y)}{\partial x}\) and \(\frac{\partial f(x,y)}{\partial y}\)) have prominent white values and pixels with little or no change are black.

Figure 3.b. represents the binarized gradient magnitude image with 0.1 as the threshold. This helped us reduce noise and expose the strong edges of the image.

1.2 Derivative of Gaussian (DoG)

In this section the camera man image in Figure 1 was convolved with a Gaussian filter,and then the steps done in the previous section were repeated to compare the outputs of two approaches.

A 2D Gaussian filter was computed by using cv2.getGaussianKernel() with \(ksize=2\) and \(sigma=10\).

Figure 4: a)Partial Derivative wrt x b)Partial Derivative wrt y

Figure 5: a)Partial Derivative wrt x b)Partial Derivative wrt y

Figure 6: a)Gradient Maganitude Image b) Binarized Gradient Maganitude Image

The partial derivatives of the smoothed image captures more vertical and horizontal line the one without. There is less noise captured in the binarized gradient magnitude image for the smoothed image compared to the one without smoothing from the previous section.

There’s a theorem called the Derivative Theorem of Convolution which can help with reducing the run-time for generating gradient magnitude images. The theorem states that

\[\frac{\partial}{\partial x}(h*f) =(\frac{\partial}{\partial x}h)*f \]

(Szeliski, 2020)

This means one can perform a single convolution instead of two by creating a derivative of Gaussian filters. Figure 7 shows that the results from a single convolution and two convolutions are the same, which confirms the theorem.

Figure 7:a) Double Convolution b) Single Convolution

2 Fun with Frequencies

2.1 Image Sharpening

In this section we focus on sharpening images. We start off with sharpening blurry images by following these steps:

Apply a Gaussian filter (\(g\)) on the image of interest (\(f)\). Gaussian is a low pass filter that outputs a low frequency image (\(I_{low}\)) which is the blurry version of the original image: \[ I_{low} = f*g \]
Subtract the low frequency image from the original image (\(f\)) to get the high frequencies of the image (\(I_{high}\)):
\[ I_{high} = f - I_{low} \]

\[ I_{high} = f - f*g \]

Lastly to make the image sharp, add the high frequencies with a factor \(\alpha\) to the original image : \[I_{sharp} = f + \alpha I_{high}\] \[ I_{sharp} = f + \alpha(f - f*g) \]

(Efros and Kanazawa, 2021)

Image Sharpening Implementation

The following parameters were used to sharpen the Taj image below:

Gauss Filter: {\(ksize =7 , sigma =2\)}
Sharpening factor:{\(\alpha =3\)}

a)Original Image b) Blurred Image

c)High Frequency Image d) Sharp Image

Additional Sharpening Example

The following parameters were used to sharpen the fox image below:

Gauss Filter: {\(ksize =7 , sigma =3\)}
Sharpening factor:{\(\alpha =2\)}

a)Original Image b) Blurred Image

c)High Frequency Image d)Sharp Image

Unsharp Mask Filter Implementation

There’s another sharpening technique called Unsharp Mask Filter defined as:

\[ (1+\alpha)f -\alpha f*g\] (Efros and Kanazawa, 2021)

We implemented the unsharp mask filter on same images from the previous section to compare the differences. There is a small improvement when using the unsharp mask filter that is observed in terms of sharpness. See below for unsharp mask filter results.

The following parameters were used to sharpen the Taj image below:

Gauss Filter: {\(ksize =7 , sigma =2\)}
Sharpening factor:{\(\alpha =2\)}