Romil Bhardwaj
The partial derivative in x and y computed as the convolution of the finite difference operator with an input image. Here's the input and the x derivative and y derivative:
After computing the partial derivatives, the magnitude of the gradient can be computed as L2 distance of the x and y derivatives (ie., sqrt(dx^2 + dy^2)).
To extract edges from this gradient magnitude image, we must threshold this image. After some fine-tuning, I used a threshold of 0.28 which gave me the following result:
To remove the noise from these gradients and get more robust edges, we can use Derivative of Gaussian (DoG) filters. Gaussian filtering blurs the image, removing noise, and then we can use the finite difference operator to find edges. We observe that the edges are thicker and stronger. Here's the result:
In order to reduce convolution operations, we can combine the two operations in a single derivative of gaussian filter. Here's a visualization of the X and Y DoG filters:
On convolving with these filters, we get the same result as with separate convolutions.
To straighten an image, we can compute the angle between the x and y derivatives of the images at each pixel using np.arctan2(dy/dx). This will yield the orientations at each pixel, which can then be histogrammed. Since right angles are considered to be "straight", we try different orientations which maximize the number of elements in the -90, +90, -180, +180 buckets. This is done by computing the histogram with np.hist(), and then comparing the histogram vector with an ideal vector (which has density 1 at -90, +90, -180 and +180) by taking their dot product. This gives us a score which we can use to rank different rotations. Here are the results:
Failure case - this running track at MLK Jr. School in Berkeley has curving lines on the track and a foggy horizon which makes it hard for our algorithm to find edges to straighten.
An image can be sharpened by boosting the higher frequencies. To get the higher frequencies, we can subtract a blurred image from the original image. This can be combined into a single unsharp filter using a weighted combination of unit impulse function and gaussian filter. The results below show the effect of the unsharp filter on sharpening an image and what happens if you blur an image and try to sharpen it again (spoiler alert: blurring is lossy, thus resharpening doesn't help and the image loses detail).
A hybrid image is a combination of two images which is interpreted differently by humans at different viewing distances. This effect is achieved by combining high frequencies from image with lower frequencies from another image. At closer distances, higher frequencies are prominent, while at larger distances lower frequencies dominate our interpretation of the image. Here are some hybrid images:
"Hello There." "General Palpatine!"
Raw Input Darth Vader Image | LPF Filtered Darth Vader Image - Higher frequencies are removed and lower frequencies are allowed |
Raw Input Luke Skywalker Image | HPF Filtered Luke Skywalker Image - Higher frequencies are allowed and lower frequencies are removed |
I implemented Gaussian stacking by repeatedly blurring an image with a kernel of size 15 and sigma = 2. Here are the gaussian and Laplacian stacks for Salvador Dali's "Lincoln in Dalivision" and Luke Vader image from the previous part. As get deeper into the gaussian stack, the original low frequency image emerges more clearly. In the laplacian stack, we can see the higher frequency image clearly in the first few images (eg. You can see Luke skywalker more clearly at depth=1).
Multiresolution blending seamlessly blends two images by using laplacian and gaussian stacks to create a smooth transition with a mask. Here are the results: