I began this project by taking an image and sharpening it using a Gaussian kernel which would then be filtered over the image to produce a blurrier image. The high frequencies are extracted by taking the difference between the original and blurrier image. To get the final sharpened image, the high frequency image is multiplied by an arbitrary scalar amount, alpha, and then added back to the original, which is summed up in the following formula:
The two photos above were taken by me. :)
Next was to create a hybrid image, which is an image that can be seen differently based on the
viewer's proximity to the image. At a close range, the high frequencies of an image will dominate but at a
further distance, the low frequencies of another image will be seen.
To get the hybrid image, I had to low-pass filter one image and then high-pass filter a second image.
I used the techniques in the sharpening image warmup to create the low-pass filter (which uses a Gaussian kernel and
a filter function to blur the entire image). The high-pass filter builds off my low-pass filter which passes the image
through the low-pass filter and then subtracts the filtered image from original. The cut-off frequency, sigma,
determines how many low frequencies or high frequencies are let through (in other words how blurry or detailed an image is)
and was chosen with much experimentation.
Here from close up and far away, we don't see just a corgi or just a tiger but an actual hybrid with a corgi's head and a tiger's body. This is because the tiger has a lot of high frequencies which overpower the corgi's overall low frequency image. A simple switch to low-pass filter the corgi and high-pass filter the tiger results in a more successful hybrid.
This is my favorite hybrid result followed by the frequency analysis using a Fourier transformation.
In this part, I implemented Gaussian and Laplacian stacks. To create the Gaussian stacks, I applied the low-pass filter to an image N amount of times, and to create the Laplacian stacks, I took the difference between the x and x+1 levels of the Gaussian stack. The last image in the Laplacian stack is the last image of the Gaussian stack (the blurriest image).
Here I aimed to seamlessly blend two images together using a multiresolution blending technique which takes advantage of the use of an image spline, which is a seam that joins two images together such that one image smoothly transforms into another image. The spline is computed for both images using the multiresolution blending technique at each level of frequencies so that when the images are stacked together, we get a seamless transition.
Using color definitely enhances the effect. After separating the images and masks into the three color channels, I performed my algorithm over each channel then stacked them together. Blending in color allows us to better see the cool transitions between one image and another (especially for my favorite result, Neptars).
All the previous parts have led us here to gradient-domain processing which is another technique used for blending.
These last parts of the project focus on "poisson blending" which uses gradients and specified pixels intensities to
blend an object from a source image into a target image as if the object was in the environment of the target image
in the first place.
Formulated as a least squares problem depicted below, our objective was to solve for the new pixel values "v" given the
pixel intensities of the source image "s" and the target image "t."
To figure out all the math involved, I started off with the toy example. My implementation for this example computes the x and y gradients of every pixel in a source image s. Then I combined all of the gradients plus one pixel intensity to reconstruct the original image v.
Finally, I attempted to extend my implementation in the Toy Problem to more complex images, i.e. using a mask to seamlessly blend an object from image s into the target image t. One problem I immediately ran into was runtime. My application kept on crashing because the size of my input images were massive compared to the image used in the previous part. This was solved by vectorizing my code using scipy.sparse.csr_matrix(data, (row, col)) which also required a lot of experimentation to find out how to maximize the function's use. However, I was unable to successfully blend an image into the target background. The source object turned out dark in the target image.
Although I was unable to successfully create a blended image, I did have some thoughts on which blending techniques would work best. If one would like to keep the integrity of an image, such as the color of an object, Laplacian blending would be the best approach as this technique smooths out the edges. Using my Neptars hybrid image as an example, Laplacian blending would keep this seamless transition whereas I imagine Poisson blending would cause Mars to take on a more blueish hue to accomodate the target image which is Neptune. Thus, if the desired result is to have the object blend with the background, Poisson blending is the best approach.
This project was very fun using different methods to blend two images together that produce different results. What I liked most is that this project focused on experimentation and interacting directly with the images. There is no right way to code, and I was able to explore different Python libraries. One thing I learned on a practical level is how to vectorize code in Python which proved to be very helpful in what I could get done in Part 2.2. However, the amount of experimentation and finding better approaches proved to be very time consuming and difficult. Nevertheless, it was interesting to see what can be done when playing around with frequencies of an image.