Overview
Tools like image blending and frequency manipulation are used often at the high level, so its necessary to understand the building blocks behind these tools. Throughout this project, the primary tools that were explored were: 1. Gaussian blurring in order to sharpen images 2. Creating hybrid images from a mix of two differing frequencies 3. Blending two images across a seam using image pyramids or gaussian/laplacian stacks 4. Fusion using the Gradient comparison across each pixel
1.1 Image Sharpening
I started by passing in an image, and computing the gaussian blurring of the image. I did this by using a gaussian kernel mask across the image, which could be computed automatically with a gaussian_filter generator in skimage, but I ended up creating my own, which worked fine. Essentially, the difference between the image and the gaussian blur is effectively an edge detector, which could be added on to the original image in order to enhance the edges, thus giving the perception of sharpening the image. Below, you can see the difference between the original image and the sharpened one.
1.2 Hybrid Images
With hybrid images, the goal was to take two images, and essentially combine them into one such that from far away it would look like one image, but close up it would look like the other. This was done by first aligning the images such that they are relatively aligned by features, trying different gaussian kernels, and then applying them in order to create blurred images like before. The low frequency image is simply the blurred one, and to create the high frequency image, I found the difference between the original and the gaussian-blurred, like an edge detector. Then I took the average of the two images, and rescaled it to the rgb scale, which produced some scary, but effective looking hybrid pictures. Some of these turned out better than others, so for my favorite one I also depicted the frequency analysis throughout the process.
Frequency Analysis
1.3 Gaussian and Laplacian Stacks
Similar to image pyramids, stacks are effectively the same image filtered over and over again. Essentially, the image is going to become blurrier each time, thus creating a gaussian stack. The laplacian stack is similar, except the gaussian is subtracted from the original. The gaussian and laplacian stack is displayed for varying gaussian filters, each with 2x the sigma compared to the previous filter.
1.4 Multiresolution Blending
Multiresolution blending is essentially combining two images across a seam. My initial attempt was to simply take half of one image, and half of another using a mask, and simply copy over the source image onto the other half of the target image. However, the line or seam that separated the two was extremely prominent, thus I opted for an approach where I find a window near the seam, and within that window take a certain percentage from each image, which created a smooth transition for a blending effect. I tested my approach on the apple and orange, and tried some irregular masks as shown below. - Masks were effectively numpy binary arrays where a 1 represented taking from the source image, while a 0 represented taking from the other. - The last image definitely isn't perfect, and is rather creepy, but I learned that images and its features need to be aligned correctly and have similar proportions in order to blend well without resizing.
2.1 Gradient Domain Fusion, Toy Problem
With gradient domain fusion, the idea is to take an image, and insert into another picture smoothly. If you just inserted it in, the boundaries would be off, the colors wouldn't match, and the background would probably not be blended in smoothly. This idea of gradient domain fusion allows us to carve out the source image, and insert it nicely into our target image. The idea is to compare each pixel to its neighbors, essentially creating 4 * pixels number of equations. This can then be mapped to a linear system of equations, which I solved using a sparse matrix A, and the sparse library of scipy.
