|
|
|
The goal of this assignment is to take the digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce a color image with as few visual artifacts as possible. In order to do this, I extract the three color channel images, place them on top of each other, and align them so that they form a single RGB color image.
|
|
|
|
I use L2 Norm , Edge Detection and Image Pyramid for image alignment, and implemented the options of Auto Cropping the image according to the shifting in channels alignment as well as improving the image quality with Histogram Equalization algorithm.
The algorithm I use to determine the similarity between two image matrix is Sum of Squared Differences (SSD), also known as L2 norm. The formula is simply sum(sum((image1-image2).^2)) where the sum is taken over the pixel values. The naive way of comparing the channels is basically using raw pixel values as inputs to the SSD function after shifting. But this method does not perform very well since there can be significant differences in an area of the image between the raw matrix of channels. For example, if there’s a large area of green on the RGB picture, then there would be a bunch of relatively high pixel value at the same area on green channel and relatively low value at that area on the other two channels. To solve this issue, I use the filter [-1, 0, 1] along the rows and columns to extract the edge information vertically and horizontally, and with the help of SSD, use the resulting pixel value matrices as reference to find the displacement value of the green and red channels.
1. Before aligning the channels, 8% of area along each edge is cropped in order to eliminate the noise.
2. The filter [-1, 0, 1] is applied to the pixel matrices. The advantage of using this filter over other convolution
kernels is that the result of this filter can be expressed as M[2:,:] - M[:-2,:] for extracting horizontal features and M[:,2:] - M[:, :-2]
for vertial features. Considering the images are represented as numpy arrays, this operations performs excellent on the aspect
of runtime and avoid explict for loops.
3. Use image pyramid to search for the correct displacement between color channels, which significantly narrow the space
of possible displacement values that are searched through. The matrix sizes between adjacent layers have ratio of 2 along the axes. Inside each
layer, I
4. Use the displacement value found in the previous step, green and red channels are shifted relative to blue channel. Image
channels are stacked back together after shifting.
Here is a demonstration of the improvement in alignment when edge detection is used.
Results From emir.tif
|
|
|
|
Results From lady.tif
|
|
|
|
After aligning the image channels, I use Histogram Equalization algorithm to imporve contrast in image. This technique effectively spreading out the most frequent intensity values, i.e. stretching out the intensity range of the image.
1. A histogram of pixel values is made.
2. Cumulative sum along x-axis of the histogram is computed.
3. Re-normalize cumsum values to be between 0-255.
3. Get the values from cumulative sum for every index in flat, and set that as the new image.
Here is a demonstration of the improvement of image quality in contrast.
Results From workshop.tif
|
|
|
|
In the example above, the improvement after Histogram Equalization is quite significant. The original picture lack contrast between the darkness and brightness, which makes it hard to see some details. For instance, the boundaries between bricks on the left side of the image is not shown clealy in the oringinal image, while the result after Histogram Equalization successfully make those details visible.
Because of channel shifting, there are areas near the edges of produced image that are not perfectly covered by all three channels. But using the displacement information calculated in the alignment process, we have clue about which part should be automaticly cropped. For example, for an image, if displacement of the green channel is [-3, 6] and [1, -5] for the red, I take the maximum displacement on every axis and both negatively and positively, which is 1 and -3 for axis 0, 6 and -5 for axis 1. Then I use this information to crop the prodeced image.
Here is a demonstration of the auto image cropping.
Results From harvesters.tif
|
|
In the example above, the displacement of green channel is [60, 15] and displacement of red channel is [123, 12]. Therefor, 123 rows of pixels on the top edge and 15 columns of pixels on the left edge are automatically cropped.