The displacement vector of green color image is: (2, 5)
The displacement vector of red color image is: (3, 12)
The goal is to take the digitized Prokudin-Gorskii glass plate images and, using image processing techniques, automatically produce a color image with as few visual artifacts as possible.
I first read the image, divide it to three different color plates (RGB), and then I crop each of them off fixed dimension.
Next step is that I need to choose a base color plate and align the other two color plates to that base plate. Specifically, I need to find out the displacement vector for the other two color matrices. I exhaustively search over a window of possible displacements (from -15 px to 15px, inclusive). Using the Sum of Squared Differences (SSD) distance (of two color matrices) as the metric, I find th best displacement vector in that window, and it should give the smallest sum. To move the matrix, I use np.roll.
Finally, I shift the other two color matrices and stack all three together. Notice that when I align the color plates, I use the original plates instead of the cropped ones and they give exactly the same result. I crop the final image again at the end so that it looks better.
The colorful images are below with their displacement vectors for red and green plates.
For larger image, I come up with a faster search procedure, which is the image pyramid. Different levels of the image pyramid have different levels of resolutions. They are resized from the original image. Starting from the smallest image at the top, I choose a scale factor of 2 and a search window of size [-15px, 15px] (same as before). Notice that in each level of pyramid image, I only use the center to find displacement vector because it gives better result. If the image is too small, then I use the entire image. After that, I find the best displacement vector for this level, and move on to the next level, I shift the scaled matrix first (with displacement vector also scaled by 2), and find the best displacement vector for the current level. All operations in different levels can be done using recursion with each level using the alignment function defined before.
After finding the final displacement vector (for the bottom level of the pyramid), I do the same thing mentioned before to align three color plates.
The colorful images are below with their displacement vectors for red and green plates.
(Notice that it does not work well for emir.tiff, and what I do for now is just changing the base color to green, which helps a lot)
For automatic contrast, I use skimage.exposure.equalize_hist method, which returns the image after histogram equalization.
For edge detection, I use skimage.filters.scharr method, which returns the edge map containing edge magnitude using the Scharr transform. I can use this new matrix for better alignment.
I need to automatically detects the border which is of color black or white. I calculate the variance of each row and column and throw away the ones that don't meet (less than) the threshhold. Finally, I find the cropping index by choosing the maximum of three color plates for left x and upper y and the minimum of three color plates for right x and lower y.
Below are some examples before and after my extra methods apply.
The images are more vivid and they are closer to the modern images (in my opinion).
