|
|
|
|
|
|
|
|
|
Prokudin-Gorskii created color photos back in 1907 by taking photographs with different exposures of each scene using red, green, and blue filters. But each of the raw glass negatives lacks proper alignment and optimization for digital display.
We demonstrate a fully automated colorization approach for separating three color components and applying image processing and techniques to align them together and reproduce full-color images.
The colorization program is written in Python 3 and requires a recent version of numpy, scikit-image and opencv-python.
python main.py [edge] [crop]Then this program will generate a new folder and output image into the folder
The goal of single-scale alignment is to find a displacement for green and red components over blue components.
The first step is removing 20 pixels length of margin so that the borders are removed. Then we go through the image to find a best displacement vector. Here we use Sum of Squared Differences(SSD), which can be defined as below. The lower the ssd, the better the alignment.
Once the alignment is found, we return a new image as a final result. Some of them is presented below.
| Raw Image | Output Image |
|---|---|
|
|
|
|
|
|
However, when we apply the single-scale on large image, such as
emir.tif, we find it takes a long time to complete the
alignment. Therefore, we propose pyramid alignment below to solve this
problem.
An image pyramid represents the image at multiple scales (usually scaled by a factor of 2) and the processing is done sequentially starting from the coarsest scale (smallest image). We found that it is really quick to find a displacement within (-15,+15) when the image scale is 400*400.
We first remove 400 pixels length of margin from the image. Then we rescale it into a half size. When the size of image is less than , we use single-scale alignment method on it and get a displacement. Then we return the displacement back to larger image recursively.
In this way, we can get a displacement from a large image more quickly, which also reaches a high accuracy of alignment. Some of the result is listed below.
| Raw Image | Output Image |
|---|---|
|
|
|
|
|
|
However, there is a problem with emir.tif . Though it can find a displacement within 10s, the displacement is not so accurate. This is because the images to be matched do not actually have the same brightness values. So we need to select other features to solve this problem.
We add edge detection as an additional step in image pre-procession stage. We first use GaussianBlur on it and then we use Canny filter to get the edge image.
We use pyramid alignment on the edge image, then we get a better alignment of emir.tif.
|
|
|---|---|
In order to get a better view of image, we enhance the the contrast of image by Gamma Correction with a power of 1.2. This provided a slight but noticeable increase in contrast without producing artifacts on the image.
This is the most creative part of our project. We found that the margin is quite different from the image and search some method to detect it. The we found Hough Transform, which in its simplest form is a method to detect straight lines [1].
We use Probabilistic Hough Transform to detect the boundaries in approximately 10% of the image margin. The line in this area if part of cropping if it's vertical or horizontal. One of the example is presented below.
This method can crop the image automatically. But there are some images over-cropped because of some sepecific properties of the image.
| Before Cropping and Contrast Enhance | After Cropping and Contrast Enhance |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|---|---|---|
| G[48, 23], R [74, 50] | G[48, 8], R[112, 10] | G [49, 17], R[108, 13] |
|
|
|
| G[5, 2], R[12, 3] | G[-3, 2], R[3, 2] | G[3, 3], R[6, 3] |
|
|
|
| G[60, 18], R[124, 17] | G[40, 18], R[89, 24] | G[86, 0], R[179, 12] |
|
|
|
| G[50, 26], R[108, 36] | G[77, 29], R[175, 37] | G[42, 6], R[85, 32] |
|
|
|
| G[64, 12], R[136, 23] | G[52, 0], R[105, -12] |
|
|
|---|---|
| G[21, 36], R[67, 49] | G[29, -9], R[108, -19] |
|
|
| G[-13, 10], R[13, 20] | G[55, 15], R[125, 24] |