The goal of the project is to reconstruct the RGB image from three grey level glass plate images taken with three filters. The raw images are stacked vertically with misplacements and artifacts. In order to reproduce the original vivid scene based on these grey glass plates, various imaging processing methods are used here to achieve this goal effectively and efficiently. In part 2, comparsions of reconstructed RGB images using SSD and image pyramid methods are demonstrated. All raw images have been appropriately cropped using auto-cropping algorithm which will be covered in detail in part 3 (Extra Credit part). Further modification results such as constrast enhancement using histogram eqalization and edge-based SSD are also shown in part 3.
Reconstructed RGB images based on one-scale SSD use a for loop searching over a fixed window size to find the minimum SSD value between Red/Green and Blue channels. Result images and corresponding offsets are shown in the left column. The image pyramid which find the displacements sequentially from the coarsest scale to finest scale is more effective for large images. Results are shown in the right column with the same window size.
As we can see in the following image-paris, especially for those high-resolution ones, SSD fails to align RBG channels very well but results from Image pyramid method are more perceptually acceptable. The underlying reason lies in the limited size of displacement window ([-15,15] here). The relative displacements between channels usually exceed this window size which inevitablly cause misalignment. The Image pyramid method can alleviates the issue in two aspects. First, the recursive implementation of alignment at different scale effectively widen the preset window size and accumulate the displacement along the way. Second, finding displacements from small to large image scale is equivalent to finding the largest displacemnt in the coarsest way first and then improve it by smaller displacements in a finner manner which is more efficient compared to one-scale SSD. (For larges images, only results from Image Pyramid are shown to reduce file size)
| SSD: | Image pyramid: |
|---|---|
| Cathetral | |
| Offset G:[-2,2], R:[-2,3] | Offset G:[-2,2], R:[-2,3] |
|
|
| Tobolsk | |
| Offset G:[-1,2], R:[-2,3] | Offset G:[-1,2], R:[-2,3] |
|
|
| Monastery | |
| Offset G:[-10,2], R:[-11,2] | Offset G:[-10,2], R:[-11,2] |
|
|
| Emir | |
| Offset G:[-15,15], R:[-15,-15] | Offset G:[-27,24], R:[-48,44] |
|
|
| Harvesters | |
| Offset G:[-10,16], R:[-16,13] | |
|
|
| Icon | |
| Offset G:[0,17], R:[9,23] | |
|
|
| Lady | |
| Offset G:[-21,8], R:[-38,11] | |
|
|
| Onion_church | |
| Offset G:[-27,26], R:[-50,36] | |
|
|
| Self_portrait | |
| Offset G:[-17,29], R:[-17,37] | |
|
|
| Train | |
| Offset G:[-29,5], R:[-57,32] | |
|
|
| Workshop | |
| Offset G:[-2,0], R:[-5,-12] | |
|
|
| Village | |
| Offset G:[-4,12], R:[0,22] | |
|
|
| Three_generations | |
| Offset G:[-24,14], R:[-44,11] | |
|
|
| Melons | |
| Offset G:[-14,10], R:[-13,13] | |
|
| Altar |
| Offset G:[-35,16], R:[-50,17] |
 |
| Judgement |
| Offset G:[-21,13], R:[-29,19] |
 |
| Wooden_church |
| Offset G:[-30,19], R:[-45,39] |
 |
Raw glass plate images are first bounded by a white frame which is probably added during digitalization processes. Then, the dark borders surrounding invididual grey images are the photographic films used by Prokudin Gorskii. Due to the strong constrast and nonegligible width of these borders, they will contribute significantly in the SSD results and cause the main part of the images (inside the borders) to be aligned. In this regard, cropping before alignment is necessary.
The auto cropping method applied here takes two vaiables. The first is the image and the second is an estimated ratio of 'roughness' of the border. In the border-cropping function, edges of the image are detected using Canny method and the outermost one is identified. Being multiplied by input ratio, different kinds of border can be cropped accordingly.
For the raw image, 0.2% is used as the input ratio to effective crop relatively smooth white border. After channel separation, black borders are cropped with 5% roughness ratio which is used for all images in this work. The chosen ratio here works well due to the consistency of all raw data and preserves as much as information near the boundary. Several examples using the same Image pyramid method with and without borders are shown below. Dramatic misalignments can be observed in the RGB images with borders, in constrast to well-aligned images with cropped borders.
| Without cropping: | Auto cropping: |
|---|---|
| Cathetral | |
| Offset G:[1,-1], R:[7,-1] | Offset G:[-2,2], R:[-2,3] |
 |
|
| Emir | |
| Offset G:[-3,7], R:[105,17] | Offset G:[-27,24], R:[-48,44] |
 |
|
| Wooden_chair | |
| Offset G:[12,-1], R:[45,-7] | Offset G:[-30,19], R:[-45,39] |
 |
|
Previously, SSD calculate the difference in terms of intensity of grey images. Alternatively, we can use edge information in the Image pyramid method. Edges are detected using Canny method. Result images are shown below. Compared to intensity based alignment, the calculated displacements are slightly different and the reconstructed images are slightly worse.
| Judgement |
| Offset G:[-19,11], R:[-29,19] |
 |
| Icon |
| Offset G:[-2,16], R:[10,22] |
 |
Histogram equalization is used here to improve the constrast in the images by stretching out the distribution of intensity distribution. Converting from RGB to HSV color space and manipulate v and s channels can also achieve the contrast improvement. Results are shown below.
| Histogram equalization: | |||
| Cathedral | |||
| Before | After | ||
|
 |
||
| Self_portrait | |||
| Before | After | ||
|
 |
||
| HSV space: | |||
| Cathedral | |||
| Before | After | ||
|
 |
||
| Train | |||
| Before | After | ||
|
 |
||
As it can be seen in the "harvesters.tiff", local/overall stretching and rotation of the image might be present. Thus further transformations such as rotation and stretching locally or nonlocally will further improve the quality of reconstructed RGB images.
| Sections from Harvesters |
| Local stretchings |
 |
 |