Colorizing the Prokudin-Gorskii Photo Collection

Nadia Hyder

OVERVIEW

In this project, I produced color images from the Prokudin-Gorskii glass plate images by dividing the glass plate images into their three color channel images: B, G, and R. These channels are then aligned and stitched together to create a single RGB color image with as few visual artifacts as possible. Here is an example of what my code accomplishes:

 

blue channel (input)

A bench in front of a building

Description automatically generated

green channel (input)

A picture containing outdoor, photo, building, train

Description automatically generated

red channel (input)

An old photo of a person

Description automatically generated

colored image (output)

A wooden bench

Description automatically generated

 

 

METHODOLOGY

INITIAL APPROACH

My initial logic for aligning these 3 channels was to:

1.     Find the best displacement between the green and blue channel over a [-15,15] window using some image matching metric (I used sum of square distances, as the computation was faster than L2 norm)  and align the green channel relative to the blue channel

a.     the best displacement is defined as that which gives the minimum possible sum of square distances between pixel values between the two channels

2.     Find the best displacement between the red channel and the aligned green channel over a [-15,15] window and align the red channel relative to the aligned green channel.

However, this exhaustive search only worked well for smaller images (.jpg files) but was neither efficient nor thorough enough for larger images.

 

IMPROVEMENTS

1.    To (somewhat) ignore the image borders, I manually cropped the images to account for only the center 85% of the image.

2.    To more efficiently and accurately calculate the appropriate displacement, I implemented an image pyramid.
I sequentially calculated the appropriate displacement over a [-15, 15] window over multiple scales (scaled by a factor of 2)– scaling from the coarsest image to the actual image size, aligning the actual image using the pixel displacement scaled to the appropriate factor relating the smaller image and the actual image.  This significantly reduced runtime and produced good alignments, even for the Emir of Bukhara.

3.    (Bells and Whistles) To align well regardless of color channels and RGB similarity, I aligned the images using canny edge detection (reference: https://scikit-image.org/docs/dev/auto_examples/edges/plot_canny.html).
Comparing the edges rather than pixel values allows for more accurate alignment, and the images matched do not need to have the same brightness values. Though the difference is just noticeable, alignment using edge detection worked better than alignment using raw pixel values.

 

The difference after applying canny edge detection is just noticeable. Comparing the appearance of the eyes in these two pictures eyes reveals edge detection worked better– there is no redness in the eyes in the image on the right, the eyes are well defined because the image is better aligned.

Emir of Bukhara using SSD on pixel values

A close up of a person wearing a hat

Description automatically generated

Emir of Bukhara using SSD on edges

A close up of a person wearing a hat

Description automatically generated

 

 

 

OUTCOMES & OFFSETS

 

 

stitching without alignment

 

alignment using SSD on raw pixel values

 

alignment using SSD on edges

monastery.jpg

 

A long bridge over a body of water

Description automatically generated

 

A bridge over a body of water

Description automatically generated

 

Green dx: 2, dy: -2

Red dx: 2, dy: 4

A bridge over a body of water with a city in the background

Description automatically generated

 

Green dx: 2, dy: -4

Red dx: 4, dy: 2

tobolsk.jpg

 

A harbor with a boat in a body of water

Description automatically generated

 

A harbor with a boat in the water

Description automatically generated

 

Green dx: 2, dy: 2

Red dx: 2, dy: 6

A harbor with a boat in the water

Description automatically generated

 

Green dx: 2, dy: 2

Red dx: 2, dy: 6

cathedral.jpg

 

A close up of a colorful sunset

Description automatically generated

 

A large green field

Description automatically generated

 

Green dx:2, dy: 4

Red dx: 2, dy: 10

A large green field

Description automatically generated

 

Green dx:2, dy: 4

Red dx: 2, dy: 10

workshop.tif

 

A blurry image of a living room

Description automatically generated

 

A dining room table

Description automatically generated

 

Green dx: 0, dy: 52

Red dx: -10, dy: 104

A dining room table

Description automatically generated

 

Green dx: -2, dy: 52

Red dx: -12, dy: 104

emir.tif

 

A person standing in front of a door

Description automatically generated

 

A person standing in front of a building

Description automatically generated

 

Green dx: 24, dy: 48

Red dx: 40, dy: 48

A person standing in front of a building

Description automatically generated

 

Green dx: 24, dy: 40

Red dx: 40, dy: 108

three_generations.tif

 

A group of people standing in a room

Description automatically generated

 

A group of people posing for a picture

Description automatically generated

 

Green dx: 14, dy: 54

Red dx: 12, dy: 112

A group of people standing in front of a building

Description automatically generated

 

Green dx: 18, dy: 56

Red dx: 12, dy: 116

castle.tif

 

A picture containing water, blurry

Description automatically generated

 

A wooden bench

Description automatically generated

 

Green dx: 2, dy: 34

Red dx: 2, dy: 98

A wooden bench

Description automatically generated

 

Green dx: 4, dy: 36

Red dx: 4, dy: 100

melons.tif

 

A picture containing colorful, colored, kite, colors

Description automatically generated

 

A store filled with lots of fresh produce

Description automatically generated

 

Green dx: 10, dy: 82

Red dx: 14, dy: 178

A store filled with lots of fresh produce

Description automatically generated

 

Green dx: 10, dy: 80

Red dx: 12, dy: 176

onion_church.tif

 

A colorful building in the background

Description automatically generated

 

A large building with a green field

Description automatically generated

 

Green dx: 26, dy: 52

Red dx: 36, dy: 108

A large building with a green field

Description automatically generated

 

Green dx: 28, dy: 52

Red dx: 36, dy: 108

train.tif

 

A train engine carrying carts down a track

Description automatically generated

 

A person sitting on a train track

Description automatically generated

 

Green dx: 4, dy: 42

Red dx: 30, dy: 86

A train on a steel track

Description automatically generated

 

Green dx: 8, dy: 40

Red dx: 34, dy: 82

icon.tif

 

A colorful building in the background

Description automatically generated

 

A picture containing building, altar, table, clock

Description automatically generated

 

Green dx: 16, dy: 40

Red dx: 22, dy: 88

A picture containing building, indoor, table, clock

Description automatically generated

 

Green dx: 16, dy: 42

Red dx: 22, dy: 90

self_portrait.tif

 

A picture containing colorful, person

Description automatically generated

 

A picture containing outdoor, grass, mountain, rock

Description automatically generated

 

Green dx: 28, dy: 78

Red dx: 36, dy: 176

A group of people walking down a rocky mountain

Description automatically generated

 

Green dx: 30, dy: 80

Red dx: 38, dy: 178

harvesters.tif

 

A blurry image of a grass covered field

Description automatically generated

 

A group of people standing in a garden

Description automatically generated

 

Green dx: 16, dy: 60

Red dx: 14, dy: 124

A group of people standing in a garden

Description automatically generated

 

Green dx: 16, dy: 60

Red dx: 14, dy: 124

lady.tif

 

A blurry image of a person

Description automatically generated

 

A person sitting in a room

Description automatically generated

 

Green dx: 8, dy: 48

Red dx: 12, dy: 110

 

A person sitting in a room

Description automatically generated

 

Green dx: 8, dy: 56

Red dx: 12, dy: 118

 

 

 

 

ADDITIONAL IMAGES

Aligned using canny edge detection

 

A rainbow over a field of green grass

Description automatically generated

 

Loparev cardboard factory

Green dx: 2, dy:24

Red dx: -6, dy: 92

 

A close up of a rock

Description automatically generated

 

Exit from the yard of the Church of Saint George

Green dx: 24, dy: 72

Red dx:38, dy: 154

 

 

A boat is docked next to a body of water

Description automatically generated

 

Fishing settlement

Green dx: -32, dy: 8

Red dx: -70, dy: 30

 

 

A castle like building with a mountain in the background

Description automatically generated

 

On the island of Capri

Green dx: -16, dy: 32

Red dx: -24, dy: 78

 

A picture containing outdoor, water, plane, flying

Description automatically generated

 

Detail of the Por-Porog waterfall

Green dx: -10, dy: 34

Red dx: -18, dy: 122

 

 

 

-