Proj 4 Part 1 - stitching panoramas

Jerry Lai

Final results first.

Part 1.1 - Lobby pictures

Pre-stitching pictures

First, I took some pictures of a lobby of my apartment.

Getting the points

Then, I used the ginput thing to select the points.

To avoid confusion, all the points will be stored in dictionaries.

So the lobby points will be in the lobby_points dictionary. The keys will be like:

{'1to2': points on image one that are meant to match with image 2,

'2to1': points on image two that are meant to match with image 1, etc.}

From running it repeatedly:

lobby_points = {
    '0to1': 
       [1385.37530635,  335.60785382],
       [1568.81359071,  339.27661951],
       [1385.37530635,  560.62548264],
       [1570.0365126 ,  564.29424832],
       [1386.59822824,  698.81565685],
       [1568.81359071,  700.03857875],
    '1to0': 
       [115.98237858, 339.27661951],
       [295.75189725, 364.95797932],
       [114.75945669, 574.07762349],
       [295.75189725, 582.63807676],
       [115.98237858, 717.15948529],
       [295.75189725, 711.04487581],
    '1to2': 
       [1299.02652789,  399.3344027 ],
       [1577.02093004,  370.3411215 ],
       [1304.14298928,  762.60316134],
       [1582.13739143,  779.65803264],
       [1321.19786058,  805.24033958],
       [1483.2191379 ,  912.68602876],
    '2to1': 
       [143.2661081 , 355.01795318],
       [413.95793769, 356.42050152],
       [141.86355976, 735.10855328],
       [411.15284101, 737.91364996],
       [157.2915915 , 775.78245514],
       [311.57190888, 865.54554889],
}

Projective transformation from $n$ points?

$$\begin{bmatrix} wx'\\ wy'\\ w \end{bmatrix} = \begin{bmatrix} a &b &c \\ d &e &f \\ g &h &1 \end{bmatrix} \begin{bmatrix} x\\ y\\ 1 \end{bmatrix}$$

To phrase this into equations:

$$x'w = ax + by + c$$$$y'w = dx + ey + f$$$$w = gx + hy + 1$$

However, we need to remove $w$ from this set of equations, since we do not know the value of $w$.

So:

$$w - (gx + hy) = 1$$$$w - 1 = (gx + hy)$$$$x' = x'w - x'(w-1)$$$$x' = x'w - x'(gx + hy)$$$$x' = ax + by + c - gx'x - hx'y$$

By the same logic, $$y' = dx + ey + f - gxy'x - hy'y$$

This allows us to use the following system of equations:

$$ \begin{bmatrix} x'_0\\ y'_0\\ x'_1\\ y'_1\\ \vdots\\ x'_n\\ y'_n \end{bmatrix} = \begin{bmatrix} x_0 & y_0 & 1 & 0 & 0 & 0 & -x'_0x_0 & -x'_0y_0\\ 0 & 0 & 0 & x_0 & y_0 & 1 & -y'_0x_0 & -y'_0y_0\\ x_1 & y_1 & 1 & 0 & 0 & 0 & -x'_1x_1 & -x'_1y_1\\ 0 & 0 & 0 & x_1 & y_1 & 1 & -y'_1x_1 & -y'_1y_1\\ &&&&\vdots&&& \\ x_n & y_n & 1 & 0 & 0 & 0 & -x'_nx_n & -x'_ny_n\\ 0 & 0 & 0 & x_n & y_n & 1 & -y'_nx_n & -y'_ny_n \end{bmatrix} \begin{bmatrix} a\\ b\\ c\\ d\\ e\\ f\\ g\\ h \end{bmatrix} $$

Warped pictures of the lobby

Note the orange and blue dots on each picture. They are the centroids of their respective sets of points. They represent a point where each pair of images are supposed to align to.

Positioning the images correctly, based on anchors

The orange and blue dots serve as anchors for when we position the images against each other.

Using the positions of those dots with respect to the size of each image, I calculated the total size of the mosaic. Then, using those positions again, I pasted the warped image onto the mosaic in the correctly aligned positions.

Seperate low and high frequencies

I used a gaussian filter (sigma = 20) to get the blurry components of each image then subtracted it from the original to get the sharp components.

Blend using weighted averaging for low freq, hard border for high freq

For the blurry components, I used a linear alpha along the overlaps between each pair of pictures.

For the sharp components, I just picked an x value. Then, I just put one image on the left, and one image on the right, with no averaging.

Part 1.2 - Roof pictures

My next set of pictures were taken from the roof of the building where I live. It's of the area north of downtown berkeley

Original pictures:

Marked points:

Warped images:

Positioning the images

Blending the low and high frequencies