### CS 194-26 Project 1: Coloring the Prokudin-Gorskii Collection

##### Nick Titterton 26608602, cs194-26-ago, 2018 November 10

#### Overview

If you have multiple images from the same XYZ location, but different rotations, you can combine the images using a projective transform.

#### Calculating Homographies

For a projective transform, given initial coordinates \(x\) and \(y\), we want to calculate matrix \(H\) such that
$$ \begin{bmatrix}
a & b & c \\
d & e & f \\
g & h & 1 \end{bmatrix}
\begin{bmatrix} x \\ y \\ 1 \end{bmatrix} =
\begin{bmatrix} wx' \\ wy' \\ w \end{bmatrix}$$
That gives us the equations
$$ax + by + c = wx'$$
$$dx + ey + f = wy'$$
$$hx + gy + 1 = w$$
\(w\) is not a variable we want to solve for (since it is not an element of the homography), so we convert the above 3 equations into 2 using \(w = hx + gy + 1\)
$$ax + by + c - hxx' - gyx' - x' = 0$$
$$dx + ey + f - hxy' - gyy' - y' = 0$$
We want the bottom right parameter of the matrix to equal 1, so we add a \(1000i = 1000\) constraint (1000 so that it is sufficiently weighted vs the other pixel-valued constraints), and then use least squares on a sufficient number of points (at least 4, I used 6 points each time).

#### Performing the Warp

Once we have a homography \(H\) converting points from image 1 to image 2, we can perform the warp. The pseudocode goes like this:

- Find bounding box by looking at transformed corners of image (after normalizing out depth \(w\))
- Compute interpolation of original image (
`sk.RectBivariateSpline`

)
- Compute \(H^{-1}\)
- For each pixel in the warped image, use \(H^{-1}\) and the interpolation to compute the value

This is very computationally expensive without vectorization. I had a very nicely vectorized Project 4, but haven't vectorized my code for this yet.
#### Creating Mosaics

Once you have a warped image, you can combine the warped image and the image it was warped to to create a mosaic! Here are some examples

There are some issues with the borders here: I used a simple take-the-pixel-by-pixel-max blend, which worked pretty well except for the chairs in the Soda 6th floor image and teh portrait in the pool table image.

#### What I learned

I learned a lot about homographies and transforms: esp. how to solve for a homography using least squares.