### CS 194-26 Project 1: Coloring the Prokudin-Gorskii Collection

#### Overview

If you have multiple images from the same XYZ location, but different rotations, you can combine the images using a projective transform.

#### Calculating Homographies

For a projective transform, given initial coordinates $$x$$ and $$y$$, we want to calculate matrix $$H$$ such that $$\begin{bmatrix} a & b & c \\ d & e & f \\ g & h & 1 \end{bmatrix} \begin{bmatrix} x \\ y \\ 1 \end{bmatrix} = \begin{bmatrix} wx' \\ wy' \\ w \end{bmatrix}$$ That gives us the equations $$ax + by + c = wx'$$ $$dx + ey + f = wy'$$ $$hx + gy + 1 = w$$ $$w$$ is not a variable we want to solve for (since it is not an element of the homography), so we convert the above 3 equations into 2 using $$w = hx + gy + 1$$ $$ax + by + c - hxx' - gyx' - x' = 0$$ $$dx + ey + f - hxy' - gyy' - y' = 0$$ We want the bottom right parameter of the matrix to equal 1, so we add a $$1000i = 1000$$ constraint (1000 so that it is sufficiently weighted vs the other pixel-valued constraints), and then use least squares on a sufficient number of points (at least 4, I used 6 points each time).

#### Performing the Warp

Once we have a homography $$H$$ converting points from image 1 to image 2, we can perform the warp. The pseudocode goes like this:

1. Find bounding box by looking at transformed corners of image (after normalizing out depth $$w$$)
2. Compute interpolation of original image (sk.RectBivariateSpline)
3. Compute $$H^{-1}$$
4. For each pixel in the warped image, use $$H^{-1}$$ and the interpolation to compute the value
This is very computationally expensive without vectorization. I had a very nicely vectorized Project 4, but haven't vectorized my code for this yet.

#### Creating Mosaics

Once you have a warped image, you can combine the warped image and the image it was warped to to create a mosaic! Here are some examples

There are some issues with the borders here: I used a simple take-the-pixel-by-pixel-max blend, which worked pretty well except for the chairs in the Soda 6th floor image and teh portrait in the pool table image.

#### What I learned

I learned a lot about homographies and transforms: esp. how to solve for a homography using least squares.