Part 1: Image Warping and Mosaicing
Overview
In this section, we utilize a homography transformation to rectify images and produce warped versions for image stitching. Here points are manually selected, but we will begin to uncover the secrets of auto-stitching in the next section.
How Do We Recover Homographs?
It's easy - you just call the function we defined for this section in main.py. What lies underneath the delicate numpy calls are significant amount of math.
Notice firstly that any two images on the same surface in space are related by a homography. Specifically, we define a \(3 \times 3\) homography transformation as follows:
This homography transformation has \(8\) degrees of freedom (as the last entry is fixed to be 1) and requires four correspondence pairs. Using only four correspondence pairs can be noisy, so we use more pairs. However, then, the system is overdetermined, and we use least squares to minimize: $$\sum_{i=1}^{n} \lVert H \begin{bmatrix} x_{1,i} \\ y_{1,i} \\ 1 \end{bmatrix} - \begin{bmatrix} wx_{2,i} \\ wy_{2,i} \\ w \end{bmatrix} \rVert_2^2$$
Further algebra gives us that the matrix we solve for, then, is: $$ \begin{bmatrix} x_{2,1} \\ y_{2,1} \\ \vdots \\ x_{2,n} \\ y_{2,n} \end{bmatrix} = \begin{bmatrix} x_{1,1} & y_{1,1} & 1 & 0 & 0 & 0 & -x_{1,1} x_{2,1} & -y_{1,1} x_{2,1} \\ 0 & 0 & 0 & x_{1,1} & y_{1,1} & 1 & -x_{1,1} y_{2,1} & -y_{1,1} y_{2,1} \\ & & & & \vdots & & & \\ x_{1,n} & y_{1,n} & 1 & 0 & 0 & 0 & -x_{1,n} x_{2,n} & -y_{1,n} x_{2,n} \\ 0 & 0 & 0 & x_{1,n} & y_{1,n} & 1 & -x_{1,n} y_{2,n} & -y_{1,n} y_{2,n} \\ \end{bmatrix} \begin{bmatrix} a \\ b \\ c \\ d \\ e \\ f \\ g \\ h \end{bmatrix} $$
Image Rectification
We take images and rectify them. This is done by computing the homography matrix from original image (manually selected points) and a rectangle.
|
|
|
|