Part 1: Image Warping and Mosaicing
Recovering Homographies
Any two images of the same planar surface in space are related by a homography, which can be represented by a homography matrix.
$$H = \begin{bmatrix} a & b & c \\ d & e & f \\ g & h & 1 \end{bmatrix} $$This method technically only requires four point correspondence pairs to be recovered, but in practice, least square regression on many pairs of points works better. $$\begin{bmatrix} x_{1,1} & y_{1,1} & 1 & 0 & 0 & 0 & -x_{1,1} x_{2,1} & -y_{1,1} x_{2,1} \\ 0 & 0 & 0 & x_{1,1} & y_{1,1} & 1 & -x_{1,1} y_{2,1} & -y_{1,1} y_{2,1} \\ & & & & \vdots & & & \\ x_{1,n} & y_{1,n} & 1 & 0 & 0 & 0 & -x_{1,n} x_{2,n} & -y_{1,n} x_{2,n} \\ 0 & 0 & 0 & x_{1,n} & y_{1,n} & 1 & -x_{1,n} y_{2,n} & -y_{1,n} y_{2,n} \\ \end{bmatrix} \begin{bmatrix} a \\ b \\ c \\ d \\ e \\ f \\ g \\ h \end{bmatrix} = \begin{bmatrix} x_{2,1} \\ y_{2,1} \\ \vdots \\ x_{2,n} \\ y_{2,n} \end{bmatrix}. %]]> $$
Using the equations above, we can calculate the homography matrix. For this part, point correspondences are manually chosen.
Image Rectification
We can now perform (inverse) image warping: for every coordinate value in the output result, multiply by the (inverse) homography matrix, taking care to normalize w to be 1, to recover the coordinates in the original image. One application of a homography is image rectification: given an image containing a planar surface, warp it so that it is frontal-parallel. We simply select 4 points representing the corners of the plane in the original image and choose the correponding points in the output to be the corners of a rectangle. Below we illustrate some examples.
