Recovering Homographies

To compute the homography matrix mapping a set of points x y to another x' y, we solve the below equations:

d * x'y' = xy @ H.T, where H[-1, -1] = 1

-AKA-

ax + by + c = dx'

dx + ey + f = dy'

gx + hy + 1 = d

Substituting for d and isolating x' y' terms on one side

ax + by + c = (gx + hy + 1) x'

dx + ey + f = (gx + hy + 1) y'

ax + by + c - gxx' - hyx'= x'

dx + ey + f - gxy' - hyy'= y'

Which can be reqritten as a least-squares problem solving a system of eqns for H (flattened homography matrix)

M @ H = x'y'

Warping the images

Image warping is done by inverse sampling, that is for every pixel in the target image, look up the corresponding source point(s) and billinearly interpolate over them. This requires computing and applying the homography matrix from the target image to the source image.

See below resuts where some tiles and a book are "rectified" to be on a plane paralell to the camera's.

Image Mosaicing

For mosaicing, all images are warped to be aligned with the target image, then blended using multi-frequency blending which we implemented in project 2.

Note how there are slight artifacts due to camera postprocessing / exposure being different in the two frames. It's most visible in the kitchen scene, where the two images have visibly different "warmth" or blue-orange balance.

One thing I learned here is how importaint it is that the constraints of projective alignment are respected. (co-planar images if camera pos varies, or camera can only rotate) If they aren't then nasty artifacts will appear due to the 3d structure of the scene making a valid homography unsolveable.