{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Problem 7 Image Stitching\n",
    "\n",
    "This section of the notebook continues the image stiching problem. Be sure to have a `figures` folder in the same directory as the notebook. The `figures` folder should contain the files:\n",
    "\n",
    "    Berkeley_banner_1.jpg\n",
    "    Berkeley_banner_2.jpg\n",
    "    stacked_pieces.jpg\n",
    "    lefthalfpic.jpg\n",
    "    righthalfpic.jpg\n",
    "    \n",
    "Note: This structure is present in the provided HW2 zip file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the next block of code before proceeding\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "tags": [
     "worksheet-0"
    ]
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import numpy.matlib\n",
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "from numpy import pi, cos, exp, sin\n",
    "import matplotlib.image as mpimg\n",
    "import matplotlib.transforms as mtransforms\n",
    "\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "#loading images\n",
    "image1=mpimg.imread('figures/Berkeley_banner_1.jpg')\n",
    "image1=image1/255.0\n",
    "image2=mpimg.imread('figures/Berkeley_banner_2.jpg')\n",
    "image2=image2/255.0\n",
    "image_stack=mpimg.imread('figures/stacked_pieces.jpg')\n",
    "image_stack=image_stack/255.0\n",
    "\n",
    "\n",
    "image1_marked=mpimg.imread('figures/lefthalfpic.jpg')\n",
    "image1_marked=image1_marked/255.0\n",
    "image2_marked=mpimg.imread('figures/righthalfpic.jpg')\n",
    "image2_marked=image2_marked/255.0\n",
    "\n",
    "def euclidean_transform_2to1(transform_mat,translation,image,position,LL,UL):\n",
    "    new_position=np.round(transform_mat.dot(position)+translation)\n",
    "    new_position=new_position.astype(int)\n",
    "\n",
    "    \n",
    "    if (new_position>=LL).all() and (new_position<UL).all():\n",
    "        values=image[new_position[0][0],new_position[1][0],:]\n",
    "    else:\n",
    "        values=np.array([2.0,2.0,2.0])\n",
    "    \n",
    "    return values\n",
    "\n",
    "def euclidean_transform_1to2(transform_mat,translation,image,position,LL,UL):\n",
    "    transform_mat_inv=np.linalg.inv(transform_mat)\n",
    "    new_position=np.round(transform_mat_inv.dot(position-translation))\n",
    "    new_position=new_position.astype(int)\n",
    "    \n",
    "    if (new_position>=LL).all() and (new_position<UL).all():\n",
    "        values=image[new_position[0][0],new_position[1][0],:]\n",
    "    else:\n",
    "        values=np.array([2.0,2.0,2.0])\n",
    "    \n",
    "    return values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will stick to a simple example and just consider stitching two images (if you can stitch two pictures, then you could conceivably stitch more by applying the same technique over and over again).\n",
    "\n",
    "Professor Alon decided to take an amazing picture of the Campanile overlooking the bay. Unfortunately, the field of view of his camera was not large enough to capture the entire scene, so he decided to take two pictues and stich them together. \n",
    "\n",
    "The next block will display the two images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "tags": [
     "worksheet-0"
    ]
   },
   "outputs": [],
   "source": [
    "\n",
    "plt.figure(figsize=(20,40))\n",
    "\n",
    "plt.subplot(311)\n",
    "plt.imshow(image1)\n",
    "\n",
    "plt.subplot(312)\n",
    "plt.imshow(image2)\n",
    "\n",
    "plt.subplot(313)\n",
    "plt.imshow(image_stack)\n",
    "\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once you apply Marcela's algorithm on the two images you get the following result (run the next block):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "plt.figure(figsize=(20,30))\n",
    "\n",
    "plt.subplot(211)\n",
    "plt.imshow(image1_marked)\n",
    "\n",
    "plt.subplot(212)\n",
    "plt.imshow(image2_marked)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see Marcela's algorithm was able to find four common points between the two images. These points expressed in the coordinates of the first image and second image are\n",
    "\n",
    "$\\begin{align} \n",
    "&&\\vec{p_1}=\\begin{bmatrix} 200 \\\\ 700 \\end{bmatrix}\n",
    "&&\\vec{p_2}=\\begin{bmatrix} 310 \\\\ 620 \\end{bmatrix} \n",
    "&&\\vec{p_3}=\\begin{bmatrix} 390 \\\\ 660 \\end{bmatrix}\n",
    "&&\\vec{p_4}=\\begin{bmatrix} 460 \\\\ 630 \\end{bmatrix}\\\\\n",
    "&&\\vec{q_1}=\\begin{bmatrix} 162.2976 \\\\ 565.8862 \\end{bmatrix}\n",
    "&&\\vec{q_2}=\\begin{bmatrix} 285.4283 \\\\ 458.7469 \\end{bmatrix} \n",
    "&&\\vec{q_3}=\\begin{bmatrix} 385.2465 \\\\ 498.1973 \\end{bmatrix}\n",
    "&&\\vec{q_4}=\\begin{bmatrix} 465.7892 \\\\ 455.0132 \\end{bmatrix}\n",
    "\\end{align}$\n",
    "\n",
    "It should be noted that in relation to the image the positive x-axis is down and the positive y-axis is right. This will have no bearing as to how you solve the problem, however it helps in interpreting what the numbers mean relative to the image you are seeing.\n",
    "\n",
    "Using the points determine the parameters $R_{11},R_{12},R_{21},R_{22},T_x,T_y$ that map the points from the first image to the points in the second image by solving an appropriate system of equations. Hint: you do not need all the points to recover the parameters. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Note that the following is a general template for solving for 6 unknowns from 6 equations represented as Az = b.\n",
    "# You do not have to use the following code exactly. \n",
    "# All you need to do is to find parameters R_11, R_12, R_21, R_22, T_x, T_y. \n",
    "# If you prefer finding them another way it is fine.\n",
    "\n",
    "# fill in the entries\n",
    "A = np.array([[?,?,?,?,?,?],\n",
    "              [?,?,?,?,?,?],\n",
    "              [?,?,?,?,?,?],\n",
    "              [?,?,?,?,?,?],\n",
    "              [?,?,?,?,?,?],\n",
    "              [?,?,?,?,?,?]])\n",
    "\n",
    "# fill in the entries\n",
    "b = np.array([[?],[?],[?],[?],[?],[?]])\n",
    "\n",
    "A = A.astype(float)\n",
    "b = b.astype(float)\n",
    "\n",
    "# solve the linear system for the coefficiens\n",
    "z = np.linalg.solve(A,b)\n",
    "\n",
    "#Parameters for our transformation\n",
    "R_11 = z[0,0]\n",
    "R_12 = z[1,0]\n",
    "R_21 = z[2,0]\n",
    "R_22 = z[3,0]\n",
    "T_x  = z[4,0]\n",
    "T_y  = z[5,0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Stitch the images using the transformation you found by running the code below.\n",
    "\n",
    "### Note that it takes about 40 seconds for the block to finish running on a modern laptop.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "matrix_transform=np.array([[R_11,R_12],[R_21,R_22]])\n",
    "translation=np.array([T_x,T_y])\n",
    "\n",
    "#Creating image canvas (the image will be constructed on this)\n",
    "num_row,num_col,blah=image1.shape\n",
    "image_rec=1.0*np.ones((int(num_row),int(num_col),3))\n",
    "\n",
    "#Reconstructing the original image\n",
    "\n",
    "LL=np.array([[0],[0]]) #lower limit on image domain\n",
    "UL=np.array([[num_row],[num_col]]) #upper limit on image domain\n",
    "\n",
    "for row in range(0,int(num_row)):\n",
    "    for col in range(0,int(num_col)):\n",
    "        #notice that the position is in terms of x and y, so the c  \n",
    "        position=np.array([[row],[col]])       \n",
    "        if image1[row,col,0] > 0.995 and image1[row,col,1] > 0.995 and image1[row,col,2] > 0.995:\n",
    "            temp = euclidean_transform_2to1(matrix_transform,translation,image2,position,LL,UL)\n",
    "            image_rec[row,col,:] = temp\n",
    "        else:\n",
    "            image_rec[row,col,:] = image1[row,col,:]\n",
    "        \n",
    "\n",
    "plt.figure(figsize=(20,20))\n",
    "plt.imshow(image_rec)\n",
    "plt.axis('on')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  },
  "name": "graphs_for_SOE.ipynb"
 },
 "nbformat": 4,
 "nbformat_minor": 0
}