Quadratic Approximation of the Log-Sum-Exp Function

Symmetric Matrices > Definitions > Example

As seen here, the log-sum-exp function $mbox{lse} : mathbf{R}^2 rightarrow mathbf{R}$ , with values

$mbox{lse}(x) := log( e^{x_1}+e^{x_2} )$

admits the following gradient and Hessian at a point :

$nabla mbox{lse}(x) = frac{1}{z_1+z_2} left(begin{array}{c} z_1 z_2 end{array} right) , ;; nabla^2 mbox{lse}(x) = frac{z_1z_2}{(z_1+z_2)^2} left(begin{array}{cc} 1 & -1 -1 & 1 end{array} right) , mbox{ where } z_i := e^{x_i}, ;; i=1,2.$

Hence, the quadratic approximation of the log-sum-exp function at a point x=(x_1,x_2) is given by

$begin{array}{rcl} mbox{lse}(x+h) &approx & mbox{lse}(x) + h^Tnabla mbox{lse}(x) + frac{1}{2} h^T nabla^2 mbox{lse}(x) h &=& displaystylembox{lse}(x) + frac{h_1 e^{x_1} + h_2 e^{x_2}}{e^{x_1} + e^{x_2}} + frac{e^{x_1+x_2}}{2(e^{x_1}+e^{x_2})^2} (h_1-h_2)^2 . end{array}$