Negative Entropy Function and Properties

The negative entropy function in \(\mathbf{R}^n\) is the function \(f : \mathbf{R}^n \rightarrow \mathbf{R}\), with domain the set of vectors with positive components, and values on the domain given by

\[ f(x) = -\sum_{i=1}^n x_i \log x_i . \]

This function is convex.

Proof: Since the function is a sum of functions, each of which depends on one variable not appearing in the others, we just need to check the convexity of the function of one variable

\[ f(\xi) = \left\{ \begin{array}{ll} -\xi \log \xi & \mbox{if } \xi > 0, \\ +\infty & \mbox{otherwise.} \end{array} \right. \]

The convexity of the latter derives directly from the second-order condition:

\[ \frac{d^2}{d\xi^2} f(\xi) = \frac{1}{\xi} > 0 . \]