Linear FunctionsVectors > Basics | Scalar product, Norms | Projection on a line | Orthogonalization | Hyperplanes | Linear functions | Application
Linear and affine functionsDefinitionLinear functions are functions which preserve scaling and addition of the input argument. Affine functions are ‘‘linear plus constant’’ functions. Formal definition, linear and affine functions. A function is linear if and only if preserves scaling and addition of its arguments:
A function is affine if and only if the function with values is linear. An alternative characterization of linear functions is given here. Examples: Consider the functions with values
The function is linear; is affine; and is neither. Connection with vectors via the scalar productThe following shows the connection between linear functions and scalar products. Theorem: Representation of affine function via the scalar product:
A function is affine if and only if it can be expressed via a scalar product: for some unique pair , with and , given by , with the -th unit vector in , , and . The function is linear if and only if . The theorem shows that a vector can be seen as a (linear) function from the ‘‘input“ space to the ‘‘output” space . Both points of view (matrices as simple collections of numbers, or as linear functions) are useful. Gradient of an affine functionThe gradient of a function at a point , denoted , is the vector of first derivatives with respect to (see here for a formal definition and examples). When (there is only one input variable), the gradient is simply the derivative. An affine function , with values has a very simple gradient: the constant vector . That is, for an affine function , we have for every : Example: gradient of a linear function. InterpretationsThe interpretation of are as follows.
Example: Beer-Lambert law in absorption spectrometry. First-order approximation of non-linear functionsMany functions are non-linear. A common engineering practice is to approximate a given non-linear map with a linear (or affine) one, by taking derivatives. This is the main reason for linearity to be such an ubiquituous tool in Engineering. One-dimensional caseConsider a function of one variable , and assume it is differentiable everywhere. Then we can approximate the values function at a point near a point as follows: where denotes the derivative of at . Multi-dimensional caseWith more than one variable, we have a similar result. Let us approximate a differentiable function by a linear function , so that and coincide up and including to the first derivatives. The corresponding approximation is called the first-order approximation to at . The approximate function must be of the form where and . Our condition that coincides with up and including to the first derivatives shows that we must have where the gradient, of at . Solving for we obtain the following result: Theorem: First-order expansion of a function.
The first-order approximation of a differentiable function at a point is of the form where is the gradient of at . Example: a linear approximation to a non-linear function. Other sources of linear modelsLinearity can arise from a simple change of variables. This is best illustrated with a specific example. Example: Power laws. |