Auto-regressive models for time series prediction

A popular model for the prediction of time series is based on the so-called auto-regressive (AR) model

$y_t = theta_1 y_{t-1} + ldots + theta_m y_{t-m} , ;; t=1,ldots,m,$

where theta_i 's are constant coefficients, and is the memory length of the model. The interpretation of the model is that the next output is a linear function of the past. Elaborate variants of auto-regressive models are widely used for prediction of time series arising in finance and economics.

To find the coefficient vector theta in $mathbf{R}^m$ , we collect observations $(y_t)_{0 le t le T}$ (with T ge m ) of the time series, and try to minimize the total squared error in the above equation:

$min_{theta} : sum_{t=m}^T (y_t - theta_1 y_{t-1} - ldots - theta_m y_{t-m})^2 .$

This can be expressed as a linear least-squares problem, with appropriate data A,y .