← Back to Model Library

AR Model

Autoregressive Model

Overview

The Autoregressive (AR) model is one of the simplest and most fundamental models in time series analysis. It forecasts future values based on a linear combination of its own past values. The core idea is that the current value of a time series can be explained by a weighted sum of its previous values plus a random error term. AR models are a component of more complex models like ARIMA.

Architecture & Components

An AR model is defined by a single parameter, p, which is the order of the autoregression. This parameter indicates the number of lagged (past) observations that are included in the model.

Mathematical Formulation

An AR(p) model is mathematically expressed as:

$ Y_t = c + \phi_1 Y_{t-1} + \phi_2 Y_{t-2} + ... + \phi_p Y_{t-p} + \epsilon_t $

Where:

  • $Y_t$ is the value of the time series at time $t$.
  • $c$ is a constant (intercept).
  • $\phi_1, \phi_2, ..., \phi_p$ are the autoregressive coefficients, representing the weights of the past observations.
  • $Y_{t-1}, Y_{t-2}, ..., Y_{t-p}$ are the past observations (lagged values) up to order $p$.
  • $\epsilon_t$ is the white noise error term at time $t$, assumed to be independently and identically distributed with a mean of zero and constant variance.

For an AR(1) model, the current value depends only on the immediately preceding value: $Y_t = c + \phi_1 Y_{t-1} + \epsilon_t$.

When to Use AR Models

AR models are suitable for:

  • Time series data that is stationary (mean, variance, and autocorrelation are constant over time). If the data is non-stationary, differencing is usually applied first.
  • Data where the current value is linearly dependent on a few recent past values.
  • As a simple baseline model for time series forecasting.
  • When interpretability is important, as the coefficients directly show the influence of past values.

Pros and Cons

Pros

  • Simple & Interpretable: Easy to understand how past values influence current predictions.
  • Foundation for Complex Models: Forms the basis for ARIMA and SARIMA models.
  • Efficient to Fit: Relatively quick to estimate parameters, especially for lower orders.
  • Provides Confidence Intervals: Allows for quantification of forecast uncertainty.

Cons

  • Assumes Stationarity: Requires the time series to be stationary, or to be made stationary through transformations.
  • Does Not Handle Trend or Seasonality Directly: Cannot directly model trends or seasonal patterns; these must be removed or accounted for before applying AR.
  • Limited to Linear Relationships: Struggles with non-linear dependencies in the data.
  • Sensitive to Outliers: Can be sensitive to extreme values in the past data.

Example Implementation

Here's an example of implementing an AR model using the `statsmodels` library in Python. We'll generate a simple autoregressive time series and then fit the model.


# Import necessary libraries
import pandas as pd
import numpy as np
from statsmodels.tsa.ar_model import AutoReg
import matplotlib.pyplot as plt

# 1. Generate sample data (AR(1) process)
# Y_t = 0.7 * Y_{t-1} + epsilon_t
np.random.seed(42)
n_samples = 100
data = np.zeros(n_samples)
data[0] = 10 # Initial value
for i in range(1, n_samples):
    data[i] = 0.7 * data[i-1] + np.random.normal(0, 1, 1) # AR(1) with noise

series = pd.Series(data, index=pd.date_range(start='2020-01-01', periods=n_samples, freq='D'))

# 2. Split data into train and test
train_size = 80
train, test = series[0:train_size], series[train_size:n_samples]

# 3. Fit the AR model
# lags=5 means we are fitting an AR(5) model
model = AutoReg(train, lags=5)
model_fit = model.fit()

# 4. Make a forecast
forecast_steps = len(test)
# The predict method can take start and end indices
forecast = model_fit.predict(start=len(train), end=len(train) + forecast_steps - 1)

# 5. Display the forecast (conceptual, actual plotting requires matplotlib setup)
print("AR Model Summary:")
print(model_fit.summary())
print("\nAR Model Forecast:")
print(forecast)

# Example plotting (uncomment and run in a Python environment with matplotlib)
# plt.figure(figsize=(12, 6))
# plt.plot(train.index, train, label='Training Data')
# plt.plot(test.index, test, label='Actual Data', color='orange')
# plt.plot(forecast.index, forecast, label='AR Forecast', color='green', linestyle='--')
# plt.title('AR(5) Model Forecast')
# plt.xlabel('Date')
# plt.ylabel('Value')
# plt.legend()
# plt.grid(True)
# plt.show()

Dependencies & Resources

Dependencies: pandas, numpy, statsmodels, matplotlib (for plotting).