Time Series Forecasting

Time series forecasting is the task of estimating future values of one or more variables that are observed sequentially over time, conditional on past observations and optionally on additional information such as covariates and known future events. It is one of the oldest applied areas of statistics, with foundations in 20th-century work by Yule, Box, Jenkins, Holt, Winters, and others, and it remains a central activity in commercial demand planning, energy systems operation, financial risk management, public health surveillance, and supply chain management. The field has been transformed in the past decade by the application of machine learning and, more recently, foundation models for time series.

Problem Formulation

A time series is a sequence of observations recorded at regular or irregular time intervals. Forecasting tasks vary along several dimensions: forecast horizon (one step ahead vs many steps), granularity (hourly, daily, monthly, annually), number of series (univariate vs multivariate; single series vs panel of related series), the presence of seasonality and exogenous drivers, and whether the output is a point forecast, an interval, or a full predictive distribution. Probabilistic forecasting, which produces distributions or quantiles rather than single point estimates, has become standard in many domains because downstream decisions depend on quantified uncertainty.

Classical Methods

Exponential Smoothing

Exponential smoothing methods, including simple exponential smoothing, Holt's linear trend method, and the Holt-Winters seasonal method, compute forecasts as weighted averages of past observations with weights that decay exponentially over time. The ETS family — Error, Trend, Seasonal — formalises these methods within a state-space framework that supports likelihood-based estimation and prediction intervals.

ARIMA

The autoregressive integrated moving average (ARIMA) family of models, popularised by Box and Jenkins in 1970, expresses each observation as a linear combination of past observations and past forecast errors after differencing to handle non-stationarity. Seasonal ARIMA (SARIMA) extends the framework with seasonal autoregressive and moving average components. ARIMA models remain a strong baseline and benchmark for many forecasting tasks.

State-Space Models

State-space and structural time series models, including the Kalman filter framework and dynamic linear models, represent time series as latent state variables that evolve according to specified equations. They support principled handling of missing data, multiple seasonalities, regression effects, and combinations of components.

GARCH

Generalised autoregressive conditional heteroskedasticity (GARCH) models, introduced by Engle and Bollerslev, address the time-varying volatility characteristic of financial returns and form a basis for value-at-risk and option pricing applications.

Machine Learning and Deep Learning Methods

Tree-Based Models

Gradient boosting frameworks such as LightGBM, XGBoost, and CatBoost have become widely used for time series forecasting, particularly for retail demand, transportation, and energy applications with large panels of related series. Lag features, rolling statistics, calendar features, and categorical group identifiers are engineered as inputs, with one model trained jointly across all series.

Recurrent and Convolutional Networks

Recurrent neural networks, particularly LSTMs and GRUs, can model long-range dependencies in sequences. Sequence-to-sequence architectures and temporal convolutional networks (TCNs) have been applied to multi-step forecasting.

Transformer-Based Models

Transformer architectures adapted to time series include Informer, Autoformer, FEDformer, PatchTST, and the Temporal Fusion Transformer (TFT). These models support long forecast horizons and incorporation of static and dynamic covariates. The TFT, introduced by Lim et al. in 2019, combines variable selection networks, LSTM encoders, and attention layers for interpretable multi-horizon forecasting.

Foundation Models for Time Series

A new generation of pre-trained foundation models for time series, including TimeGPT by Nixtla, Lag-Llama, Chronos by Amazon Web Services, Moirai by Salesforce Research, and Google's TimesFM, applies the foundation model paradigm to forecasting. These models are pre-trained on large heterogeneous corpora of time series and produce zero-shot or few-shot forecasts on new series without per-series training, lowering the barrier to scalable forecasting.

Evaluation

Common evaluation metrics include mean absolute error (MAE), root mean squared error (RMSE), mean absolute percentage error (MAPE), symmetric MAPE (sMAPE), and continuous ranked probability score (CRPS) for probabilistic forecasts. Evaluation must respect the temporal structure of the data: cross-validation uses rolling-origin or expanding-window schemes rather than random splits, ensuring that training data precedes evaluation data in time. The M-competitions, including the M4 and M5 competitions, have established widely used benchmarks for forecasting accuracy across many domains.

Applications

Time series forecasting supports demand planning in retail and consumer goods, electricity and gas load forecasting in utilities, capacity planning in cloud computing, traffic forecasting in transportation, financial risk modelling, public health surveillance such as influenza and dengue forecasting, predictive maintenance of industrial equipment, water resource management, and climate and weather prediction.

References

1. Hyndman, R. J., and Athanasopoulos, G. (2021). Forecasting: Principles and Practice (3rd ed.). OTexts.
2. Box, G. E. P., Jenkins, G. M., Reinsel, G. C., and Ljung, G. M. (2015). Time Series Analysis: Forecasting and Control (5th ed.). Wiley.
3. Lim, B., Arik, S. O., Loeff, N., and Pfister, T. (2021). Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting. International Journal of Forecasting, 37(4), 1748–1764.
4. Makridakis, S., Spiliotis, E., and Assimakopoulos, V. (2022). The M5 Accuracy Competition: Results, Findings, and Conclusions. International Journal of Forecasting.
5. Department of Statistics Malaysia. (2024). Methodology Reports on Seasonal Adjustment and Nowcasting. Putrajaya: DOSM.