Time Series Analysis and Forecasting

Stationarity

• A time series is stationary when it has neither a deterministic trend nor a stochastic trend.
• Deterministic trend: Data consistently increases or decreases over time.
• Stochastic trend: Data exhibits a random walk, fluctuating up and down unpredictably.
• Strict Stationarity: The joint distribution of the time series (e.g., y1, y2, …) does not change with time shifts. Difficult to assume in practice.
• Weak Stationarity: Easier to assume. Requires a constant mean and constant variance across time shifts.
• Unit root tests check for weak stationarity without assuming specific distributions.

MA Process (Moving Average)

• Time series generated from a process.
• Pure random process (white noise) is notated as \epsilon_t.
• An MA process of order q follows the equation: yt = \theta0 \epsilont + \theta1 \epsilon{t-1} + \theta2 \epsilon{t-2} + … + \thetaq \epsilon_{t-q}
• y_t is the sum of weighted random shocks over time.
• q represents the number of time periods to go back into the past.
• An MA process is created by short-term shocks affecting a short period of time without persistent trends.
• Series generated are a moving average of shocks across time.
• MA processes possess autocorrelation due to the use of lagged random shock values.
• MA processes are stationary.

Real-World Examples

• Manufacturing (Quality Control):
  • Error in a machine weighing candy packs.
  • Additional weight added to packs due to machine error.
  • Even after fixing the machine, leftover errors can persist in subsequent operations.
  • Tomorrow's packs might contain new errors plus leftover errors from today (MA(1) process).
• Restaurant Sales:
  • Social media influencers featuring restaurants can cause a sales increase.
  • The increase is a random shock (\epsilon_t).
  • The popularity lasts for a short time (e.g., one to two weeks).
  • Sales become the weekly average of the initial shock plus the following weeks affected by the increase (MA(2) process).

Generating Simulated Data (Stata)

• Set up time series structure with 200 observations (e.g., using command gen t = _n).
• Generate white noise (random errors) using rnormal function (e.g., gen e = rnormal(0,1) for standard normal distribution).
• Create lagged values of the error term (e.g., L.e for lag of e) after setting the time series (e.g., tsset t).
• Simulate the MA process (e.g., gen y = e + 0.6*L.e for an MA(1) process).
• Plot the generated process and check autocorrelation function (ACF) and Augmented Dickey-Fuller test.
  • Autocorrelation. The correlation between the current and the logged value of order one.
  • Augmented Dickey-Fuller (ADF) test. Unit roots are rejected (stationarity).

AR Process (Autoregressive Model)

• Time series is a function of its own past values.
• An AR process of lag p follows the equation: yt = \rho1 y{t-1} + \rho2 y{t-2} + … + \rhop y{t-p} + \epsilont where \epsilon_t is the random shock from the same period.
• y_t is a weighted average of its own past values.
• Random shocks from the past are not considered (unlike MA).
• GDP growth rate, inflation rate often follow AR processes.
• Similar to sticky prices in macroeconomics.
• Each value remembers and tries to stick to the previous one, but with some decay.
• Autocorrelation exists.
• Not automatically stationary if it has unit roots.
• Unit roots/random walk process comes from an AR process when \rho=1.

Possible Examples

• Stock Returns: Similar to past values, unless there's a significant shock.
• Inflation Rates: Related to sticky prices; difficult to experience a very high change in price quickly.
• Unemployment Rate: Stays around the natural rate of unemployment.
• Environment: Temperature anomalies can be modeled as AR(1) or AR(2) processes.
• Audio signal.

Generation of AR one (1) Process (Stata)

• Setup time series structure.
• Generate three types of AR(1) processes with low, high, and unit persistence.
• Generate the error term (random shocks).
• Create variables for each AR process with different persistence levels.
• Example. AR(1) process with low persistence:
  • gen e1 = rnormal(0,1)
  • gen y1 = .
  • replace y1 = e1[1] (first value)
  • Loop to generate subsequent values:
  • forvalues i = 2/200 {
  • replace y1[``i''] = 0.3*y1[``i''-1] + e1[``i'']
  • }
• AR(1) with high persistence (\rho=0.95) and random walk (\rho=1) follow similar logic.
• Plot the processes to visualize differences.
• ACF plots show autocorrelation.
• Augmented Dickey-Fuller test confirms stationarity (or non-stationarity) empirically.

ARMA Process

• Combination of AR and MA processes.
• Past values and past random shocks both affect the value today.
• Mathematical representation:
• yt = \rho1 y{t-1} + \rho2 y{t-2} + … + \rhop y{t-p} + \theta0 \epsilont + \theta1 \epsilon{t-1} + \theta2 \epsilon{t-2} + … + \thetaq \epsilon_{t-q}

When are ARMA Models Useful?

• If the series is stationary.
• If stationarity is not well-described by just AR or MA models alone.
• Share prices: Stickiness (AR) and shocks to the company (MA).

ARIMA Process

• A more general version of ARMA.
• Takes non-stationary series into account.
• AutoRegressive Integrated Moving Average.
• Includes autoregressive features, moving average components, and order of integration.
• Notation: ARIMA(p, d, q), where:
  • p = lag order for AR components
  • d = order of integration (number of times differenced to achieve stationarity)
  • q = lag order for MA components
• If the series is nonstationary, difference it until it becomes stationary.
• Then, fit the differenced series with an ARMA model.

Extensions to ARIMA Processes

• ARIMAX: Includes exogenous variables (e.g., interest rates, policy changes).
• SARIMA: Seasonal ARIMA; takes into account seasonal components.
• SARIMAX: Seasonal ARIMA with exogenous variables.

ARIMA Modeling and Forecasting: Box-Jenkins Methodology

• Developed by George Box and G.M. Jenkins in the 1970s.
• A procedural approach for identifying, specifying, estimating, and diagnosing ARIMA models.
• Relies on autocorrelations and partial autocorrelations to determine lag orders.
• Box quote: All models are wrong, but some are useful.
  • Modeling is never be perfect, so aim for parsimony (simpler models).

Process of Statistical Modeling

• Model Identification:
  • Initial Check: Determine stationarity through visual inspection and ADF tests. Apply differencing if needed to get to a stationary level.
  • Specify the differencing (d).
  • Choose p and q using ACF and PACF. AC is for MA terms. PACF is for AR terms
• Estimation: Estimate candidate models based on ACF and PACF:
  • Model downwards.
• Diagnostics.
• Forecasting.

Implementing ARIMA Forecasting in Stata

• Use DTA dataset of Philippine GDP growth rates from 1961 to 2023.
• Step 1: Initial check with tsline to visualize and Augmented Dickey-Fuller test for stationarity.
• Step 2: (If needed) Apply first difference if non-stationary.
• Step 3: Model identification using ACF and PACF to choose p and q.
  • Example: corrgram gdpgr.
• Estimate candidate ARIMA models using the arima command:
  • arima gdpgr, ar(1) ma(1) (ARMA(1,1))
• Save estimated models using estimates save.
• Step 4: Diagnostics (residuals analysis) using estimates use and predict commands.
• Step 5: Forecasting using the chosen model.
  • Ask Stata to add more time periods.
  • predict predicted values for the growth rate.
  • Create a forecasting model.
  • Forecast the estimates for the exiting values, it will creates partially fitted values.
  • Solve for the rest of the years.
• Plot actual vs. forecasted value. A stagnate growth rate will be shown, since stationary series to begin with.