As mentioned in previous post, I have been working with Autoregressive and Moving Average simulations.
To test the correctness of estimations by our simulations, we employ acf(Autocorrelation) and pacf(partial autocorrelation) to our use. For different order of AR and MA, we get the varying visualizations with them, such as:
- Exponential decreasing curves.
- Damped sine waves.
- Positive and negative spikes, etc.
While analyzing and writing tests for same, I also took some time to visualize that data on ilne and bar charts to get a clearer picture:
AR(1) process is the autoregressive simulation with order p = 1, i.e, with one value of phi.
Ideal AR(p) process is represented by:
To simulate this, install
statsample-timeseries from here.
1 2 3 4
Here, number of observations, n = 1500 (greater value is preferrable for best fit), p = 1, with phi = [0.9].
To generate it’s autocorrelation
For an AR(1) process,
acf must exponentially decay if
phi > 0, or alternate in sign if
phi < 0Ref. Go through the analysis above. It can be visualized as:
phi > 0, acf decreases exponentially:
phi < 0, you get the alternate acf lags:
To generate it’s partial autocorrelation:
For AR(1) process,
pacf must have a spike at lag 1, then 0. Former spike must be positive if
phi > 0, otherwise, negative spike. Have a look at the pacf series generated above. On visualizing the data:
phi > 0, positive lag at 1 and 0(contains 1.0):
phi < 0, negative lag at 1:
Here is the representation of ideal acf-vs-pacf for positive phi in AR(1):
Simulation of AR(p) process is similar as AR(1).
1 2 3 4 5 6
acf must give a damping sine wave. The pattern is greatly dependent on the value and sign of phi parameters.
When positive content in phi coefficients is more, you will get a sine wave starting from positive side, else, sine wave will start from negative side.
Notice, the damping sine wave starting from positive side here:
and negative side here..
pacf gives spike at lag 0(value = 1.0, default) and from lag 1 to lag k. The example above, features AR(2) process, for this, we must get spikes at lag 1 - 2 as:
MA(1) process is the moving average simulation with order
q = 1, i.e, with one value of
To simulate this, use
ma_sim method from
1 2 3 4 5 6
theta > 0, for
MA(1), we must get a positive spike at lag 1 as:
theta < 0, the spike at lag 1 must be in negatie direction as:
When I put these two visualizations aside each other, the visualization seems quite fit:
Order = q => Number of theta coefficients = q.
Ideal MA(q) process is represented by:
Similar to AR(1) simulation, it will have spikes for lag 1 - lag p as :
In pacf of MA(q) simulation, we observe exponentially decaying/damping sine wave.
ARMA(p, q) process
ARMA(p, q) is combination of autoregressive and moving average simulations.
q = 0, the process is called as pure autoregressive process; when
p = 0, the process is purely moving average.
The simulator of ARMA can be found as
ARMA(1, 1) process, here are the comparisons of the visualizations from
R and this code, which just made my day :)
- Ankur Goel