introduction to times series
DESCRIPTION
A good introduction to times series.TRANSCRIPT
-
Introduction to Time Series Analysis. Lecture 4.Peter Bartlett
Last lecture:
1. Sample autocorrelation function
2. ACF and prediction
3. Properties of the ACF
1
-
Introduction to Time Series Analysis. Lecture 4.Peter Bartlett
1. Review: ACF, sample ACF.
2. Properties of estimates of and .
3. Convergence in mean square.
2
-
Mean, Autocovariance, Stationarity
A time series {Xt} has mean function t = E[Xt]and autocovariance function
X(t+ h, t) = Cov(Xt+h, Xt)= E[(Xt+h t+h)(Xt t)].
It is stationary if both are independent of t.Then we write X(h) = X(h, 0).The autocorrelation function (ACF) is
X(h) =X(h)
X(0)= Corr(Xt+h, Xt).
3
-
Estimating the ACF: Sample ACF
For observations x1, . . . , xn of a time series,
the sample mean is x = 1n
nt=1
xt.
The sample autocovariance function is
(h) =1
n
n|h|t=1
(xt+|h| x)(xt x), for n < h < n.
The sample autocorrelation function is
(h) =(h)
(0).
4
-
Properties of the autocovariance function
For the autocovariance function of a stationary time series {Xt},1. (0) 0,2. |(h)| (0),3. (h) = (h),4. is positive semidefinite.
Furthermore, any function : Z R that satisfies (3) and (4) is theautocovariance of some stationary (Gaussian) time series.
5
-
Introduction to Time Series Analysis. Lecture 4.1. Review: ACF, sample ACF.
2. Properties of estimates of and .
3. Convergence in mean square.
6
-
Properties of the sample autocovariance function
The sample autocovariance function:
(h) =1
n
n|h|t=1
(xt+|h| x)(xt x), for n < h < n.
For any sequence x1, . . . , xn, the sample autocovariance function satisfies
1. (h) = (h),2. is positive semidefinite, and hence
3. (0) 0 and |(h)| (0).
7
-
Properties of the sample autocovariance function: psd
n =
(0) (1) (n 1)(1) (0) (n 2)
.
.
.
.
.
.
.
.
.
.
.
.
(n 1) (n 2) (0)
=1
nMM , (see next slide)
so ana =1
n(aM)(M a)
=1
nM a2
0,
i.e., n is a covariance matrix. It is also important for forecasting.
8
-
Properties of the sample autocovariance function: psd
M =
0 0 0 X1 X2 Xn0 0 X1 X2 Xn 00 X1 X2 Xn 0 0.
.
.
.
.
.
X1 X2 Xn 0 0
.
and Xt = Xt .
9
-
Estimating
How good is Xn as an estimate of ?
For a stationary process {Xt}, the sample average,
Xn =1
n(X1 + +Xn) satisfies
E(Xn) = , (unbiased)
var(Xn) =1
n
nh=n
(1 |h|
n
)(h).
10
-
Estimating the ACF: Sample ACF
To see why: var(Xn) = E
(1
n
ni=1
Xi ) 1
n
nj=1
Xj
=1
n2
ni=1
nj=1
E(Xi )(Xj )
=1
n2
i,j
(i j)
=1
n
n1h=(n1)
(1 |h|
n
)(h).
11
-
Estimating
Since var(Xn) =1
n
nh=n
(1 |h|
n
)(h),
if limh
(h) = 0, var(Xn) 0.
12
-
Estimating
Also, since var(Xn) =1
n
nh=n
(1 |h|
n
)(h),
ifh
|(h)|
-
Estimating
n var(Xn) 2
h=
(h).
i.e., instead of var(Xn) 2
n, we have var(Xn)
2
n/,
with =
h (h). The effect of the correlation is a reduction of samplesize from n to n/ . (c.f. mixing time.)
14
-
Estimating : Asymptotic distribution
Why are we interested in asymptotic distributions?
If we know the asymptotic distribution of Xn, we can use it toconstruct hypothesis tests,e.g., is = 0?
Similarly for the asymptotic distribution of (h),e.g., is (1) = 0?
Notation: Xn AN(n, 2n) means asymptotically normal:Xn n
n
d Z, where Z N(0, 1).
15
-
Estimating for a linear process: Asymptotically normal
Theorem (A.5) For a linear process Xt = +
j jWtj ,
ifj 6= 0, then
Xn AN(x,
V
n
),
where V =
h=
(h)
= 2w
j=
j
2
.
(X AN(n, n) means 1n (Xn n) d Z.)
16
-
Estimating for a linear process
Recall: for a linear process Xt = +
j jWtj ,
X(h) = 2w
j=
jh+j ,
so limn
n var(Xn) = limn
n1h=(n1)
(1 |h|
n
)(h)
= limn
2w
j=
j
n1h=(n1)
(j+h |h|
nj+h
)
= 2w
j=
j
2
.
17
-
Estimating the ACF: Sample ACF for White Noise
Theorem For a white noise process Wt,if E(W4
t)
-
Sample ACF and testing for white noise
If {Xt} is white noise, we expect no more than 5% of the peaks of thesample ACF to satisfy
|(h)| > 1.96n.
This is useful because we often want to introduce transformations thatreduce a time series to white noise.
19
-
Sample ACF for white Gaussian (hence i.i.d.) noise
20 15 10 5 0 5 10 15 200.2
0
0.2
0.4
0.6
0.8
1
1.2
20
-
Estimating the ACF: Sample ACF
Theorem (A.7) For a linear process Xt = +
j jWtj ,
if E(W4t
)
-
Sample ACF for MA(1)
Recall: (0) = 1, (1) = 1+2 , and (h) = 0 for |h| > 1. Thus,
V1,1 =h=1
((h+ 1) + (h 1) 2(1)(h))2 = ((0) 2(1)2)2 + (1)2,
V2,2 =h=1
((h+ 2) + (h 2) 2(2)(h))2 =1
h=1
(h)2.
And if is the sample ACF from a realization of this MA(1) process, thenwith probability 0.95,
|(h) (h)| 1.96Vhhn.
22
-
Sample ACF for MA(1)
10 8 6 4 2 0 2 4 6 8 100.2
0
0.2
0.4
0.6
0.8
1
1.2ACFConfidence intervalSample ACF
23
-
Introduction to Time Series Analysis. Lecture 4.1. Review: ACF, sample ACF.
2. Properties of estimates of and .
3. Convergence in mean square.
24
-
Convergence in Mean Square
Recall the definition of a linear process:
Xt =
j=
jWtj
What do we mean by these infinite sums of random variables?i.e., what is the limit of a sequence of random variables?
Many types of convergence:1. Convergence in distribution.2. Convergence in probability.3. Convergence in mean square.
25
-
Convergence in Mean Square
Definition: A sequence of random variables S1, S2, . . .converges in mean square if there is a random variable Yfor which
limn
E(Sn Y )2 = 0
26
-
Example: Linear Processes
Xt =
j=
jWtj
Then if
j= |j |
-
Example: Linear Processes (Details)
(1) P (|Xt| ) 1
E|Xt| (Markovs inequality)
1
j=
|j |E|Wtj |
j=
|j | (Jensens inequality)
0.
28
-
Example: Linear Processes (Details)
For (2):The Riesz-Fisher Theorem (Cauchy criterion):Sn converges in mean square iff
limm,n
E(Sm Sn)2 = 0.
29
-
Example: Linear Processes (Details)
(2) Sn =n
j=n
jWtj converges in mean square, since
E(Sm Sn)2 = E m|j|n
jWtj
2
=
m|j|n
2j2
2 m|j|n
|j |
2
0.
30
-
Example: AR(1)
Let Xt be the stationary solution to Xt Xt1 =Wt, whereWt WN(0, 2).If || < 1,
Xt =j=0
jWtj
is a solution. The same argument as before shows that this infinite sumconverges in mean square, since || < 1 impliesj0 |j |
-
Example: AR(1)
Furthermore, Xt is the unique stationary solution: we can check that anyother stationary solution Yt is the mean square limit:
limn
E
(Yt
n1i=0
iWti
)2= lim
nE(nYtn)2
= 0.
32
-
Example: AR(1)
Let Xt be the stationary solution to
Xt Xt1 =Wt,
where Wt WN(0, 2).If || < 1,
Xt =
j=0
jWtj .
= 1? = 1?|| > 1?
33
-
Introduction to Time Series Analysis. Lecture 4.1. Properties of estimates of and .
2. Convergence in mean square.
34