stat 516: multivariate distributions
TRANSCRIPT
STAT 516: Multivariate DistributionsLecture 7: Convergence in Probability and Convergence in
Distribution
Prof. Michael Levine
November 12, 2015
Levine STAT 516: Multivariate Distributions
Convergence in Probability
I A sequence Xnp→ X (converges in probability to X ) if, for
any ε > 0limn→∞
P(|Xn − X | ≥ ε) = 0
I In this context, X may be a constant a - a degenerate randomvariable
I Chebyshev’s inequality is a common way of showingconvergence in probability
Levine STAT 516: Multivariate Distributions
Examples
1. Let Xn = X + 1n where X ∼ N(0, 1)
2. Easy to verify that (by Chebyshev’s inequality) Xnp→ X
1. For Xn s.t. the mean µ and variance σ2 are finite
Xnp→ µ
2. The weak law of large numbers - second moment must exist;strong law does not require that - will not be proved in thiscourse
Levine STAT 516: Multivariate Distributions
Convergence in probability is closed under linearity
I Xnp→ X and Yn
p→ Y implies Xn + Ynp→ X + Y
I If a is a constant and Xnp→ X aXn
p→ aX
I Conclusion: convergence in probability is closed under linearity
Levine STAT 516: Multivariate Distributions
Continuous Mapping Theorem for Convergence inProbability
I If g is a continuous function, Xnp→ X then g(Xn)
p→ g(X )
I We only prove a more limited version: if, for some constant a,g(x) is continuous at a, g(Xn)
p→ g(a)
I Can be viewed as one of the statements of Slutsky theorem -the full theorem to be stated later
Levine STAT 516: Multivariate Distributions
Another useful property
I If Xnp→ X and Yn
p→ Y , XnYnp→ XY
I Only prove in this form but can be generalized tog(Xn,Yn)
p→ g(X ,Y )
Levine STAT 516: Multivariate Distributions
Consistency and convergence in probability
I For X ∼ F (x ; θ), θ ∈ Ω a statistic Tn is a consistentestimator of θ if
Tnp→ θ
I Weak Law of Large Numbers → X is a consistent estimator ofµ
I A sample variance S2 = 1n−1
∑ni=1(Xi − X )2
p→ σ2
I By continuous mapping theorem Sp→ σ
Levine STAT 516: Multivariate Distributions
Example
I Let X1, . . . ,Xn ∼ Unif [0, 1] and Yn = maxX1, . . . ,XnI The cdf of Yn is FYn(t) =
(tθ
)nfor 0 < t ≤ θ
I Check that EYn =(
nn+1
)θ - Yn is a biased estimator
I Direct computation implies that Yn is consistent...and so isthe unbiased estimator
(n+1n
)Yn
Levine STAT 516: Multivariate Distributions
Convergence in Distribution
I If for a sequence Xn with cdf FXn(x) , and a randomvariable X ∼ FX (x)
limn→∞
FXn(x) = FX (x)
for all points of continuity of FX (x), XnD→ X - Xn converges
in distribution or in law to X
I We say that FX (x) is the asymptotic distribution or thelimiting distribution of Xn
I Occasional abuse of notation: Xn → N(0, 1)
Levine STAT 516: Multivariate Distributions
Convergence in distribution and convergence in probability
I Convergence in distribution is only concerned withdistributions and not at all with random variables
I For a symmetric fX (x), X and −X have the same distribution
I Let
Xn =
X if n is odd−X if n is even
I Clearly, Xn
D→ X but there is no convergence in probability!
Levine STAT 516: Multivariate Distributions
Example
I Let X ∼ N(0, σ2/n)
I Check that
limn→∞
Fn(x) =
0 x < 012 x = 01 x > 0
I Conclude that Fn(x) converges to the point mass at zero
Levine STAT 516: Multivariate Distributions
Example
I Convergence of pdfs/pmfs does NOT mean convergence indistribution!
I Define the pmf
pn(x) =
1 x = 2 + 1
n0 elsewhere
I limn→∞ pn(x) = 0 for any x
I However, the limiting function of cdf’s is F (x) = 0 if x < 2and F (x) = 1 if x ≥ 2 which is a cdf!
I Convergence in distribution does take place!
Levine STAT 516: Multivariate Distributions
Example
I However...for Tn ∼ tn we have
Fn(t) =
∫ t
−∞
Γ[(n + 12)]
√πnΓ
(n2
) 1
(1 + y2/n)(n+1)/2dy
I Stirling’s formula:
Γ(k + 1) ≈√
2πkk+1/2 exp (−k)
I The limit under the sign of integral is the normal pdf...so
limn→∞
Fn(t) =
∫ t
−∞
1√2π
exp (−y2/2) dy
I The limiting distribution of tn is N(0, 1)
Levine STAT 516: Multivariate Distributions
Example
I Recall that for X1, . . . ,Xn ∼ Unif [0, θ] Yn = max1≤i≤n is theconsistent estimator of θ
I Now we can say more...let Zn = n(θ − Yn)
I For any t ∈ (0, nθ)
P(Zn ≤ t) = P(Yn ≥ θ − (t/θ)) = 1−(
1− t/θ
n
)n
I Since limn→∞ P(Zn ≤ t) = 1− exp (−t/θ) for some
Z ∼ exp(θ) ZnD→ Z
Levine STAT 516: Multivariate Distributions
Relationship between convergence in probability andconvergence in distribution
I If Xnp→ X , Xn
D→ X
I The converse is not true in general - see an earlier example!
I However, if for a constant b XnD→ b it also true that Xn
p→ b
Levine STAT 516: Multivariate Distributions
Basic properties of convergence in distribution
I If XnD→ X and Yn
p→ 0, Xn + YnD→ X
I This is the magic wand if it is hard to show that XnD→ X but
easy to show that some other YnD→ X and Xn − Yn
p→ 0
I If XnD→ X and g(x) is a continuous function on the support
of X ,
g(Xn)D→ g(X )
I If a and b are constants, XnD→ X , An
p→ a, and Bnp→ b,
An + BnXnD→ a + bX
Levine STAT 516: Multivariate Distributions
Boundedness in probability
I For any X ∼ FX (x), we can always find η1 and η2 s.t.FX (x) < ε/2 for x ≤ η1 and FX (x) > 1− ε/2 for x ≥ η2
I Thus, for η = max|η1|, |η2|
P[|X | ≤ η] ≥ 1− ε
I Formal definition: Xn is bounded in probability if for anyε > 0 there exist Bε > 0 and an integer Nε s.t. if n ≥ NεP[Xn ≤ Bε] ≥ 1− ε
I Can show immediately that if XnD→ X then Xn is bounded
in probability...the converse is not always true
Levine STAT 516: Multivariate Distributions
Why a sequence that is bounded in probability may notconverge
I Define X2m = 2 + 12m and X2m−1 = 1 + 1
2m w.p.1
I All of the mass of this sequence is concentrated in [1, 2.5] andso it is bounded in probability
I Xn consists of two subsequences that converge to degenerateRV’s Y = 2 and W = 1 in distribution
Levine STAT 516: Multivariate Distributions
A useful property
I Xn a sequence of random variables bounded in prob. and Yn asequence that converges to zero in probability
I Then, XnYnp→ 0
I Analog from the world of calculus: if A is a constant andξn = 1
n then limn→∞1n = 0 and limn→∞
An = 0
Levine STAT 516: Multivariate Distributions
MGF technique
I If Xn has mgf MXn(t) for |t| ≤ h, X has mgf MX (t) for|t| ≤ h1 < h, and limn→∞MXn(t) = M(t) for |t| ≤ h1, then
XnD→ X
I Take Yn ∼ b(n, p) with fixed µ = np for every n
I Check that MYn(t) = [(1− p) + pet ]n =[1 + µ(et−1)
n
]n
Levine STAT 516: Multivariate Distributions
Poisson approximation of the binomial: an example
I Y ∼ b(50, 1
25
);
I P(Y ≤ 1) =(2425
)50+ 50
(125
) (2425
)49= 0.4000
I Since µ = np = 2, we have the Poisson approximation
e−2 + 2e−2 = 0.406
Levine STAT 516: Multivariate Distributions
Central Limit Theorem (CLT)
I If X1, . . . ,Xn ∼ N(µ, σ2) we know that X ∼ N(µ, σ
2
n
)I CLT: if X1, . . . ,Xn are independent, bE Xi = µ and
Var Xi = σ2, we have
√n(X − µ)
σ∼ N(0, 1)
Levine STAT 516: Multivariate Distributions