Download - Random Sequences
Random Sequences
Gaurav S. Kasbekar
Dept. of Electrical Engineering
IIT Bombay
Recall
β’ We started with one r.v. π on a probability space (Ξ©, β±, π)
β’ Then, two r.v.s π and π on a common probability space (Ξ©, β±, π)
β’ Then, a vector (π1, β¦ , ππ) of r.v.s on (Ξ©, β±, π)
β’ Next: an infinite sequence π1, π2, π3, β¦ of r.v.s on (Ξ©, β±, π)
β’ Weβll study convergence of such a sequence
Motivation
β’ Two important results have to do with convergence of random sequences:
1) Law of Large Numbers
2) Central Limit Theorem
Law of Large Numbers β’ Recall motivation of the definition of πΈ(π)
β’ π independent trials of experiment performed
β’ Average of values of π in the π trials used to motivate expression for πΈ(π)
β’ Let π1, π2, π3, β¦ be i.i.d. with mean π
β’ limπββ
π1+β―+ππ
π:
intuitively, π
β’ That is, letting π π =π1+β―+ππ
π, the sequence
π 1, π 2, π 3, β¦ converges to the constant π
β’ To state this result formally, need to define convergence
Central Limit Theorem
β’ π1, π2, π3, β¦ i.i.d. with mean π and variance π2
β’ Informally, for large π, the CDF of π1 +β―+ ππ is approximately Gaussian
β’ That is, letting ππ = π1 +β―+ ππ, the distribution of ππ converges to a Gaussian distribution as π β β
Convergence of Real Numbers
β’ π₯1, π₯2, π₯3, β¦: a sequence of real numbers
β’ limπββ
π₯π = π₯ if:
for every π > 0, there exists ππ such that |π₯π β π₯| < π for all π β₯ ππ
β’ E.g., limit of π₯π =(β1)π
π:
0
β’ E.g., limit of π₯π = (β1)π:
does not exist (sequence oscillates)
Convergence of Random Variables
β’ π1, π2, π3, β¦ r.v.s on (Ξ©, β±, π)
β’ Want to define convergence of this sequence
β’ Recall: ππ is a function from Ξ© to β
β’ So convergence of r.v.s similar to convergence of functions
β’ Simplest notion: point-wise convergence
called sure convergence in r.v. terminology
Sure Convergence
β’ Definition: π1, π2, π3, β¦ converges surely to π if for every Ο β Ξ©, lim
πββππ(Ο) = π(Ο)
β’ E.g.: box initially has π€ white and π black balls
β’ At each step π = 1,2,3, β¦ one ball is drawn at random without replacement (if any left)
β’ ππ: number of white balls left after πβth draw
β’ Convergence behaviour of π1, π2, π3, β¦ :
converges surely to 0
Example
β’ A fair coin tossed an infinite number of times
β’ ππ = 1 if at least one of tosses 1,β¦ , π results in heads and ππ = 0 else
β’ Convergence behaviour of π1, π2, π3, β¦ :
with probability 1, converges to 1
but for Ο = "πππβ¦ " β Ξ©, ππ Ο = 0 for all π
does not converge surely to 1
β’ "πππβ¦ " is an event of probability 0
β’ Typically we donβt care about 0 probability events
Almost Sure Convergence
β’ Definition: π1, π2, π3, β¦ converges almost surely to π if for every Ο β π΄ β Ξ©, limπββ
ππ(Ο) = π(Ο), where P π΄ = 1
β’ In coin tossing example, π1, π2, π3, β¦ converges a.s. to 1
Example
β’ Ξ© = 0,1 , β± = β¬, π π, π = π β π, 0 β€ π β€ π β€ 1
β’ ππ Ο = Οπ
β’ For fixed Ο β Ξ©, limπββ
ππ(Ο) :
0, 0 β€ Ο < 11, Ο = 1.
β’ π1, π2, π3, β¦ converges a.s. to: 0
β’ Thus, one way to show a.s. convergence of π1, π2, π3, β¦ to π:
identify π΄ such that limπββ
ππ(Ο) = π(Ο) for all Ο β π΄
show that P π΄ = 1
Almost Sure Convergence
β’ In several examples where intuitively π1, π2, π3, β¦ seems to converge to π,
a.s. convergence does not hold
Example β’ Ξ© = 0,1 , β± = β¬, π π, π = π β π, 0 β€ π β€ π β€ 1
Ref: Hajek, Chapter 2
Example (contd.)
β’ limπββ
ππ(Ο):
does not exist for any Ο β Ξ©!
β’ So π1, π2, π3, β¦ does not converge a.s. to any r.v. π
β’ But intuitively, the sequence seems to be converging to 0