time to equilibrium for finite state markov chain
Post on 06-Jan-2016
23 Views
Preview:
DESCRIPTION
TRANSCRIPT
Time to Equilibrium for Finite State Markov Chain
許元春（交通大學應用數學系）
a finite set ( state space) a sequence of valued random variables ( random process, stochastic process)
( finite-dimensional distribution)
( Here )
:SS
),,,( 1100 nn iXiXiXP
),|()|()( 110022001100 iXiXiXPiXiXPiXP
),,,|(),,,|( 111100110011 nnnnkkkk iXiXiXiXPiXiXiXiXP
)(
)()|(
AP
BAPABP
:,,,,, 210 nXXXX
What is ?Among all possibilities, the following two
are the simplest： (i.i.d.)
where is a probability measure on
• Example: ( Black-Scholes-Merton Model)
the price of some asset at time t
),,,|( 110011 kkkk iXiXiXiXP
)(),,|( 10011 kkkkk iPiXiXiXPP
S
)(
))1((ln
nS
nSX n
)(tS
Here is a stochastic matrix
(i.e. and )
In this case,
is the transition probability for the Markov chain
),(),,|( 10011 kkkkkk iiKiXiXiXP
K
|||| SS0),( jiK
i
j,
Sj
jiK 1),(
),( yxK
0}{ nnX
Example: ( Riffle Shuffles )
(Gilbert, Shannon ‘55, Reeds ‘81) n
k kn
n
k
n
kn
a
b
ba
a
ba
b
n
n
k
n
2
binomial
n
k
Markov Chain with transition kernel K and initial distribution λ
This implies
In particular, we observe
Here and
),(),(),()(),....,,( 1211001100 nnnn iiKiiKiiKiiXiXiXP
),(),(),()|,....,( 121100011 nnnn iiKiiKiiKiXiXiXP
),()|( 0 jiKiXjXP nn KK 1
KKK nn 1
What is the limiting distribution of given ? (i.e. What is the limiting behavior for ?)
Example: ( Two State Chain )
1
0
1
1
0
1
,10
10
nX iX 0
nK
K
0 I
20 II
1 III
10
01nK
nn
nn
nK)1()1(
)1()1(
n
IK n 2
KK n 12
n
nK
lim
does not exist
invariant/equilibrium/stationary distribution
Suppose for some , that
for all
Then
Sx)(),( yyxK
nn
Sy
K ..ei
)(),()( yyxKxSx
Sy
Ergodic Markov Chain
Assume is aperiodic and irreducible.
Then there admits a unique invariant
distribution λ and
How the distribution of converge to its
limiting distribution?
K
)(),( yyxK n Syx ,
nX
Distance between two probability measures ν and μ on S . ( total variation distance )
( distance )
( Note that )
)}()({sup AASA
TV
PL
Px
x
x
x
Pxx
x
x
xP
Sx
P
P
,)(
)(
)(
)(sup
1,)()(
)(
)(
)(/1
,
1,2
1
TV
For
is a non-increasing sub-additive function
( )
This implies that if
for some and
then
p1n
P
n
SxxK
,)(),(sup
..ei )()()( mfnfmnf
mn,
P
m
SxxK
,)(),(sup
m
10
nn
m
n
P
n
Sx
m
n
xK
,)(),(sup
1
lnexpm
n
n
n
mn
1
ln/
exp
We say is reversible if it satisfies the detailed balance condition
Assume is reversible, irreducible and aperiodic.Then there exists eigenvalue and for any corresponding orthonormal basis of eigenvectors with , we have
and
K
),()(),()( xyKyyxKx Syx ,K
1....1 012||1|| SS
1||
0
)()()(
),( S
iii
ni
n
yxy
yxK
1||
1
222
2,|)(|||),(
S
ii
ni
n xxK
}{ i 10
the smallest non-zero eigenvalue of = the spectral gap of
where
is the smallest constant satisfying the Poincare inequality
Holding for all
11 KI
),( K
0)(|)(
),(inf fVar
fVar
ff
2)()( fEfEfVar
yx
xyxKxfyfff,
2 )(),(|)()(|2
1),(
"1
" A
),()( ffAfVar f
'
Setting .Then
( The Divichlet form associated with the semigroup )
and
Note that
Hence
0
)(
!m
mtKIt
t m
KeeH
),)((1
lim),(200
2
2
ffHIt
fHt
ff tttt
2
2
2
2)()( ffH
dt
dffH
dt
dtt
)(2
))(),((2
fHVar
ffHffH
t
tt
)( fHVardt
dt
)()( 22
2fVareffH t
t
tH
),(22
2fHfHfH
t ttt
Theorem:
The mixing time is given by
• Theorem:
where
)(),(
2, x
exH
t
t
tt e
x
yyyxH
)(
)()(),(
2T
exHtT t
Sx
1),(max|0inf
2,2
)1
log2
11(
11
*2
T
)(min* xSx
Consider the entropy – like quantity
And
The log-Sobolev constant is given by the
Formula Hence is the smallest constant satisfying
the log-Sobolev inequality
holding for all function
)(|)(|
log|)(|)(2
2
22 s
f
sfsff
Ss
,
)(
),(inf{
f
ff
}0)( f
1 A
),()( ffAf
f
Theorem:
• •
2
)1
loglog4
11(
1
2
1*2 T
2T
)1
loglog2
12(
1*
)1
log2
11(
1*
1
21
)1
loglog4
11(
1*
Can one compute or estimate the constant ?The present answer is that it seems to be a very difficult
problem to estimate . Lee-Yau(1998), Ann. of probability symmetric simple exclusion/random transpositionDiaconis-Saloff-Coste(1996), Ann. Of Applied
Probability . For simple random walk on cycle,
. The exact value of for with all rows equal to Chen-Sheu(03), Journal of Functional Analysis when and is even
n
2
1~nn
K
)2
log1(2
1
nn
4n
n
Who Cares ?
a set. a group.
Action of group on set :
Orbit of for some
What’s the number of orbits (or patterns) ?
:S
:G
yxSyOx gx |{
}Gg
Z
G S
SxSGxg g ),(
Example ( balls, boxes, Bose-Einstein distribution)
Polya’s theory of counting
(See Enumerative Combinatorics, Vol II, by R. Stanley, Sec7.24)
Burnside Process (Jerrum and Goldberg)
n
k,][ nkS
},...,2,1{][ kk
nSG
}|{ xxGgG gx
}|{ ssSsS gg
n
l
Diaconis (‘03) ( balls, boxes)
for all
yx GGg g
x
SG
OyxK
||
1
||
||),(
zOx
x
1
||
1)(
zOx
1)( x
|1
),(||)(),(|z
OxKOOxK yl
yyl
TV
l xK ),(
n
k
,))(1(),0( l
TV
l kCK !
1~)(k
kC
)( nk
0ddK
TV
l ),0(
nl log
Cut-off phenomenon
Bayer and Diacoins (’86)
The total variation distance for riffle shuffles of 52 cards
“neat riffle shuffles”?
l
top related