on semi-markov and semi-regenerative processes i

Z. Wahrscheinlichkeitstheorie verw. Gebiete 42, 2 6 1 - 2 7 7 (1978)

Zeitschrift ft~r

Wahrscheinl ichkei t s theor ie und verwandte Gebiete

�9 by Springer-Verlag 1978

On Semi-Markov and Semi-Regenerative Processes I*

David McDonald

Depar tment of Mathematics, University of Ottawa, Ottawa, Canada K 1N 6N5

Introduction

Consider a particle which jumps from one state to another in a measurable space {H,N} with real valued, random sojourn times in between. Denote the state entered after the n th sojourn by I n and denote the duration of the n th sojourn by X n. The successive states are to form a Markov chain and the distribution of a sojourn time depends on the state being visited as well as the next state to be entered. The pair (I,, X,)n~o is Markovian and is called a semi- Markov chain. If the X, are all integer valued we are in the lattice case; otherwise it is the continuous case. As a convention we let R represent the real line in the continuous case or the integers in the lattice case. If the X, are strictly positive we call (I,, Xn)n~ 0 a positive semi-Markov process, and we may define the age process (assuming Xo= 0) :

(I (t), Z(t)) = (I ,_ 1, t - S n_ 1)

where

S n= ~ X k and Sn-1--<t<S n. k=0

A strong mixing condition is imposed on the sojourn times (see Definitions 1 and 3). This condition is weaker than demanding that no matter which suc- cession of states be entered the support of the distributions of the (Xn),~ o must not asymptotically be contained in a subgroup of R. If a further uniformity condition is imposed on the tails of the distributions of the (X,),~ 0 then Theorem 3 classifies the ergodic properties of the age process (in fact only the lattice case is proved. The continuous analogue is in a subsequent paper). Slightly rewritten Theorem 3 says that if F ~ is the distribution of the sojourn

* This work was done in the author 's doctoral thesis at the Universit6 de Montreal under support from the Canada Council. It was extended at Cornell University under a Bourse de Perfectionnement from the Gov. of Qu6bec

0044-3719/78/0042/0261/$03.40

262 D. McDonald

time in the state re, if #~ is the mean of F ~ and if m is counting measure in the lattice case, Lebesgue measure in the continuous case then

lim P r o b { I ( t ) = d r t , Z ( t ) = d x } (1-F=(x)) t--+ o3 #7~ t e R

m ( d x ) . Prob {I(t) = dn} = 0,

where I[ II is the total variation. The equilibrium measure of a renewal age (1 - F~(Tr))

process is implicit in the rn(dx) factor.

A semi-regenerative process (Vt)t~R+ (see [1], p. 343) has an embedded semi- Markov process (I,, X , )~ 0; hence at times S. the entire past of the V t process is summarized by I,. Since the age process tends to equilibrium (in the sense of Theorem 3) and since

Prob { 14,~AlI(t) = re, Z( t ) = s} = Prob { ~ e A J(Io, Xo) = (re, 0), X 1 > S}

where A is some measurable set in the range of V, it is clear that V~ tends to equilibrium with (1(0, Z(t)). Theorem 4 says

lim Prob{VteA } - ~ ~ . P r o b { l ( t ) = d r c } =0

t e R

(in fact a stronger result is proven), where A~ is the mean time V t is in A during a sojourn in the state re. Renewal theorems for independent, non-identical random variables are also given.

The chief novelty of this paper is in the treatment of transient semi-Markov processes. One hopes to see applications to regenerative processes whose stochastic mechanism changes in time and which therefore may be pictured as a semi-regenerative process with a transient, embedded semi-Markov process.

Section I

Throughout we maintain a dichotomy between the lattice and continuous cases. In the former case R (R§ is the integers (the non-negative integers); B (B+) is the a-field of subsets of R (R+) and m is counting measure. In the latter case R (R+) is ( - 0% oo) ([0, oo)); B (B+) is the a-field of Borel sets on R (R+), and m is Lebesgue measure. Future references to continuity shall, in the lattice case, be treated as vacuous. Let (P,.~§ be a probability transition semi-group defined on a measure space (S, 9.I) admitting for any initial probability c~ on (S, 9.I) the construction of a Markov process on S:

{~, ~ (P~), (x,),~,<, (~),~,~+, (0,),~ + }.

(X,)t~R+ is defined on {fa,~,P~}. (0,),~R+ is the shift operator. G=a(Xs)s_<, (the oo

a-field generated by X s, s < t ) a # -a(X,),>__,, ~,~oo= ~o~t . G is the (shift) t = 0

invariant a-field. Throughout we will denote by the same symbol both the

On Semi-Markov and Semi-Regenerative Processes I 263

probability measure and the expectation operator derived from a probability transition kernel. Henceforth denote pex by px.

Defining

S = S x R + , B(S)=B(S)xB(R+)

and

/~((x, s), (B x {t +s}))= Pt, t+s(x, B)

where xeS, t, seR+ and BegI gives the space-time transition semi-group which admits the construction of a Markov process

Remark 1. We remark that given { O , ~ , P ~, (Xt)t~R+, (~)t~R+, (Ot)t~R+} we could define J(~=(X t, t) thus giving a space-time process defined on { s ~} with probability transition function /~ and initial distribution ~ where c~{A x 0} =c~{A} for Aeg.I. By [7] (see p. 16) for any tail event T e f f ~ of (Xt),~R+ there is an invariant event [ e Y of (J(,)~R+ such that P~ {[A T} = 0.

The following Theorem is included for reference.

Theorem I (Orey's Theorem). The following are equivalent: (a) all bounded harmonic functions on S (space-time harmonic) are constant. (b) for all probabilities ~ and fl on (S, 9.1)

lira Jr~Po,,(dx)-fieo,~(dx)ll =0. t ~ o o

Proof. See [7] Proposition 4.3 for discrete time Markov chains with stationary transition probabilities and arbitrary measurable state space. See [2] for an extension to continuous time. [53 gives the (obvious) extension to non-stationary transition probabilities.

The following is a useful weakening of Orey's theorem.

Theorem 2. I f (Ps,~+t)t,s~n+ is a homogeneous probability semi-group, that is Ps, s+t =Po,t =Pt, and if the tail field of the Markov process (Xt)t~e+ is invariant a.s.-P ~ (that is for any T E Y ~ there exists an I e Z such that P~(TAI)=O, where A is the symmetric difference) then

VseR+ lira IreP~(dx)-~P~+~(dx)]] =0 t ~ o o t e R +

(I] I] is the total variation).

Proof. Let IeZ, the invariant c-field.

Hence P~I=P~O-'I . Let A(A)=P'(O-~A) for all Ae~I. Therefore A is a probability measure. Also define ~ ~ L = g P +~A and 7(dx )=L{Xoedx }. P~ and L

agree on invariant events and hence on tail events. Next e ~ y so set f = a ~ Clearly If] <2 dT"

264

lim sup IL{f(X0). ZP} -L{F}I < lira j [L{f(Xo)I ~-'} - 11. dL

= ~ [L(f(Xo)]~-~ ~) - 11. dL

= ~ ] L { f ( X o l ~ } - l ] . d E .

L{f(Xo)]~5 } - 1 is 3; measurable and if I'~%

j [ L { f ( X o ) [ ~ } - 1] dL = L { f ( X o ) . Zr} - L { I ' } I '

= ~ nx (I') f ( x ) 7 (dx) - L {I'}

= W I ' - L I ' =0

since P~ and L agree on invariant events. Thus

, IL{f(Xo)l~2} - iI = 0 a.s.-L.

Hence

lim sup ]L{ f (Xo) . ZF} -- L{F}I = O. t ~ 00 F ~ t

Moreover L { f ( X o ) . ZF} = W F since A = P ~e" by the homogeneity of (Pt)t~R. ; SO

lim sup [ W ( X , ~ A ) - W(O- ~(Xt~A)) I = O. t ~ o o A~~

Hence

lira II c~ P~(dx) - ~ P~ +~ (dx ) I [ = 0 . t ~ o O

D. McDonald

Section II

Following [3, 4] let (H, f#) be a measure space. Let

(E, $ ) = ( H x R, f~ |

(E+, g+)=(H x R+, N| +).

A transition kernel H on (E, g) is called semi-markovian if H satisfies

lI(rc, x; drc', dx')= II (rc, 0; drc', dx'),

that is the transition is independent of x. Given an initial probability measure c5(~,~ on (E,g) we may construct a

X ~ probability space {A, d , H (~'~)} on which a Markov chain (1 n, ,),=o (called a semi-Markov chain) is defined having initial distribution c5(~,~ and transition

probability kernel //. If, moreover, S ,= ~ Xk then (I,),~ 0 and (I,, S,)[= o are l l

k = 0

both Markov chains defined on {A, d , II~'~'~}. Let N be the transition kernel of I oo ( ,),= o. Therefore

~(Tr; dTr') =H(~, 0; dTz' x R).


Let

~"{I.--d~'} __//~, o~ {(1., X.)ed~' x 8}

=//(~'*) {(I., X.)ed~' x R}.

I ~ Hence we may consider ( . ) .=0 to be defined on {A,sr162 Let Q be the transition kernel of (I., Sn).~ o. Therefore

Q(rc, s; d~', s + dx')= lI(rc, s; drc', dx').

We call the chain lattice if the (X.).% 0 take values in the integers. We now prove a result analogous to Theorem 2 in [8]. First: Let s T ~ be the coordinate functions defined on the probability space t n ~ n = 0

{ R % B % F } ( R ~ = R x R x ..., B ~ 1 7 6 ...) and suppose the {T~}~=o are independent. Following [6] let K, be a partition of {0, 1, 2, ...} of the form K, ={ili, < i< i,+~}.

1 Given K, define Y,= ~ T~. For d > 0 and e=~rr, r an integer, set

i ~ K n

&(e)={xl -e<x-2~a=<e} ,

q,~(e, d)=min[F{K~Bk(e)}, r{Y,-d~Bk(e)}],

q.(e, d) = ~ q.k(~, d). k = - - o v

Definition 1. The sequence {T~}.~ o is called strongly d-mixing if Ve there exists a sequence K. such that

q~ d)= ~ .

Furthermore the sequence {T,,}~= 0 is called strongly mixing if the closure of the smallest subgroup containing

T ~ {d{{ .}.= 0 is strongly d-mixing}

is R. The measure F on {R ~, B ~} is called strongly d-mixing (strongly mixing) if the coordinate functions { .}.= o are strongly d-mixing (strongly mixing).

In the lattice case Definition i may be expressed as follows: The sequence {Tn}[= o is strongly d-mixing if there is a partition K. of {0, 1, 2, ...} such that if

we define Y.= ~ T~ then lira ~ min{F{Y.=k}, r { Y . = k + d } } = o o . i s K n N ~ co n ~ 0 k = co

In practice it suffices to impose some uniformity conditions on the support of the distributions of the {T.}~= o to ensure strong d-mixing (strong mixing). In the lattice case, for instance, strong mixing follows if inf rain(F{ T~ =j.}, F{T. =j. + d } ) > 0 for some sequence {j,}~= 1.

T ~ Example I. Consider a sequence { ,},=o of integer valued, independent random variables such that

C{T.=2"- 1} = �89 n -0 , 1,2, ....

266 D. McDonald

This sequence is strongly 1-mixing by the above remark.

Definition 2. Consider the family {fp}p~A of real valued functions on R which are T. ~o uniformly equicontinuous. { ,},= o satisfies condition Md for the family {fp}p~A if

lim sup [F fp (x + U, +k - Uk_ 1) -- F fp (x + U, + k - Uk_ l + d)] = 0 n ~ p e a

for all x~R and for all k where U, = ~ T k. k = O

Proposition 1. I f {T,}~=~o is strongly d-mixing and if {fp};~d is a uniformly bounded, uniformly equicontinuous family of functions from R to the reals then {T,},~= 0 satisfies condition M d for the family {f;}p~A.

Proof. The proof follows from the "if" portion of the proof of Theorem 1 in [-6].

Restricting to the lattice case, Proposition 1 implies that if {T,}~= o satisfies condition Mt then:

lim sup [rfp(Un)- Ff;(U, + 1)] = 0 n ~ o 3 p~A

where {fp};~A is a family of bounded functions on R (the integers). That is

lim [f(U, = d x ) - r ( u n + 1 =dx)[[ =0. n ---~ c ~

Definition 3. The semi-Markov kernel H is called strongly d-mixing (respectively strongly mixing) if V(u, s)~E any version of the regular conditional probability F(~,s ) defined on (R ~, B ~~ by

F(~,s){Bo xBz • '"} =II("s){XoeBo, XteB1 ... . [1o, 11, ""}

for each trajectory (Io, I t . . . . ) is strongly d-mixing (respectively strongly mixing) X oo a.s.-~ ~. This means the sojourn times ( ,),=0 conditioned on knowing the

trajectory of the semi-Markov process (i.e. (I,),~=o) are strongly d-mixing (respectively strongly mixing).

F(~,s) exists since R ~ is a Polish space and B ~~ is Borel o--algebra on R ~. See Appendix I for a discussion of regular conditional distributions.

Example 2. Let {T,}~=o be a sequence of independent, integer valued random variables defined on {R~,B~,F} . Also suppose {T,}~=o is strongly 1-mixing. Define

/ / = { 1 , 2 , 3 . . . . ),

I I (n , s ;n+l , ds')=F{T,=ds'} n = 1 ,2 ,3 . . . . .

For any point (k, s)~E, construct the semi-Markov chain (I,, X,)~= o defined on {A, d,//(k,~)}. Since

H(k's){I o=k ,1 l = k + 1 . . . . } =1 ,


we have

F(~,~){B 0 xB 1 x .--} =V{TkeBo, Tk+leB 1 . . . . }.

Therefore, since { T,},~ o is strongly 1-mixing, H is also strongly 1-mixing. Note that Q may be viewed as the space-time transition kernel associated

with the Markov chain U, = ~ T k (i.e. the chain having kernel k = 0

P,,,+,(s, ds')=F{T,=ds'} n=1,2 , ...

and initial distribution F{To =ds}).

Example 3. Let H be a semi-Markov kernel on (E, g). Let 0 be a non-trivial, o-- finite measure on (// ,~). For any rceH construct the semi-Markov chain (I,, X,),~ o defined on {A, ag, H ~=' 0)}.

One may check that H is strongly d-mixing if the following conditions hold

(a) I , is 9-recurrent.

(b) The support of the semi-Markov chain contains (d, m) (see p. 87 of [3] for the definition of support).

By 0-recurrence the trajectories of I , enter D (see [3]) infinity often w.p.1. Moreover, since (d,m) is in the support of the semi-Markov chain, any such trajectory, say {rc,}~= o, has, w.p.l., a subsequence {re,k}2 = 1 such that

{rc,k,r~,~+m, rc,~+2m}cD and nk+2m<nk+ 1 for all k.

The sojourn times conditioned on the trajectory {7c,}~= o form an independent sequence of random variables {T,}2=0. We must show {T,}~~ is strongly d- mixing. Define a blocking

nl

Yl= 2 rk, k = 0

n ~ + 2 m

k = n l + l

n2

Z k ~ n l + 2rn+ l

n 2 + 2 m

k ~ n 2 + l

T~,

and so on.

Since (d, m) is in the support of the semi-Markov chain it is clear that that for all k

min [F(~ o) { Y2k~(d)}, �9 F(,~,o){Y2k-d~(d)}]> for I.--r~., n=O, 1 . . . .

where w.l.o.g, we assume, sup{pm(~, re'); re, ~'sD} < M (6(d), 0 and pm are as in [3]). It follows that {Tn},~176 1 is strongly d-mixing.

268 D. McDonald

Definition 4. Let W be the set of real valued functions such that for all h e w

(a) h is a bounded, measurable function on (E, N),

(b) h=Q h, (c) h(rc, s) is continuous in s for all re.

Proposition 2. I f the semi-Markov kernel H is strongly d-mixing then every h ~ has period d.

Proof. Let h ~ . Thus

h(~z, s)= ~ h (~z', s + x) H(~z, s; d rc', dx)

= ~ h(rc', S+X)//(n, 0; dn', dx).

By regularization there is a family h~ of measurable functions on (E, N) such that {h~(rc, . ) } ~ , is a uniformly equicontinuous family such that

h~(~, s)= ~ h~(n', s + x) H(rc, 0; drc', dx)

and limh~(rc, s)=h(rc, s) (since h is continuous in s). We may thus suppose

{h(rc, ")}~a forms a uniformly equicontinuous family. (In the lattice case this procedure is superfluous.)

For (n0, so)~E

h(no, So)- h(Tro, So + d)

= lira H ~~176 {h(I~, S,) - h(I,, S, + d)} n ~ a o

-- lira ~ o { / / ( . . . . ~ S , ) -h ( I , , S ,+d)l lo, I~, ...}}. n ~ o o

Now

H(~~176 {h(I,, S , ) - h ( l , , S, + d) lI o, 11,...}

< sup H (~~ ~o) {h(~, S,) - h(rc, S, + d) I Io, I1,. . .}.

Now the family {h(n , ' )}~n is uniformly equicontinuous and uniformly bounded. Moreover F(~o, ~o) is strongly d-mixing by hypothesis, and so also are the coordinate

o~ B ~~ E ~ ~ Therefore functions {T,},= o defined on {R ~176 , (~o, so)J.

lim sup H ( . . . . o) {h(n, S,) - h (n, S, + d) I Io, I1, ...}

i h = lira supF(~o,~o ) n, Tk - h re, Tk+d n ~ o e ~t k - k = O

= 0 by Proposit ion 1.

By dominated convergence then

lim H (~~ ~o){h(I,, S,) - h(I,, S. + d)} = 0, n ~ GO

so h(~zo, So) =h(no, s o +d).


Example 4. Consider the special semi-Markov chai n (In, Xn)2= 0 given in Exam, ple 2. S i n c e / / i s strongly 1-mixing and since we are restricted to the lattice case, Proposition 2 implies that the set of bounded, measurable functions h defined on E satisfying h = Q h are such that

h(n,s)=h(n,O) for all (n,s)eE.

Moreover it is clear that bounded measurable solutions k of k = ~ k are constants, hence bounded, measurable solutions to h=Qh are constants. We

Z ~ may conclude that if { n},=o is a strongly 1-mixing sequence of independent, integer valued random variables then the space-time harmonic functions for the

Markov chain Un = L Tk are constants. By Theorem 1, then k = O

lira IFF(Un = dx) - F(U, + 1 = dx)rJ = 0 n ~ cg)

(see also the remark after Definition 2). Moreover we may conclude the tail field of {U,},~= o is trivial for any starting measure F{Uo =dx} (see Proposition 4.3 in [7]).

The sequence of random variables {T,},~ o defined on { R % B ~~ F} given in Example 1 is strongly 1-mixing. By the above remarks the tail field and the

U ~ invariant field of { n},= 0 are trivial a.s.-F. However the conclusion of Theorem 2 is not verified since

U, > 2 n a.s.-F

and

Un_ 1 < 1 + 2 + . . . +2n-1 = 2 n - 1 a.s.-F.

Hence

HF(Un=dx)-F(Un_I=dX)I[ =2 for all n.

We conclude the Theorem 2 is, in general, false for non-homogeneous Markov chains.

Proposition 3. I f 17l is strongly mixing and xeR+ then if heJt~ h(~, x) =h(rc, 0),

Proof. Obvious.

S oo X oo and( I , , ,)n=o. With initial measure 6(~, o) construct//(~' o), {A, d } (I,~ n)n= 0 We suppose henceforth that the chain is positive, that is Xn > 0 Vn. Moreover we demand that//(=,o){ lira S, = c~} = 1. It is clear that this condition is aut0mati-

n ~ o o

cally fulfilled i f / / i s strongly mixing. We define the associated age process

( I ( t ) ,Z( t ) )=(In_, , t -Sn_x) S n _ , < t < S ,

teR+ (t, S, are both integers in the lattice case). This process defined on {A, d , H (~'~ is a homogeneous Markov process with right continuous trajectories. Define

(J(t), Y(t)) = (I,, S, - t), S ,_ , < t < S,.

270

This is the residual age process. Also define

1 - Fp ~ (s) = 11 ~' o~ { X l > s lI1 = p } .

Now define the transition function (H~)~R+ H(~'~ > X} >O:

Ht(rc, x; F) = H (~' o) {(I(t + x), Z(t + x))eFlX a > x} ,

In practice the state space of the age process is

{(~, x): //~' ~ >x} >0}

D. McDonald

on (E+,N+). F~g+. For

x~R+.

but in order to define Ht(zc, x; F) for all (rc, x)eE+ we add: For//{='~ x } =0:

Ht(n ,x ;n ,x+t )=l if t < [ x ] + l - x

(this is vacuous in the lattice case)

Ht(~z,x;F)=Ht_(Exl+l_x)(n,O;F) if t > [ x ] + l - x .

([x] is the greatest integer in x.) (Ht)t~R+ is quickly seen to be a probability transition kernel on (E+, ~+) and

for any initial probability a on (E+, ~+) we may construct a Markov process on (E+, 8+) (as in Section I):

{o, g , H ~, (W,)~R+, (~),~R +, (03,~R+}.

Moreover if a = 6(~.x)let H~= H (~'~). Clearly (I(t), Z(t)) defined on {A, d,//(~o, o)} and (W~) defined on {s W, H (~~176 have the same distributions on (E+, g+)Vt. In this sense the process (I(t), Z(t))e~R+ is embedded in (Wt)t~R+. Henceforth redefine (I(0, z ( t ) ) - (wO.

For t~R+, define U(t) on {s by

U(t) =(I(qt), q t - t)

where qt=inf{s>t[Z(s)=O}. Again (J(t), Y(t)) defined on {A,s/ ' ,H (~~176 and U(t) defined on {s ~ H ~~ o)} have the same distributions on (E+, g+). Hence- forth redefine

(J(t), Y(t)) - (Ut).

We may also define the corresponding space-time transition function (IZlt)teR+ on (/~.., g. . )=(E+ x R+, #__ x B+). For A~#+ define:

/], ((re, x, s); (A x (s + t)) = Ht((n, x) ; A).

Now, as in Section Ii given an initial distribution ~ on (/~+,d~+) we may construct the corresponding space-time Markov process:

{ ~ , ~ I~I8, ( ]~t)teR + , ( ~ t ) t sR + , ( Ot)teR+ } "

Again if ~ = 6( . . . . . ) let/qa = / ~ . . . . . ). Also let

I(z~,x,s)=7c, Z(zc, x,s)=x, T(~,X,S)=S.


Section III

In this section we restrict ourselves entirely to the lattice case.

Lemma 1. (a) I f the semi-Markov kernel 1I is strongly mixing and lattice and if is bounded and harmonic for VVtt then there exists a measurable function h on (E +, g+) such that

/~(~z, x, t )=h(~, x) Vt.

(b) I f also the only bounded, measurable solutions to k = N k are constants then is a constant.

Proof. Let z= in f{ t>OJZ(@)=O}. {(h(~)),~R+,(~),~R+} is a bounded mar- tingale. Thus

fi(~, 0, t) = lq ~, o , , ) f i (~)

= ~ I~(~', O, s') Q(7~, t; dx', ds').

Let h*(Tr, s)=/~(n, 0, s). Hence h* =Qh*. By Proposit ion 3 it is clear that h*(7~, s) =h*(~ ,0 )=k(~) for some k. Hence k = ~ k and /~(7~,0, t) is independent of t. Fur thermore

/~(~, x, t) = /~( . . . . ~)/~(~)

=/~( .. . . o ]~(I(I~dO, O, T(ITV~)) =/q( .. . . t) k (I(WO).

This is clearly independent of t, so write

t~(~, x, t)=h(~, x).

(b) If k is constant so is h* hence so is/~.

Note that if N defines a 0-recurrent Markov chain then the only bounded, measurable solutions to k = ~ k are constants.

Proposition 4. (a) I f the semi-Markov kernel El is strongly mixing and lattice then Vs~R+ and for any initial probability distribution ~ on E+

lira I[ ~Ht (dzc, dx) - c~Ht +s (dzc, dx) lP = O. t ~ o O t ~ R +

(b) If, moreover, the only bounded measurable solutions to k = N k are constants then for any initial distributions ~ and fi on E+

lira Ir ~Ht(dz~, dx) - fiHt(d~, dx)[[ = O. t ~ O 0 t ~ R +

Proof. (a) Given (I(o,Z(o)t~g § defined on {f2,ff, H ~} we may proceed as in Remark 1, to construct the space-time process defined on {~2,W,H~}. For any tail event T ~ of (I(t), Z(t))t~R+ there is an invariant event [ ~ f f of (lYVtt)t~R § such that H~{TA[} =0. Now there exists a space-time harmonic function I~ on /~+ such that [ = lim/~(r ) a.s.-H ~.

t ~ o o

272 D. McDonald

However by L e m m a 1./~(l~0 =h(Wt ) for some measurable, bounded function h on (E+,#+). l i m h ( W t ) = I exists a.s.-H ~ and defines an invariant event of

t ~ OO

(l(t), Z(t))t~R+. Hence H~(TA1)=0. The result now follows by Theorem 2.

(b) follows from L e m m a 1 (b) and Theorem 1.

We now come to our principal Theorem. First for A~#+ define

A': = {xi(rc, x ) sA} .

Moreover define

v'~(s)=Q(rc, O;rl x [0, s]).

F" is a proper, strictly positive distribution function. Moreover for s fixed F~(s) is a measurable function in 7:. Finally for any probabil i ty c~ on (E+, #+) define

at(A) = ~ ~ (1 - F ' ( s ) ) m(ds), aHt(du) Fl A ~ f i n

oo

where aHt(drc ) = aHt(du, R +), #~ = ~ sF'(ds). 0

Definition 5. The semi-Markov chain (I,, X,) ,~ o is regular if H is strongly mixing and if

F~(s)>G(s) VTr

where G is a distribution with finite mean.

Theorem 3. I f the semi-Markov chain (I,, X,)~= o is regular and lattice then for all probability measures ~ on E+

lira ]h c~Ht(dn, dx) - ~t(dn, dx)II = O. t s R +

Proof. sup 5 [aH,(d~, d x ) - ( 1 -F=(x)) �9 m(dx). ~H,(d~z, 0)J

A ~ g + A

= sup ~ (1 - F'(x)) . [aHt_ ~(du, O) - o~Ht(du, 0)]. m(dx) A E g + A

since aHt(dTc, d x )=(1 -F~(x ) ) �9 aHt(drc, O)m(dx) (if at t ime t the sojourn in state ~c is x then at t ime t - x the sojourn time is 0).

Now for x fixed

11(1 - F~(x)) �9 [c~Ht_ x(d~z, O) - c~H~(d~, 0)] II

<__ (1 - ~ (x)) . il ~ U , _ ~ (d ~, O) - ~H~ ( d ~, O) ql.

Therefore sup S [c~ Ht(dg, d x ) - ( 1 - F~(x)) . m(dx) . ~ H,(dTr, 0)3

A ~ g + A

<= S (1 - G(x)). I[ ~Ht_ x(d~, O) - aHt(d~z, O)ll" m(dx)

=< ~ (1 - G(x)) . ilaHt_ ~(d~, d s ) - aHt(dTr, ds)ll . m ( d x ) ~ O (,)


by dominated convergence (since G has finite mean). Also (,) implies

lira I[c~Ht(d~) -i~,~ c~H,(d~z, O) l ] =0 . t ~ o o t ~ R +

(**)

Hence

c~Ht(dn, dx) (1-F=(x))#~ m(dx), o:Ht(drc)

<= [I c~Ht( d~, dx) - (1 - F=(x)) m(dx) . aHt(d~, O)II

+ (1 -F~(x)) m(dx), c~Ht(d=, O) (1 -F=(x))t~= m(dx), aHt(dn)

0)

(2)

(1) tends to 0 as t tends to infinity by (,). Also

rl(1 -F~(x))[#~. c~Ht(d~, O)- c~Ht (d~c)] I]

<(1 - G(x)). Jlu~" ~U~(d~, 0) - ~Ht(d~)/I

so (2) is dominated by

~ . [[#~. aHt(d~, 0 ) - c~Ht(d~ )[] . (1 - G(x)) m(dx). R + llJ-I p1,7r

7~

This tends to 0 as t-~ oo by (**) and dominated convergence.

Corollary 1. I f the hypotheses of Theorem 3 hold and if the only bounded measurable solutions to k = ~ k are constants then for all probability measures c~ and fl on E+

lim I/~Ht (drc, dx) - fit(d~, dx) l[ = O. t ~ c O t ~ R +

Proof Follows from Theorem 3 and Proposition 4.

Corollary 2. Under the hypothesis of Theorem 3

lim [[P~ {I(t)=du, Z(t) =dz, J(t) =dp, Y(t)=dy} t ~ o o t s R +

1 - - - ~ ( ~ , dp) F~" (z + dy) m (d z) c~H~ (d ~)11 = 0.

#,

Proof It is easy to check that

P~{I(t) =du, Z(t)=dz, J(t)=dp, Y(t)-=dy}

= ~(~, dp) &" (z + dy) ~H,_ ~ (d~, O) m (dz).

Next, by Theorem 3,

c~Ht_z(d~, O) - l~c~Ht(drc ) lim ~ O . t~r # ,

274 D. McDonald

Hence,

lira ~(~, dp) F2(z + dy). c~H t_ z(drc, 0). m(dz)

- ~(~, dp) Fp~(z + dy). m(dz)c~Ht(drc )

lira S(1 - G(z))bl~Ht_z(d~z, o ) - l c~Ht (d~ ) --0. < t--* oO

The theorem follows.

Definition& A delayed semi-regenerative process (Vt)t~R+ with an embedded semi-Markov chain having k e r n e l / / i s a real valued stochastic process defined on a measurable space (g2,~) such that for any initial measure a on (E+,g+) there exists a probability measure P~ on {O, 2 } such that:

(a) An age process (I(t),Z(t)),~R+ associated with the semi-Markov k e r n e l / / can be defined {O,~,,P~} such that

P~ {I (t) = drc, Z (t) = dz} = aHt (drc, dz).

If c~=b(~,o ) denote W by p(~,0). Also define X 1 on (fL o~p(~,o)) by

X a = inf{t > 01Z(t) = 0}.

(b) For W-almost all tong? the function t~V~(co) is right continuous with left hand limits. (This condition is vacuous in the lattice case).

(c) The family of probabilities on (R +, B +) defined by P(~' o) { V~ = dzlX 1 > s} is such that for any AsB+, the function P(~'~ >s} is a jointly measurable function of (re, s) defined on (E+, g+).

(d) For all t~R+ there is a version of the regular conditional distribution

P~{Vt=dzII(t)=n,Z(t)=s}

such that

W { V~=dz]I (t)=n,Z(t)-- s} = P(~'~ { Vs=dZlX 1 >s};

that is they are one and the same probability measure. The essential property of (Vt)~R+ is given in d. This is essentially condition (c)

in the definition of a semi-regenerative process given on p. 343 of [1]. Let cg be the measurable sets in the range of (V~),~R+.

X ~ is regular and lattice Theorem4. I f the embedded semi-Markov chain (I,, ,),=o then

lim P~{Vt~A,I(t)=d~z}-A~.aHt(dn) =0 t e R +


oo

where A~=~P(~ ,~ that is A~ is the mean time V t 0

before a regeneration.

Proof Let GeN

is in A

P~ { Vt~A,I (t)eG } = P~ { VteA, I ( t )~G,Z (t)eR + }

= ~ P~{VteAlI(t)=r~,Z(t)=sI '~Ht(drc, ds) O x R +

= ~ P (~ '~ G x R +

Therefore

P~{V~sA, co ds) sup l(t) =dry} -- ~ P(='~ V, eAIX a >s}-~t(dzc, AEcg 0

< l] aHt (dzc, ds) - a t (drc, ds) l[

since P(~'~ 1 >s} < 1. Also

O3

P('~' ~ { V~AIX 1 >s}. o~t(dzc, ds ) 0

= S P(~'~ eA, X1 >s} (1-F~s))

o (1 - f ~ ( s ) ) #~

_ A~ c~Hr(dn).

�9 m(ds). ~Ht(d~z )

The result follows.

Corollary3. Under the hypotheses of Theorem4 if the only bounded measurable solutions to k = N k are constants then for all probability measures ~, fl on (E +, g + )

P~ { V t cA, I (t) = dzc} - ~ flH t(drc) = O. lim t ~ c o t ~ R +

Proof Immediate.

Corollary4. Let {T~},~__I be a sequence of independent strictly positive integer valued random variables with distributions {F~}~= 1 such that

F~ ( s) > G ( s) V n and V s ~ R + G has fini te mean.

Furthermore suppose {T~}~= 1 is strongly mixing and that the mean of F~ is #~. Then

Prob{renewal at t } - ~ __t P r o b { S , _ l < t < S , } =0. lira t e R +

276 D. McDonald

(So=0, S , = ~ Tk, Prob{renewal at t } : ~ P r o b { S , : t } ) . k = l n=O

Proof Let H = {1,2, 3, ...}. Define

II(n, s; n + 1, ds') = Prob { T, = ds'}.

X ~ The semi-Markov chain (I,, ,),=o defined on hypothesis so by Theorem 3

lim Ht((1,O);(n,s) ) (1 -F"(s) t~oz n = l s ]An tER +

{ A , d , H (~'~ is regular by

�9 Ht((1,0);(n,R+) ) =0.

Therefore

lim ~_ H,((1,O);(n,O))- ~, __1 .Ht((1, O);(n,R+))=0. t ~ c o n = l ]An t e R +

The result follows since

Prob {renewal at t} = ~ H~((1, 0);(n, 0)). n = l

Example2 in [9] shows that the renewal theorem in Corollary4 may not hold if there does not exist a distribution G with finite mean such that F,(s) > G(s) for all n and s. This implies the analogous condition in Theorem 3 is essential.

I should like to thank Michel Roussignole of the University of Paris, and Harry Kesten of Cornell University, for their invaluable help. Thanks also to the Depar tment of Mathematics at Cornell for its hospitality.

Je tiens aussi A remercier le referee de ses remarques judicieuses.

Appendix I

Let ((2, ~,, P) be a probability space, let M be a polish space and let B represent the Borel sets on M. Let X be a random variable from (O, ~ , P) to (M, B). Let Y be a random variable from (f2, Y, P) to a measurable space (g2', ~-'). It is well known that there exists a regular conditional distribution P(X=dxIY=y) with the following properties:

(a) For y~f2' fixed P(X=dxlY=y) is a probability measure on (M,B). (b) For AeB fixed, P(XeAI Y =y) is a measurable function on (~', Y) and

P(XeA, Y+F)= ~ P(xeA] Y=y) P(Y=dy). g2'


Bibliography

1. ~inlar, E.: Introduction to stochastic processes. Englewood Cliffs, N.J.: Prentice Hall 1974 2. Duflo, M., Revuz, D.: Propri6t6s asymptotiques des probabilit6s de transition des processus de

Markov r6currents. Ann. Inst. H. Poincar6, t. 5, n ~ 3, 233-244 (1969) 3. Jacod, J.: Th6or6mes de renouvellement et classification pour les chaines semi-markoviennes.

Ann. Inst. H. Poincar6, Sect. B7, 83-129 (1971) 4. Jacod, J.: Corrections et compl6ments ~ l'article Th6or6me de renouvellement et classification

pour les chaines semi-markoviennes. Ann. Inst. H. Poincar6, Vol. X, n o 2, 201~09 (1974) 5. McDonald, D.: La th6orie de renouvellement. Thesis, U. de Montr6al (1975) 6. Mineka, J.: A Criterion for Tail Events for Sums of Independent Random Variables. Z.

Wahrscheinlichkeitstheorie verw. Gebiete 25, 163-170 (1973) 7. Orey, S.: Limit Theorems for Markov Chain Transition Probabilities. New York: Van Nostrand

Reinhold Math. Studies 1971 8. Orey, S.: Tail events for sums of independent random variables. J. Math. Mech. 15, 937452 (1966) 9. Williamson, J.A.: A class of limit laws and a renewal theorem. Illinois J. Math. 10, 21~219 (1966)

Received November 15, 1975; in revised form May 29, 1977

on semi-markov and semi-regenerative processes i

Documents