acc10presentation

8/7/2019 ACC10presentation

1/37

Stochastic Hybrid Systems with

Renewal Transitions

Duarte Antunes Joao Hespanha Carlos Silvestre

Institute for Systems and Robotics, Instituto Superior Tecnico

Dep. of Electrical and Computer Eng., UC Santa Barbara

July 1, 2010

2010 American Control ConferenceBaltimore, Maryland, USA

Stochastic Hybrid Systems with Renewal Transitions 1/19


2/37

Outline

Stochastic Hybrid Systems (SHS)MotivationSHS definitionLiterature Review

Main resultsTransitory analysisAsymptotic analysis

Example and ConclusionsNetworked Control ExampleConclusions and Future Work



3/37

Stochastic Hybrid Systems (SHS)

Outline






4/37

Stochastic Hybrid Systems (SHS) Motivation

Networked Control

Control loops closed through shared network.



5/37

St h ti H b id S t (SHS) M ti ti


6/37


Networked Control

Control loops closed through shared network. For simplicity: Focus on single loop & Only the control law is sent.

Need network abstraction.Stochastic Hybrid Systems with Renewal Transitions 4/19



7/37


Network abstraction

User (sensor, controller,...) waits independent and identically distributed(i.i.d.) times to gain access to network.




8/37


Network abstraction


Example- CSMA-type protocol




9/37


Network abstraction






10/37


Network abstraction






11/37

y y ( )

Network abstraction



Key: model time intervals tk+1 tk as i.i.d.




12/37

y y ( )

Network abstraction




Prob. Distribution of the access times

Npairs of nodesimplement aboveprotocol.

Back-offs

Uniform([0, hs])Stochastic Hybrid Systems with Renewal Transitions 5/19



13/37

y y ( )

Network abstraction




Prob. Distribution of the access times

Npairs of nodesimplement aboveprotocol.

Back-offs

Uniform([0, hs])Stochastic Hybrid Systems with Renewal Transitions 5/19



14/37

SHS modelAfter waiting i.i.d. time to obtain access to the network:

Case I

controller transmitsdata collected at timeit initially tried totransmit data.

x = (xP, u , v)A models system andcontroller dynamics

J1 models samplingv(rk) = KxP(r

k )

J2 models control updateu(sk) = v(s

k )




15/37

SHS modelAfter waiting i.i.d. time to obtain access to the network:

Case I

controller transmitsdata collected at timeit initially tried totransmit data.

x = (xP, u , v)A models system andcontroller dynamics

J1 models samplingv(rk) = KxP(r

k )

J2 models control updateu(sk) = v(s

k )

Case IIcontroller (re)samplessensor, computescontrol, and transmits

most recent data.

x = (xP, u)Jmodels sample and controlupdateu(t

k) = Kx

P(tk

)


Stochastic Hybrid Systems (SHS) SHS definition


16/37

SHS definition

Differential Eqs.

x(t) = Aq(t)x(t)

Reset maps

(q(tk), x(tk)) = (l(q(tk )), Jq(t

k),lx(tk ))

Transition distributions

Fi,l

1 l nl, 1 q(t), i nq




17/37

SHS definition

Differential Eqs.

x(t) = Aq(t)x(t)

Reset maps

(q(tk), x(tk)) = (l(q(tk )), Jq(t

k),lx(tk ))


Fi,l


Execution:Transition times tk+1 determined by first hq(tk)j Fq(tk)j to trigger.




18/37

SHS definition

Differential Eqs.

x(t) = Aq(t)x(t)

Reset maps

(q(tk), x(tk)) = (l(q(tk )), Jq(t

k),lx(tk ))


Fi,l


Execution:Transition times tk+1 determined by first hq(tk)j Fq(tk)j to trigger.See paper for assumptions on atom points of the trans. distributions.


Stochastic Hybrid Systems (SHS) Literature Review


19/37

Literature Review

DynamicsWiener

Process

Stochastic

Intervals

Stochastic

Transitions

SHS1 Non-linear Yes Guards NoSHS2 Non-linear Yes Yes-Exponential No

SHS3 Non-linear No/Yes* Yes-SD No/Yes*MJLS Linear No/Yes* Yes-Exponential No/Yes*PDPs Non-linear No/Yes* Yes-SD Yes-SD

Our work Linear No/Yes* Yes No/Yes*

SD-state dependent*See [SHS3]

SHS1 -Hu et all, Towards a Theory of Stochastic Hybrid Systems, Workshop on Hybrid Systems: Computation andControl,2000.

SHS2 -Ghosh et all, Ergodic Control Of Switching Diffusions, SIAM Journal of Control and Optimization,1997.SHS3 -Hespanha, A Model for Stochastic Hybrid Systems with Application to Communication Networks,Special Issue on

Hybrid Systems, 2005.MJLS -Mariton, Jump Linear Systems in Automatic Control, 1990, Marcel Dekker Inc.

PDPs -Davis, Markov Models and Optimization, Chapman & Hall, 1993.


Main results


20/37

Outline





Main results Transitory analysis


21/37

Moment analysis

Theorem-Any state moment can be obtained through Volterra Eq.

The following holds

E[xi1(t)m1xi2(t)

m2 . . . xir(t)mr ] =

nq

i=1czmi (t), for

mj = m, mj > 0,

where

zmi (t) =

t0

i(ds)zmi (t s) + h

mi (t) (Volterra equation), (1)

i(ds) is a matrix-measure. For AC distributions i(ds) = Ki(s)ds.




22/37

Moment analysis


The following holds

E[xi1(t)m1xi2(t)

m2 . . . xir(t)mr ] =

nq

i=1czmi (t), for

mj = m, mj > 0,

where

zmi (t) =

t0

i(ds)zmi (t s) + h



Example: x(t) = Ax(t), x(tk) = J x(tk ), tk+1 tk F




23/37

Moment analysis


The following holds

E[xi1(t)m1xi2(t)

m2 . . . xir(t)mr ] =

nq

i=1czmi (t), for

mj = m, mj > 0,

where

zmi (t) =

t0

i(ds)zmi (t s) + h



Example: x(t) = Ax(t), x(tk) = J x(tk ), tk+1 tk F

E[x(t)x(t)] = x0

Z(t)x0, where

Z(t) = t

0

(JeAs)Z(t s)(JeAs)F(ds) + eAteAt(1 F(t)).




24/37

Why moment analysis

Computing E[a(t)], a(t) = xi1(t)m1xi2(t)

m2 . . . xir (t)mr allows to

Provide probability bounds at time t. Chebychevs inequality:

Prob[|b(t)| > ] E[a(t)]

2, b(t)2 = a(t)

Compute probability distribution of Markov process at time t(moment problem).


Main results Asymptotic analysis


25/37

StabilityTheorem

Suppose that transition distributions have finite support1

. Then, under easy totest technical conditions, the SHS is MSS (E[x(t)x(t)] 0) if and only if

det(I K(z)) = 0, [z] 0.

K(z)-Laplace transform of the Kernel of the Volterra Eq. describing E[x(t)x(t)].

1See the paper for the infinite support case2[1] Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors

webpages. Stochastic Hybrid Systems with Renewal Transitions 12/19



26/37

StabilityTheorem

Suppose that transition distributions have finite support1

. Then, under easy totest technical conditions, the SHS is MSS (E[x(t)x(t)] 0) if and only if

det(I K(z)) = 0, [z] 0.

K(z)-Laplace transform of the Kernel of the Volterra Eq. describing E[x(t)x(t)].

Nyquist Criterion! Example taken from our previous work2

nl = 1, nq = 1, A, J

given in2

F Unif([0, ])

1See the paper for the infinite support case2[1] Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors

webpages. Stochastic Hybrid Systems with Renewal Transitions 12/19



27/37

Equivalent stability conditions

TheoremThe following are equivalent

(A) det(I K(z)) = 0, [z] 0,

(B) (M) < 1, M:= K(0) and denotes the spectral radius.

(C) There exists a set of matrices P:= {Pi > 0, i Q} such that for every i Q

Li(P)Pi < 0,

where Li(P) is a finite-dimension operator.3

(D) For every set of matrices {Qi0,iQ} the solution to

Li(P) Pi = Qi, i Q

is unique and satisfies {Pi0,iQ}

3See the paper for the expression.Stochastic Hybrid Systems with Renewal Transitions 13/19


28/37



29/37

PerformanceTheorem

If SHS is MSS and trans. distribution have finite support, then for [0, max)E[x(t)x(t)] cetx

0x0,

where max =

, ifdet(I K(a)) = 0, a (, 0),

max{a < 0 : det(I K(a)) = 0}, otherwise.

Recall that convergence rate ofE[x(t)x(t)] implies convergence rate forprobabilities bounds (performance).

Theorem

Under easy to test technical conditions, if SHS is not MSS then there exists one

unstable pole z1 ofdet(I K(zi)) = 0, i = 1, . . . , nz;, which is real andsatisfies [zi] z1, i;

Recall that E[x(t)x(t)] = x0

Z(t)x0

and from results of Volterra equation

Z(t) =

nzi=1

mi1j=0

Ri,jtj

ezit

+(t),

(t) 0.




30/37

Positive kernel

Special property of Volterra Eq. lies at the heart of the results.

Example nq = 1, nl = 1

Z(t) = t0

(JeAs)Z(t s)(JeAs)F(ds) + eAteAt(1 F(t))

The kernel is positive (leaves a cone invariant) since

s, (JeAs)X(JeAs) 0, ifX 0

The results can be generalized to Volterra Eqs. with positive operators:4

4Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors

webpages.Stochastic Hybrid Systems with Renewal Transitions 15/19

Example and Conclusions


31/37

Outline





Example and Conclusions Networked Control Example

E l


32/37

ExampleRecall

Plant connected to controller though a shared network.

Only control law is sent through the network. i.i.d. random access time to the network. Consider two cases:

Case I: controller transmits data collected at time it initially tried to

transmit data.Case II:controller (re)samples sensor, computes control, and transmitsmost recent data.

Moreover

Plant: Inverted pendulum. Network distribution- uniform withsupport T.

LQR control law minimizes

e(t) = xP(t)xP(t) + u(t)

2

and yields rate of convergence = 2 ((x(t)

x(t) ce2t

)).Stochastic Hybrid Systems with Renewal Transitions 17/19


R l


33/37

ResultsTransitory (support T= 0.1)



R l


34/37

ResultsTransitory (support T= 0.1)

AsymptoticTable: Exponential decay rates E[e(t)] cet

(a) Case IT 0.1 0.2 0.3 0.4 0.5 > 0.521

2.000 2.000 2.000 0.849 0.118 NOT MSS(b) Case II

T 0.4 0.6 0.8 1.0 1.2 > 1.211

2.000 2.000 1.969 0.477 7.63 105 NOT MSS

Conclusion: Better to transmit most recent data.Stochastic Hybrid Systems with Renewal Transitions 18/19

Example and Conclusions Conclusions and Future Work

C l i d F W k


35/37

Conclusions and Future WorkKey ideas:

Piecewise deterministic systems (Davis, Markov Models and Optimization,Chapman & Hall, 1993).

Volterra analysis to specific class of SHS with renewal transition providesdifferent results.

Volterra equation with positive kernel structure.

Networked control modeled by SHSs.



C l i d F W k


36/37


Piecewise deterministic systems (Davis, Markov Models and Optimization,

Chapman & Hall, 1993).



Networked control modeled by SHSs.Future work:

Compare method of obtaining the probability density function thought alarge number of momemnts with solution methods to Fokker-Plank equation.

Asymptotic for other higher order functions (other thanE

[x

(t)x(t)])-Volterra equations has positive kernel in the sense that leaves the cone ofpositive definite tensors (instead of positive semi-definite matrices) invariant.



C l i d F t W k


37/37


Piecewise deterministic systems (Davis, Markov Models and Optimization,

Chapman & Hall, 1993).



Networked control modeled by SHSs.Future work:

Compare method of obtaining the probability density function thought alarge number of momemnts with solution methods to Fokker-Plank equation.

Asymptotic for other higher order functions (other thanE

[x

(t)x(t)])-Volterra equations has positive kernel in the sense that leaves the cone ofpositive definite tensors (instead of positive semi-definite matrices) invariant.

Thank you.Stochastic Hybrid Systems with Renewal Transitions 19/19

acc10presentation

Documents