Download - ACC10presentation
-
8/7/2019 ACC10presentation
1/37
Stochastic Hybrid Systems with
Renewal Transitions
Duarte Antunes Joao Hespanha Carlos Silvestre
Institute for Systems and Robotics, Instituto Superior Tecnico
Dep. of Electrical and Computer Eng., UC Santa Barbara
July 1, 2010
2010 American Control ConferenceBaltimore, Maryland, USA
Stochastic Hybrid Systems with Renewal Transitions 1/19
-
8/7/2019 ACC10presentation
2/37
Outline
Stochastic Hybrid Systems (SHS)MotivationSHS definitionLiterature Review
Main resultsTransitory analysisAsymptotic analysis
Example and ConclusionsNetworked Control ExampleConclusions and Future Work
Stochastic Hybrid Systems with Renewal Transitions 2/19
-
8/7/2019 ACC10presentation
3/37
Stochastic Hybrid Systems (SHS)
Outline
Stochastic Hybrid Systems (SHS)MotivationSHS definitionLiterature Review
Main resultsTransitory analysisAsymptotic analysis
Example and ConclusionsNetworked Control ExampleConclusions and Future Work
Stochastic Hybrid Systems with Renewal Transitions 3/19
-
8/7/2019 ACC10presentation
4/37
Stochastic Hybrid Systems (SHS) Motivation
Networked Control
Control loops closed through shared network.
Stochastic Hybrid Systems with Renewal Transitions 4/19
-
8/7/2019 ACC10presentation
5/37
St h ti H b id S t (SHS) M ti ti
-
8/7/2019 ACC10presentation
6/37
Stochastic Hybrid Systems (SHS) Motivation
Networked Control
Control loops closed through shared network. For simplicity: Focus on single loop & Only the control law is sent.
Need network abstraction.Stochastic Hybrid Systems with Renewal Transitions 4/19
Stochastic Hybrid Systems (SHS) Motivation
-
8/7/2019 ACC10presentation
7/37
Stochastic Hybrid Systems (SHS) Motivation
Network abstraction
User (sensor, controller,...) waits independent and identically distributed(i.i.d.) times to gain access to network.
Stochastic Hybrid Systems with Renewal Transitions 5/19
Stochastic Hybrid Systems (SHS) Motivation
-
8/7/2019 ACC10presentation
8/37
Stochastic Hybrid Systems (SHS) Motivation
Network abstraction
User (sensor, controller,...) waits independent and identically distributed(i.i.d.) times to gain access to network.
Example- CSMA-type protocol
Stochastic Hybrid Systems with Renewal Transitions 5/19
Stochastic Hybrid Systems (SHS) Motivation
-
8/7/2019 ACC10presentation
9/37
Stochastic Hybrid Systems (SHS) Motivation
Network abstraction
User (sensor, controller,...) waits independent and identically distributed(i.i.d.) times to gain access to network.
Example- CSMA-type protocol
Stochastic Hybrid Systems with Renewal Transitions 5/19
Stochastic Hybrid Systems (SHS) Motivation
-
8/7/2019 ACC10presentation
10/37
Stochastic Hybrid Systems (SHS) Motivation
Network abstraction
User (sensor, controller,...) waits independent and identically distributed(i.i.d.) times to gain access to network.
Example- CSMA-type protocol
Stochastic Hybrid Systems with Renewal Transitions 5/19
Stochastic Hybrid Systems (SHS) Motivation
-
8/7/2019 ACC10presentation
11/37
y y ( )
Network abstraction
User (sensor, controller,...) waits independent and identically distributed(i.i.d.) times to gain access to network.
Example- CSMA-type protocol
Key: model time intervals tk+1 tk as i.i.d.
Stochastic Hybrid Systems with Renewal Transitions 5/19
Stochastic Hybrid Systems (SHS) Motivation
-
8/7/2019 ACC10presentation
12/37
y y ( )
Network abstraction
User (sensor, controller,...) waits independent and identically distributed(i.i.d.) times to gain access to network.
Example- CSMA-type protocol
Key: model time intervals tk+1 tk as i.i.d.
Prob. Distribution of the access times
Npairs of nodesimplement aboveprotocol.
Back-offs
Uniform([0, hs])Stochastic Hybrid Systems with Renewal Transitions 5/19
Stochastic Hybrid Systems (SHS) Motivation
-
8/7/2019 ACC10presentation
13/37
y y ( )
Network abstraction
User (sensor, controller,...) waits independent and identically distributed(i.i.d.) times to gain access to network.
Example- CSMA-type protocol
Key: model time intervals tk+1 tk as i.i.d.
Prob. Distribution of the access times
Npairs of nodesimplement aboveprotocol.
Back-offs
Uniform([0, hs])Stochastic Hybrid Systems with Renewal Transitions 5/19
Stochastic Hybrid Systems (SHS) Motivation
-
8/7/2019 ACC10presentation
14/37
SHS modelAfter waiting i.i.d. time to obtain access to the network:
Case I
controller transmitsdata collected at timeit initially tried totransmit data.
x = (xP, u , v)A models system andcontroller dynamics
J1 models samplingv(rk) = KxP(r
k )
J2 models control updateu(sk) = v(s
k )
Stochastic Hybrid Systems with Renewal Transitions 6/19
Stochastic Hybrid Systems (SHS) Motivation
-
8/7/2019 ACC10presentation
15/37
SHS modelAfter waiting i.i.d. time to obtain access to the network:
Case I
controller transmitsdata collected at timeit initially tried totransmit data.
x = (xP, u , v)A models system andcontroller dynamics
J1 models samplingv(rk) = KxP(r
k )
J2 models control updateu(sk) = v(s
k )
Case IIcontroller (re)samplessensor, computescontrol, and transmits
most recent data.
x = (xP, u)Jmodels sample and controlupdateu(t
k) = Kx
P(tk
)
Stochastic Hybrid Systems with Renewal Transitions 6/19
Stochastic Hybrid Systems (SHS) SHS definition
-
8/7/2019 ACC10presentation
16/37
SHS definition
Differential Eqs.
x(t) = Aq(t)x(t)
Reset maps
(q(tk), x(tk)) = (l(q(tk )), Jq(t
k),lx(tk ))
Transition distributions
Fi,l
1 l nl, 1 q(t), i nq
Stochastic Hybrid Systems with Renewal Transitions 7/19
Stochastic Hybrid Systems (SHS) SHS definition
-
8/7/2019 ACC10presentation
17/37
SHS definition
Differential Eqs.
x(t) = Aq(t)x(t)
Reset maps
(q(tk), x(tk)) = (l(q(tk )), Jq(t
k),lx(tk ))
Transition distributions
Fi,l
1 l nl, 1 q(t), i nq
Execution:Transition times tk+1 determined by first hq(tk)j Fq(tk)j to trigger.
Stochastic Hybrid Systems with Renewal Transitions 7/19
Stochastic Hybrid Systems (SHS) SHS definition
-
8/7/2019 ACC10presentation
18/37
SHS definition
Differential Eqs.
x(t) = Aq(t)x(t)
Reset maps
(q(tk), x(tk)) = (l(q(tk )), Jq(t
k),lx(tk ))
Transition distributions
Fi,l
1 l nl, 1 q(t), i nq
Execution:Transition times tk+1 determined by first hq(tk)j Fq(tk)j to trigger.See paper for assumptions on atom points of the trans. distributions.
Stochastic Hybrid Systems with Renewal Transitions 7/19
Stochastic Hybrid Systems (SHS) Literature Review
-
8/7/2019 ACC10presentation
19/37
Literature Review
DynamicsWiener
Process
Stochastic
Intervals
Stochastic
Transitions
SHS1 Non-linear Yes Guards NoSHS2 Non-linear Yes Yes-Exponential No
SHS3 Non-linear No/Yes* Yes-SD No/Yes*MJLS Linear No/Yes* Yes-Exponential No/Yes*PDPs Non-linear No/Yes* Yes-SD Yes-SD
Our work Linear No/Yes* Yes No/Yes*
SD-state dependent*See [SHS3]
SHS1 -Hu et all, Towards a Theory of Stochastic Hybrid Systems, Workshop on Hybrid Systems: Computation andControl,2000.
SHS2 -Ghosh et all, Ergodic Control Of Switching Diffusions, SIAM Journal of Control and Optimization,1997.SHS3 -Hespanha, A Model for Stochastic Hybrid Systems with Application to Communication Networks,Special Issue on
Hybrid Systems, 2005.MJLS -Mariton, Jump Linear Systems in Automatic Control, 1990, Marcel Dekker Inc.
PDPs -Davis, Markov Models and Optimization, Chapman & Hall, 1993.
Stochastic Hybrid Systems with Renewal Transitions 8/19
Main results
-
8/7/2019 ACC10presentation
20/37
Outline
Stochastic Hybrid Systems (SHS)MotivationSHS definitionLiterature Review
Main resultsTransitory analysisAsymptotic analysis
Example and ConclusionsNetworked Control ExampleConclusions and Future Work
Stochastic Hybrid Systems with Renewal Transitions 9/19
Main results Transitory analysis
-
8/7/2019 ACC10presentation
21/37
Moment analysis
Theorem-Any state moment can be obtained through Volterra Eq.
The following holds
E[xi1(t)m1xi2(t)
m2 . . . xir(t)mr ] =
nq
i=1czmi (t), for
mj = m, mj > 0,
where
zmi (t) =
t0
i(ds)zmi (t s) + h
mi (t) (Volterra equation), (1)
i(ds) is a matrix-measure. For AC distributions i(ds) = Ki(s)ds.
Stochastic Hybrid Systems with Renewal Transitions 10/19
Main results Transitory analysis
-
8/7/2019 ACC10presentation
22/37
Moment analysis
Theorem-Any state moment can be obtained through Volterra Eq.
The following holds
E[xi1(t)m1xi2(t)
m2 . . . xir(t)mr ] =
nq
i=1czmi (t), for
mj = m, mj > 0,
where
zmi (t) =
t0
i(ds)zmi (t s) + h
mi (t) (Volterra equation), (1)
i(ds) is a matrix-measure. For AC distributions i(ds) = Ki(s)ds.
Example: x(t) = Ax(t), x(tk) = J x(tk ), tk+1 tk F
Stochastic Hybrid Systems with Renewal Transitions 10/19
Main results Transitory analysis
-
8/7/2019 ACC10presentation
23/37
Moment analysis
Theorem-Any state moment can be obtained through Volterra Eq.
The following holds
E[xi1(t)m1xi2(t)
m2 . . . xir(t)mr ] =
nq
i=1czmi (t), for
mj = m, mj > 0,
where
zmi (t) =
t0
i(ds)zmi (t s) + h
mi (t) (Volterra equation), (1)
i(ds) is a matrix-measure. For AC distributions i(ds) = Ki(s)ds.
Example: x(t) = Ax(t), x(tk) = J x(tk ), tk+1 tk F
E[x(t)x(t)] = x0
Z(t)x0, where
Z(t) = t
0
(JeAs)Z(t s)(JeAs)F(ds) + eAteAt(1 F(t)).
Stochastic Hybrid Systems with Renewal Transitions 10/19
Main results Transitory analysis
-
8/7/2019 ACC10presentation
24/37
Why moment analysis
Computing E[a(t)], a(t) = xi1(t)m1xi2(t)
m2 . . . xir (t)mr allows to
Provide probability bounds at time t. Chebychevs inequality:
Prob[|b(t)| > ] E[a(t)]
2, b(t)2 = a(t)
Compute probability distribution of Markov process at time t(moment problem).
Stochastic Hybrid Systems with Renewal Transitions 11/19
Main results Asymptotic analysis
-
8/7/2019 ACC10presentation
25/37
StabilityTheorem
Suppose that transition distributions have finite support1
. Then, under easy totest technical conditions, the SHS is MSS (E[x(t)x(t)] 0) if and only if
det(I K(z)) = 0, [z] 0.
K(z)-Laplace transform of the Kernel of the Volterra Eq. describing E[x(t)x(t)].
1See the paper for the infinite support case2[1] Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors
webpages. Stochastic Hybrid Systems with Renewal Transitions 12/19
Main results Asymptotic analysis
-
8/7/2019 ACC10presentation
26/37
StabilityTheorem
Suppose that transition distributions have finite support1
. Then, under easy totest technical conditions, the SHS is MSS (E[x(t)x(t)] 0) if and only if
det(I K(z)) = 0, [z] 0.
K(z)-Laplace transform of the Kernel of the Volterra Eq. describing E[x(t)x(t)].
Nyquist Criterion! Example taken from our previous work2
nl = 1, nq = 1, A, J
given in2
F Unif([0, ])
1See the paper for the infinite support case2[1] Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors
webpages. Stochastic Hybrid Systems with Renewal Transitions 12/19
Main results Asymptotic analysis
-
8/7/2019 ACC10presentation
27/37
Equivalent stability conditions
TheoremThe following are equivalent
(A) det(I K(z)) = 0, [z] 0,
(B) (M) < 1, M:= K(0) and denotes the spectral radius.
(C) There exists a set of matrices P:= {Pi > 0, i Q} such that for every i Q
Li(P)Pi < 0,
where Li(P) is a finite-dimension operator.3
(D) For every set of matrices {Qi0,iQ} the solution to
Li(P) Pi = Qi, i Q
is unique and satisfies {Pi0,iQ}
3See the paper for the expression.Stochastic Hybrid Systems with Renewal Transitions 13/19
-
8/7/2019 ACC10presentation
28/37
Main results Asymptotic analysis
-
8/7/2019 ACC10presentation
29/37
PerformanceTheorem
If SHS is MSS and trans. distribution have finite support, then for [0, max)E[x(t)x(t)] cetx
0x0,
where max =
, ifdet(I K(a)) = 0, a (, 0),
max{a < 0 : det(I K(a)) = 0}, otherwise.
Recall that convergence rate ofE[x(t)x(t)] implies convergence rate forprobabilities bounds (performance).
Theorem
Under easy to test technical conditions, if SHS is not MSS then there exists one
unstable pole z1 ofdet(I K(zi)) = 0, i = 1, . . . , nz;, which is real andsatisfies [zi] z1, i;
Recall that E[x(t)x(t)] = x0
Z(t)x0
and from results of Volterra equation
Z(t) =
nzi=1
mi1j=0
Ri,jtj
ezit
+(t),
(t) 0.
Stochastic Hybrid Systems with Renewal Transitions 14/19
Main results Asymptotic analysis
-
8/7/2019 ACC10presentation
30/37
Positive kernel
Special property of Volterra Eq. lies at the heart of the results.
Example nq = 1, nl = 1
Z(t) = t0
(JeAs)Z(t s)(JeAs)F(ds) + eAteAt(1 F(t))
The kernel is positive (leaves a cone invariant) since
s, (JeAs)X(JeAs) 0, ifX 0
The results can be generalized to Volterra Eqs. with positive operators:4
4Antunes, Hespanha, Silvestre, Volterra Integral Approach to Impulsive Renewal Systems, 2010 available at authors
webpages.Stochastic Hybrid Systems with Renewal Transitions 15/19
Example and Conclusions
-
8/7/2019 ACC10presentation
31/37
Outline
Stochastic Hybrid Systems (SHS)MotivationSHS definitionLiterature Review
Main resultsTransitory analysisAsymptotic analysis
Example and ConclusionsNetworked Control ExampleConclusions and Future Work
Stochastic Hybrid Systems with Renewal Transitions 16/19
Example and Conclusions Networked Control Example
E l
-
8/7/2019 ACC10presentation
32/37
ExampleRecall
Plant connected to controller though a shared network.
Only control law is sent through the network. i.i.d. random access time to the network. Consider two cases:
Case I: controller transmits data collected at time it initially tried to
transmit data.Case II:controller (re)samples sensor, computes control, and transmitsmost recent data.
Moreover
Plant: Inverted pendulum. Network distribution- uniform withsupport T.
LQR control law minimizes
e(t) = xP(t)xP(t) + u(t)
2
and yields rate of convergence = 2 ((x(t)
x(t) ce2t
)).Stochastic Hybrid Systems with Renewal Transitions 17/19
Example and Conclusions Networked Control Example
R l
-
8/7/2019 ACC10presentation
33/37
ResultsTransitory (support T= 0.1)
Stochastic Hybrid Systems with Renewal Transitions 18/19
Example and Conclusions Networked Control Example
R l
-
8/7/2019 ACC10presentation
34/37
ResultsTransitory (support T= 0.1)
AsymptoticTable: Exponential decay rates E[e(t)] cet
(a) Case IT 0.1 0.2 0.3 0.4 0.5 > 0.521
2.000 2.000 2.000 0.849 0.118 NOT MSS(b) Case II
T 0.4 0.6 0.8 1.0 1.2 > 1.211
2.000 2.000 1.969 0.477 7.63 105 NOT MSS
Conclusion: Better to transmit most recent data.Stochastic Hybrid Systems with Renewal Transitions 18/19
Example and Conclusions Conclusions and Future Work
C l i d F W k
-
8/7/2019 ACC10presentation
35/37
Conclusions and Future WorkKey ideas:
Piecewise deterministic systems (Davis, Markov Models and Optimization,Chapman & Hall, 1993).
Volterra analysis to specific class of SHS with renewal transition providesdifferent results.
Volterra equation with positive kernel structure.
Networked control modeled by SHSs.
Stochastic Hybrid Systems with Renewal Transitions 19/19
Example and Conclusions Conclusions and Future Work
C l i d F W k
-
8/7/2019 ACC10presentation
36/37
Conclusions and Future WorkKey ideas:
Piecewise deterministic systems (Davis, Markov Models and Optimization,
Chapman & Hall, 1993).
Volterra analysis to specific class of SHS with renewal transition providesdifferent results.
Volterra equation with positive kernel structure.
Networked control modeled by SHSs.Future work:
Compare method of obtaining the probability density function thought alarge number of momemnts with solution methods to Fokker-Plank equation.
Asymptotic for other higher order functions (other thanE
[x
(t)x(t)])-Volterra equations has positive kernel in the sense that leaves the cone ofpositive definite tensors (instead of positive semi-definite matrices) invariant.
Stochastic Hybrid Systems with Renewal Transitions 19/19
Example and Conclusions Conclusions and Future Work
C l i d F t W k
-
8/7/2019 ACC10presentation
37/37
Conclusions and Future WorkKey ideas:
Piecewise deterministic systems (Davis, Markov Models and Optimization,
Chapman & Hall, 1993).
Volterra analysis to specific class of SHS with renewal transition providesdifferent results.
Volterra equation with positive kernel structure.
Networked control modeled by SHSs.Future work:
Compare method of obtaining the probability density function thought alarge number of momemnts with solution methods to Fokker-Plank equation.
Asymptotic for other higher order functions (other thanE
[x
(t)x(t)])-Volterra equations has positive kernel in the sense that leaves the cone ofpositive definite tensors (instead of positive semi-definite matrices) invariant.
Thank you.Stochastic Hybrid Systems with Renewal Transitions 19/19