情報理論的アプローチによる制御システムの性能限...

情報理論的アプローチによる

制御システムの性能限界解析

石井秀明

東京工業大学

情報理工学院

電子情報通信学会 IT/SIP/RCS 研究会

広島市青少年センター

2020年1月23日(木)

2

大規模なネットワーク化システム

CPS, IoT, 超スマート社会：ネットワーク構造を持つ複雑システム

広域に渡る多数のエージェントが協調

高度な知能化や学習の実現：種々のフィードバック制御機構

共通する制御，数理・データ科学の課題

piqsels

3

大規模ネットワーク系の解析と制御

制御工学 1. 通信制約

2. 大規模ネットワーク系

通信を介した遠隔制御

量子化，データ損失，情報理論

マルチエージェント系

協調制御，分散最適化

複雑ネットワーク

ロバスト制御，最適化

ディジタル制御，…

3. セキュリティ・学習

最近の研究

サイバー攻撃下の制御系

攻撃検知，耐故障性

プライバシー保護下の分散学習

制御ルール

入力・作用センサ出力

ダイナミカルシステム

フィードバック制御

自動車，飛行機，ロケット，

ロボット，電力システム…

望みの動きを実現する

システム論アプローチ：共通する方法論を構築する

数学的モデル

（運動方程式など）

制御ルール

入力・作用センサ出力

ダイナミカルシステム

ネットワークを介した制御

通信ネットワーク

従来のフィードバック制御系＋通信路

通信路：どう帯域制限を制御系の設計で考慮するか？

制御理論と通信理論の境界領域！

6

(1) 制御と量子化の同時設計

通信路：有限の通信レート [ビット/秒]

信号の量子化: システムに関する本質的な情報とは？

最小データレート定理 Wong & Brockett (2000)

D/A

制御対象

A/D

符号器

通信路

TH TS

BuAxx +=

E

)(tx)(tu

D復号器

サンプル周期 :T

制御器

K

制御対象に不確かさ

7

不確かさを持つシステム

通信路に不確かさ

D/A

制御器

制御対象

A/DTH TS

BuAxx += )(tx)(tu

K量子化器

Q通信路

不確かさ→大，フィードバックすべき情報量→大

量子化器が定める「情報の粗さ」の限界導出

適応・学習制御やロバスト制御に基づく手法

T. Hayakawa, H. Ishii, & K. Tsumura (Automatica 2009)

K. Okano & H. Ishii (Automatica 2017), K. Takijiri & H. Ishii (Sys. Cont. Lett 2018)

パケット損失の確率的情報を用いる手法

H. Hoshina, K. Tsumura, H. Ishii (Automatica 2009)

8

無線通信の広がり → 攻撃者によるパケット損失・データ改ざん

制御系の性能の劣化，不安定化

ITベースの手法によらない，動的モデルに基づくセキュリティ対策

ジャミング攻撃

(2) ネットワーク化制御に対するサイバー攻撃

D/A

サンプル周期 :T

制御器

制御対象

A/DTH TS

BuAxx +=)(tx)(tu

K量子化器

Q通信路

A. Cetinkaya, H. Ishii & T. Hayakawa (IEEE TAC 2017 & 2019)(SIAM J. Cont. Opt. 2018a & 2018b)

フィードバック制御で越えられない本質的限界

ボードの積分定理

情報理論的な定式化による一般化

9Oberwolfach Mini-Workshop: Entropy, Information, and Control

(March 2018)

(3) 制御システムにおける性能限界

Performance Limitation Analysis of Control Systems

Based on an Information Theoretic Approach

Hideaki ISHIIDepartment of Computer Science

Tokyo Institute of Technology

Joint Work with

Song FANG (New York University)

Jie CHEN (City University of Hong Kong)

IEICE IT/SIP/RCS Workshop

Hiroshima, January 2020

1 / 27

Limitations of Control: Bode IntegralLimitations of Control: Bode Integral

K(z) P(z)

d

e yz

_

Figure: A linear, time-invariant (LTI) feedback system

Song FANG (TokyoTech) SICE ISCS 2017 8 / 54

Linear, time-invariant (LTI) feedback system

Disturbance rejection: Reduce the influence of d on e via feedback

Question: What are the limitations on the performance?

Sensitivity function (from d to e): S(z) = 11+K(z)P(z)

In general, want low sensitivity in low freqencies

2 / 27

Limitations of Control: Bode Integral

No free lunch theorem of feedback mechanism

“Waterbed effect”

Conservation of “dirt”: G. Stein (2003)

3 / 27

Limitations of Control: Bode Integral

Bode (1945), Freudenberg & Looze (1985), Sung & Hara (1988)

Assumptions:

- K (z)P (z): Strictly proper with poles at λi- Closed-loop system: Stable

Lemma

1

2π

∫ π

−πlog

∣∣∣∣1

1 + K (ejω)P (ejω)

∣∣∣∣ dω =m∑

i=1

max {0, log |λi |}

Based on transfer functions and complex analysis (Jensen’s formula)

Not possible to achieve arbitrary sensitivity reduction, e.g.,|sensitivity| < 1 over the whole frequency range

λi (A): Plant more unstable → worse

4 / 27

Control vs. Communication

Control

Information utilization

Delay are detrimental, maycause instability

Feedback is essential

⇒ “Systems” approach

Communication

Information transmission

Delays are tolerable, coding witharbitrarily long block lengths

Feedback is scattered, e.g., TCP

⇒ “Signals” approach

5 / 27

Control and Communication in the Early Days

There is an obvious analogy between the problem of smoothing the datato eliminate or reduce the effect of tracking errors and the problem ofseparating a signal from interfering noise in communications systems.

- R. Blackman, H. Bode, C. Shannon (1946)

6 / 27

Information Theory: Basic Concepts

Differential entropy of a random vector x ∈ Rm with density px (a):

h (x) = −∫

px (a) log px (a) da

Entropy rate of a stochastic process {xk}:

h∞ (x) = lim supk→∞

h (x0,...,k)

k + 1

Mutual information between two random vectors x ∈ Rm1 , y ∈ Rm2

with densities px (a), py (b) and joint density px,y (a, b):

I (x; y) =

∫px,y (a, b) log

px,y (a, b)

px (a) py (b)dadb

Mutual information rate between two stochastic processes {xk}, {yk}:

I∞ (x; y) = lim supk→∞

I (x0,...,k ; y0,...,k)

k + 1

7 / 27

Stationary Processes

A zero-mean stochastic process {xk} , xk ∈ Rm is asymptoticallystationary if the following limit exists for every k :

Rx (k) = limi→∞

E[xix

Ti+k

]

Asymptotic power spectrum:

Sx (ω) =∞∑

k=−∞Rx (k) e−jωk

8 / 27

Bode-Type Integrals via Information TheoryBode-Type Integrals

d

e yz

Figure: A feedback system

LTI plant, causal controller


Figure: A feedback system

Plant: LTI

Controller: Causal

9 / 27

Sensitivity Case Martins, Dahleh, & Doyle (2007, 2008)

Assumptions:

- Plant:[xk+1

yk

]=

[A BC 0

] [xkek

], xk ∈ Rm, ek ∈ R, yk ∈ R

- λi (A): eigenvalues of A, i = 1, . . . ,m

- Controller: zk = Kk (y0,...,k)

- {dk}: Gaussian asymptotically stationary

- {ek}: Asymptotically stationary

- |h(x0)| <∞ ⇒ Not completely known nor unknown

- {dk}, x0 independent

- Closed-loop system is MS stable: supk E[xTk xk

]<∞

10 / 27

Sensitivity Case Martins, Dahleh, & Doyle (2007,2008)

Lemma

1

2π

∫ π

−πlog

√Se (ω)

Sd (ω)dω ≥

m∑

i=1

max {0, log |λi (A)|}

Stochastic version of Bode integral by analyzing entropies

General class of controllers (nonlinear and time varying)

In the LTI case, in general,

√Se (ω)

Sd (ω)=

∣∣∣∣1

1 + K (ejω)P (ejω)

∣∣∣∣ ,

but in transfer function, |h(x0)| <∞ does not hold.

Not clear when equality holds.

11 / 27

Derivation for the Sensitivity Case

Step 1: Power spectrum and entropy rates

Entropy rate

h∞(x) = lim supk→∞

h(x0,...,k)

k + 1

For an asymptotically stationary signal:

h∞(x) ≤ 1

2π

∫ π

−πlog√

2πeSx(ω)dω,

where equality holds if Gaussian.

For system’s input and output:

1

2π

∫ π

−πlog

√Se (ω)

Sd (ω)dω ≥ h∞(e)− h∞(d)

12 / 27


Step 2: Conservation law of entropy

How does h(d0,...,k) affect h(e0,...,k)?

h(e0,...,k)− h(d0,...,k) = I (e0,...,k ; x0)

⇒ Mutual information between the initial state and output

Then, divide this by k + 1 and take the limsup:

h∞(e)− h∞(d) ≥ lim infk→∞

I (e0,...,k ; x0)

k + 1

13 / 27


Step 3: Relation to unstable poles

If the closed loop is stable, then

lim infk→∞

I (e0,...,k ; x0)

k + 1≥

m∑

i=1


Implication: As the poles become more unstable, signals in the loopcontain more info about x0.

Lemma

1

2π

∫ π

−πlog

√Se (ω)

Sd (ω)dω ≥

m∑

i=1


14 / 27

Bode-Type Integrals for Networked Feedback SystemNetworked Feedback Systems

d

e y

Channel

w z

,fu v n

n

uv

Figure: Control over an uplink channel

LTI plant, causal controller, causal channel


Figure: Control over an uplink channel

LTI plant

Causal controller & causal channel

15 / 27

Bode-Type Integrals

Assumptions:

- Plant:[xk+1

yk

]=

[A BC 0

] [xkek

], xk ∈ Rm, ek ∈ R, yk ∈ R

- Controller: wk = Kk(y0,...,k)

- Channels: uk = fk(v0,...,k ,n0,...,k)

- Encoder/Decoder: vk = Ek(w0,...,k), zk = Dk(u0,...,k)

- {dk}, {ek}: asymptotically stationary,h(d0,...,k)

k+1 converges as k →∞- |h(x0)| <∞- {nk}, {dk}, and x0 are mutually independent.

- Closed-loop system is MS stable: supk E[xTk xk

]<∞

16 / 27

Bode-Type Integrals

Theorem

1

2π

∫ π

−πlog

√Se (ω)

Sd (ω)dω ≥ I∞ (n; e) +

m∑

i=1


Se (ω) /Sd (ω): Power spectra ratio, entire frequency range

I∞ (n; e): Mutual information rate of channels

Implications:

I∞ (n; e): Channel noisier → worse

λi (A): Plant more unstable → worse

Fang, Chen, & I. (Automatica 2017)

17 / 27

Mutual Information Rate and Channel Blurredness

Theorem

If D (·) is further assumed injective, then

supd

I∞ (n; e) ≥ B

B: Blurredness of the channel

18 / 27

Channel Blurredness: How Blurred/Noisy?

Definition (Channel blurredness)

For a general causal noisy channel, input {vk}, output {uk}, and noise{nk},

B , infpv

I∞ (n;u) = infpv

lim supk→∞

I (n0,...,k ;u0,...,k)

k + 1

Cf.: Channel capacity

C = suppv

I∞ (v;u) = suppv

lim supk→∞

I (v0,...,k ;u0,...,k)

k + 1

How blurred vs. how transparent

19 / 27

AWGN Channel

Additive white Gaussian noise (AWGN) channel: uk = vk + nk

- {nk}: zero-mean AWGN with variance N

- Power constraint: limk→∞1

k+1

∑ki=0 v

2i ≤ P

Theorem (AWGN channel blurredness)

B =1

2log

(1 +

N

P

)

Channel capacity:

C =1

2log

(1 +

P

N

)

Duality:

B =1

2log

(1 +

1

22C − 1

)

20 / 27

AWGN ChannelPREPRINT SUBMITTED TO IEEE TRANSACTIONS ON AUTOMATIC CONTROL 5

0 1 2 3 4 5 6 7 8 9 100

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Power Constraint

Cha

nnel

Blu

rred

ness

and

Cap

acity

Blurredness

Capacity

Fig. 2. Channel blurredness vs. channel capacity of a scalar AWGN channel.

In what follows, we show that LTI stable systems donot change the non-Gaussianity of asymptotically stationaryprocesses.

Proposition 3: Consider an m-input, m-output LTI stablesystem L(z). If the input process {xk} is asymptoticallystationary, then the output process {yk} is also asymptoticallystationary. Furthermore,

J∞ (y) = J∞ (x) . (8)

Proof: It is known from [19], [40], [46] that

h∞ (y) = h∞ (x) +1

2π

∫ π

−πlog∣∣detL

(ejω)∣∣dω.

Hence, since det Φy(ω) =∣∣detL

(ejω)∣∣2 det Φx(ω), we have

J∞ (y) =1

2π

∫ π

−πlog

√(2πe)

m |detL (ejω)|2 det Φx(ω)dω

− h∞ (y)

=1

2π

∫ π

−πlog√

(2πe)m

det Φx (ω)dω − h∞ (x)

= J∞ (x) .

B. Channel Blurredness

Channel blurredness was first introduced in [35] for scalarchannels and is herein generalized to MIMO channels.

Definition 4: Consider a general causal noisy channel withinput process {vk}, output process {uk} and noise process{nk}. Its channel blurredness, measured in bits, is defined as

B , infpvI∞ (n;u) = inf

pvlim supk→∞

I (n0,...,k;u0,...,k)

k + 1, (9)

where the infimum is taken over all possible densities pv ofthe input process allowed for the channel.

Unlike channel capacity, channel blurredness is defined asthe infimum of mutual information (rate) between the channelnoise and output, and thus lends a direct characterization of the

noise effect. For comparison, consider a scalar AWGN channelwith noise variance N and power constraint P . It was foundin the authors’ earlier work [35] that

B =1

2log

(1 +

N

P

).

A simple calculation shows that for this channel, the channelblurredness is related to the channel capacity by

B =1

2log

(1 +

1

22C − 1

).

Fig. 2 shows the blurredness and capacity of a sample AWGNchannel with noise variance N = 1. It is worth pointing out,however, that such a relationship may not be available forother types of channels. Analytical expressions of the channelblurredness will be developed in the sequel for more generalMIMO channels with no linkage to channel capacity.

IV. BODE-TYPE INTEGRAL INEQUALITIES

We now derive MIMO Bode-type integral inequalities fornetworked systems controlled over uplink and downlink noisychannels. Consider the system depicted in Fig. 3. In this setup,the reference signal r is assumed to be known. The plant Pis an LTI system with its state-space model given by

[xk+1

yk

]=

[A BC 0

] [xkek

],

where xk ∈ Rm is the state, ek ∈ Rl the control input, andyk ∈ Rp the plant’s output. The system matrices are A ∈Rm×m,B ∈ Rm×l, and C ∈ Rp×m. The initial state x0 isassumed to be a random vector with a finite entropy h (x0).Let λi (A), i = 1, . . . ,m, denote the eigenvalues of the matrixA. The Mahler measure [47]–[49] of A is then defined as

M (A) =

m∏

i=1

max {1, |λi (A)|} .

Note that an attempt was made in [50] to extend the Mahlermeasure to an instability measure for nonlinear systems. Thecontroller K is assumed to be causal. In other words, thecontroller can be nonlinear and time-varying, as long as it iscausal and stabilizes the plant P , whereas K is said to stabilizeP if the closed-loop system in Fig. 3 is mean-square stable,i.e., supk∈N E

[xTk xk

]<∞. The uplink (ul) and downlink (dl)

channels are assumed to be causal with inputs{vulk}

,{vdlk}

,outputs

{uulk}

,{udlk}

, and noises{nulk}

,{ndlk}

, respectively,where uulk ,v

ulk ,n

ulk ∈ Rl and udlk ,v

dlk ,n

dlk ∈ Rp. The uplink

and downlink encoders and decoders Eul, Dul, Edl, Ddl areassumed to be causal as well. Furthermore,

{nulk}

,{ndlk}

,{dk} ,vdl0 , x0 are assumed to be mutually independent. Notethat if the feedthrough matrix D 6= 0 in the plant, thenthe controller should be assumed strictly causal in order toensure the strict causality of the open-loop system, so as toprevent dk, k ∈ N, and z0 from being dependent. It should beemphasized that both the physical-layer models and the higher-layer input-output properties of the communication channelscan be dealt with in this general setup.

The following information-theoretic inequality provides anentropy domain characterization of disturbance attenuation,

Analysis on additive colored Gaussian noise (ACGN) channels andfading channels

In general, difficult to obtain explicit relations with channel capacity

Fang, I., & Chen (IEEE TAC 2017)

21 / 27

Further Studies

Analysis of MIMO systems

Power allocation: “Fire-Quenching” ↔ “Water filling”Fang, I., & Chen (IEEE TAC 2017, CDC 2017)

Bode’s perspective on estimation problems

22 / 27

Extentions to Estimation Problems 3

_

kx kz

kv

kw

kx ky ke ku

1z I−1k+x

1z I−1k+x

ky

A

A

C

C

kK

Fig. 1. The Kalman filter in observer form.

zeros on the unit circle. Let the relative degree of f (z) begiven by ν. Then,

1

2π

∫ π

−πlog∣∣f(ejω)∣∣dω

= limz→∞

log |zνf (z)|+∑

i

max {0, log |ϕi|}

−∑

j

max {0, log |ηj |} , (3)

where ϕi denote the zeros of f (z) and ηj denote its poles.

C. Kalman Filter

We next give a brief review of the Kalman filter. Considerthe observer form of the Kalman filtering system [1], [2], [4]as depicted in Fig. 1, where the system is LTI with its state-space model given by

{xk+1 = Axk + wk,

yk = Cxk + vk,

where xk ∈ Rm is the state to be estimated, yk ∈ Rlis the system output, wk ∈ Rm is the process noise, andvk ∈ Rl is the measurement noise. The system matricesare A ∈ Rm×m and C ∈ Rl×m, and (A,C) is assumedto be detectable. Suppose that {wk} and {vk} are whiteGaussian with covariance matrices W = E

[wkw

Tk

]≥ 0 and

V = E[vkv

Tk

]> 0, respectively, and that the initial state x0

is Gaussian with covariance Σx0 satisfying 0 < det Σx0 <∞.Furthermore, {wk}, {vk}, and x0 are assumed to be mutuallyuncorrelated.

The Kalman filter is given by

xk+1 = Axk + uk,yk = Cxk,ek = yk − yk,uk = Lkek,

where xk ∈ Rm, yk ∈ Rl, ek ∈ Rl, and uk ∈ Rm. In addition,

Kk = APkCT(CPkC

T + V)−1

, (4)

where Pk denotes the state estimation error covariance matrixdefined as

Pk = E[(xk − xk) (xk − xk)

T].

Herein, Pk is obtained iteratively using the Riccati equation

Pk+1 = APkAT +W

−APkCT(CPkC

T + V)−1

CPkAT

with P0 = E[x0x

T0

]. Note that by a slight abuse of notation,

the Kk given in (4) is indeed the observer gain [41] for theobserver form of the Kalman filter, different from the Kalmangain [1], [2] given by PkCT

(CPkC

T + V)−1

.It is known that the Kalman filtering system converges, i.e.,

the estimator is asymptotically stable, the state estimation error{xk − xk} and the innovation process {ek} are asymptoti-cally stationary, and {ek} is asymptotically white, under theassumption that the system (A,C) is detectable. Moreover,in steady state, the optimal state estimation error covariancematrix

P = limk→∞

E[(xk − xk) (xk − xk)

T]

attained by the Kalman filter satisfies the algebraic Riccatiequation

P = APAT +W −APCT(CPCT + V

)−1CPAT , (5)

whereas the steady-state observer gain is given by

K = APCT(CPCT + V

)−1. (6)

In addition, the optimal steady-state innovation covariancematrix is found to be

Σe = limk→∞

E[eke

Tk

]

= limk→∞

E[(yk − yk) (yk − yk)

T]

= CPCT + V. (7)

It is worth mentioning that since {ek} is (asymptotically)white, CPCT+V denotes not only its (asymptotic) covariancematrix Σe, but also its (asymptotic) power spectrum Φe (ω).Moreover, when l = 1, (7) reduces to

σ2e = lim

k→∞E[e2k

]= limk→∞

E[(yk − yk)

2]

= CPCT + σ2v.

(8)

On the other hand, it can be shown that

Σz−y = limk→∞

E[(zk − yk) (zk − yk)

T]

= CPCT , (9)

which reduces to σ2z−y = CPCT when l = 1.

III. OPTIMAL ERROR COVARIANCE BY KALMAN FILTER

We now provide an explicit expression of the determinant ofthe optimal innovation covariance matrix Σe by the Kalmanfilter; it is determined by the plant dynamics and the noisestatistics in an integral characterization, which is obtainedby examining the algebraic Riccati equation using analyticfunction theory.

System to be estimated xk+1 = Axk + wk , yk = Cxk + vkwhere xk ∈ Rm and yk ∈ R, and {wk} and {vk} are white Gaussianwith covariance matrices W and V , respectively.

(Standard) Kalman filter

xk+1 = Axk + uk , yk = Cxk

ek = yk − yk , uk = Kek .

23 / 27

Extentions to Estimation Problems

Estimation error covariance matrix

P = limk→∞

E[(xk − xk)(xk − xk)T

]

It can be obtained by the Riccati equation:

P = APAT + W − APCT (CPCT + V )−1CPAT

Kalman gain K = APCT (CPCT + V )−1

Question: Is there an analytic expression for the optimal steady-stateinnovation covariance matrix?

Σek = limk→∞

E[eke

Tk

]= lim

k→∞E[(yk − yk)(yk − yk)T

]

= CPCT + V

24 / 27

Analysis on the Innovation Covariance Matrix

Theorem

The determinant of the optimal output estimation error covariance matrixby the Kalman filter is given by

det Σe =m∏

i=1

max{1, |λi (A)|2}2 12π

∫ π−π log det[C(ejω I−A)−1W (ejω I−A)−TCT ]dω

When A is stable, it becomes the Wiener-Masani formula (1957). ForSISO case, it is also known as the Kolmogorov-Szego formula.

In that case, C (ejωI − A)−1W (ejωI − A)−TCT becomes the powerspectrum of yk .

Non-stationary generalization obtained through the Bode integral

Fang, I., & Chen (ACC 2018)

25 / 27

Summary

Interplay of control and communication

Networked feedback systems

Bode-type integrals by evaluating entropies of signals

New notions: Channel blurredness, Negentropy rate, Power gain

Ongoing work: State estimation

- Analytic expressions for error covariance- Continuous-time case

26 / 27

References

J. Chen, S. Fang, and H. Ishii, Fundamental limitations and intrinsiclimits of feedback: An overview in an information age, AnnualReviews in Control, 47: 155-177, 2019.

S. Fang, J. Chen, and H. Ishii, Towards Integrating Control andInformation Theories: From Information-Theoretic Measures toControl Performance Limitations, Lecture Notes in Control andInformation Sciences, vol. 465, Springer, 2017.

S. Fang, H. Ishii, and J. Chen, An integral characterization of optimalerror covariance by Kalman filtering, Proc. American ControlConference, 2018.

S. Fang, M. Skoglund, K. H. Johansson, H. Ishii, and Q. Zhu, Genericvariance bounds on estimation and prediction errors in time seriesanalysis: An entropy perspective, IEEE Information Theory Workshop(ITW), 2019.

27 / 27

情報理論的アプローチによる 制御システムの性能限...

Documents

情報理論的アプローチによる制御システムの性能限...