a study on iterative learning control with adjustment of learning interval for monotone convergence...

A STUDY ON ITERATIVE LEARNING CONTROL WITH

ADJUSTMENT OF LEARNING INTERVAL FOR MONOTONE

CONVERGENCE IN THE SENSE OF SUP-NORM

Kwang-Hyun Park and Zeungnam Bien

ABSTRACT

It has been found that some huge overshoot in the sense of sup-norm maybe observed when typical iterative learning control (ILC) algorithms areapplied to LTI systems, even though monotone convergence in the sense ofλ-norm is guaranteed. In this paper, a new ILC algorithm with adjustment oflearning interval is proposed to resolve such an undesirable phenomenon, andit is shown that the output error can be monotonically converged to zero in thesense of sup-norm when the proposed ILC algorithm is applied. A numericalexample is given to show the effectiveness of the proposed algorithm.

KeyWords: Iterative learning control, monotone convergence, learning interval,sup-norm.

Manuscript received October 31, 2000; accepted February 6,2001.

The authors are with Division of EE, Department of EECS,Korea Advanced Institute of Science and Technology, 373-1Kusong-dong, Yusong-gu, Taejon, 305-701, Korea.

Asian Journal of Control, Vol. 4, No. 1, pp. 111-118, March 2002 111

I. INTRODUCTION

There has been a number of studies in designingadvanced control systems to guarantee that a given outputtrajectory will be executed by the system with acceptableaccuracy from the demand for high precision controltechnique. As one of the alternatives, the iterative learningcontrol (ILC) method has been developed by which com-plete tracking performance can be achieved as the giventask is imposed iteratively [1-5].

The important task in designing of iterative learningcontroller is to find an algorithm for generating the nextinput in such a way that the output error is reduced on suc-cessive trials. This is usually quantified by showing thatthe error converges in the sense of some norm. In most ofthe investigations, the λ-norm is adopted as a measure ofdistance between two time functions to prove the conver-gence of the ILC algorithms.

For a vector function h : [0, T] → R n , h(t) = (h1(t), …hn(t))T and a real number λ > 0, the formal definition of theλ-norm [6] is given by

h( ⋅ )λ

= sup0 ≤ t ≤ T

e – λt h(t)∞.

where h(t) ∞ = sup1 ≤ i ≤ n

h i(t)∞. From the definition of

λ-norm, it is easily shown that

h( ⋅ )λ ≤ sup

0 ≤ t ≤ Th(t)

∞ ≤ e λT h( ⋅ )λ

and thus the λ-norm is equivalent to the sup-norm definedby

h[0, T]sup

= sup0 ≤ t ≤ T

h(t)∞

where h[0, T] denotes a time function h defined over timeinterval [0, T].

Thus, the convergence property in the sense of sup-norm seems to be equivalently obtained in the sense ofλ-norm. We can, however, observe some huge overshootin the sense of sup-norm even though the monotoneconvergence is guaranteed in the sense of λ-norm.

We remark that, in the real-world applications, themaximum absolute magnitude of the error signal may beof major concern, which can cause failure of hardwarecomponents. In analyzing the behavior of a system equipedwith an ILC algorithm, it is more practical and sometimesnecessary to investigate the routes of convergence in thesense of sup-norm.

Such undesirable phenomenon of the λ-norm wasfirst observed by Lee and Bien [7], and it was reported in[8] that the pure error term of a PD-type ILC algorithmplays an important role in a bound of the interval where themonotone convergence is guaranteed in the sense of sup-norm. To be more specific, consider the linear systemdescribed by (1) and the PD-type ILC law described by (2).

112 Asian Journal of Control, Vol. 4, No. 1, March 2002

nonnegative integers k, i ≥ 0, and let α and β be nonnega-tive constants. Suppose ak, i = 0, ∀ i < 0 and 0 ≤ ρ < 1. Then,the inequality

0 ≤ a k + 1, i ≤ ρa k, i + β α i – jΣj = 0

i – 1

a k, j, ∀ k, i ≥ 0

implies

limk → ∞

a k, i = 0, ∀ i ≥ 0. (3)

Proof. For the proof, we employ the method of methema-tical induction. For each i ≥ 0, let Pi be the statement that

limk → ∞

a k, i = 0.

From the conditions, we obtain that 0 ≤ ak + 1, 0 ≤ ρak, 0 and0 ≤ ρ < 1. So, the statement P0 is true. That is

limk → ∞

a k, 0 = 0,

Now suppose that statement Pn is true for every interger nwith 0 ≤ n < m. Then, it is easily seen that

limk → ∞

β αm – jΣj = 0

m – 1

a k, j = 0.

This implies that for any : > 0, there exists a positive Ksuch that

β αm – jΣj = 0

m – 1

a k, j < :

for all k ≥ K. This gives us

ak + 1, m < ρak, m + :, ∀ k ≥ K.

We can choose :′ as follows:

a k, m – ρka 0, m +ρk

1 – ρ: <:

1 – ρ = :′.

This means that for any :′, there exists a positive K′ suchthat

a k, m – ρka 0, m +ρk

1 – ρ: < :′

for all k ≥ K′. So, we can write

limk → ∞

a k, m = limk → ∞

ρka 0, m – limk → ∞

ρk

1 – ρ:.

Since 0 ≤ ρ < 1, we can conclude that

x(t) = Ax(t) + Bu(t)

y(t) = Cx(t) (1)

u k + 1(t) = u k(t) + Γ(e k(t) – Re k(t)) (2)

Here, ek(t) = yd(t) – yk(t) is the output error, and x ∈ R n,u ∈ R r and y ∈ R m denote the state, the input and the output,respectively. A, B and C are matrices with appropriatedimensions and it is assumed that CB is a full rank matrix.Let yd(⋅) be the desired output trajectory, and ud(⋅) andxd(⋅) be the corresponding input trajectory and statetrajectory, respectively. It is shown by Lee and Bien [8]that if the desired output trajectory is given on the intervalt ∈ [0, Tsup] where Tsup is bounded by

T sup < 1

A ∞

ln 1 +A ∞(1 – ρ)

Γ(CA – RC)∞

B ∞

then, there exists ρu < 1 such that

∆u k + 1[0, T sup]sup

≤ ρu ∆u k[0, T sup]sup

where ∆uk(⋅) = ud(⋅) – uk(⋅).Thus, we find that the upper bound of Tsup depends on

CA – RC ∞: if R is chosen such that CA – RC ∞

is arb-

itrarily small, then Tsup becomes very large, implying thatthe interval for monotone convergence in the sense of sup-norm becomes wider. If the given time interval Tsup isfixed, on the other hand, we have to obtain an accuratemodel of the plant in order to get a desired error conver-gence behavior. Furthermore, when the upper bound T ofthe given time interval [0, T] is larger than Tsup, we can notguarantee the monotone convergence of the output error.

In this paper, a new ILC algorithm is proposed withadjustment of learning interval, which is found to be morerobust against parameter uncertainty, and achieves mono-tone convergence of the output error in the sense of sup-norm.

II. MAIN RESULT

In this section, monotone convergence of outputerror in the sense of sup-norm is shown for LTI systems.First, we show that the convergence can be proved directlyfrom the sup-norm, not by using λ-norm. For this end,consider LTI system described by (1) and the ILC algo-rithm described by (2).

Before showing convergence of the ILC law (2), weneed the following Lemma 1, whose result is utilized in theproof of the convergence in the sense of sup-norm.

Lemma 1. Let ak, i be a nonnegative function for every

K.H. Park and Z. Bien: A Study on Iterative Learning Control with Adjustment of Learning Interval 113

limk → ∞

a k, m = 0

which establishes the truth of the statement Pm. Bymathematical induction, (3) is true. This completes theproof. ■

Now, the convergence of the ILC law (2) will beshown.

Theorem 1. Suppose that the update law (2) is applied tothe system (1) and that the initial state at each iteration isthe same as the desired initial state, i.e., xk(0) = xd(0) fork = 0, 1, 2, …. If

I – CBΓ∞

≤ ρ < 1

then,

limk → ∞

e k[0, T]sup

= 0. (4)

Proof. It follows from (2) that

e k + 1(t) = e k(t) – C

0

t

e A(t – τ)BΓ(e k(τ ) – Re k(τ )) dτ

= (I – CBΓ)e k(t) – C

0

t

e A(t – τ)(ABΓ – BΓR)e k(τ )dτ .

(5)

Let : be a real number satisfying the following inequality:

0 < : < 1

a ln 1 +a(1 – ρ)

C ∞ ABΓ – BΓR∞

and

ti + 1 = ti + :, i = 0, 1, …, N

t0 = 0, tN + 1 = T,

where a = A ∞ . Then, from (5) we find that, for t ∈ [i:,(i + 1):],

e k + 1(t) = (I – CBΓ)e k(t) – C

0

t 1e A(t – τ)(ABΓ – BΓR)e k(τ )dτ

– C

t 1

t 2e A(t – τ)(ABΓ – BΓR)e k(τ )dτ

–

– C

t i

t

e A(t – τ)(ABΓ – BΓR)e k(τ )dτ . (6)

Taking the sup-norm on both sides of (6), we find that

e k + 1[t i, t i + 1] sup≤ ρ e k[t i, t i + 1] sup

+ 1a C ∞ ABΓ – BΓR

∞e a(i – j):Σ

j = 0

i

(e a: – 1) e k[t i, t j + 1]sup

.

Here,

ρ0 = ρ + 1a C ∞ ABΓ – BΓR

∞(e a: – 1)

< ρ + 1

a C ∞ ABΓ – BΓR∞

1 +a(1 – ρ)


– 1

= 1.

Since 0 ≤ ρ0 < 1, from Lemma 1 we can conclude that

limk → ∞

e k[t i, t i + 1] sup= 0, ∀ i ∈ {0, 1, …, N}.

This completes the proof. ■

Theorem 1 shows that the convergence of the ILCalgorithm can be directly proved from the sup-norm,which is different from conventional proof of the conver-gence using the λ-norm. From the proof, we can observethat the convergence in the sense of sup-norm in the ithsubinterval, [ti, ti +1], is guaranteed by the convergence inthe prior subintervals, [t0, t1], …, [ti – 1, ti]. This means thatthe output trajectory converges one after another to thedesired output trajectory from the prior time to the end ofthe given time interval while the ILC algorithm is appliedrepetitively.

Now, we propose a new type of ILC algorithm,which guarantees monotone convergence of the outputerror in the sense of sup-norm. Consider the followingPD-type ILC algorithm with adjustment of learning interval.

u k + 1(t) = u k(t) + Γ[e k(t) – Re k(t)], 0 ≤ t ≤ t kλ (7)

Here, t kλ is the maximum value among the time when

e–λt e k(t) ∞

takes its maximum value over the given time

interval [0, T], that is,

t k

λ = sup t′ e – λt′ e k(t′) ∞= sup

0 ≤ t ≤ Te – λt e k(t) ∞

where λ is a real number satisfying the following inequality:

λ > a +


1 – ρ , (8)

and


e k + 1(t) = e k(t), t kλ ≤ t ≤ T.

Monotone convergence of the proposed ILC algo-rithm is presented in the following theorem.

Theorem 2. Suppose that the update law (7) is applied tothe system (1) and that the initial state at each iteration isthe same as the desired initial state, i.e., xk(0) = xd(0) fork = 0, 1, 2, …. If

I – CB∞

≤ ρ < 1,

then there exists a constant ρ0, 0 ≤ ρ0 < 1, such that

sup0 ≤ t ≤ t k

λe k + 1(t) ∞

≤ ρ0 sup0 ≤ t ≤ t k

λe k(t) ∞

.

Proof. Similarly in the proof of the Theorem 1, we canobtain that

e k + 1(t) = e k(t) – C

0

t

e A(t – τ)BΓ[e k(τ ) – Re k(τ )]dτ

= (I – CBΓ)e k(t) + C

0

t

e A(t – τ)(ABΓ – BΓR)e k(τ )dτ(9)

Taking the λ-norm on both sides of (9), we find that

e k + 1( ⋅ )λ

≤ ρ e k( ⋅ )λ

+ C ∞

0

t

e – (a – λ)(t – τ) ABΓ – BΓR∞dτ e k( ⋅ )

λ

≤ ρ + 1

λ – aC ∞ ABΓ – BΓR

∞e k( ⋅ )

λ. (10)

From (8) and (10), we further find that

ρ0 = ρ + 1λ – a


< ρ +1 – ρ



= 1.

Let t k + 1s be the time when e k + 1(t) ∞

takes its maximumvalue on [0, t k

λ]. Then,

e – λt k + 1s

e k + 1(t k + 1s )

∞≤ e k + 1( ⋅ )

λ≤ ρ0 e k( ⋅ )

λ

= ρ0e– λt k

λe k(t k

λ)∞

≤ ρ0e– λt k

λsup

0 ≤ t ≤ t kλ

e k(t) ∞. (11)

From (11), we can find that

sup0 ≤ t ≤ t k

λe k + 1(t) ∞

= e k + 1(t k + 1s )

∞

≤ ρ0e– λ(t k

λ – t k + 1s ) sup

0 ≤ t ≤ t kλ

e k(t) ∞. (12)

Since t kλ ≥ t k + 1

s , we can conclude that

sup0 ≤ t ≤ t k

λe k + 1(t) ∞

≤ ρ0 sup0 ≤ t ≤ t k

λe k(t) ∞

This completes the proof. ■

A simple method that ensures the monotone conver-gence is to divide the given time interval [0, T] into sub-intervals by Tsup and to apply the ILC algorithm (2) in asubinterval after the output error converges to zero in priortime interval. Theorem 2 shows, however, that if the pro-posed ILC algorithm (7) is applied, then the control inputcan be updated ensuring the monotone convergence, eventhough the output error in the time interval [0, Tsup] is notconverged to zero. We remark that t k

λ approaches to T as

k → ∞, since e k( ⋅ ) λ converges to zero.

The upper bound of the learning interval, t kλ depends

on the parameters of the plant and the learning gain R, andwe can choose R as follow as commented in [8]:

R = BΓ

TBΓ

– 1

BΓTABΓ,

where A and B are models of A and B of the system (1),respectively.

III. NUMERICAL EXAMPLE

The following example is given to illustrate theeffectiveness of the proposed algorithm.

Example 1. Consider the following linear time-invariantsystem:

x(t) = 0 0.1

0.02 – 0.03x(t) +

– 11

u(t)

y(t) = – 0.1 0.2 x(t).

Let the desired output trajectory be given as follows.

yd(t) = t(6 – t), 0 ≤ t ≤ 6


Based on the result in [8], suppose that the following ILCalgorithm is applied:

u k + 1(t) = u k(t) + 3(e k(t) + 0.7e k(t)). (13)

Here, Γ is chosen as 3 under the assumption of 10% un-certainty of the system parameters, and R is chosen as–0.7. It is already known that the ILC algorithm (13)makes the output error monotonically decrease in thesense of λ-norm as shown in Fig. 1. In Figs. 2(a), (b), (c)and (d), the output trajectories at the 2nd, 4th, 5th and 7thiteration are shown, respectively. We can easily observethat the output trajectory converges to the desired outputtrajectory from the forepart of the time interval to the endpart of the time interval. Figure 3 shows, however, thatthere is a huge overshoot in the sense of supnorm, eventhough the output error monotonically decrease in the

0 5 10 150

0.5

1

1.5

2

2.5

iteration no. k

||ek(

⋅)|| λ

0 1 2 3 4 5 630

25

20

15

10

5

0

5

10

15

20

25Output

y

time

yd(t)

y(t)

0 1 2 3 4 5 630

25

20

15

10

5

0

5

10

15

20

25Output

y

time

yd(t)

y(t)

0 1 2 3 4 5 630

25

20

15

10

5

0

5

10

15

20

25Output

y

time

yd(t)

y(t)

0 1 2 3 4 5 630

25

20

15

10

5

0

5

10

15

20

25Output

y

time

yd(t)

y(t)

Fig. 2. The output trajectory at each iteration. (a) The output at the 2nd iteration; (b) The output at the 4th iteration; (c) The output at the 5th iteration;(d) The output at 7th iteration.

(c) (d)

(b)(a)

Fig. 1. Trend of convergence of the output error in the sense of λ-norm.


sense of λ-norm as shown in Fig. 1.Now, consider the proposed ILC algorithm with the

same learning gains with the ILC algorithm (13) with λ =0.2:

u k + 1(t) = u k(t) + 3(e k(t) + 0.7e k(t)), 0 ≤ t ≤ t kλ.

Figures 4(a), (b), (c) and (d) show the output trajec-tories at 2nd, 4th, 6th and 8th iteration. In Fig. 5, we caneasily observe that there is no overshoot and the outputerror converges to zero monotonically in the sense of sup-norm for the proposed ILC algorithm, while the conven-tional PD-type ILC algorithm causes a huge overshoot.Since the control input is not updated over time interval[t k

λ , T] during the prior some iterations, the sup-norm of theoutput error in whole time interval [0, T] may not decreasewhere the sup-norm over [t k

λ , T] is larger than the sup-normover [0, t k

λ ].

0 1 2 3 4 5 64

2

0

2

4

6

8

10Output

y

time

yd(t)

y(t)

0 1 2 3 4 5 64

2

0

2

4

6

8

10Output

y

time

yd(t)

y(t)

0 1 2 3 4 5 64

2

0

2

4

6

8

10Output

y

time

yd(t)

y(t)

0 1 2 3 4 5 64

2

0

2

4

6

8

10Output

y

time

yd(t)

y(t)

(c) (d)

(b)(a)

Fig. 4. The output trajectory at each iteration. (a) The output at the 2nd iteration; (b) The output at the 4th iteration; (c) The output at the 6th iteration;(d) The output at the 8th iteration.

0 5 10 150

5

10

15

20

25

iteration no. k

sup 0

≤ t

≤ T||e

k(t)

|| ∞

Fig. 3. Trend of convergence in the sense of sup-norm for the conven-tional ILC algorithm.


IV. CONCLUDING REMARK

In this paper, we first presented a new proof of theconvergence by direct use of the sup-norm, which isdifferent from the conventional proof using the λ-norm.Then, we proposed a new type of ILC algorithm withadjustment of learning interval to ensure the monotoneconvergence of the output error and investigated the rela-tion between the upper bound of the learning interval, t k

λ ,and the learning gain.

When the initial state at each iteration can be differ-ent from the desired initial state, i.e., xk(0) = x0 ≠ xd(0) fork = 0, 1, 2, …, then we can easily show that, based on theresult in [9], there exists a constant ρ1, 0 ≤ ρ1 < 1, such that

sup0 ≤ t ≤ t k

λe k + 1(t) sup

≤ ρ1 sup0 ≤ t ≤ t k

λe k(t) sup

where

y a(t) = y d(t) + e RtC(x 0 – x d(0))

e k(t) = y a(t) – y k(t).

It is remarked that the monotone convergence in thesense of sup-norm for nonlinear systems is open to furtherinvestigation.

REFERENCES

1. Arimoto, S., T. Naniwa, and H. Suzuki, “Robustnessof P-type Learning Control with a Forgetting Factorfor Robotic Motions,” Proc. 29th IEEE Conf. Decis.Contr., Honolulu, Hawaii, pp. 2640-2645 (1990).

2. Sugie, T. and T. Ono, “An Iterative Learning ControlLaw for Dynamical Systems,” Automatica, Vol. 27,

pp. 729-732 (1991).3. Amann, N., D.H. Owens, E. Rogers and A. Wahl, “An

H∞ Approach to Linear Iterative Learning ControlDesign,” Int. J. Adaptive Contr. Signal Process., Vol.10, No. 6, pp. 767-781 (1996).

4. Lee, H.S. and Z. Bien, “Study on Robustness ofIterative Learning Control with Non-zero InitialError,” Int. J. Contr., Vol. 64, No. 4, pp. 345-359(1996).

5. Park, K.H., Z. Bien and D.H. Hwang, “Design of anIterative Learning Controller for a Class of LinearDynamic Systems with Time-delay,” IEE Proc.-PartD, Vol. 145, No. 6, pp. 507-512 (1998).

6. Arimoto, S., S. Kawamura and F. Miyazaki, “Better-ing Operation of Robots by Learning,” J. Rob. Syst.,Vol. 1, No. 123, pp. 123-140 (1984).

7. Lee, H.S. and Z. Bien, “A Note on ConvergenceProperty of Iterative Learning Controller with Re-spect to Sup-Norm,” Automatica, Vol. 33, No. 8, pp.1591-1593 (1997).

8. Bien, Z. and J.X. Xu, Eds., Iterative Learning Control:Analysis, Design, Integration and Applications,Kluwer Academic Publishers, The Netherlands (1998).

9. Park, K.H., Z. Bien and D.H. Hwang, “A Study on thePID-Type Iterative Learning Controller Against Ini-tial State Error,” Int. J. Syst. Sci., Vol. 30, No. 1, pp.49-59 (1999).

Zeungname Bien received the B.S.degree in electronics engineeringfrom Seoul National University,Seoul, Korea, in 1969 and the M.S.and Ph.D. degrees in electrical engi-neering from the University of Iowa,Iowa City, Iowa, U.S.A., in 1972 and1975, respectively. During 1976-

1977 academic years, he taught as assistant professor at theDepartment of Electrical Engineering, University of Iowa.Then, Dr. Bien joined Korea Advanced Institute of Sci-ence and Technology, summer, 1977, and is now Profes-sor of Control Engineering at the Department of ElectricalEngineering and Computer Science, KAIST.

Dr. Bien was the president of the Korea Fuzzy Logicand Intelligent Systems Society during 1990-1995 andalso, the general chair of IFSA World Congress 1993, andfor FUZZ-IEEE99, respectively. He is currently co-Editor-in-Chief for International Journal of Fuzzy Sys-tems (IJFS), Associate Editor for IEEE Transactions onFuzzy Systems, and a regional editor for the InternationalJournal of Intelligent Automation and Soft Computing.He has been serving as Vice President for IFSA since1997, and is now Chief Chairman of Institute of Electron-ics Engineers of Korea and Director of HumanfriendlyWelfare Robot System Research Center. His current

Fig. 5. Trend of convergence int the sense of sup-norm for the proposedILC algorithm.

0 5 10 150

1

2

3

4

5

6

7

8

9

10

iteration no. k

sup 0

≤ t

≤ T||e

k(t)

|| ∞


research interests include intelligent control methods withemphasis on fuzzy logic systems, service robotics andrehabilitation engineering, and large-scale industrial con-trol systems.

Kwang-Hyun Park received theB.S., M.S. and Ph.D. degrees in elec-trical engineering and computer sci-ence from KAIST, Korea, in 1994,19997 and 2001, respectively. He isnow a researcher at Human-friendlyWelfare Robot System ResearchCenter. His research interests in-

clude learning control, machine learning, human-friendlyinterfaces and service robotics.

a study on iterative learning control with adjustment of learning interval for monotone convergence...

Documents