optimal decision policy for deteriorating systems under variable operating conditions &...

Optimal Decision Policy for Deteriorating Systems Under Variable

Operating Conditions &Markovian Environment Conditions

The University of Electro-CommunicaionsSuzuki Lab

Dinesh Rajapaksha2015-02-04

1

In order to prevent fetal failures,effective Preventive maintenance is important

It is difficult to completely eliminate accidents or breakdowns in design phase

BackgroundWith the development of science and technology,

systems get more massive and complicated

2

- Exhaust gas temperature

- Engine vibration

- Type and amount of metal

in engine oil

- Amount of engine oil

consumption

- etc.

Operating

conditions

- Purpose of use

- Frequency of use

- Load from associated system

- Operation (standard/mistake)

- Fuel(standard/bio/mistake)

- storage (place, method), etc.

Environmental

conditions

Failure mode

Failure

mechanism

- Stress from repetition

- Stress from overload

- Stress from shock

- Stress from cooling cycle

- Temperature, humidity,

electrical

- Stress, wave, shock, solar

radiation, thunder, rain, hail,

snow

- Dust, saltwater, PH, animal,

earthquake, altitude

- Crack/Leak

- Thinning damage

- Clogging

- Deformation/Change

- Slack

Effects

E (t): effect oncustomers

M (t): Alternative characteristic value ofFM (t) obtained by monitoringBig data acquired in real time

{X (t), M (t), FM (t), A (t), TM (t), and E (t)}

Pro

du

ctio

nh

isto

ry Top event mode

TM(t)

FM(t)

History/

Preventive

maintenance

A(t)

©K. Suzuki 2014

On-line Monitoring[1]X (t): covariate

3

Induction Motors

Production Line[2]

Induction Motors are widely used in Industrial Applications

Breakdowns can cause Higher Losses

Speed Load High Speeds / High Loads

Low Costs / High profits

4

High Productivity

[2] Continuous Production Line manufacturer, DUT co.,LTDhttp://dutkorea.en.ec21.com/PU_Continuous_Production_Line--5442100_5442281.html

http://dutkorea.en.ec21.com/PU_Continuous_Production_Line--5442100_5442281.html

Induction Motor Failures[3]

Fig. 1 Statistics related to motors failures

1. Motor faults related to bearing 41%2. Motor faults related to stator 37%3. Motor faults related to rotor 10%4. Other faults 12%

[3]Irfan.,M. Saad., N, Asirvadam, A. S.,“ Development of an Intelligent Condition Monitoring System for AC Induction Motors using PLC” IEEE Business Engineering and Industrial Applications Colloquium(BEIAC), vol. 28(2013), No. 3, pp. 3194-3203.[5] 5

Mechanical failure Thermal failure

51% to 64% 35% to 37%

Table 1. Major motor failures

6

Motor Operation

Operating Conditions a Deterioration state i

),...,2,1(, nSSi

: Motor is new

: Most deteriorated 1

n

New Most Deterioratedi

Internal Environment l zEEl ,,2,1,

1 : environment is in the best level : environment is in the worst level

Best Worstl

Speed / LoadHIGH

Speed / LoadLOW

2a z

)2,1(, AAa

1a

Higher Productivity

Low Cost

Lower Productivity

High Cost

1 n

z1

Purpose Of The Research

Deterioration state i

Inte

rnal

en

viro

nm

en

t

l

1 2 3

1

2

3Total possible status of the system : 9

Induction motor with 3 deterioration states,3 internal environment levels and 3 operating conditions.

Total possible policies : 683,1939

To provide sufficient conditions for an optimal policy to

be given by a Monotone Policy.

All possiblepolicies

Optimal policy

Monotone Policy

status of the system

7

Purpose

Total Monotone policies : 175

Example

High Speed

Low Speed

Medium Speed

State i

Inte

rnal

En

viro

nm

ent

Op. Cond. k Op. Cond. k’

Monotonically Increasing Operating Conditions

8

i1 i

'l

l

i

l

Operating condition with a higher number becomes better with the

deterioration of the System

Op. Cond. k

Op. Cond. k’’

Op. Cond. k’

Op. Cond. k’ Op. Cond. k’’

Op. Cond. k’’Op. Cond. k’’

hkkk 1kkk ,,

kli ),(

kli ),(

kli ),( kli ),( kli ),(

kli ),(

kli ),( kli ),(

kli ),(

Monotone Policy

Operating conditions :

l

Operating condition at status : ),( li),( li

Fig. 1 Concept of monotone policy

Markov Decision Process

• True status can be known exactly by inspections.

• System deteriorates according to a Markov chain.

• Operator maker may choose any operating

condition based on system’s status.9

Optimal decision making problem for adeteriorating system under variable

operating conditions and environment conditions

Markov Decision Process(MDP)

Research Model Operator’s action(Operating conditions)

EnvironmentalCondition

Derman(1963)[4] Perfect Observation(MDP)

•Keep•Replace

Not Considered

Table 1. Previous research on optimal decision making for system with stationary environmental conditions

Previous Research

10

State i

Cost

)(iV Keep

Replace

*iOptimal action change

Keep Replace

Fig. 2 Monotone Policy

System

Internal Environment

Deterioration

• Stochastically Increasing

• Not deteriorating

Previous Research

Research Model Operator’s action(Operating conditions)


Derman (1963) Perfect Observation(MDP)

•Keep•Replace

Not Considered

Kurt &

Kharufeh(2010) [5]

Perfect Observation(MDP)

•Keep•Replace

Considered Internal

environment

State i

Optimal action change

Keep

Replace


l

Fig.2 Monotone Policy

System


Deterioration



Table 2. Previous research on optimal decision making for system with stationary environmental conditions

11

Research Model Operator’s action

(Operating conditions)


Kurt. M.,

Kharufeh.P.j.,(2010)[5]

Perfect Observation(MDP)

•Keep•Replace

Consider InternalEnvironment

This Research Perfect Observation(MDP)

Multiple Actions

(Operating Conditions)Consider Internal

Environment

Comparison with Previous Research

System


Deterioration



Operating Condition affects Internal Environment of the System

12


Operating Condition

Denotation

: Deterioration state due to usage

: environment’s state

: Operating Condition

: Transition probability of the system state from i to j when op. condition is a

: Transition probability of the system state from l to l’ when op. condition is a

: Transition probability matrix of system;

: Internal Environment transition matrix

: Operating cost per period when state is , internal environment is and

operating condition is

: Discount factor 0 < < 1

a

}{ a

ij

a pP

i

a

a

ijp

aP

a

liC ,

i

l

aQ }{ 'll

aa qQ

13

l

a

llq

Operating Conditions a

Speed h / Load h

Speed 2 / Load 2

.

.

.

HIGH

LOW

Operating Condition : 1


Operating Condition : h

),...,2,1(, hAAa

Speed 1 / Load 1

Operating Conditions a

Speed / LoadHIGH

Speed / LoadLOW

2a

)2,1(, AAa

1a

14

Model Description Deterioration state i

),...,2,1(, nSSi

: Motor is new

: Most deteriorated1

n

New Most deterioratedi

Internal Environment l zEEl ,,2,1,

1 : environment is in the best level: environment is in the worst level

Best Worstl

z

DeteriorationTransition from state i to j

operating condition a

aijp

NewMost

deterioratedi j

Best Worstl l’

Internal Environment Transition from l to l’

Operating condition. a

a

llq

a

nn

a

nj

a

n

a

in

a

ij

a

i

a

n

a

j

a

t

t

t

a

ppp

ppp

ppp

nS

iS

S

1

1

1111

)(

1

P

nSjSS ttt 111 1

New

Most

Deteriorated

New

) cond. Op.,|Pr( 1 aiSjSp tt

a

ij

Transition Probability Matrix

15

Most

Deteriorated

)()()(

)()()(

)()()(

0

0

000

aa

nj

a

aaa

a

n

aa

a

nnn

iniji

j

ppp

ppp

ppp

P

increasing Stochastic:SI

≧

≧

)(1 ippn

lj

a

ji

n

lj

a

ij

Probability toDeteriorate further or Breakdown Probability to

Deteriorate further or Breakdown

NEW System USED System

More deteriorated systems are more likely to go to a worse deteriorated state

Current time period

Remain time periods

Next time period

Current status

tt ElSit li ,),(

h

a

Cond. Op.

Cond. Op.

1 Cond. Op.

)},|{Pr( 1

)( aSS tt

a

P

Updated status

Transition Process

a Cond. Op.

)},|{Pr( 1

)( aEE tt

a

Q

Operator Selects Operating

condition based on the status 17

11 ,1 ),(

tt ElSjt lj

System



Operating Conditions & the System

1ijp

18

t 1t1

,lic

:S

:E l

State of the system

Environment state 'l

One period

1

llq

i j

2

,lic

2ijp

2

llq

Operating Condition : h

・・・

19

Op. k leads to faster cost increasing and faster internal environment and state deterioration rate than Op. k’

as the system deteriorates

Inte

rnal

Enviro

nm

ent

Dete

riora

tion ra

te

Slow

Fast

Cost in

creasin

g

Fast

Slow

Sta

te

Dete

riora

tion ra

te

Slow

Fast

Ordering of Operating Conditions

h

k

k

Cond. Op.

Cond. Op.

Cond. Op.

1 Cond. Op.

hkk 1

Model Formulation( Internal Environmental Conditions & Operational Conditions)

),,(),(

)2,,(),(

)1,,(),(

min,

1 1

)(

,

1 1

22)2(

,

1 1

11)1(

,

hlivljVpqc

livljVpqc

livljVpqc

liV

z

l

n

j

h

ij

h

ll

h

li

z

l

n

j

ijllli

z

l

n

j

ijllli

20

Total expected discounted Cost

Cost of 1periodExpected Cost for future Periods up to infinity

+=

: Cost for operating under operating condition 1

Conditions

C-1

C-2

C-3

C-4

haSIPa ,...,2,1 )(ippn

lj

a

ji

n

lj

a

ij

Probability of going to a worse state is increased with state i

zaSIQa ,...,2,1

As the internal environment gets bad, it is more likely to go to a worse level

)(',1' lqqz

lj

lla

z

lj

lla

21

n

kj

a

ij

n

kj

a

ij pp 1

z

kl

a

ll

z

kl

a

ll qq 1

''

Probability of going to a worse state is reduced when the system is operating under a superior operation which has a greater number.

probability of going to a worse environment is reduced when the system is operating under a superior operation which has a greater number.

C-5

C-6

C-7

C-8 ),(1

),(),( icc a

li

a

li

,)(, ica

li

As state increases, increment of deterioration rate for current operating condition is greater than that for a superior operation which has a greater number.

Operating cost is increased with state and internal environment of the system

As the system deteriorates, merit of changing operating conditions becomes greater than using the same operating condition.

)(. lca

li

)(1

,, lcc a

li

a

li

As environment level increases, increment of deterioration rate for current operating condition is greater than that for a superior operation which has a greater number..

22

)(1 ippn

kj

a

ij

a

ij

)('

1

'' lqqz

kl

a

ll

a

ll

Conditions )(1 ippn

kj

a

ij

a

ij

i

l

)('

1

'' lqqz

kl

a

ll

a

ll

)(1

,, lcc a

li

a

li

l

a

lic ,

1

,

a

lic

Properties of the Cost Function 1/2)(),,( ialiv 1)

2)

Is a non-decreasing function of , when operation cond. a &internal environment state l is given.

i

Is non-decreasing function of when operation cond. a & a+1, internal environment state l is given.

i

ialivaliv )1,,(),,(

23

)(),,( laliv 3)

4)

Is a non-decreasing function of internal environment level l when operating condition is a & state i is given.

Is a non-decreasing function of state l for operation condition a, a+1 when state i is given

lalivaliv )1,,(),,(

),,( aliv ),,( aliv

)1,,( aliv

il

cost

Internal environment levelState

Optimal Policy Structure

state i

Inte

rnal

en

viro

nm

en

t

Op. Cond.1 Op. Cond.2 Op. Cond.3

Op. Cond. 1 Op. Cond.2 Op. Cond.3

Op. Cond. 2 Op. Cond.3

Monotonically Increasing in both Environmental and Operating Conditions.



24

l

High SpeedMedium

Speed Low Speed

High Speed Medium

Speed Low speed

Medium Speed Low Speed

New

Induction Motor

Op. Cond.1High

Speed

Op. Cond.2Medium

Speed

Op. Cond.3Low

Speed

z

l

n

j

ijllli

z

l

n

j

ijllli

Vpqc

ljVpqc

liV

1 1

22)2(

,

1 1

11)1(

,

)0,0(

),(

min,

;2

, Rc li

001

001

0012

P

Replacement cost per period

SIpij

...

..

...11

P

Keep

Replace

Ahh ,2

25

• Derman (1963)’s[4] result is a special case of this research

Discussion

When,

Operating Conditions are limited to two ,

State i

)(iV Keep

Replace

*i

Keep Replace

•

•

•

100

010

0011

Q

001

001

0012

Q

Keep Replace1 2

cost

1/2

z

l

n

j

ijllli

z

l

n

j

ijllli

Vpqc

ljVpqc

liV

1 1

22)2(

,

1 1

11)1(

,

)0,0(

),(

min,

;2

, Rc li Replacement cost per period

SIqSIp ijij

...

..

...

,

...

..

...1111

QP

Keep

Replace

Ahh ,2

26

• Kurt (2010)’s[5] result is a special case of this research

Discussion

When,

Operating Conditions are limited to two ,•

•

•

State i

Keep

Replace

Internal Environmentl

001

001

0012

P

001

001

0012

Q

Keep Replace1 2

2/2

Numerical Example 1/2

Transition Probability matrixes

Internal Environment Transition probability

matrixes

Costs for Operating Conditions

The above data satisfy conditions (C-1) to (C-8)27

Parameters

a=1l i 1 2 3

1 30 36 70

2 35 49 81

3 79 85 100

a=2l i 1 2 3

1 40 45 79

2 45 57.5 85

3 70 75 89

a=3l i 1 2 3

1 48 52.5 86

2 52.5 62.5 90

3 75 80 92

aP a

Q

a

lic ,

a=1

0.9 0.1 0

0.39 0.4 0.21

0.1 0.06 0.84

a=2

0.9 0.1 0

0.5 0.4 0.1

0.25 0.03 0.72

a=3

0.99 0.01 0

0.6 0.3 0.1

0.4 0.3 0.3

a=1

0.8 0.2 0

0.3 0.49 0.21

0 0.06 0.94

a=2

0.9 0.1 0

0.4 0.5 0.1

0.1 0.08 0.82

a=3

0.99 0.01 0

0.5 0.4 0.1

0.2 0.2 0.6

Op. Cond.3329.5

Op. Cond.3344.0

Op. Cond.3365.0

Op. Cond.1266.8

Op. Cond.2293.4

Op. Cond.3333.3

Op. Cond.1236.6

Op. Cond.2262.8

Op. Cond.3312.7

l=3 340.9 354.2 386.1

l=2 268.0 293.4 350.0

l=1 243.9 262.8 330.0

i=1 i=2 i=3

l=3 359.3 376.5 408.0

l=2 266.8 299.1 359.5

l=1 236.6 263.8 331.8

i=1 i=2 i=3

Costs values for ( Operating Condition a=1 )

Optimal Decision Policy

Inte

rnal

En

viro

nm

ent

state i

1

2

3

1 2 3

l

Result

28

For 10 periods time

l=3 329.5 344.0 365.0

l=2 270.7 293.5 333.3

l=1 247.4 265.5 312.7

i=1 i=2 i=3

)1,,( liv Costs values for ( Operating Condition a=2 )

)2,,( liv Costs values for (Operating Condition a=3 )

)3,,( liv


Op. Cond.1High

Speed

Op. Cond.2Medium

Speed

Op. Cond.3Low

Speed


New


Transition Probability matrixes

Internal Environment Transition probability

matrixes

Costs for Operating Conditions

The above data does not satisfy conditions (C-3) and (C-5)

29

Parameters

a=1

l i 1 2 3

1 30 36 70

2 35 49 81

3 79 85 100

a=2

l i 1 2 3

1 40 45 79

2 45 57.5 85

3 70 75 89

a=3

l i 1 2 3

1 48 52.5 86

2 52.5 62.5 90

3 75 80 92

aP a

Qa

lic ,

a=1

0.9 0.1 0

0.39 0.4 0.21

0.1 0.06 0.84

a=2

0.9 0.1 0

0.5 0.4 0.1

0.25 0.03 0.72

a=3

0.99 0.01 0

0.6 0.3 0.1

0.4 0.3 0.3

a=1

0.8 0.2 0

0.3 0.49 0.21

0 0.06 0.94

a=2

0.99 0.01 0

0.5 0.4 0.1

0.2 0.2 0.6

a=3

0.9 0.1 0

0.4 0.5 0.1

0.1 0.08 0.82

Op. Cond.3298.2

Op. Cond.3322.2

Op. Cond.2380.6

Op. Cond.2256.4

Op. Cond.2288.5

Op. Cond.2355.8

Op. Cond.1235.1

Op. Cond.2264.9

Op. Cond.2340.6

l=3 309.5 331.4 380.6

l=2 256.4 288.5 355.8

l=1 240.0 264.9 340.6

i=1 i=2 i=3

l=3 330.1 358.8 420.3

l=2 257.7 297.9 382.9

l=1 235.1 268.7 359.9

i=1 i=2 i=3

Costs values for Operating Condition a=1

Optimal Decision Policy

Inte

rnal

En

viro

nm

ent

state i

1

2

3

1 2 3

l

Result

30

For 10 periods time

l=3 298.2 322.2 385.9

l=2 264.4 294.3 376.0

l=1 248.6 273.3 363.8

i=1 i=2 i=3

)1,,( liv Costs values for Operating Condition a=2

)2,,( liv Costs values for Operating Condition a=3

)3,,( liv

3,3,2 Op. Cond.1

High Speed

Op. Cond.2

Medium Speed

Op. Cond.3

Low Speed

New

NOT Monotonically Increasing


Conclusions

• This research deals with an deteriorating system under variable operating condition and environment conditions.

• Provide sufficient conditions for the existence of an optimal monotone policy.

• Limiting the optimal procedure to the set of monotone procedures would greatly reduce the tremendous amount of calculation time required to find the optimal decision procedure.

31

• Simplify the algorithm.• Reduce the computation error.

• Reference

[1] 鈴木和幸,椿広計(2013) : “トラブル未然防止への全体スキームとオンラインモニタリング情報の活用” ,

電気通信大学大学院情報システム学研究科シンポジウム第17回「信頼性とシステム安全学」

[3] Irfan.,M. Saad., N, Asirvadam, A. S.,“ Development of an Intelligent Condition Monitoring System for AC Induction Motors using PLC” IEEE Business Engineering and Industrial Applications Colloquium(BEIAC), vol. 28(2013), No. 3, pp. 3194-3203.

32

[5] Kurt, M., Kharoufeh, J., P.(2013) : “ Monotone Optimal Replacement Policies for a Markovian Deteriorating System in a Controllable Environment ”, Operations

Research Letters, , vol.38, pp 273-279.

[2] Continuous Production Line manufacturer, DUT co.,LTDhttp://dutkorea.en.ec21.com/PU_Continuous_Production_Line--5442100_5442281.html

[4] Derman,C. ”On optimal replacement rules when changes of state are Markovian”, in: R.Bellman(ed.), Mathematical optimization techniques, Univ. of California Press. Berkeley, CA, 1963.

http://dutkorea.en.ec21.com/PU_Continuous_Production_Line--5442100_5442281.html

THANK YOU