lec4psr

EE5712 Power System Reliability:: Introduction to Stochastic Process

Panida Jirutitijaroen

19/3/2010 EE5712 Power System Reliability

Announcement

• In-class mid-term exam: September 17th.

– Material: from the beginning to last week.

• Homework2 due today.

• No new homework assignment today!

• See updated syllabus.


Last Lecture

• Reliability theory– Failure distribution– Survival function– Hazard rate function

• Reliability Measure• Simple reliability analysis

– Probability convolution method– Series/parallel system– Conditional probability approach– Cut-set/Tie-set method

• Input: failure probability of each component• Output: failure probability of a system


Structure

• System is called ‘structure’ by definition when the followings apply.

– Only 2-state components: Up or Down.

– System has only 2 states: Up or Down.

4

UP DOWN

λ(t)

μ(t)


Observation of A Component

5

Time

Z(t)

0

1

2

Time

Z(t)

0

1

2


Today’s Lecture

• So far, we only concern about a random variable at a fixed time.

• Today, we will consider a random variable that evolves with time.

• With multiple states components/subsystems.• Question:

– What is the probability distribution at time t?– What is the average time that a system will visit a

certain state?

• We’ll use stochastic process to describe the system’s behavior.


OUTLINE

Stochastic process

Markov chain

Markov process


STOCHASTIC PROCESS

Definition

Types of stochastic process

Independent process

Markov property

Time-homogeneous property


Recall: Two Transmission Lines

Line 1

Line 2

Line 1

Line 2

Line 1

Line 2

Line 1

Line 2

(1 Up, 2 Up)

(1 Up, 2 Down)

(1 Down, 2 Up)

(1 Down, 2 Down)

Outcomes Random Variable

Z(t) = 0

Z(t) = 1

Z(t) = 299/3/2010 EE5712 Power System Reliability

Example of Stochastic Process

• Three identical and independent systems of two transmission lines. At time 0, both links are up. Observe three systems in 24 hours.

Time

Z(t)

0

1

2

Realizations of a stochastic process109/3/2010 EE5712 Power System Reliability

Stochastic Process

• Z(t) is a state of the process at time t.

• Z(t) is a random variable.

• Discrete time process, t is discrete.

• Continuous time process, t is continuous.

A collection of random variables.

TttZ ,


Types of Stochastic Process

Continuous RV

Continuous time process

Discrete RV

Continuous time process

Continuous RV

Discrete time process

Discrete RV

Discrete time process

Primary interest of power system reliability129/3/2010 EE5712 Power System Reliability

Discrete RV Discrete Time Process

Primary interest is to determine probability distribution of Zn+1.

3

2

1

Z0 Z1 Zn Zn+1

Pr{Z0 = 3} = 0.0

Pr{Z0 = 2} = 0.0

Pr{Z0 = 1} = 1.0

3

2

1

3

2

1

3

2

1

Pr{Zn = 3} = 0.2

Pr{Zn = 2} = 0.4

Pr{Zn = 1} = 0.4

Pr{Z1 = 3} = 0.1

Pr{Z1 = 2} = 0.1

Pr{Z1 = 1} = 0.8

Pr{Zn+1 = 3} = ?

Pr{Zn+1 = 2} = ?

Pr{Zn+1 = 1} = ?

Time


Discrete RV Continuous Time Process

Primary interest is to determine probability distribution of Z(t).

3

2

1

Z(0) Z(t)

Pr{Z0 = 3} = 0.0

Pr{Z0 = 2} = 0.0

Pr{Z0 = 1} = 1.0

3

2

1

Pr{Zt = 3} = 0.2

Pr{Zt = 2} = 0.4

Pr{Zt = 1} = 0.4

Time


Probability distribution of Z(t)

• Joint distributions of random variables of the process.

• Or, Bayes’ theorem based on conditional probability concept.


Independent Process

• If we assume that , the probability of being in state i at the next time step (n) does not depend on the current (n-1) and previous states (n-2,…,0)

16

Not quite realistic enough to model any physical system


Markov Property

• Discrete time process

• Continuous time process

• “Memoryless process”

Probability of being in future state is conditional on the present state of the system.


Discrete Time Markov Chain

• Discrete state space (discrete RV)

• Discrete time process

• Probability of being in state i at the next time step (n+1) depends only on the current (n), not the previous states (n-1,…,0)


Continuous Time Markov Chain

• Discrete state space (discrete RV)

• Continuous time process

• Probability of being in state i Z(s+t) at time s+tdepends only on the present state Z(s) at time s.


Time-Homogenous Property

• Or, equivalently, “Stationary process”• Discrete time case:

• Continuous time case:

• All Markov Chains considered in this class are time-homogeneous process.

Probability of moving from one state to another state is constant over time.


DISCRETE TIME MARKOV CHAIN

Transition probability

State transition diagram

Chapman-Komogorov equation


Discrete Time Stochastic Process

• Discrete state space and discrete time process

• Markov property

• Time-homogenous property


Pr{Zn = 3} = 0.2

Pr{Zn = 2} = 0.4

Pr{Zn = 1} = 0.4

A Discrete Time Markov Chain

The probability distribution of Zn+1 is dependent only on the current state Zn.

Zn Zn+1

3

2

1

3

2

1

Pr{Zn+1 = 3} = ?

Pr{Zn+1 = 2} = ?

Pr{Zn+1 = 1} = ?

Use concept of transition probability


Transition Probability

• Time-homogenous process => Pij is constant.

The probability of transition from state i to state j in one step, Pij


Transition Probability Matrix

• Given Pij and an initial state, we can find probability of being in any state at the subsequent time steps.


State Transition Diagram

• Stationary process

• Show all possible states and the transition probabilities.


Example 1

• (From [1]) A person is practicing firing. If he misses, he becomes nervous and the probability of the next shot being a hit reduces to ½ , but a hit bolsters his confidence and the chance of the next shot being a hit increases to ¾.

– Draw state transition diagram.

– What is transition probability matrix?


Example 2

• Find transition probability of going from state 1 to state 3 in 2 time steps, P13,(2).


Example 2

• Find transition probability of going from state 1 to state 3 in 2 time steps, P13,(2).

n = 0 n = 2

3

2

1

3

2

1

n = 1

1

2

3

State 1

State 2

State 3

P13,(2) = (P11 × P13 )+ (P12 × P23) + (P13 × P33)


Example 2

9/3/2010 EE5712 Power System Reliability 30

We can write this in a matrix form as follows,

Transition Probability in n Steps

• Probability of transition from state i to state j in n time steps.

• Multiply transition probability matrix n times.


?

Transition Probability in 0 Step


The process does not make any transition, it stays wherever it is.

Chapman-Komogorov Equation


Assume that at time m, the process is at state k,

For a time homogeneous process,

Transition probability in n+m steps,

What about a probability at each time step??

DISCRETE-TIME MARKOV CHAIN –PROBABILITY DISTRIBUTION

Initial distribution

Probability distribution at n time step

Equilibrium distribution


Initial Distribution

• Give probability of being in each state at the beginning of the process.

• Example:

– p0 = [0 1 0]

– p0 = [1 0 0 0]


Probability Distribution at n Time Step

• Give the probability of being in each state after n time steps

• Multiply the transition probability matrix after n time steps with the initial probability vector.

• Example: – p2 = p0 × P × P

– p5 = p0 × P × P × P × P × P


Example 3

• From Example 1, A person is practicing firing. If he misses, he becomes nervous and the probability of the next shot being a hit reduces to ½ , but a hit bolsters his confidence and the chance of the next shot being a hit increases to ¾. – If the initial shot is a hit, what is the probability of

a hit on the fourth shot?

– Calculate the same probability if the initial shot is a miss.


Probability Distribution at Step ‘n’

• Since transition probability matrix is a square matrix, we can use matrix transformation,

– where V is a matrix that columns are found from eigen vector of R, and D is a diagonal matrix of eigen value of R.

• Then,


Example 4

• From Example 1, A person is practicing firing. If he misses, he becomes nervous and the probability of the next shot being a hit reduces to ½ , but a hit bolsters his confidence and the chance of the next shot being a hit increases to ¾. – Find probability distribution after n time step, if

the initial shot is a miss.

– What happens to transition probability when n → ∞?


Equilibrium Distribution

• Give steady state probability of being in each state

• From previous example, when n →∞, the steady state probability distribution is not affected by initial state.

• Sometimes called ‘equilibrium distribution’.

• If the process is ergodic, then the equilibrium distribution exists and is independent of where the process starts from.


Equilibrium Dist. Calculation


Then, the equilibrium distribution is the unique solution of,

Example 5

• From Example 1, A person is practicing firing. If he misses, he becomes nervous and the probability of the next shot being a hit reduces to ½ , but a hit bolsters his confidence and the chance of the next shot being a hit increases to ¾.

– Find equilibrium distribution.


DISCRETE-TIME MARKOV CHAIN –SOME RELIABILITY INDEXES

Absorbing state

First passage time

Mean first passage time


Absorbing State

• In reliability analysis, apply this concept to “failure states”

States in the system that once entered, can not leave unless the system restarts.


First Passage Time

• From state i to state j

• Random variable

• Interest to find Mean First Passage Time

• In reliability analysis, interested to find “Mean time to the first failure, MTTFF”

The number of steps to reach state j starting from state i for the first time.


Mean First Passage Time Calculation

• Define absorbing state j

• Let Q be a truncated transitional probability matrix (P) by removing rows and columns associated with the absorbing states

• Let N be the average number of time steps


‘N’ is called Fundamental matrix.

Fundamental Matrix ‘N’

• nik represents average number of time steps in state k before absorption to state j, given that the system starts from state i

• Mean first passage time, ni, the average time step that the system spend in state i before going to absorbing state j


Example 6

• Find mean first passage time if absorbing state is state 1,

– when the system starts from state 0.

– when the system starts from state 2.


CONTINUOUS TIME MARKOV CHAIN

Transition probability

Chapman-Komogorov equation

Transition rate matrix

State transition diagram


Continuous Time Stochastic Process

• Discrete state space and continuous time process

• Markov property

• Time homogeneous property


Pr{Z(s) = 3} = 0.2

Pr{Z(s) = 2} = 0.4

Pr{Z(s) = 1} = 0.4

A Continuous time Markov Chain

The probability distribution of Z(s+t) is dependent only on the current state Z(s).

Z(s) Z(s+t)

3

2

1

3

2

1

Pr{Z(s+t) = 3} = ?

Pr{Z(s+t) = 2} = ?

Pr{Z(s+t) = 1} = ?


Chapman-Kolmogorov Equation


Transition Probability


Recall from Markov property,

Transition probability is time-dependent!

Let denote the time that the process spend to transit from state i to state j.

Transition Rate

• Define a transition rate from state i to state j,

• as Δt → 0,

• Or,


Transition Rate


Recall,

Since

Transition Rate Matrix


State Transition Diagram

• Represent transition rates among states in a system

1 2

3

0.01

0.01 0.02

0.5 0.5


What is transition rate matrix of this diagram?

Transition Probability VS Rate


Example 7

• Assume that the system started in state 1, find P13(t+Δt).

t = 0 Xt+Δt

3

2

1

3

2

1

Xt

1

2

3


• From conditional probability as Δt → 0,

P13(t+Δt) =

t = 0 Xt+Δt

3

2

1

3

2

1

Xt

1P11(t) P13(Δt)

2P12(t) P23(Δt)

3P13(t) P33(Δt)

+ P12(t) P23(Δt)P11(t) P13(Δt) + P13(t) P33(Δt)

= P11(t) λ13 Δt + P12(t) λ23 Δt + P13(t) (1+λ33 Δt)


• Since P31(Δt) + P32(Δt) + P33(Δt) = 1, then

λ33 Δt + λ32 Δt + λ31 Δt = 1

λ33 Δt = 1 - λ32 Δt - λ31 Δt

• We have,

P13(t+Δt) = P11(t)λ13 Δt + P12(t) λ23 Δt

+ P13(t) (1 - λ32 Δt - λ31 Δt)

• Then,

[P13(t+Δt) - P13(t) ]/ Δt

= P11(t) λ13 + P12(t) λ23 + P13(t) (- λ31 -λ32) = P’13(t)


• Thus,

• Where

• Similarly, we have


Example 8

• Suppose a continuous time Markov chain enters state i at time 0, and stays in this state for 10 mins. What is the probability that the process will not leave state i during the next 5 mins?

– Let Ti be the time the process spent in state i, then Pr{Ti > 15|Ti > 10} = Pr{Ti > 5}

– In general, Pr{Ti > s+t|Ti > s} = Pr{Ti > t}.

• Memoryless property, what is the distribution of Ti??


Another Definition

• A continuous-time Markov Chain is a stochastic process with the following property.

– Amount of time spent in the state before making a transition is exponentially distributed.

– When a process leaves state i, it enters state j with some probability (transition probability) Pij.

• A Markov Chain with exponentially distributed RV of time spent in each state. (RVs of time spent in each state are also independent)


Recall: Hazard Function

• From hazard function as Δx → 0,

• This gives probability of failure of a component in interval (x, x+Δx).

xXxxXxxx |Pr

x

x x+ Δx

xXxxXx |Pr


Transition Rate VS. Hazard Function

• Let Xij be the random variable of time duration from state i to j, then

Pij(Δt) = Pr{ t < Xij < t+ Δt | Xij > t }

• Hazard Function of a random variable Xij, as Δt → 0,

φij(t)Δt = Pr{ t < Xij < t+ Δt | Xij > t }

• If the process is markovian, φij(t) = φij (constant and independent of t)

The transition rate is therefore the hazard function

This also means that the underlying random variable Xij

of Markov process must be exponentially distributed669/3/2010 EE5712 Power System Reliability

Recall: Exponential Distribution

• Mean time to failure is • n = number of failures

• λ is called “Failure rate”.• The same is also applied for μ “repair rate”, found from

the reciprocal of ‘mean time to repair’.

x1 x2 x3 x4 x5

n

xxx n

21

n timeobservatio Total

1 n

MTTF

Assume that the repair time for this system is negligible


CONTINUOUS TIME MARKOV CHAIN –PROBABILITY DISTRIBUTION

Initial distribution

Probability distribution at time t

Equilibrium distribution


Initial Distribution

• Give probability of being in each state at the beginning of the process.

• Example:

– p0 = [0 1 0]

– p0 = [1 0 0 0]


Probability Distribution at Time t

• Give the probability of being in each state after time t.

• In this case, we need to obtain .


Transition Probability at Time t

• From

• Since , the solution to the first order equation is,

• We can use matrix transformation,

– where V is a matrix that columns are found from eigen vector of R, and D is a diagonal matrix of eigen value of R.


Transition Probability Approximation

• Recall the derivative,

• And

• Use one step transition probability matrix in Discrete time Markov chain and divide time t to small equal intervals of Δt.

• Assume Δt → 0,


Transition Probability (Δt)


This gives the transition probability approximation of time jΔt using the approximation of time step Δt .

Example 9

• A two-state model is commonly used in reliability study to represent a status of a component

• Find– Transition rate matrix

– Transition probability at time t

– State probability at time t

DownUp

λ = failure rate

μ = repair rate


Equilibrium Distribution

• Give steady state probability of being in each state

• Similar to discrete time case, if the process is ergodic, then the equilibrium distribution exists and is independent of where the process starts from.

• In example 9, observe the transition probability matrix at time t→∞.


Equilibrium Dist. Calculation


Let t →∞, we have,

From

At steady state,

Example 10

• From example 9, find equilibrium distribution

DownUp

λ = failure rate

μ = repair rate


CONTINUOUS TIME MARKOV CHAIN –SOME RELIABILITY INDEXES

First passage time

Mean first passage time


First Passage Time

• Random variable, denote by Tij

• Calculate as ‘Mean first passage time’

A time to enter state j starting from state ifor the first time.


Mean First Passage Time Calculation

• From discrete time case, , gives number of time steps to absorbing states.

• Approximate Q by [I + RΔt], the time spent is found by multiplying N by Δt.

• The mean first passage time is,

1 QIN


Time Dependent System Availability

• Summation of state probability at time t of success states

Probability of being found in the success states at some time t.


System Availability

• Summation of state equilibrium probability of success states

Probability of being found in the success states at steady state.


A Two-State Component Example

• Assume that system starts from up state at time t = 0.

• Time to change state is random variable with exponential distribution.

DOWNUP

λ = failure rate

μ = repair rate

tetR

Time to change state from up to down

Probability of staying in the up state given that the system started from this state at time t = 0.

tup etptA

Probability of being found in the up state at time t, given that the system started from up state at time t = 0.


Summary

• Stochastic process

• Markov process

• Markov chain

• Transition probability/ state probability

• Transition probability matrix

• Transition rate matrix

• Equilibrium distribution

• First passage time/ mean first passage time


Application

• Most repairable systems are described by Markov processes

• We will use this to represent different outage models

• To compute the state probability and state equilibrium probability and mean first passage time (mean time to the first failure)

• Interested to know a frequency of encountering a system state or average duration spent in each state– How often the system fails

• This will be in the next lecture.


Reading Materials

• Review today’s lecture• Chapter 2: The Preliminaries “System Reliability Modeling and

Evaluation”, http://www.ece.tamu.edu/People/bios/singh/sysreliability/ch2.pdf

• Lecture notes on this part will be uploaded.• Sheldon M. Ross, “Introduction to Probability Models”, 8th edition.• Markov chain

– Transition probability matrix– Equilibrium state probability– Mean first passage time

• Markov process– Transition rate matrix– Equilibrium state probability– Mean first passage time


Limitations

1

2

5

3

4

Series or Parallel??

2

8

5

3

9

1

7

4

10

6

What about other reliability measure? Now we can only compute for system availability/unavailability.


About Next Lecture

• Analytical methods for reliability analysis for frequency of failure

– State space approach -> similar to the example in lecture 2.

– Conditional probability approach

– Network reduction approach

– Frequency balance approach


lec4psr

Documents