stoc proc kg

8/10/2019 Stoc Proc KG

1/18

Author: Dr. K. Gururajan

Assistant Professor

Department of Mathematics

Malnad College of Engineering, Hassan573 201

UNIT 8:

Jointly Distributed Random Variables

Introduction: So far the study was restricted to one dimensional random variable,distribution function, and other related aspects. However, in real world, one come

across a number of situations where we do find the presence of two or correlated random

variables. For example, consider the experiment of finding how far e - learningprogramme initiated by VTU Belgaum has become popular among the engineering

students? To find this say, authorities collect feed back from the students by visiting their

institutions. Let the problem be about finding the opinion of students regarding two

parameters; (i) quality of transmission from studio situated at Bangalore which we call it

as X and (ii) students interest in this kind of programs which we shall refer it to as Y.For convenience of discussion, authorities visit seven colleges located at different parts of

the state and results are in the following table. We assume that these are given in terms ofpercentages.

Engg.

Colleges

PGA SSIT BVB GSSIT AIT SBMJCE KLE

X X1 x2 x3 x4 x5 x6 x7

Y y1 y2 y3 y4 y5 y6 y7

In problems like this, we/authorities are certainly interested to learn the mood of the

students/teachers about the e learning programme initiated by us of course with hugecost. It is known to you that one satellite channel has been completely dedicated for this

purpose in India. Many people are involved in this programme to reach the unreached

and needy.

One comes across many illustrative examples like that. Therefore, there is a necessity to

extend the study beyond single random variable.

This chapter is devoted to a discussion on jointly related variables, their distribution

functions and other important characteristics. First we shall have a discussion on discrete

case.

(Dr. K. Gururajan, MCE, Hassan page 1)


2/18


3/18

Note: One caution with discrete random variables is that probabilities of events must be

calculated individually. From the preceding sections, it is clear that in the current

problem, there totally m n events. Thus, it is necessary to compute the probability of

each and every event. This can also be shown by means of table.

Y

X

y1 y2 y3 . . . . ny

x1 ,h x y1 1 ,h x y1 2 ,h x y1 3 . . . . , nh x y1x2 ,h x y2 1 ,h x y2 2 ,h x y2 3 . . . . , nh x y2

x3 ,h x y3 1 ,h x y3 2 ,h x y3 3 . . . . , nh x y3

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

mx ,mh x y1 ,mh x y2 ,mh x y3 . . . . ,m nh x y

Note: When the problem is based on discrete random variables, it is

necessary to compute the probability and each and every event

separately.

Illustrative examples:

A fair coin is tossed 3 times. Let X denote the random variable equals to 0 or 1

accordingly as a head or a tail occurs on the first toss, and let Y be the random

variable representing the total number of heads that occurs. Find the joint

distribution function of (X, Y).

Solution: S = {HHT, HHT, HTH, THH, THT, TTH, HTT, TTT}. Thus, |S| = 8.

Here, X takes the values 0 or 1 accordingly as a H appears on the I toss or a Tail appearson the I toss, while Y takes the values 0, 1, 2, 3 where these numbers represent the

number of heads appearing in the experiment. Observe that joint variables(X, Y) assume

eight values. Thus, there are (0, 0), (0, 1), (0, 2), (0, 3), (1, 0), (1, 1), (1, 2), (1, 3) totally 8 events. Thus, we need to find the probabilities of all these 8 events. First we

shall list the respective events corresponding to X and Y.



4/18

[X = 0] = {HHH, HHT, HTH, HTT},

[X = 1] = {THH, TTH, THT, TTT}

[Y = 0] = {TTT},

[Y = 1] = {HTT, HTH, TTH],

[Y = 2] = {HHT, HTH, THH}

[Y = 3] = {HHH}

Therefore, [X = 0, Y = 0] = { }, a null event, hence [ , ] ( , )P X Y h0 0 0 0 0 .

Similarly, [ , ]X Y0 1 = {HTT}, so [ , ] ( , )P X Y h 10 1 0 1 8 . Note that

[ , ]X Y0 2 = {HHT, HTH}, thus [ , ] ( , )P X Y h 20 2 0 28

. Another example,

consider the event [X = 1, Y = 2] = {THH} implying that [ , ] ( , )P X Y h 11 2 1 28

.

Continuing like this, one can compute the probabilities of all the events. The results are

shown in the form of the following table;

Y

X

0 1 2 3

0 ( , )h 0 0 0 ( , )h 10 18

( , )h 20 28

( , )h 10 38

1 ( , )h 11 08

( , )h 21 18

( , )h 11 28

( , )h 1 3 0

Two cards are selected at random from a box which contains 5 cards numbered 1, 1,

2, 2, and 3. Let X denote the sum selected, and Y the maximum of two numbers

drawn. Determine the joint probability function of X and Y.

Solution: It is known that 2 cards can be selected from the available 5 in C (5, 2) = 10

ways. With cards being numbered as 1, 1, 2, 2, and 3, clearly, S = {(1, 1), (1, 2), (1, 2),

(1, 2), (1, 2), (1, 3), (1, 3), (2, 2), (2, 3), (2, 3)}. As X denote the sum of numbered

chosen, X takes the values 2, 3, 4, and 5. Y is defined as the maximum of the two

numbers, so Y takes the values 1, 2, 3. Therefore, (X, Y) assumes totally 12 values. It

can be shown as



5/18


6/18

Y

X

0 1 2 3

0 ( , )h 0 0 0 ( , )h 10 18

( , )h 20 28

( , )h 10 38

1 ( , )h 11 0 8 ( , )h 21 1 8 ( , )h 11 2 8 ( , )h 1 3 0

The distribution function of X is got by adding the probabilities row wise. Thus,

:

1 1:

2 2

i

i

x

f x

0 1

and the marginal distribution function of Y is to be obtained by the adding theprobabilities column wise. Therefore,

iy 0 1 2 3

jg y 183

83

81

8

The Co Variance of X and Y is given as Cov( , ) ( ) ( ) ( )X Y E XY E X E Y . The

regression coefficient between X and Y can be written asCov( , )

( , )x y

X Yx y

. Here,

and x y are standard deviation of X and Y series respectively.

ILLUSTRATIVE EXAMPLES:

Consider the joint distribution of X and Y

Y -4 2 7

1 1/8 1/8

5 2/8 1/8 1/8

Compute E(X), E(Y), E (XY), Covar (X, Y),y

,x and ( , ).x y

Solution: First, we obtain the distribution functions of X and Y. To get the distribution

of X, it is sufficient to add the probabilities row wise. Thus,


X

Y


7/18

X 1 5

if x 1/2 1/2

( ) ( / ) ( / )E X 1 1 2 5 1 2 3

( / ) ( / )E X2 1 1 2 25 1 2 13

Var( ) ( )X E X E X22 4 and x 2 .

Similarly,

Y -4 2 7

jg y 3/8 3/8 2/8

( ) ( / ) ( / ) ( / ) .E Y 4 3 8 2 3 8 7 2 8 1 0

( / ) ( / ) ( / ) .E Y2 16 3 8 4 3 8 49 2 8 39 5

Var( ) ( )Y E Y E Y 22 = 38.5

.y 38 5 = 6.2048.

Consider ( ) ( , )x y

E XY x y h x y = (2)(1)(0.2) + (3)(2)(0.4) + (4)(2)(0.1) +

(4)(3)(0.2) + (5)(3)(0.2) = 8.8. With E(X) = 3, and E(Y) = 1, it follows that

Cov( , ) ( ) ( ) ( )X Y E XY E X E Y = 5.8 andCov( , ) .

( , ) .( . ) ( . )

X Yx y

x y

5 80 4673

2 0 6 2048

. As

Cov( , ) is not zero, we conclude that and are not Independent.X Y X Y

It may be noted that so far the study on probability theory was concentrated on either a

single random variable or a two random variables (correlated variables) defined on the

same sample space, S of some random experiment. It is interesting to see that in a

practical situation, we come across numerous occasions involving a family or a collection



8/18

of random variables. This is especially true in problems of queuing theory and in

networking. As todays problems are mainly based on theory of queues, thus, there is a

strong necessity to study the same. Also, this has enormous amount of applications in

engineering and technology. With these in view, we shall consider a brief discussion on

stochastic processes. Among various types of stochastic processes available in literature,Markov processes have found significant applications in many fields of science and

engineering; therefore, current study will be restricted to this only.

A discussion on Independent Random Variables

Let X and Y be two discrete random variables. One says that X and Y are independent

whenever the joint distribution function is the product of the respective marginal

distribution functions. Equivalently, suppose that ( ) ( ) ( ),, g andi j i jf x y h x y denoterespectively, the distribution function of X, Y and that of X and Y,

and ( ) ( ) ( ), gi j i jh x y f x y= , then we say that X and Y are independent random

variables.

Note: If it is known that X and Y are independent random variables, then it is very easyto compute the joint distribution function. This may be obtained by just multiplying the

respective probabilities in the corresponding row and column of the table. For example,consider

X 0 1 2 3

( )if x 0.2 0.2 0.5 0.1

y 3 5 6

( )jg y 0.2 0.5 0.3

Then, joint distribution function of X and Y may be obtained as follows:

( ),i jh x y 3 5 6

0 0.04 0.10 0.06

1 0.04 0.10 0.06

2 0.10 0.25 0.153 0.02 0.05 0.03



9/18


10/18

not on the previous or past records. Mathematically, this can be explained as

( ) | , , . . .1 1 0n n n n o P X t x X t x X t x X t x

( ) |

n n

P X t x X t x

i.e. here, behavior of the stochastic process is such that probability distributions for its

future development depend only the present state and not on how the process arrived in

that state. Also, state space, I is discrete in nature.

Equivalently, in a Markov chain, (a Markov process where state space, I takes discrete

values), the past history is completely summarized in the current state, and, future isindependent of its past but depends only on present state.

First we shall discuss few basic concepts pertaining to Markov chains. A vector

, , . . . ,1 2 3

na a a a a is called a probability vector if all the components are nonnegative and their sum is 1.

A matrix, A such that each row is a probability vector, then A is called stochastic matrix.IfA and B are stochastic matrices, then AB is a stochastic matrix. In fact, any power of

A is a a stochastic matrix.

A stochastic matrix A is said to be regular if all the entries of A are positive for some

power mA of A where m is a positive integer.

Result 1: Let A be a regular stochastic matrix. Then A has a unique fixed probability

vector, t. i.e. One can find a unique fixed vector t such that t t A .

Result 2: The sequence , , , . . .2 3A A A of powers ofA approaches the matrix T whose

rows are each the fixed point t.

Result 3: Ifq is any vector, then , , , .2 3q A q A q A approaches the fixed point t.

Transition Matrix of a Markov Chain:

Consider a Markov chain a fine stochastic process consisting of a finite sequence of

trials, whose outcomes say, , , . . .1 2 3x x x satisfy the following properties:

(Dr. K. Gururajan, MCE, Hassanpage 10)


11/18

Each outcome belongs to the state space, m{ , , . . . a }1 2 3I a a a . The outcome of any

trial depends, at most, on the outcome of the trial and not on any other previous

outcomes. If the outcome in thn trial isi

a then we say that the system is ini

a state at

the thn stage. Thus, with each pair of states ,i ja a , we associate a probability value

ijp which indicate probability of system reaching the state ia from the state ja in nsteps. These probabilities form a matrix called transition probability matrix or just

transition matrix. This may be written as

. . .

. . .

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .

. . .

11 12 13 1

21 22 23 2

1 2 3

n

n

n n n n n

p p p p

p p p p

M

p p p p

Note: Here, thi row ofM represents the probabilities of that system will change from

ia to , , , . . . .1 2 3 na a a a Equivalently, | 1i j n n p P x j x i .

nstep transition probabilities:

The probability that a Markov chain will move from state I to state j in exactly n steps

is denoted by ( ) ( ) |ij

n

i j m n m p p n P x j x i

Evaluation of nstep transition probability matrix

If M is the transition matrix of a Markov chain, then the n step transition matrix may beobtained by taking nth power of M. Suppose that a system at time t = 0 is in the state

n, , . . . a1 2 3a a a a where the process begins, then corresponding probability vector

n

( ) ( ) ( ) ( ) (0), , . . . a1 2 3

0 0 0 0a a a a denotes the initial probability distribution. Similarly, the

step transition probability matrix may be given as n

( ) ( ) ( ) ( ) (n), , . . . a1 2 3

n n n n a a a a . Now,

the (marginal) probability mass function of the random variable may be obtainedfrom the nth step transition probabilities and the initial distribution function as follows:

( ) ( ) ( ) ( )0 0n n na a M a M . Thus, the probability distribution of a homogeneous Markov

chain is completely determined from the one step transition probability matrix M and theinitial probability distribution (0 )a .

Note: The same may be explained as



12/18

( ) ( )1 0a a M

( ) ( ) ( )2 1 0 2a a M a M

( ) ( ) ( )3 2 0 3a a M a M

Some illustrative examples on Markov chains:

1. Determine which of the following are stochastic matrices?

(i)

1 1 1

3 3 3

1 1 02 2

(ii)

3 1

4 4

2 23 3

(iii)

3 1 -

4 4

2 13 3

Solution: (i) is a stochastic matrix as all the entries are nonnegative and the sum of allvalues in both the rows equals 1. (ii) This is not a stochastic matrix, since the second row

is such that sum exceeding 1 (iii) this is again not a stochastic matrix, because one of the

entry is negative.

2. Determine which of the following are regular stochastic matrices?

(i)

1 1 1

2 4 4

0 1 0

1 1 0

2 2

A

(ii)

1 1 1

2 4 4

0 0 1

0 1 0

B

Solution: A is not a regular stochastic matrix, as it has a 1 on the main diagonal. B is a

regular stochastic matrix, Compute 3B , you would notice that all the entries are positive.

3. Find the unique fixed probability vector of the following stochastic matrices.

(i)23

0

13

1

A

(ii)

3 1

4 4

1 0

2

1 0

0

1

2

0

(iii)

1 0

1 1

2 3

2 1

3 3

0

1

6

0

A



13/18

Solution: Here, to find a unique fixed probability vector ( , ) such thatt x y t t A .

As t is a probability vector, one can take ( , )1t x x . Consider the matrix equation,

2

( , ) ( , ) 3

0

1

1 1 3

1

x x x x

; A multiplication of RHS matrices, yields,

( )1

13

x x x and ( )2

13

x x . Thus, one can find that .0 6x , hence the

required unique fixed probability vector ( . , . )0 6 0 4t .

(ii) We shall set up ( , , )t x y z as the unique fixed probability vector. Again this is a

probability vector, therefore, ( , , )1t x y x y . Now, consider the matrix equation,

3 1

4 4

1( , , ) ( , , ) 02

1 0

0

11 12

0

x y x y x y x y

Multiplying the matrices on the right hand side gives,

1

2x y

( )3 1

1

4 2

y x y x y

Solving these two equations, one obtains the unique fixed probability vector.

(iii) Here, the unique fixed probability vector may be obtained as ( . , . , . )0 1 0 6 0 3t .

5. A boys studying habits are as follows. If he studies one night, he is 70 percent

sure not to study the next night. On the other hand, if he does not study one night,

he is only 60 percent sure not to study the next night as well. In the long run, how

often does the boy studies?

Solution: First we construct transition probability matrix. The problem here is about

whether the boy studies or not on consecutive nights. As his study is given to bedependent on study in one night or not, therefore, it may be modeled in terms of Markov

chain. Thus, first we construct the required transition probability matrix. The following

table indicates the pattern of the study of the boy:



14/18

I night

II night

Transition probability Boy Studying Boy not studying

Boy Studying 0.3 0.7

Boy not studying 0.4 0.6

Therefore, it is clear that required stochastic matrix is0.3 0.7

0.4 0.6M

. To solve the

given problem, it is sufficient to compute a unique fixed probability vector, ( , )1t x x

such that t t M . Using the above, one obtains . . ( )0 3 0 4 1x x x . From this

equation, we can calculate4

11x . Thus, a chance of boy studying in the long run is

36.7%.

6. A persons playing chess game or tennis is as follows: If he plays chess game one

week then he switches to playing tennis the next week with a probability 0.2. On the

other hand, if he plays tennis one week, then there is a probability of 0.7 that he will

play tennis only in the next week as well. In the long run, how often does he play

chess game?

Solution: Like in the previous case, this one too is based on Markov process. Here,

parameters are about a player playing either chess or tennis game. Also, playing the

game in the next week is treated to be dependent on which game player plays in theprevious week. As usual, first we obtain transition probability matrix.

I Week

II Week

Transition

probabilities

Player Playing Chess Player Playing Tennis

Player Playing

Chess 0.8 0.2

Player Playing

Tennis 0.3 0.7

(Dr. K. Gururajan, MCE, Hassan, page number 14)


15/18

Clearly, the transition probability matrix is0.8 0.2

0.3 0.7M

. Here, too, the problem is

to find a unique fixed probability vector, ( , )1t x x such that t t M . Consider the

matrix equation;0.8 0.2

- ) ( - )

0.3 0.7

1 1x x x x

. A simple multiplication work

results in . . ( )0 8 0 7 1x x x from which we get .0 6x . Hence, we conclude that in

the long run, person playing chess game is about 60%.

7. A sales man S sells in only three cities, A, B, and C. Suppose that S never sells in

the same city on successive days. If S sells in city A, then the next day S sells in the

city B. However, if S sells in either B or C, then the next day S is twice likely to sell

in city A as in the other city. Find out how often, in the long run, S sells in each city.

Solution: First we shall obtain transition probability matrix which will have 3 rows and

3 columns. As a salesman does not sell in a particular city on consecutive days, clearly,

main diagonal is zero. It is given that if S sells in city A on first day, then next day hewill sell in city B, therefore, first row is 0, 1, 0. next, one can consider two cases: (i) S

selling city B or (ii) S selling in city C. It is also given that if S sells in city B with

probability p, then his chances of selling in city A, next day is 2p. There is no way, hewill sell in city C. Thus, we have p + 2p + 0 = 1 implying that p = 1/3. Therefore middle

row of the matrix is 2/3, 1/3, 0. Similarly, if S sells in city C with probability q, then his

chances of selling in city A, next day is 2q. Again, a chance of selling city B is 0. Thus,last row is 2/3, 0 , 1/3. The probability transition matrix is

I day

Next Day

A B C

A 0 1 0

B 23

0 13

C 23

13

0

(Dr. K. Gururajan, MCE, Hassan, page number 15 , 2405 2-008)


16/18

The matrix of the Markov process is

0 1 0

2 1 03 3

2 1 03 3

M

. As in the previous two

problems, we need to find a unique fixed probability vector, ( , , )t x y z such that

0 1 0

2 1 1 - - 1 - - 03 3

2 1 03 3

x y x y x y x y

Or

2 2 (1 )3 3

x y x y and 1 (1 )3

y x x y . Solving these two equations,

one obtains 0.4, 0.45 and 0.15x y z . Hence, chances of a sales man selling in

each of the cities is 40%, 45% and 15% respectively.

8. Marys gambling luck follows a pattern. If she wins a game, the probability of

winning the next game is 0.6. However, if she loses a game, the probability of losing

the next game is 0.7. There is an even chance that she wins the first game. (a) Find

the transition matrix M of the Markov process. (b) Find the probability that she

wins the second game? (c) Find the probability that she wins the third game? Find

out how often, in the long run, she wins?

10. Each year Rohith trades his car for a new car. If he has a Maruti, he trades it in

for a Santro. If he has a Santro car, then he trades it in for a Ford. However, is he

has a Ford car; he is just as likely to trade it in for a new Ford as to trade it in for a

Maruti or for a Santro. In 1995, he bought his first car which was a Ford.

(a) Find the probability that he has bought (i) a 1997 Buick, (ii) a 1998 Plymouth,(iii) a 1998 Ford?

(b) Find out how often, in the long run, he will have a Ford?

Definition of various states:

Transient state: A state i is said to be transient (or nonrecurrent) if and only if there is

a positive probability that process will not return to this state. For example, if we modela program as a Markov chain all but the final state will be transient. Otherwise program

ends in an infinite loop.

Recurrent state: A state i is said to be recurrent if and only if process returns to this

state with probability one.

Periodic state: A recurrent state which returns to the state i at regular periods of time.

Absorbing state: A state i is said to be absorbing if and only if 1ii

p . Here, once a

Markov chain enters a state, and it remains there itself.

(Dr. K. Gururajan, MCE, Hassan, page number 16, 2405 2-008)


17/18

PLEDGE

I commit to excel, in all I do.I will apply myself actively to making a difference at

Malnad College of Engineering, Hassan.

I will use every opportunity presented to me by my

superiors from this moment to make that difference.

For myself, for colleagues, and for my students

(Dr. K. Gururajan)


18/18

stoc proc kg

Documents