2016 lecture 3 ry

8/19/2019 2016 Lecture 3 Ry

1/39

Yates & Goodman 3e 19

Figure 1.1

A B1

B2

B3

B4 C 1 C 2 C 3 C 4

In this example of Theorem 1.8, the partition is B = {B1, B2, B3, B4} and

C i = A∩Bi for i = 1, . . . ,4. It should be apparent that A = C 1∪C 2∪C 3∪C 4.

8/19/2019 2016 Lecture 3 Ry

2/39


Theorem 1.10 Law of Total Probability

For a partition {B1, B2, . . . , Bm} with P[Bi] > 0 for all i,

P [A] =mX

i=1

P [A|Bi] P [Bi] .

8/19/2019 2016 Lecture 3 Ry

3/39


Example 1.19 Problem

A company has three machines B1, B2, and B3 making 1 kΩ resistors.

Resistors within 50 Ω of the nominal value are considered acceptable. It

has been observed that 80% of the resistors produced by B1 and 90% of

the resistors produced by B2 are acceptable. The percentage for machine

B3 is 60%. Each hour, machine B1 produces 3000 resistors, B2 produces

4000 resistors, and B3 produces 3000 resistors. All of the resistors are

mixed together at random in one bin and packed for shipment. What is

the probability that the company ships an acceptable resistor?

8/19/2019 2016 Lecture 3 Ry

4/39


Example 1.19 Solution

Let A = {resistor is acceptable}. Using the resistor accuracy information

to formulate a probability model, we write

P [A|B1] = 0.8, P [A|B2] = 0.9, P [A|B3] = 0.6. (1)

The production figures state that 3000 + 4000 + 3000 = 10,000 resis-

tors per hour are produced. The fraction from machine B1 is P[B1] =

3000/10,000 = 0.3. Similarly, P[B2] = 0.4 and P[B3] = 0.3. Now it is a

simple matter to apply the law of total probability to find the acceptable

probability for all resistors shipped by the company:

P [A] = P [A|B1] P [B1] + P [A|B2] P [B2] + P [A|B3] P [B3] (2)

= (0.8)(0.3 ) + ( 0.9)(0.4 ) + ( 0.6)(0.3) = 0.78. (3)

For the whole factory, 78% of resistors are within 50 Ω of the nominal

value.

8/19/2019 2016 Lecture 3 Ry

5/39



In Example 1.19 about a shipment of resistors from the factory, we learned

that:

• The probability that a resistor is from machine B3 is P[B3] = 0.3.

• The probability that a resistor is acceptable — i.e., within 50 Ω

of the nominal value — is P[A] = 0.78.

• Given that a resistor is from machine B3, the conditional probability

that it is acceptable is P[A|B3] = 0.6.

What is the probability that an acceptable resistor comes from machine

B3?

8/19/2019 2016 Lecture 3 Ry

6/39


Theorem 1.11 Bayes’ theorem

P [B|A] = P [A|B] P [B]

P [A] .

8/19/2019 2016 Lecture 3 Ry

7/39



Now we are given the event A that a resistor is within 50 Ω of the nominal

value, and we need to find P[B3|A]. Using Bayes’ theorem, we have

P [B3|A] = P [A|B3] P [B3]

P [A]. (1)

Since all of the quantities we need are given in the problem description,

our answer is

P [B3|A] = (0.6)(0.3)/(0.78) = 0.23. (2)

Similarly we obtain P[B1|A] = 0.31 and P[B2|A] = 0.46. Of all resistors

within 50 Ω of the nominal value, only 23% come from machine B3(even though this machine produces 30% of all resistors). Machine B1

produces 31% of the resistors that meet the 50 Ω criterion and machineB2 produces 46% of them.

8/19/2019 2016 Lecture 3 Ry

8/39

!"#$%&'$ )*&+ ,-# .#/+012304

565 778

9*&303121,: 0';

8/19/2019 2016 Lecture 3 Ry

9/39

!"#$%&'(&)*+$,--

/ 0$1231(4 #56 7389$: %'+;&'(%3$>9$: 7+2 %+ )3=);=3%& %7&?'+ +$& +@ &3)7 (73?& 9( )+=+'&>4 %7&?'+ $+ (E;3'&( 3'& )+=+'&>M4 %7&?'+

8/19/2019 2016 Lecture 3 Ry

10/39

!""#$%&' )*+,+ !,+ - .%,./+" !&0 - "1#!,+"2 !&0 3&+ 34 +!.*

"*!5+ %" .3/3,+02 )*+ 5,36!6%/%)7 34 5%.8%&' ! .3/3,+0 .%,./+ %"

9:.%,./+; < 9:.3/3,+0; = >?@AB < >?@-B = ?@?AC

9:.%,./+; = ?@A

9:.3/3,+0; = A@?A9:.%,./+2.3/3,+0;=A@?A

8/19/2019 2016 Lecture 3 Ry

11/39

!"#$%$&' )* +*",-$& .+/$ #-$&$ ,-$&$ +&$ ,-$ /+0$ *102$&/ "3

$+.- /-+4$/' 21, ,#" .)&.5$/ +&$ ."5"&$6 7+*6 *" /81+&$/ +&$

."5"&$69' ,-$ 4&"2+2)5),: "3 4).;)*< + ."5"&$6 .)&.5$ )/

!"# =>.)&.5$? @ =>."5"&$6? A 7BCD9 @ 7DCBD9 A BCBD

=>.)&.5$? A BCD

=>."5"&$6? A DCBD=>.)&.5$'."5"&$6?ADCBD

8/19/2019 2016 Lecture 3 Ry

12/39


Section 1.6

Independence

8/19/2019 2016 Lecture 3 Ry

13/39


Definition 1.6 Two Independent Events

Events A and B are independent if and only if

P [AB] = P [A] P [B] .

8/19/2019 2016 Lecture 3 Ry

14/39

!""#$%&' )*+,+ !,+ - .%,./+" !&0 - "1#!,+"2 !&0 3&+ 34 +!.*

"*!5+ %" .3/3,+02 )*+ 5,36!6%/%)7 34 5%.8%&' ! .3/3,+0 .%,./+ %"

9:.%,./+; < 9:.3/3,+0; = >?@AB < >?@-B = ?@?AC

9:.%,./+; = ?@A

9:.3/3,+0; = A@?A9:.%,./+2.3/3,+0;=A@?A

8/19/2019 2016 Lecture 3 Ry

15/39

!"#$%$&' )* +*",-$& .+/$ #-$&$ ,-$&$ +&$ ,-$ /+0$ *102$&/ "3

$+.- /-+4$/' 21, ,#" .)&.5$/ +&$ ."5"&$6 7+*6 *" /81+&$/ +&$

."5"&$69' ,-$ 4&"2+2)5),: "3 4).;)*< + ."5"&$6 .)&.5$ )/

!"# =>.)&.5$? @ =>."5"&$6? A 7BCD9 @ 7DCBD9 A BCBD

=>.)&.5$? A BCD

=>."5"&$6? A DCBD=>.)&.5$'."5"&$6?ADCBD

8/19/2019 2016 Lecture 3 Ry

16/39


1.6 Comment:

Independent vs. Mutually

Exclusive

Keep in mind that independent and mutually exclusive are not syn-onyms.

In some contexts these words can have similar meanings, but this is

not the case in probability. Mutually exclusive events A and B have

no outcomes in common and therefore P[AB] = 0. In most situationsindependent events are not mutually exclusive! Exceptions occur only

when P[A] = 0 or P[B] = 0. When we have to calculate probabilities,

knowledge that events A and B are mutually exclusive is very helpful.

Axiom 3 enables us to add their probabilities to obtain the probability of

the union. Knowledge that events C and D are independent is also veryuseful. Definition 1.6 enables us to multiply their probabilities to obtain

the probability of the intersection.

8/19/2019 2016 Lecture 3 Ry

17/39



Suppose that for the experiment monitoring three purchasing decisions in

Example 1.9, each outcome (a sequence of three decisions, each either

buy or not buy) is equally likely. Are the events B2 that the second

customer purchases a phone and N 2 that the second customer does not

purchase a phone independent? Are the events B1 and B2 independent?

8/19/2019 2016 Lecture 3 Ry

18/39


Example 1.21 SolutionEach element of the sample space S = {bbb, bbn,bnb, bnn,nbb,nbn,nnb,nnn} has probabil-ity 1/8. Each of the events

B2 = {bbb, bbn,nbb,nbn} and N 2 = {bnb, bnn,nnb,nnn} (1)contains four outcomes, so P[B2] = P[N 2] = 4/8. However, B2∩N 2 = ∅ and P[B2N 2] =0. That is, B2 and N 2 are mutually exclusive because the second customer cannot bothpurchase a phone and not purchase a phone. Since P[B2N 2] 6= P[B2] P[N 2], B2 and N 2are not independent. Learning whether or not the event B2 (second customer buys aphone) occurs drastically aff ects our knowledge of whether or not the event N 2 (secondcustomer does not buy a phone) occurs. Each of the events B1 = {bnn, bnb, bbn, bbb}

and B2 = {bbn, bbb,nbn,nbb} has four outcomes, so P[B1] = P[B2] = 4/8 = 1/2. In thiscase, the intersection B1 ∩ B2 = {bbn, bbb} has probability P[B1B2] = 2/8 = 1/4. SinceP[B1B2] = P[B1] P[B2], events B1 and B2 are independent. Learning whether or notthe event B2 (second customer buys a phone) occurs does not aff ect our knowledge of whether or not the event B1 (first customer buys a phone) occurs.

8/19/2019 2016 Lecture 3 Ry

19/39



Integrated circuits undergo two tests. A mechanical test determines

whether pins have the correct spacing, and an electrical test checks the

relationship of outputs to inputs. We assume that electrical failures and

mechanical failures occur independently. Our information about circuit

production tells us that mechanical failures occur with probability 0.05

and electrical failures occur with probability 0.2. What is the probability

model of an experiment that consists of testing an integrated circuit and

observing the results of the mechanical and electrical tests?

8/19/2019 2016 Lecture 3 Ry

20/39



To build the probability model, we note that the sample space contains

four outcomes:

S = {(ma, ea), (ma, er), (mr, ea), (mr, er)} (1)

where m denotes mechanical, e denotes electrical, a denotes accept, and

r denotes reject. Let M and E denote the events that the mechanical

and electrical tests are acceptable. Our prior information tells us that

P[M c] = 0.05, and P[E c] = 0.2. This implies P[M ] = 0.95 and P[E ] = 0.8.

Using the independence assumption and Definition 1.6, we obtain the

probabilities of the four outcomes:

P [(ma, ea)] = P [ME ] = P [M ] P [E ] = 0.95 × 0.8 = 0.76, (2)

P [(ma, er)] = P [ME c

] = P [M ] P [E c

] = 0.95 × 0.2 = 0.19, (3)P [(mr, ea)] = P [M cE ] = P [M c] P [E ] = 0.05 × 0.8 = 0.04, (4)

P [(mr, er)] = P [M cE c] = P [M c] P [E c] = 0.05 × 0.2 = 0.01. (5)

8/19/2019 2016 Lecture 3 Ry

21/39


Definition 1.7 Three Independent Events

A1, A2, and A3 are mutually independent if and only if

(a) A1 and A2 are independent,

(b) A2 and A3 are independent,

(c) A1 and A3 are independent,

(d) P[A1 ∩A2 ∩A3] = P[A1] P[A2] P[A3].

8/19/2019 2016 Lecture 3 Ry

22/39



In an experiment with equiprobable outcomes, the partition is S = {1,2,3,4}.

P[s] = 1/4 for all s ∈ S . Are the events A1 = {1,3,4}, A2 = {2,3,4}, and

A3 = ∅ mutually independent?

8/19/2019 2016 Lecture 3 Ry

23/39



These three sets satisfy the final condition of Definition 1.7 because

A1 ∩A2 ∩A3 = ∅, and

P [A1 ∩A2 ∩A3] = P [A1] P [A2] P [A3] = 0. (1)

However, A1 and A2 are not independent because, with all outcomes

equiprobable,

P [A1 ∩A2] = P [{3,4}] = 1/2 6= P [A1] P [A2] = 3/4 × 3/4. (2)Hence the three events are not mutually independent.

8/19/2019 2016 Lecture 3 Ry

24/39


Definition 1.8

More than Two Independent

Events

If n ≥ 3, the events A1, A2, . . . , An are mutually independent if and only if

(a) all collections of n− 1 events chosen from A1, A2, . . . An are mutually

independent,

(b) P[A1 ∩A2 ∩ · · · ∩An] = P[A1] P[A2] · · · P[An].

8/19/2019 2016 Lecture 3 Ry

25/39


Quiz 1.6Monitor two consecutive packets going through a router. Classify each one as video (v)if it was sent from a Youtube server or as ordinary data (d) otherwise. Your observationis a sequence of two letters (either v or d). For example, two video packets corresponds

to vv. The two packets are independent and the probability that any one of them isa video packet is 0.8. Denote the identity of packet i by C i. If packet i is a videopacket, then C i = v; otherwise, C i = d. Count the number N V of video packets in thetwo packets you have observed. Determine whether the following pairs of events areindependent:

(a) {N V = 2} and {N V ≥ 1}

(b) {N V ≥ 1} and {C 1 = v}

(c) {C 2 = v} and {C 1 = d}

(d) {C 2 = v} and {N V is even}

8/19/2019 2016 Lecture 3 Ry

26/39


Quiz 1.6 SolutionIn this experiment, there are four outcomes with probabilities

P[{vv}] = (0.8)2 = 0.64, P[{vd}] = (0.8)(0.2) = 0.16,

P[{dv}] = (0.2)(0.8) = 0.16, P[{dd}] = (0.2)2 = 0.04.

When checking the independence of any two events A and B, it’s wise to avoid intuitionand simply check whether P[AB] = P[A] P[B]. Using the probabilities of the outcomes,we now can test for the independence of events.(a) First, we calculate the probability of the joint event:

P [N V = 2,N V ≥ 1] = P [N V = 2] = P [{vv}] = 0.64. (1)

Next, we observe that P[N V ≥ 1] = P[{vd,dv,vv}] = 0.96.. Finally, we make thecomparison

P [N V = 2] P [N V ≥ 1] = (0.64)(0.96) 6= P [N V = 2,N V ≥ 1] , (2)

which shows the two events are dependent. [Continued]

&

8/19/2019 2016 Lecture 3 Ry

27/39


Quiz 1.6 Solution (Continued 2)

(b) The probability of the joint event is

P [N V ≥ 1, C 1 = v] = P [{vd,vv}] = 0.80. (3)From part (a), P[N V ≥ 1] = 0.96. Further, P[C 1 = v] = 0.8 so that

P [N V ≥ 1] P [C 1 = v] = (0.96)(0.8) = 0.768 6= P [N V ≥ 1, C 1 = v] . (4)

Hence, the events are dependent.(c) The problem statement that the packets were independent implies that the events

{C 2 = v} and {C 1 = d} are independent events. Just to be sure, we can do the

calculations to check:

P [C 1 = d, C 2 = v] = P [{dv}] = 0.16. (5)

Since P[C 1 = d] P[C 2 = v] = ( 0.2)(0.8 ) = 0.16, we confirm that the events areindependent. Note that this shouldn’t be surprising since we used the informationthat the packets were independent in the problem statement to determine theprobabilities of the outcomes. [Continued]

Y t & G d 3 27

8/19/2019 2016 Lecture 3 Ry

28/39


Quiz 1.6 Solution (Continued 3)

(d) The probability of the joint event is

P [C 2 = v, N V is even] = P [{vv}] = 0.64. (6)Also, each event has probability

P [C 2 = v] = P [{dv,vv}] = 0.8, (7)

P [N V is even] = P [{dd, vv}] = 0.68. (8)

Thus,

P [C 2 = v] P [N V is even] = (0.8)(0.68)= 0.544 6= P [C 2 = v, N V is even] . (9)

Thus the events are dependent.

Y t & G d 3 27

8/19/2019 2016 Lecture 3 Ry

29/39


Section 1.7

Matlab

Y t & G d 3 28

8/19/2019 2016 Lecture 3 Ry

30/39


Example 1.24

>> X=rand(1,4)X =

0.0879 0.9626 0.6627 0.2023>> X

8/19/2019 2016 Lecture 3 Ry

31/39



Use Matlab to generate 12 random student test scores T as described in

Quiz 1.3.


8/19/2019 2016 Lecture 3 Ry

32/39



Since randi(50,1,12) generates 12 test scores from the set {1, . . . ,50},

we need only to add 50 to each score to obtain test scores in the range

{51, . . . ,100}.

>> 50+randi(50,1,12)ans =

69 78 60 68 93 99 77 95 88 57 51 90


8/19/2019 2016 Lecture 3 Ry

33/39


Example 1.26

>> s=rng;

>> 50+randi(50,1,12)ans =

89 76 80 80 72 92 58 56 77 78 59 58>> rng(s);>> 50+randi(50,1,12)ans =

89 76 80 80 72 92 58 56 77 78 59 58


8/19/2019 2016 Lecture 3 Ry

34/39


Quiz 1.7

The number of characters in a tweet is equally likely to be any integer

between 1 and 140. Simulate an experiment that generates 1000 tweets

and counts the number of “long” tweets that have over 120 characters.

Repeat this experiment 5 times.


8/19/2019 2016 Lecture 3 Ry

35/39


Quiz 1.7 SolutionThese two matlab instructions

>> T=randi(140,1000,5);

>> sum(T>120)ans =

126 147 134 133 163

simulate 5 runs of an experiment each with 1000 tweets. In particular, we note thatT=randi(140,1000,5) generates a 1000 × 5 array T of pseudorandom integers between1 and 140. Each column of T has 1000 entries representing an experimental runcorresponding to the lengths of 1000 tweets. The comparison T>120 produces a 5×1000

binary matrix in which each 1 marks a long tweet with length over 120 characters.Summing this binary array along the columns with the command sum(T>120) counts thenumber of long tweets in each experimental run.

The experiment in which we examine the length of one tweet has sample space S ={s1, s2, . . . , s140} with si denoting the outcome that a tweet has length i. Note thatP[si] = 1/140 and thus

P [tweet length > 120] = P [{s121, s122, . . . , s140}] = 20140

= 17. (1)

Thus in each run of 1000 tweets, we would expect to see about 1/7 of the tweets, orabout 143 tweets, to be be long tweets with length of over 120 characters. However,because the lengths are random, we see that we observe in the neighborhood of 143long tweets in each run.

8/19/2019 2016 Lecture 3 Ry

36/39

PROBABILITY AND STOCHASTIC PROCESSES

A FRIENDLY INTRODUCTION FOR ELECTRICAL AND COMPUTER ENGINEERS

Third Edition

Chapter 2 Viewgraphs

Roy D. Yates & David J. Goodman


8/19/2019 2016 Lecture 3 Ry

37/39


Section 2.1

Tree Diagrams


8/19/2019 2016 Lecture 3 Ry

38/39


Example 2.1 Problem

For the resistors of Example 1.19, we used A to denote the event that

a randomly chosen resistor is “within 50 Ω of the nominal value.” This

could mean “acceptable.” We use the notation N (“not acceptable”) for

the complement of A. The experiment of testing a resistor can be viewed

as a two-step procedure. First we identify which machine (B1, B2, or B3)

produced the resistor. Second, we find out if the resistor is acceptable.

Draw a tree for this sequential experiment. What is the probability of

choosing a resistor from machine B2 that is not acceptable?


8/19/2019 2016 Lecture 3 Ry

39/39



B20.4

B10.3

H H H H H H H H B30.3

A0.8

N 0.2

A0.9

X X X X X X X X N 0.1A

0.6 X X X X X X X X N

0.4

•B1A 0.24•B1N 0.06•B2A 0.36

•B2N 0.04•B3A 0.18

•B

3N

0.12

This two-step procedure is shown

in the tree on the left. To use the

tree to find the probability of the

event B2N , a nonacceptable re-

sistor from machine B2, we start

at the left and find that the prob-

ability of reaching B2 is P[B2] = 0.4. We then move to the right to B2N

and multiply P[B2] by P[N |B2] = 0.1 to obtain P[B2N ] = (0.4)(0.1) =

0.04.

2016 lecture 3 ry

Documents