Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-1
1036: Probability & Statistics
1036: Probability & 1036: Probability & StatisticsStatistics
Lecture 5 Lecture 5 –– Some Discrete Some Discrete Probability Distributions Probability Distributions
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-2
Discrete Uniform Distribution• If a Random variable X have values of x1, x2,… xk, with equal
probabilities, then the discrete uniform distribution is given by
• the discrete uniform distribution depends on the parameter k.
• The mean and variance of the discrete uniform distribution f (x;k) are
kxxxxk
kxf ,,, ,1);( 21 K==
k
xk
ii∑
== 1µ( )
k
xk
ii∑
=
−= 1
2
2µ
σ
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-3
Bernoulli Process• The experiment consists of n repeated trials• Each trial results in an outcome that may be classified as a
success or a failure (2 possible outcomes)• The probability of success, denoted by p, remains constant
from trial to trial• The repeated trials are independent
• The number X of successes in n Bernoulli trials is called a binomial random variable.
• The probability distribution of the binomial random variable X is
nxqpqpxn
pnxb xnx ,,2,1,0 ,1 where,),;( K==+⎟⎟⎠
⎞⎜⎜⎝
⎛= −
Identically, independently, distribution (i.i.d.)
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-4
Binomial Distribution • The binomial distribution derives it name form the fact
that the n+1 terms in the binomial expansion of (p+q)n
correspond to the various values of b(x; n, p) for x=0,1,2,…,n. That is,
• For simplicity, we define
1 ),;(),;2(),;1(),;0(
210)( 221
=++++=
⎟⎟⎠
⎞⎜⎜⎝
⎛++⎟⎟
⎠
⎞⎜⎜⎝
⎛+⎟⎟
⎠
⎞⎜⎜⎝
⎛+⎟⎟
⎠
⎞⎜⎜⎝
⎛=+ −−
pnnbpnbpnbpnb
pnn
qpn
pqn
qn
qp nnnnn
L
L
∑=
=r
xpnxbpnrB
0),;(),;(
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-5
Example 5.5• The probability that a patient recovers from a rare blood
disease is 0.4. If 15 persons infected, then the probability of (a). at least 10 recovery, (b). from 3 to 8 survive, and (c). exactly 5 survive?
Sol: Let X be the number of persons that recover
( ) ( ) )40159(14.0,15;10 .15
10.,;BxbXPa
x−==≥ ∑
=
( ) ( ) )4.0,15;2()4.0,15;8(4.0,15;83 .8
3BBxbXPb
x−==≤≤ ∑
=
( ) )4.0,15;4()4.0,15;5()4.0,15;5(5 . BBbXPc −===
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-6
Example 5.6• A retailer purchases an electronic device with a defective
rate of 3%. The inspector randomly picks 20 items. What is the probability of at least 1 defective item among these 20?
• Suppose that the retailer receives 10 shipments in a month and the inspector randomly tests 20 devices per shipment. What is the probability that there will be 3 shipments containing at least one defective device?
4562.0)03.0,20;0(1)0(1)1( =−==−=≥ bXPXP
73 )4562.01(4562.03
10)3( −⎟⎟
⎠
⎞⎜⎜⎝
⎛==YP
)4562.0,10;(yb
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-7
µ, σ of the Binomial Distribution
• The mean and variance of the binomial distribution b(x;n,p)are µ=np, σ2=npq.
Proof in a binomial experiment, the number of successes can be written as the sum of the n independent indicator variables. Then
nIIIX +++= L21
npIEIEIEIIIEXE nn =+++=+++== )()()()()( 2121 LLµ
pqpIE j =×+×= 01)(
npqIII n =+++= )()()( 22
21
22 σσσσ L
pqpppqpuIEIjIJj =−=−×+×=−= 2222222 01)()(σ
We have
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-8
Multinomial Distribution• The binomial experiment becomes a multinomial experiment
if we let each trial have more than 2 possible outcomes.• Suppose that there are k possible outcomes E1, …, Ek with
probabilities p1, …, pk. For n experiments, we have that E1occurs x1 times; E2 occurs x2 times; …; Ek occurs xk times.
• Let rvs. X1, X2, …, Xk denote the number of occurrences for E1, …, Ek. Then, we have the multinomial distribution
k
k
xk
xx
k
xk
xx
kkk
pppxxx
n
pppxxx
nnpppxxxf
L
LKK
21
21
2121
21,..,21
2121
!!...!!
,),,,,;,,,(
=
⎟⎟⎠
⎞⎜⎜⎝
⎛=
∑∑==
==k
ii
k
ii pnx
11
1 and where
nkppp )( 21 +++ L
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-9
Remarks• If the binomial is applied to sampling from a lot of
items, for example deck of cards, the sampling must be done with replacement of each item after it is observed.
• What the distribution we have if the sampling done without replacement?
• In general, this is known as a hyper-geometric experiment– A random sample of size n is selected without
replacement from N items– k of N items may be classified as successes and N−k are
classified as failures
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-10
Hypergeometric Distribution• The probability distribution of the hypergeometric random variable
X, the number of successes in a random sample of size n selected from N items of which k are labeled success and N-k labeled failure, is
• Example: lots of 40 components each are called unacceptable if they contain as many as 3 defectives or more. The procedure for sampling the lot is to select 5 components at random and to reject the lot if a defective is found. What is the probability that exactly 1 defective is found in the sample if there are 3 defective in the entire lot?
• Sol:
},min{)}(,0max{ ,),,;( nkxkNn
nN
xnkN
xk
knNxh ≤≤−−
⎟⎟⎠
⎞⎜⎜⎝
⎛
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
⎟⎟⎠
⎞⎜⎜⎝
⎛
=
)3,5,40;1(h
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-11
Mean of the HypergeometricDistribution
Nnk
nN
xnkN
xk
Nkn
nN
xnkN
xk
k
nN
xnkN
xk
xknNxxhXE
n
x
n
x
n
x
n
x
=
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
=
⎟⎟⎠
⎞⎜⎜⎝
⎛
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
=
⎟⎟⎠
⎞⎜⎜⎝
⎛
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
⎟⎟⎠
⎞⎜⎜⎝
⎛
==
∑ ∑
∑ ∑
= =
= =
11
11
11
),,;()(
1 1
0 0
1
)1,1,1;(
111
)1()1(11
0
1
0
=
−−−=
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
−−−⎟⎟⎠
⎞⎜⎜⎝
⎛ −
∑∑−
=
−
=
n
y
n
y
knNyh
nN
ynkN
yk
Let y=x-1
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-12
Variance of the HypergeometricDistribution
1)1(
1
1
11
1
11
1
11
11
11
),,;()(
00
00
1
0 1
22
⎟⎠⎞
⎜⎝⎛
−−−
−⋅=
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎟⎟⎠
⎞⎜⎜⎝
⎛ −
⎟⎟⎠
⎞⎜⎜⎝
⎛ −⎟⎟⎠
⎞⎜⎜⎝
⎛−−
−−
⎟⎟⎠
⎞⎜⎜⎝
⎛
⎟⎟⎠
⎞⎜⎜⎝
⎛⎟⎟⎠
⎞⎜⎜⎝
⎛−−
=
⎟⎟⎟⎟⎟
⎠
⎞
⎜⎜⎜⎜⎜
⎝
⎛
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
⎟⎟⎠
⎞⎜⎜⎝
⎛ −⎟⎟⎠
⎞⎜⎜⎝
⎛−−
−
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
⎟⎟⎠
⎞⎜⎜⎝
⎛⎟⎟⎠
⎞⎜⎜⎝
⎛−−
=
⎭⎬⎫
⎩⎨⎧
⎟⎟⎠
⎞⎜⎜⎝
⎛ −−⎟⎟⎠
⎞⎜⎜⎝
⎛
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
=
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
==
∑∑
∑∑
∑
∑ ∑
==
==
=
= =
Nnk
nnN
Nkn
nN
Nkn
nN
xk
xnkN
xn
nN
nN
xk
xnkN
xnN
Nkn
nN
xk
xnkN
x
nN
xk
xnkN
xNkn
xk
xk
nN
xnkN
xNkn
nN
xnkN
xk
xNknknNxhxXE
n
x
n
x
n
x
n
x
n
x
n
x
n
x
⎟⎟⎠
⎞⎜⎜⎝
⎛−⎟⎟⎠
⎞⎜⎜⎝
⎛ +=⎟⎟
⎠
⎞⎜⎜⎝
⎛− r
nr
nr
n 11 ( )
NkN
Nkn
NnN
NNkNnN
Nnk
Nnk
Nnk
NknnkN
Nnk
XEX X
−⋅⋅⋅
−−
=
⎟⎟⎠
⎞⎜⎜⎝
⎛−
−−=
−−
−−+=
−=
1
1))((
1
)()( 222 µσ
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-13
Remarks
• If n<<N (The good approximation is n/N≤0.05 ), then the hypergeometric distribution approaches to the binomial distribution. That is
( )NnkXE ==µ
⎟⎠⎞
⎜⎝⎛ −⋅⋅
−−
=Nk
Nkn
NnN 11
2σ
( ) npNnkXE ===µ ⎟
⎠⎞
⎜⎝⎛ −⋅=
Nk
Nkn 12σ
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-14
Multinomial Hypergeometric Distribution
• If N items can be partitioned into k cells A1, …, Ak with elements a1, …, ak, respectively, then the probability distribution of the random variables X1, …, Xk representing the number of elements selected from A1, …, Ak in a random sample of size n is
⎟⎟⎠
⎞⎜⎜⎝
⎛
⎟⎟⎠
⎞⎜⎜⎝
⎛⎟⎟⎠
⎞⎜⎜⎝
⎛⎟⎟⎠
⎞⎜⎜⎝
⎛
=
nN
xa
xa
xa
nNaaaxxxf k
k
kk
L
KK 2
2
1
1
2121 ),,,,,;,,,(
∑∑==
==k
ii
k
ii Nanx
11
and with
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-15
Negative Binomial Distribution• Consider the binomial experiment. Instead of finding k
successes among n trials, we continue the trials until a fixed number of successes occur.
• n is not a fixed number now, but a random variable, X• we are now interested in the probability that the kth success
occurs on the xth trial.• This is called negative binomial experiments• If repeated independent trials can result in a success with
probability p and a failure with probability q, then the probability distribution of the random variable, X, the number of the trial on which the kth success occurs, is
,...1, ,11
11
),;(* 1 +=⎟⎟⎠
⎞⎜⎜⎝
⎛−−
=⎟⎟⎠
⎞⎜⎜⎝
⎛−−
= −−− kkxqpkx
pqpkx
pkxb kxkkxk
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-16
Example 5.16Playoff rule: team wins 4 games in 7 games a winnerTeam A has probability 55% of winning over team B
a. Probability that team A win in 6th gameX: number of games to win, a RV, x=6, p=0.55k=4, min. number to win
( ) 464*
1416
55.0,4;6 −⎟⎟⎠
⎞⎜⎜⎝
⎛−−
= qpb =18.5%
b. Probability that team A wins
( ) ( ) ( ) ( ) ( )55.0,4;755.0,4;655.0,4;555.0,4;44 **** bbbbXP +++=≥
=60.8%
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-17
Geometric Distribution• Consider the special case of the negative binomial
distribution with k=1, we have a probability distribution for the number of trials required for a single success.
• The function is just the probability distribution of the random variable X, the number of the trial on which the first success occurs.
• The mean and variance of the geometric distribution are
,...3,2,1 ,);(),1;(* 1 === − xpqpxgpxb x
22 1 ,1
pp
p−
== σµ
Proof ….
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-18
Geometric Distribution1
11
1 =−
== ∑∑∞
=
−
qpq
qppq
x
x
x
x
pupqxpqxpq
qxpqqxpq
x
x
x
x
x
x
x
x
x
x
===−
=⇔=
∑∑∑
∑∑
∞
=
−∞
=
∞
=
−
−−
1
have wesides,both gsubtractin
1
1
11
1
11 µµ
pu 1=right shift
right shift
L
L
++++=
++++=432
32
432 432
pqpqpqpqqpqpqpqp
µ
µ
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-19
σ2 of Geometric Distribution• Instead of considering E(X2), we consider the following
2222
2
2
32
432
32
32
)(
)()(21)]1([
)32(2 642)]1([)]1([
34231201 )]1([34231201 )]1([
34231201)]1([
pqXE
XEXEpq
pXXE
pqpqqqpqpqpqXXqEXXE
pqpqpqpqXXqEpqpqpqpXXE
pqpqpqpXXE
=−=
−=×=−
+++=
+++=−−−
+⋅⋅+⋅⋅+⋅⋅+⋅⋅=−
+⋅⋅+⋅⋅+⋅⋅+⋅⋅=−
+⋅⋅+⋅⋅+⋅⋅+⋅⋅=−
µσ
L
L
L
L
L
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-20
Poisson process• Poisson experiments : Number of outcomes X
occurs during a given time interval• Properties of Poisson process:
– Number of outcomes occurring in one time interval isindependent of the number that occurs in any other time interval no memory
– Probability that a single outcome occurs in a short timeinterval is proportional to the length of the time interval and does not depend on the number of outcomes occurring outsides this time interval
– Probability that > 1 outcomes occur in such a short timeinterval is negligible
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-21
Poisson Distribution• The probability distribution of the Poisson random variable X,
representing the number of outcomes occurring in a given time interval or specified region denoted by t, is
where λ is the average number of outcomes per unit
• The mean and variance of the Poisson distribution both have the value λt
( ) ( ) ,...2,1,0 ,!
; ==−
xx
tetxpxt λλ
λ
Proof….
rate !!
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-22
Poisson Distribution
µµµ
µµµµµ
µ
µµµµ
==
−=
−===
∑
∑∑∑∑∞
=
−
∞
=
−−∞
=
−∞
=
−∞
=
−
0
1
1
110
!
!)1(!)1(!!)(
y
y
x
x
x
x
x
x
x
x
ye
xe
xe
xex
xexXE
µµσ
µµµ
µµµµµ
µ
µµµµ
=−=∴
==
−=
−=−=−=−
∑
∑∑∑∑∞
=
−
∞
=
−−∞
=
−∞
=
−∞
=
−
222
2
0
2
2
22
220
)(
!
!)2(!)2(!)1(
!)1()]1([
XE
ye
xe
xe
xexx
xexxXXE
y
y
x
x
x
x
x
x
x
x
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-23
Theorem 5.6• Let X be a binomial random variable with
probability distribution b(x; n, p). When n→∞, p→0, and µ=np remains constant, we have
);(),;( µxppnxb →
Proof….
Prob. & Stat. Lecture05 - discrete probability distribution ([email protected])
5-24
Proof
);( !
1lim!
11lim!
),;(lim
have weconstant,remain and while, As
1!
)11()11(1
1!
)1()1(),;(
µ
µµµµµµ
µ
µµ
µµ
xpxe
nxnnxpnxb
xnnxn
xn
nnxxnnnqp
xn
pnxb
uxn
n
xxn
n
x
n
xnx
xnxxnx
=
=⎟⎠⎞
⎜⎝⎛ −=⎟
⎠⎞
⎜⎝⎛ −⎟
⎠⎞
⎜⎝⎛ −=
∞→
⎟⎠⎞
⎜⎝⎛ −
−−−=
⎟⎠⎞
⎜⎝⎛ −⎟
⎠⎞
⎜⎝⎛+−−
=⎟⎟⎠
⎞⎜⎜⎝
⎛=
−
∞→
−
∞→∞→
−
−−
L
L