foundations of computer science lecture 21magdon/courses/focs-slides/slideslect20.pdf · creator:...
TRANSCRIPT
![Page 1: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/1.jpg)
Foundations of Computer ScienceLecture 21
Deviations from the MeanHow Good is the Expectation as a Sumary of a Random Variable?
Variance: Uniform; Bernoulli; Binomial; Waiting Times.
Variance of a Sum
Law of Large Numbers: The 3-σ Rule
![Page 2: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/2.jpg)
Last Time
1 Expected value of a Sum.◮ Sum of dice◮ Binomial◮ Waiting time◮ Coupon collecting.
2 Build-up expectation.
3 Expected value of a product.
4 Sum of Indicators.◮ Random arrangement of hats on heads.
Creator: Malik Magdon-Ismail Deviations from the Mean: 2 / 13 Today →
![Page 3: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/3.jpg)
Today: Deviations from the Mean
1 How well does the expected value (mean) summarize a random variable?
2 Variance.
3 Variance of a sum.
4 Law of large numbersThe 3-σ rule.
Creator: Malik Magdon-Ismail Deviations from the Mean: 3 / 13 Up to Now →
![Page 4: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/4.jpg)
Probability For Analyzing a Random Experiment.
Experiment(random)
![Page 5: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/5.jpg)
Probability For Analyzing a Random Experiment.
Experiment(random)
Outcomes(complex)
![Page 6: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/6.jpg)
Probability For Analyzing a Random Experiment.
Experiment(random)
Outcomes(complex)
Measurement X(random variable)
![Page 7: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/7.jpg)
Probability For Analyzing a Random Experiment.
Experiment(random)
Outcomes(complex)
Measurement X(random variable)
Summary E[X](expectation)
![Page 8: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/8.jpg)
Probability For Analyzing a Random Experiment.
Experiment(random)
Outcomes(complex)
Measurement X(random variable)
Summary E[X](expectation)
How goodis E[X]?
Creator: Malik Magdon-Ismail Deviations from the Mean: 4 / 13 Average of n Dice →
![Page 9: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/9.jpg)
Probability For Analyzing a Random Experiment.
Experiment(random)
Outcomes(complex)
Measurement X(random variable)
Summary E[X](expectation)
How goodis E[X]?
Experiment. Roll n dice and compute X, the average of the rolls.
Creator: Malik Magdon-Ismail Deviations from the Mean: 4 / 13 Average of n Dice →
![Page 10: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/10.jpg)
Probability For Analyzing a Random Experiment.
Experiment(random)
Outcomes(complex)
Measurement X(random variable)
Summary E[X](expectation)
How goodis E[X]?
Experiment. Roll n dice and compute X, the average of the rolls.
E[average]
Creator: Malik Magdon-Ismail Deviations from the Mean: 4 / 13 Average of n Dice →
![Page 11: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/11.jpg)
Probability For Analyzing a Random Experiment.
Experiment(random)
Outcomes(complex)
Measurement X(random variable)
Summary E[X](expectation)
How goodis E[X]?
Experiment. Roll n dice and compute X, the average of the rolls.
E[average] = E[
1n· sum
]
Creator: Malik Magdon-Ismail Deviations from the Mean: 4 / 13 Average of n Dice →
![Page 12: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/12.jpg)
Probability For Analyzing a Random Experiment.
Experiment(random)
Outcomes(complex)
Measurement X(random variable)
Summary E[X](expectation)
How goodis E[X]?
Experiment. Roll n dice and compute X, the average of the rolls.
E[average] = E[
1n· sum
]
= 1n· E [sum]
Creator: Malik Magdon-Ismail Deviations from the Mean: 4 / 13 Average of n Dice →
![Page 13: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/13.jpg)
Probability For Analyzing a Random Experiment.
Experiment(random)
Outcomes(complex)
Measurement X(random variable)
Summary E[X](expectation)
How goodis E[X]?
Experiment. Roll n dice and compute X, the average of the rolls.
E[average] = E[
1n· sum
]
= 1n· E [sum] = 1
n× n× 31
2
Creator: Malik Magdon-Ismail Deviations from the Mean: 4 / 13 Average of n Dice →
![Page 14: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/14.jpg)
Probability For Analyzing a Random Experiment.
Experiment(random)
Outcomes(complex)
Measurement X(random variable)
Summary E[X](expectation)
How goodis E[X]?
Experiment. Roll n dice and compute X, the average of the rolls.
E[average] = E[
1n· sum
]
= 1n· E [sum] = 1
n× n× 31
2= 31
2.
Creator: Malik Magdon-Ismail Deviations from the Mean: 4 / 13 Average of n Dice →
![Page 15: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/15.jpg)
Average of n Dice
Number of dice n
Ave
rage
roll
1 10 102 103 104 1051
2
33.5
4
5
6
![Page 16: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/16.jpg)
Average of n Dice
Number of dice n
Ave
rage
roll
1 10 102 103 104 1051
2
33.5
4
5
6
Average of 4 dice
Pro
bab
ilit
y
1 2 3 3.5 4 5 60
0.1 σ
Average of 100 dice
Pro
bab
ilit
y
1 2 3 3.5 4 5 60
0.1 σ
Creator: Malik Magdon-Ismail Deviations from the Mean: 5 / 13 Variance →
![Page 17: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/17.jpg)
Variance: Size of the Deviations From the Mean
X = sum of 2 dice. E[X] = 7 ← µ(X)
Creator: Malik Magdon-Ismail Deviations from the Mean: 6 / 13 Risk →
![Page 18: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/18.jpg)
Variance: Size of the Deviations From the Mean
X = sum of 2 dice. E[X] = 7 ← µ(X)
X 2 3 4 5 6 7 8 9 10 11 12
∆ −5 −4 −3 −2 −1 0 1 2 3 4 5 ← X− µ
PX136
236
336
436
536
636
536
436
336
236
136
Pop Quiz. What is E[∆]?
Creator: Malik Magdon-Ismail Deviations from the Mean: 6 / 13 Risk →
![Page 19: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/19.jpg)
Variance: Size of the Deviations From the Mean
X = sum of 2 dice. E[X] = 7 ← µ(X)
X 2 3 4 5 6 7 8 9 10 11 12
∆ −5 −4 −3 −2 −1 0 1 2 3 4 5 ← X− µ
PX136
236
336
436
536
636
536
436
336
236
136
Pop Quiz. What is E[∆]?
Variance, σ2, is the expected value of the squared deviations,
σ2 = E[∆2] = E[(X− µ)2] = E[(X− E[X])2]
σ2 = E[∆2] = 136·25 +
Creator: Malik Magdon-Ismail Deviations from the Mean: 6 / 13 Risk →
![Page 20: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/20.jpg)
Variance: Size of the Deviations From the Mean
X = sum of 2 dice. E[X] = 7 ← µ(X)
X 2 3 4 5 6 7 8 9 10 11 12
∆ −5 −4 −3 −2 −1 0 1 2 3 4 5 ← X− µ
PX136
236
336
436
536
636
536
436
336
236
136
Pop Quiz. What is E[∆]?
Variance, σ2, is the expected value of the squared deviations,
σ2 = E[∆2] = E[(X− µ)2] = E[(X− E[X])2]
σ2 = E[∆2] = 136·25 +
236·16 +
Creator: Malik Magdon-Ismail Deviations from the Mean: 6 / 13 Risk →
![Page 21: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/21.jpg)
Variance: Size of the Deviations From the Mean
X = sum of 2 dice. E[X] = 7 ← µ(X)
X 2 3 4 5 6 7 8 9 10 11 12
∆ −5 −4 −3 −2 −1 0 1 2 3 4 5 ← X− µ
PX136
236
336
436
536
636
536
436
336
236
136
Pop Quiz. What is E[∆]?
Variance, σ2, is the expected value of the squared deviations,
σ2 = E[∆2] = E[(X− µ)2] = E[(X− E[X])2]
σ2 = E[∆2] = 136·25 +
236·16 +
336·9 +
436·4 +
536·1 +
636·0 +
536·1 +
436·4 +
336·9 +
236·16 +
136·25
Creator: Malik Magdon-Ismail Deviations from the Mean: 6 / 13 Risk →
![Page 22: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/22.jpg)
Variance: Size of the Deviations From the Mean
X = sum of 2 dice. E[X] = 7 ← µ(X)
X 2 3 4 5 6 7 8 9 10 11 12
∆ −5 −4 −3 −2 −1 0 1 2 3 4 5 ← X− µ
PX136
236
336
436
536
636
536
436
336
236
136
Pop Quiz. What is E[∆]?
Variance, σ2, is the expected value of the squared deviations,
σ2 = E[∆2] = E[(X− µ)2] = E[(X− E[X])2]
σ2 = E[∆2] = 136·25 +
236·16 +
336·9 +
436·4 +
536·1 +
636·0 +
536·1 +
436·4 +
336·9 +
236·16 +
136·25
= 556.
Creator: Malik Magdon-Ismail Deviations from the Mean: 6 / 13 Risk →
![Page 23: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/23.jpg)
Variance: Size of the Deviations From the Mean
X = sum of 2 dice. E[X] = 7 ← µ(X)
X 2 3 4 5 6 7 8 9 10 11 12
∆ −5 −4 −3 −2 −1 0 1 2 3 4 5 ← X− µ
PX136
236
336
436
536
636
536
436
336
236
136
Pop Quiz. What is E[∆]?
Variance, σ2, is the expected value of the squared deviations,
σ2 = E[∆2] = E[(X− µ)2] = E[(X− E[X])2]
σ2 = E[∆2] = 136·25 +
236·16 +
336·9 +
436·4 +
536·1 +
636·0 +
536·1 +
436·4 +
336·9 +
236·16 +
136·25
= 556.
Standard Deviation, σ, is the square-root of the variance,
σ =√
E[∆2] =√
E[(X− µ)2] =√
E[(X− E[X])2]
σ =√
556≈ 2.52
sum of two dice rolls = 7± 2.52.
Practice. Exercise 21.2.
Creator: Malik Magdon-Ismail Deviations from the Mean: 6 / 13 Risk →
![Page 24: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/24.jpg)
Variance is a Measure of Risk
Game 1 Game 2
Creator: Malik Magdon-Ismail Deviations from the Mean: 7 / 13 More Convenient Variance →
![Page 25: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/25.jpg)
Variance is a Measure of Risk
Game 1 Game 2
X1 :win $2 probability = 2
3;
lose $1 probability = 13.
Creator: Malik Magdon-Ismail Deviations from the Mean: 7 / 13 More Convenient Variance →
![Page 26: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/26.jpg)
Variance is a Measure of Risk
Game 1 Game 2
X1 :win $2 probability = 2
3;
lose $1 probability = 13.
X2 :win $102 probability = 2
3;
lose $201 probability = 13.
Creator: Malik Magdon-Ismail Deviations from the Mean: 7 / 13 More Convenient Variance →
![Page 27: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/27.jpg)
Variance is a Measure of Risk
Game 1 Game 2
X1 :win $2 probability = 2
3;
lose $1 probability = 13.
X2 :win $102 probability = 2
3;
lose $201 probability = 13.
E[X1] = $1
Creator: Malik Magdon-Ismail Deviations from the Mean: 7 / 13 More Convenient Variance →
![Page 28: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/28.jpg)
Variance is a Measure of Risk
Game 1 Game 2
X1 :win $2 probability = 2
3;
lose $1 probability = 13.
X2 :win $102 probability = 2
3;
lose $201 probability = 13.
E[X1] = $1 E[X2] = $1
Creator: Malik Magdon-Ismail Deviations from the Mean: 7 / 13 More Convenient Variance →
![Page 29: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/29.jpg)
Variance is a Measure of Risk
Game 1 Game 2
X1 :win $2 probability = 2
3;
lose $1 probability = 13.
X2 :win $102 probability = 2
3;
lose $201 probability = 13.
E[X1] = $1 E[X2] = $1
σ2(X1) = 23· (2− 1)2 + 1
3· (−1− 1)2
= 2
Creator: Malik Magdon-Ismail Deviations from the Mean: 7 / 13 More Convenient Variance →
![Page 30: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/30.jpg)
Variance is a Measure of Risk
Game 1 Game 2
X1 :win $2 probability = 2
3;
lose $1 probability = 13.
X2 :win $102 probability = 2
3;
lose $201 probability = 13.
E[X1] = $1 E[X2] = $1
σ2(X1) = 23· (2− 1)2 + 1
3· (−1− 1)2
= 2
σ2(X2) = 23· (102− 1)2 + 1
3· (−201− 1)2
≈ 2× 104.
Creator: Malik Magdon-Ismail Deviations from the Mean: 7 / 13 More Convenient Variance →
![Page 31: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/31.jpg)
Variance is a Measure of Risk
Game 1 Game 2
X1 :win $2 probability = 2
3;
lose $1 probability = 13.
X2 :win $102 probability = 2
3;
lose $201 probability = 13.
E[X1] = $1 E[X2] = $1
σ2(X1) = 23· (2− 1)2 + 1
3· (−1− 1)2
= 2
σ2(X2) = 23· (102− 1)2 + 1
3· (−201− 1)2
≈ 2× 104.
X1 = 1± 1.41
X2 = 1± 141
For a small expected profit you might risk a small loss (Game 1), not a huge loss.
Creator: Malik Magdon-Ismail Deviations from the Mean: 7 / 13 More Convenient Variance →
![Page 32: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/32.jpg)
A More Convenient Formula for Variance
σ2 = E[(X− µ)2]
Creator: Malik Magdon-Ismail Deviations from the Mean: 8 / 13 Variance of Uniform and Bernoulli →
![Page 33: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/33.jpg)
A More Convenient Formula for Variance
σ2 = E[(X− µ)2]
= E[X2 − 2µX + µ2] ← Expand (X− µ)2
Creator: Malik Magdon-Ismail Deviations from the Mean: 8 / 13 Variance of Uniform and Bernoulli →
![Page 34: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/34.jpg)
A More Convenient Formula for Variance
σ2 = E[(X− µ)2]
= E[X2 − 2µX + µ2] ← Expand (X− µ)2
= E[X2]− 2µ E [X] + µ2← Linearity of expectation
Creator: Malik Magdon-Ismail Deviations from the Mean: 8 / 13 Variance of Uniform and Bernoulli →
![Page 35: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/35.jpg)
A More Convenient Formula for Variance
σ2 = E[(X− µ)2]
= E[X2 − 2µX + µ2] ← Expand (X− µ)2
= E[X2]− 2µ E [X] + µ2← Linearity of expectation
= E[X2]− µ2. ← E[X] = µ
Creator: Malik Magdon-Ismail Deviations from the Mean: 8 / 13 Variance of Uniform and Bernoulli →
![Page 36: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/36.jpg)
A More Convenient Formula for Variance
σ2 = E[(X− µ)2]
= E[X2 − 2µX + µ2] ← Expand (X− µ)2
= E[X2]− 2µ E [X] + µ2← Linearity of expectation
= E[X2]− µ2. ← E[X] = µ
Variance: σ2 = E[X2]− µ2 = E[X2]− E[X]2.
Creator: Malik Magdon-Ismail Deviations from the Mean: 8 / 13 Variance of Uniform and Bernoulli →
![Page 37: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/37.jpg)
A More Convenient Formula for Variance
σ2 = E[(X− µ)2]
= E[X2 − 2µX + µ2] ← Expand (X− µ)2
= E[X2]− 2µ E [X] + µ2← Linearity of expectation
= E[X2]− µ2. ← E[X] = µ
Variance: σ2 = E[X2]− µ2 = E[X2]− E[X]2.
Sum of two dice,
E[X2] =12∑
x=2PX(x) · x2
Creator: Malik Magdon-Ismail Deviations from the Mean: 8 / 13 Variance of Uniform and Bernoulli →
![Page 38: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/38.jpg)
A More Convenient Formula for Variance
σ2 = E[(X− µ)2]
= E[X2 − 2µX + µ2] ← Expand (X− µ)2
= E[X2]− 2µ E [X] + µ2← Linearity of expectation
= E[X2]− µ2. ← E[X] = µ
Variance: σ2 = E[X2]− µ2 = E[X2]− E[X]2.
Sum of two dice,
E[X2] =12∑
x=2PX(x) · x2
= 136·22 +
Creator: Malik Magdon-Ismail Deviations from the Mean: 8 / 13 Variance of Uniform and Bernoulli →
![Page 39: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/39.jpg)
A More Convenient Formula for Variance
σ2 = E[(X− µ)2]
= E[X2 − 2µX + µ2] ← Expand (X− µ)2
= E[X2]− 2µ E [X] + µ2← Linearity of expectation
= E[X2]− µ2. ← E[X] = µ
Variance: σ2 = E[X2]− µ2 = E[X2]− E[X]2.
Sum of two dice,
E[X2] =12∑
x=2PX(x) · x2
= 136·22 +
236·32 +
Creator: Malik Magdon-Ismail Deviations from the Mean: 8 / 13 Variance of Uniform and Bernoulli →
![Page 40: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/40.jpg)
A More Convenient Formula for Variance
σ2 = E[(X− µ)2]
= E[X2 − 2µX + µ2] ← Expand (X− µ)2
= E[X2]− 2µ E [X] + µ2← Linearity of expectation
= E[X2]− µ2. ← E[X] = µ
Variance: σ2 = E[X2]− µ2 = E[X2]− E[X]2.
Sum of two dice,
E[X2] =12∑
x=2PX(x) · x2
= 136·22 +
236·32 +
336·42 +
436·52 +
536·62 +
636·72 +
536·82 +
436·92 +
336·102 +
236·112 +
136·122
Creator: Malik Magdon-Ismail Deviations from the Mean: 8 / 13 Variance of Uniform and Bernoulli →
![Page 41: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/41.jpg)
A More Convenient Formula for Variance
σ2 = E[(X− µ)2]
= E[X2 − 2µX + µ2] ← Expand (X− µ)2
= E[X2]− 2µ E [X] + µ2← Linearity of expectation
= E[X2]− µ2. ← E[X] = µ
Variance: σ2 = E[X2]− µ2 = E[X2]− E[X]2.
Sum of two dice,
E[X2] =12∑
x=2PX(x) · x2
= 136·22 +
236·32 +
336·42 +
436·52 +
536·62 +
636·72 +
536·82 +
436·92 +
336·102 +
236·112 +
136·122
= 5456
Creator: Malik Magdon-Ismail Deviations from the Mean: 8 / 13 Variance of Uniform and Bernoulli →
![Page 42: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/42.jpg)
A More Convenient Formula for Variance
σ2 = E[(X− µ)2]
= E[X2 − 2µX + µ2] ← Expand (X− µ)2
= E[X2]− 2µ E [X] + µ2← Linearity of expectation
= E[X2]− µ2. ← E[X] = µ
Variance: σ2 = E[X2]− µ2 = E[X2]− E[X]2.
Sum of two dice,
E[X2] =12∑
x=2PX(x) · x2
= 136·22 +
236·32 +
336·42 +
436·52 +
536·62 +
636·72 +
536·82 +
436·92 +
336·102 +
236·112 +
136·122
= 5456
Since µ = 7
σ2 = 5456− 72 = 55
6
Creator: Malik Magdon-Ismail Deviations from the Mean: 8 / 13 Variance of Uniform and Bernoulli →
![Page 43: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/43.jpg)
A More Convenient Formula for Variance
σ2 = E[(X− µ)2]
= E[X2 − 2µX + µ2] ← Expand (X− µ)2
= E[X2]− 2µ E [X] + µ2← Linearity of expectation
= E[X2]− µ2. ← E[X] = µ
Variance: σ2 = E[X2]− µ2 = E[X2]− E[X]2.
Sum of two dice,
E[X2] =12∑
x=2PX(x) · x2
= 136·22 +
236·32 +
336·42 +
436·52 +
536·62 +
636·72 +
536·82 +
436·92 +
336·102 +
236·112 +
136·122
= 5456
Since µ = 7
σ2 = 5456− 72 = 55
6
Theorem. Variance ≥ 0, which means E[X2] ≥ E[X]2 for any random variable X.
Creator: Malik Magdon-Ismail Deviations from the Mean: 8 / 13 Variance of Uniform and Bernoulli →
![Page 44: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/44.jpg)
Variance of Uniform and Bernoulli
Uniform. We saw earlier that E[X] = 12(n + 1).
Creator: Malik Magdon-Ismail Deviations from the Mean: 9 / 13 Linearity of Variance? →
![Page 45: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/45.jpg)
Variance of Uniform and Bernoulli
Uniform. We saw earlier that E[X] = 12(n + 1).
E[X2]
Creator: Malik Magdon-Ismail Deviations from the Mean: 9 / 13 Linearity of Variance? →
![Page 46: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/46.jpg)
Variance of Uniform and Bernoulli
Uniform. We saw earlier that E[X] = 12(n + 1).
E[X2] = 1n(12 + · · · + n2)
Creator: Malik Magdon-Ismail Deviations from the Mean: 9 / 13 Linearity of Variance? →
![Page 47: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/47.jpg)
Variance of Uniform and Bernoulli
Uniform. We saw earlier that E[X] = 12(n + 1).
E[X2] = 1n(12 + · · · + n2) = 1
n× n
6(n + 1)(2n + 1) = 1
6(n + 1)(2n + 1)
Creator: Malik Magdon-Ismail Deviations from the Mean: 9 / 13 Linearity of Variance? →
![Page 48: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/48.jpg)
Variance of Uniform and Bernoulli
Uniform. We saw earlier that E[X] = 12(n + 1).
E[X2] = 1n(12 + · · · + n2) = 1
n× n
6(n + 1)(2n + 1) = 1
6(n + 1)(2n + 1)
so
σ2(Uniform) = E[X2]− E[X]2
Creator: Malik Magdon-Ismail Deviations from the Mean: 9 / 13 Linearity of Variance? →
![Page 49: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/49.jpg)
Variance of Uniform and Bernoulli
Uniform. We saw earlier that E[X] = 12(n + 1).
E[X2] = 1n(12 + · · · + n2) = 1
n× n
6(n + 1)(2n + 1) = 1
6(n + 1)(2n + 1)
so
σ2(Uniform) = E[X2]− E[X]2 = 16(n + 1)(2n + 1)− (1
2(n + 1))2
Creator: Malik Magdon-Ismail Deviations from the Mean: 9 / 13 Linearity of Variance? →
![Page 50: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/50.jpg)
Variance of Uniform and Bernoulli
Uniform. We saw earlier that E[X] = 12(n + 1).
E[X2] = 1n(12 + · · · + n2) = 1
n× n
6(n + 1)(2n + 1) = 1
6(n + 1)(2n + 1)
so
σ2(Uniform) = E[X2]− E[X]2 = 16(n + 1)(2n + 1)− (1
2(n + 1))2 = 1
12(n2 − 1).
Creator: Malik Magdon-Ismail Deviations from the Mean: 9 / 13 Linearity of Variance? →
![Page 51: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/51.jpg)
Variance of Uniform and Bernoulli
Uniform. We saw earlier that E[X] = 12(n + 1).
E[X2] = 1n(12 + · · · + n2) = 1
n× n
6(n + 1)(2n + 1) = 1
6(n + 1)(2n + 1)
so
σ2(Uniform) = E[X2]− E[X]2 = 16(n + 1)(2n + 1)− (1
2(n + 1))2 = 1
12(n2 − 1).
Bernoulli. We saw earlier that E[X] = p.
Creator: Malik Magdon-Ismail Deviations from the Mean: 9 / 13 Linearity of Variance? →
![Page 52: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/52.jpg)
Variance of Uniform and Bernoulli
Uniform. We saw earlier that E[X] = 12(n + 1).
E[X2] = 1n(12 + · · · + n2) = 1
n× n
6(n + 1)(2n + 1) = 1
6(n + 1)(2n + 1)
so
σ2(Uniform) = E[X2]− E[X]2 = 16(n + 1)(2n + 1)− (1
2(n + 1))2 = 1
12(n2 − 1).
Bernoulli. We saw earlier that E[X] = p.
E[X2]
Creator: Malik Magdon-Ismail Deviations from the Mean: 9 / 13 Linearity of Variance? →
![Page 53: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/53.jpg)
Variance of Uniform and Bernoulli
Uniform. We saw earlier that E[X] = 12(n + 1).
E[X2] = 1n(12 + · · · + n2) = 1
n× n
6(n + 1)(2n + 1) = 1
6(n + 1)(2n + 1)
so
σ2(Uniform) = E[X2]− E[X]2 = 16(n + 1)(2n + 1)− (1
2(n + 1))2 = 1
12(n2 − 1).
Bernoulli. We saw earlier that E[X] = p.
E[X2] = p · 12 + (1− p) · 02
Creator: Malik Magdon-Ismail Deviations from the Mean: 9 / 13 Linearity of Variance? →
![Page 54: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/54.jpg)
Variance of Uniform and Bernoulli
Uniform. We saw earlier that E[X] = 12(n + 1).
E[X2] = 1n(12 + · · · + n2) = 1
n× n
6(n + 1)(2n + 1) = 1
6(n + 1)(2n + 1)
so
σ2(Uniform) = E[X2]− E[X]2 = 16(n + 1)(2n + 1)− (1
2(n + 1))2 = 1
12(n2 − 1).
Bernoulli. We saw earlier that E[X] = p.
E[X2] = p · 12 + (1− p) · 02 = p
soσ2(Bernoulli) = E[X2]− E[X]2
Creator: Malik Magdon-Ismail Deviations from the Mean: 9 / 13 Linearity of Variance? →
![Page 55: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/55.jpg)
Variance of Uniform and Bernoulli
Uniform. We saw earlier that E[X] = 12(n + 1).
E[X2] = 1n(12 + · · · + n2) = 1
n× n
6(n + 1)(2n + 1) = 1
6(n + 1)(2n + 1)
so
σ2(Uniform) = E[X2]− E[X]2 = 16(n + 1)(2n + 1)− (1
2(n + 1))2 = 1
12(n2 − 1).
Bernoulli. We saw earlier that E[X] = p.
E[X2] = p · 12 + (1− p) · 02 = p
soσ2(Bernoulli) = E[X2]− E[X]2 = p− p2 = p(1− p).
Creator: Malik Magdon-Ismail Deviations from the Mean: 9 / 13 Linearity of Variance? →
![Page 56: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/56.jpg)
Linearity of Variance?
Creator: Malik Magdon-Ismail Deviations from the Mean: 10 / 13 Variance of a Sum →
![Page 57: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/57.jpg)
Linearity of Variance?
Let X be a Bernoulli and Y = a + X (a is a constant):
Y =
a + 1 with probability p;
a with probability 1− p.
E[Y] = p · (a + 1) + (1− p) · a = a + p = a + E[X] (as expected)
Creator: Malik Magdon-Ismail Deviations from the Mean: 10 / 13 Variance of a Sum →
![Page 58: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/58.jpg)
Linearity of Variance?
Let X be a Bernoulli and Y = a + X (a is a constant):
Y =
a + 1 with probability p;
a with probability 1− p.
E[Y] = p · (a + 1) + (1− p) · a = a + p = a + E[X] (as expected)
Deviations from the mean µ = a + p:
∆Y =
1− p with probability p;
−p with probability 1− p,(deviations independent of a!)
Creator: Malik Magdon-Ismail Deviations from the Mean: 10 / 13 Variance of a Sum →
![Page 59: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/59.jpg)
Linearity of Variance?
Let X be a Bernoulli and Y = a + X (a is a constant):
Y =
a + 1 with probability p;
a with probability 1− p.
E[Y] = p · (a + 1) + (1− p) · a = a + p = a + E[X] (as expected)
Deviations from the mean µ = a + p:
∆Y =
1− p with probability p;
−p with probability 1− p,(deviations independent of a!)
Therefore σ2(Y) = σ2(X).
Creator: Malik Magdon-Ismail Deviations from the Mean: 10 / 13 Variance of a Sum →
![Page 60: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/60.jpg)
Linearity of Variance?
Let X be a Bernoulli and Y = a + X (a is a constant):
Y =
a + 1 with probability p;
a with probability 1− p.
E[Y] = p · (a + 1) + (1− p) · a = a + p = a + E[X] (as expected)
Deviations from the mean µ = a + p:
∆Y =
1− p with probability p;
−p with probability 1− p,(deviations independent of a!)
Therefore σ2(Y) = σ2(X).
Pop Quiz. Y = bX. Compute E[Y] and σ2(Y).
Creator: Malik Magdon-Ismail Deviations from the Mean: 10 / 13 Variance of a Sum →
![Page 61: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/61.jpg)
Linearity of Variance?
Let X be a Bernoulli and Y = a + X (a is a constant):
Y =
a + 1 with probability p;
a with probability 1− p.
E[Y] = p · (a + 1) + (1− p) · a = a + p = a + E[X] (as expected)
Deviations from the mean µ = a + p:
∆Y =
1− p with probability p;
−p with probability 1− p,(deviations independent of a!)
Therefore σ2(Y) = σ2(X).
Pop Quiz. Y = bX. Compute E[Y] and σ2(Y).
Theorem. Let Y = a + bX. Then,
σ2(Y) = b2σ2(X).
Creator: Malik Magdon-Ismail Deviations from the Mean: 10 / 13 Variance of a Sum →
![Page 62: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/62.jpg)
Variance of a Sum
X = X1 + X2
E[X]2 = E[X1 + X2]2
Creator: Malik Magdon-Ismail Deviations from the Mean: 11 / 13 3-σ Rule →
![Page 63: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/63.jpg)
Variance of a Sum
X = X1 + X2
E[X]2 = E[X1 + X2]2 (∗)
= (E[X1] + E[X2])2
(∗) is by linearity of expectation.
Creator: Malik Magdon-Ismail Deviations from the Mean: 11 / 13 3-σ Rule →
![Page 64: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/64.jpg)
Variance of a Sum
X = X1 + X2
E[X]2 = E[X1 + X2]2 (∗)
= (E[X1] + E[X2])2 = E[X1]
2 + E[X2]2 + 2 E [X1] E [X2];
(∗) is by linearity of expectation.
Creator: Malik Magdon-Ismail Deviations from the Mean: 11 / 13 3-σ Rule →
![Page 65: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/65.jpg)
Variance of a Sum
X = X1 + X2
E[X]2 = E[X1 + X2]2 (∗)
= (E[X1] + E[X2])2 = E[X1]
2 + E[X2]2 + 2 E [X1] E [X2];
E [X2] = E[(X1 + X2)2]
(∗) is by linearity of expectation.
Creator: Malik Magdon-Ismail Deviations from the Mean: 11 / 13 3-σ Rule →
![Page 66: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/66.jpg)
Variance of a Sum
X = X1 + X2
E[X]2 = E[X1 + X2]2 (∗)
= (E[X1] + E[X2])2 = E[X1]
2 + E[X2]2 + 2 E [X1] E [X2];
E [X2] = E[(X1 + X2)2] = E[X2
1 + X22 + 2X1X2]
(∗) is by linearity of expectation.
Creator: Malik Magdon-Ismail Deviations from the Mean: 11 / 13 3-σ Rule →
![Page 67: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/67.jpg)
Variance of a Sum
X = X1 + X2
E[X]2 = E[X1 + X2]2 (∗)
= (E[X1] + E[X2])2 = E[X1]
2 + E[X2]2 + 2 E [X1] E [X2];
E [X2] = E[(X1 + X2)2] = E[X2
1 + X22 + 2X1X2]
(∗)= E [X2
1] + E[X22] + 2 E [X1X2].
(∗) is by linearity of expectation.
Creator: Malik Magdon-Ismail Deviations from the Mean: 11 / 13 3-σ Rule →
![Page 68: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/68.jpg)
Variance of a Sum
X = X1 + X2
E[X]2 = E[X1 + X2]2 (∗)
= (E[X1] + E[X2])2 = E[X1]
2 + E[X2]2 + 2 E [X1] E [X2];
E [X2] = E[(X1 + X2)2] = E[X2
1 + X22 + 2X1X2]
(∗)= E [X2
1] + E[X22] + 2 E [X1X2].
(∗) is by linearity of expectation.
σ2(X) = E[X2]− E[X]2
Creator: Malik Magdon-Ismail Deviations from the Mean: 11 / 13 3-σ Rule →
![Page 69: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/69.jpg)
Variance of a Sum
X = X1 + X2
E[X]2 = E[X1 + X2]2 (∗)
= (E[X1] + E[X2])2 = E[X1]
2 + E[X2]2 + 2 E [X1] E [X2];
E [X2] = E[(X1 + X2)2] = E[X2
1 + X22 + 2X1X2]
(∗)= E [X2
1] + E[X22] + 2 E [X1X2].
(∗) is by linearity of expectation.
σ2(X) = E[X2]− E[X]2
= (E[X21] + E[X2
2] + 2 E [X1X2])− (E[X1]2 + E[X2]
2 + 2 E [X1] E [X2])
Creator: Malik Magdon-Ismail Deviations from the Mean: 11 / 13 3-σ Rule →
![Page 70: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/70.jpg)
Variance of a Sum
X = X1 + X2
E[X]2 = E[X1 + X2]2 (∗)
= (E[X1] + E[X2])2 = E[X1]
2 + E[X2]2 + 2 E [X1] E [X2];
E [X2] = E[(X1 + X2)2] = E[X2
1 + X22 + 2X1X2]
(∗)= E [X2
1] + E[X22] + 2 E [X1X2].
(∗) is by linearity of expectation.
σ2(X) = E[X2]− E[X]2
= (E[X21] + E[X2
2] + 2 E [X1X2])− (E[X1]2 + E[X2]
2 + 2 E [X1] E [X2])
= E[X21]− E[X1]
2
︸ ︷︷ ︸
σ2(X1)
+ E[X22]− E[X2]
2
︸ ︷︷ ︸
σ2(X2)
+2 (E[X1X2]− E[X1] E [X2])︸ ︷︷ ︸
0 if X1 and X2 are independent
Creator: Malik Magdon-Ismail Deviations from the Mean: 11 / 13 3-σ Rule →
![Page 71: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/71.jpg)
Variance of a Sum
X = X1 + X2
E[X]2 = E[X1 + X2]2 (∗)
= (E[X1] + E[X2])2 = E[X1]
2 + E[X2]2 + 2 E [X1] E [X2];
E [X2] = E[(X1 + X2)2] = E[X2
1 + X22 + 2X1X2]
(∗)= E [X2
1] + E[X22] + 2 E [X1X2].
(∗) is by linearity of expectation.
σ2(X) = E[X2]− E[X]2
= (E[X21] + E[X2
2] + 2 E [X1X2])− (E[X1]2 + E[X2]
2 + 2 E [X1] E [X2])
= E[X21]− E[X1]
2
︸ ︷︷ ︸
σ2(X1)
+ E[X22]− E[X2]
2
︸ ︷︷ ︸
σ2(X2)
+2 (E[X1X2]− E[X1] E [X2])︸ ︷︷ ︸
0 if X1 and X2 are independent
Variance of a Sum. For independent random variables,the variance of the sum is a sum of the variances.
Practice. Compute the variance of 1 dice roll. Compute the variance of the sum of n dice rolls.
Creator: Malik Magdon-Ismail Deviations from the Mean: 11 / 13 3-σ Rule →
![Page 72: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/72.jpg)
Variance of a Sum
X = X1 + X2
E[X]2 = E[X1 + X2]2 (∗)
= (E[X1] + E[X2])2 = E[X1]
2 + E[X2]2 + 2 E [X1] E [X2];
E [X2] = E[(X1 + X2)2] = E[X2
1 + X22 + 2X1X2]
(∗)= E [X2
1] + E[X22] + 2 E [X1X2].
(∗) is by linearity of expectation.
σ2(X) = E[X2]− E[X]2
= (E[X21] + E[X2
2] + 2 E [X1X2])− (E[X1]2 + E[X2]
2 + 2 E [X1] E [X2])
= E[X21]− E[X1]
2
︸ ︷︷ ︸
σ2(X1)
+ E[X22]− E[X2]
2
︸ ︷︷ ︸
σ2(X2)
+2 (E[X1X2]− E[X1] E [X2])︸ ︷︷ ︸
0 if X1 and X2 are independent
Variance of a Sum. For independent random variables,the variance of the sum is a sum of the variances.
Practice. Compute the variance of 1 dice roll. Compute the variance of the sum of n dice rolls.
Example. The Variance of the Binomial (sum of independent Bernoullis)
X = X1 + · · · + Xn (sum of independent Bernoullis), and σ2(Xi) = p(1− p)
Creator: Malik Magdon-Ismail Deviations from the Mean: 11 / 13 3-σ Rule →
![Page 73: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/73.jpg)
Variance of a Sum
X = X1 + X2
E[X]2 = E[X1 + X2]2 (∗)
= (E[X1] + E[X2])2 = E[X1]
2 + E[X2]2 + 2 E [X1] E [X2];
E [X2] = E[(X1 + X2)2] = E[X2
1 + X22 + 2X1X2]
(∗)= E [X2
1] + E[X22] + 2 E [X1X2].
(∗) is by linearity of expectation.
σ2(X) = E[X2]− E[X]2
= (E[X21] + E[X2
2] + 2 E [X1X2])− (E[X1]2 + E[X2]
2 + 2 E [X1] E [X2])
= E[X21]− E[X1]
2
︸ ︷︷ ︸
σ2(X1)
+ E[X22]− E[X2]
2
︸ ︷︷ ︸
σ2(X2)
+2 (E[X1X2]− E[X1] E [X2])︸ ︷︷ ︸
0 if X1 and X2 are independent
Variance of a Sum. For independent random variables,the variance of the sum is a sum of the variances.
Practice. Compute the variance of 1 dice roll. Compute the variance of the sum of n dice rolls.
Example. The Variance of the Binomial (sum of independent Bernoullis)
X = X1 + · · · + Xn (sum of independent Bernoullis), and σ2(Xi) = p(1− p)
σ2(Binomial) = σ2(X1) + · · · + σ2(Xn) = p(1− p) + · · · + p(1− p) = np(1− p).
Creator: Malik Magdon-Ismail Deviations from the Mean: 11 / 13 3-σ Rule →
![Page 74: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/74.jpg)
3-σ Rule: X = µ(X)± σ(X)
3-σ Rule. For any random variable X, the chances are at least (about) 90% that
µ− 3σ < X < µ + 3σ or X = µ± 3σ.
Creator: Malik Magdon-Ismail Deviations from the Mean: 12 / 13 Law of Large Numbers →
![Page 75: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/75.jpg)
3-σ Rule: X = µ(X)± σ(X)
3-σ Rule. For any random variable X, the chances are at least (about) 90% that
µ− 3σ < X < µ + 3σ or X = µ± 3σ.
Lemma (Markov Inequality). For a positive random variable X,
P[X ≥ α] ≤E[X]
α.
Creator: Malik Magdon-Ismail Deviations from the Mean: 12 / 13 Law of Large Numbers →
![Page 76: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/76.jpg)
3-σ Rule: X = µ(X)± σ(X)
3-σ Rule. For any random variable X, the chances are at least (about) 90% that
µ− 3σ < X < µ + 3σ or X = µ± 3σ.
Lemma (Markov Inequality). For a positive random variable X,
P[X ≥ α] ≤E[X]
α.
Proof. E[X]
Creator: Malik Magdon-Ismail Deviations from the Mean: 12 / 13 Law of Large Numbers →
![Page 77: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/77.jpg)
3-σ Rule: X = µ(X)± σ(X)
3-σ Rule. For any random variable X, the chances are at least (about) 90% that
µ− 3σ < X < µ + 3σ or X = µ± 3σ.
Lemma (Markov Inequality). For a positive random variable X,
P[X ≥ α] ≤E[X]
α.
Proof. E[X] =∑
x≥0x · PX(x)
Creator: Malik Magdon-Ismail Deviations from the Mean: 12 / 13 Law of Large Numbers →
![Page 78: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/78.jpg)
3-σ Rule: X = µ(X)± σ(X)
3-σ Rule. For any random variable X, the chances are at least (about) 90% that
µ− 3σ < X < µ + 3σ or X = µ± 3σ.
Lemma (Markov Inequality). For a positive random variable X,
P[X ≥ α] ≤E[X]
α.
Proof. E[X] =∑
x≥0x · PX(x) ≥
∑
x≥αx · PX(x)
Creator: Malik Magdon-Ismail Deviations from the Mean: 12 / 13 Law of Large Numbers →
![Page 79: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/79.jpg)
3-σ Rule: X = µ(X)± σ(X)
3-σ Rule. For any random variable X, the chances are at least (about) 90% that
µ− 3σ < X < µ + 3σ or X = µ± 3σ.
Lemma (Markov Inequality). For a positive random variable X,
P[X ≥ α] ≤E[X]
α.
Proof. E[X] =∑
x≥0x · PX(x) ≥
∑
x≥αx · PX(x) ≥
∑
x≥αα · PX(x)
Creator: Malik Magdon-Ismail Deviations from the Mean: 12 / 13 Law of Large Numbers →
![Page 80: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/80.jpg)
3-σ Rule: X = µ(X)± σ(X)
3-σ Rule. For any random variable X, the chances are at least (about) 90% that
µ− 3σ < X < µ + 3σ or X = µ± 3σ.
Lemma (Markov Inequality). For a positive random variable X,
P[X ≥ α] ≤E[X]
α.
Proof. E[X] =∑
x≥0x · PX(x) ≥
∑
x≥αx · PX(x) ≥
∑
x≥αα · PX(x) = α · P[X ≥ α].
Creator: Malik Magdon-Ismail Deviations from the Mean: 12 / 13 Law of Large Numbers →
![Page 81: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/81.jpg)
3-σ Rule: X = µ(X)± σ(X)
3-σ Rule. For any random variable X, the chances are at least (about) 90% that
µ− 3σ < X < µ + 3σ or X = µ± 3σ.
Lemma (Markov Inequality). For a positive random variable X,
P[X ≥ α] ≤E[X]
α.
Proof. E[X] =∑
x≥0x · PX(x) ≥
∑
x≥αx · PX(x) ≥
∑
x≥αα · PX(x) = α · P[X ≥ α].
Lemma (Chebyshev Inequality).
P[|∆| ≥ tσ] ≤1
t2.
Creator: Malik Magdon-Ismail Deviations from the Mean: 12 / 13 Law of Large Numbers →
![Page 82: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/82.jpg)
3-σ Rule: X = µ(X)± σ(X)
3-σ Rule. For any random variable X, the chances are at least (about) 90% that
µ− 3σ < X < µ + 3σ or X = µ± 3σ.
Lemma (Markov Inequality). For a positive random variable X,
P[X ≥ α] ≤E[X]
α.
Proof. E[X] =∑
x≥0x · PX(x) ≥
∑
x≥αx · PX(x) ≥
∑
x≥αα · PX(x) = α · P[X ≥ α].
Lemma (Chebyshev Inequality).
P[|∆| ≥ tσ] ≤1
t2.
Proof.
P[|∆| ≥ tσ]
Creator: Malik Magdon-Ismail Deviations from the Mean: 12 / 13 Law of Large Numbers →
![Page 83: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/83.jpg)
3-σ Rule: X = µ(X)± σ(X)
3-σ Rule. For any random variable X, the chances are at least (about) 90% that
µ− 3σ < X < µ + 3σ or X = µ± 3σ.
Lemma (Markov Inequality). For a positive random variable X,
P[X ≥ α] ≤E[X]
α.
Proof. E[X] =∑
x≥0x · PX(x) ≥
∑
x≥αx · PX(x) ≥
∑
x≥αα · PX(x) = α · P[X ≥ α].
Lemma (Chebyshev Inequality).
P[|∆| ≥ tσ] ≤1
t2.
Proof.
P[|∆| ≥ tσ] = P[∆2 ≥ t2σ2]
Creator: Malik Magdon-Ismail Deviations from the Mean: 12 / 13 Law of Large Numbers →
![Page 84: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/84.jpg)
3-σ Rule: X = µ(X)± σ(X)
3-σ Rule. For any random variable X, the chances are at least (about) 90% that
µ− 3σ < X < µ + 3σ or X = µ± 3σ.
Lemma (Markov Inequality). For a positive random variable X,
P[X ≥ α] ≤E[X]
α.
Proof. E[X] =∑
x≥0x · PX(x) ≥
∑
x≥αx · PX(x) ≥
∑
x≥αα · PX(x) = α · P[X ≥ α].
Lemma (Chebyshev Inequality).
P[|∆| ≥ tσ] ≤1
t2.
Proof.
P[|∆| ≥ tσ] = P[∆2 ≥ t2σ2](a)
≤E[∆2]
t2σ2
Creator: Malik Magdon-Ismail Deviations from the Mean: 12 / 13 Law of Large Numbers →
![Page 85: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/85.jpg)
3-σ Rule: X = µ(X)± σ(X)
3-σ Rule. For any random variable X, the chances are at least (about) 90% that
µ− 3σ < X < µ + 3σ or X = µ± 3σ.
Lemma (Markov Inequality). For a positive random variable X,
P[X ≥ α] ≤E[X]
α.
Proof. E[X] =∑
x≥0x · PX(x) ≥
∑
x≥αx · PX(x) ≥
∑
x≥αα · PX(x) = α · P[X ≥ α].
Lemma (Chebyshev Inequality).
P[|∆| ≥ tσ] ≤1
t2.
Proof.
P[|∆| ≥ tσ] = P[∆2 ≥ t2σ2](a)
≤E[∆2]
t2σ2=
σ2
t2σ2
Creator: Malik Magdon-Ismail Deviations from the Mean: 12 / 13 Law of Large Numbers →
![Page 86: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/86.jpg)
3-σ Rule: X = µ(X)± σ(X)
3-σ Rule. For any random variable X, the chances are at least (about) 90% that
µ− 3σ < X < µ + 3σ or X = µ± 3σ.
Lemma (Markov Inequality). For a positive random variable X,
P[X ≥ α] ≤E[X]
α.
Proof. E[X] =∑
x≥0x · PX(x) ≥
∑
x≥αx · PX(x) ≥
∑
x≥αα · PX(x) = α · P[X ≥ α].
Lemma (Chebyshev Inequality).
P[|∆| ≥ tσ] ≤1
t2.
Proof.
P[|∆| ≥ tσ] = P[∆2 ≥ t2σ2](a)
≤E[∆2]
t2σ2=
σ2
t2σ2=
1
t2.
In (a) we used Markov’s Inequality.
To get the 3-σ rule, use Chebyshev’s Inequality with t = 3.
Creator: Malik Magdon-Ismail Deviations from the Mean: 12 / 13 Law of Large Numbers →
![Page 87: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/87.jpg)
Law of Large Numbers
Expectation of the average of n dice:
E[average] = E[ 1n× sum] = 1
n× E[sum] = 1
n× n× 31
2
Creator: Malik Magdon-Ismail Deviations from the Mean: 13 / 13
![Page 88: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/88.jpg)
Law of Large Numbers
Expectation of the average of n dice:
E[average] = E[ 1n× sum] = 1
n× E[sum] = 1
n× n× 31
2
Variance of the average of n dice:
σ2(average)
Creator: Malik Magdon-Ismail Deviations from the Mean: 13 / 13
![Page 89: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/89.jpg)
Law of Large Numbers
Expectation of the average of n dice:
E[average] = E[ 1n× sum] = 1
n× E[sum] = 1
n× n× 31
2
Variance of the average of n dice:
σ2(average) = σ2( 1n× sum)
Creator: Malik Magdon-Ismail Deviations from the Mean: 13 / 13
![Page 90: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/90.jpg)
Law of Large Numbers
Expectation of the average of n dice:
E[average] = E[ 1n× sum] = 1
n× E[sum] = 1
n× n× 31
2
Variance of the average of n dice:
σ2(average) = σ2( 1n× sum) = 1
n2× σ2(sum)
Creator: Malik Magdon-Ismail Deviations from the Mean: 13 / 13
![Page 91: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/91.jpg)
Law of Large Numbers
Expectation of the average of n dice:
E[average] = E[ 1n× sum] = 1
n× E[sum] = 1
n× n× 31
2
Variance of the average of n dice:
σ2(average) = σ2( 1n× sum) = 1
n2× σ2(sum) = 1
n2× n× σ2(one die)
Creator: Malik Magdon-Ismail Deviations from the Mean: 13 / 13
![Page 92: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/92.jpg)
Law of Large Numbers
Expectation of the average of n dice:
E[average] = E[ 1n× sum] = 1
n× E[sum] = 1
n× n× 31
2
Variance of the average of n dice:
σ2(average) = σ2( 1n× sum) = 1
n2× σ2(sum) = 1
n2× n× σ2(one die) = 1
n× σ2(one die)
Creator: Malik Magdon-Ismail Deviations from the Mean: 13 / 13
![Page 93: Foundations of Computer Science Lecture 21magdon/courses/FOCS-Slides/SlidesLect20.pdf · Creator: Malik Magdon-Ismail Deviations from the Mean: 6/13 Risk → ... lose $201 probability](https://reader033.vdocuments.pub/reader033/viewer/2022042922/5f6f18e58a49492dfb622ba3/html5/thumbnails/93.jpg)
Law of Large Numbers
Expectation of the average of n dice:
E[average] = E[ 1n× sum] = 1
n× E[sum] = 1
n× n× 31
2
Variance of the average of n dice:
σ2(average) = σ2( 1n× sum) = 1
n2× σ2(sum) = 1
n2× n× σ2(one die) = 1
n× σ2(one die)
n
3-si
gma
range [µ− 3σ, µ + 3σ]
1 10 102 103 104
1
312
6
Creator: Malik Magdon-Ismail Deviations from the Mean: 13 / 13