ellipse and gaussian distribution - i-systems.github.ioi-systems.github.io/hse545/machine learning...
TRANSCRIPT
Ellipse and Gaussian Distribution
Industrial AI Lab.
Coordinates with Basis
• Basis Ƹ𝑒1 Ƹ𝑒2 or basis ො𝑞1 ො𝑞2
2
Coordinate Transformation
3
Coordinate Transformation
• Coordinate change to basis of ො𝑞1 ො𝑞2
4
Equation of an Ellipse
• Unit circle
5
ො𝑥1
ො𝑥2
1
1
Equation of an Ellipse
• Independent ellipse
6
ො𝑥1
ො𝑥2
𝑏
𝑎
Equation of an Ellipse
• Dependent ellipse (Rotated ellipse)– Coordinate changes
• Now we know in basis ො𝑥1, ො𝑥2 = 𝐼
7
ො𝑥1
ො𝑥2
ො𝑦2
ො𝑦1
𝑏
𝑎ො𝑦1
ො𝑦2
𝑏𝑎
𝑥 = 𝑢𝑇𝑦
𝑢 = [ො𝑥1, ො𝑥2]
dependent ellipse
Equation of an Ellipse
• Then, we can find Σ𝑦 such that
8
Equation of an Ellipse (Python)
9
Equation of an Ellipse (Python)
10
Equation of an Ellipse (Python)
11
Equation of an Ellipse (Python)
12
Question (Reverse Problem)
• Given Σ𝑦−1 (or Σ𝑦), how to find 𝑎 (major axis) and 𝑏 (minor axis)
or
• How to find the proper matrix 𝑢
• Eigenvectors of Σ
13
Question (Reverse Problem)
14
ො𝑦1
ො𝑦2ො𝑥1
ො𝑥2𝑎 = 𝜆1
𝑏 = 𝜆2
Question (Reverse Problem)
15
Summary
• Independent ellipse in ො𝑥1, ො𝑥2• Dependent ellipse in ො𝑦1, ො𝑦2• Decouple– diagonalize
– eigen-analysis
16
ො𝑥1
ො𝑥2
ො𝑦2
ො𝑦1
𝑏
𝑎ො𝑦1
ො𝑦2
𝑏𝑎
𝑥 = 𝑢𝑇𝑦
𝑢 = [ො𝑥1, ො𝑥2]
dependent ellipse
Standard Univariate Normal Distribution
17
0 𝑦
𝑃(𝑦)
Standard Univariate Normal Distribution
18
Standard Univariate Normal Distribution
19
Standard Univariate Normal Distribution
20
Univariate Normal distribution
• Gaussian or normal distribution, 1D (mean 𝜇, variance 𝜎2)
21
0 𝑥
𝑃(𝑥)
Univariate Normal distribution
22
Univariate Normal distribution
23
Univariate Normal distribution
24
Multivariate Gaussian Models
• Similar to a univariate case, but in a matrix form
• Multivariate Gaussian models and ellipse– Ellipse shows constant Δ2 value…
25
Two Independent Variables
• In a matrix form
26
Two Independent Variables
• Summary in a matrix form
27
ො𝑥1
ො𝑥2
Two Independent Variables
28
Two Independent Variables
29
Two Dependent Variables in 𝒚𝟏, 𝒚𝟐
• Compute 𝑃𝑌 𝑦 from 𝑃𝑋 𝑥
• Relationship between 𝑦 and 𝑥
30
ො𝑥1
ො𝑥2
ො𝑦2 ො𝑦1
ො𝑦2
ො𝑦1
Two Dependent Variables in 𝒚𝟏, 𝒚𝟐
• Σ𝑥 : covariance matrix of 𝑥
• Σ𝑦 : covariance matrix of 𝑦
• If 𝑢 is an eigenvector matrix of Σ𝑦, then Σ𝑥 is a diagonal matrix
31
Two Dependent Variables in 𝒚𝟏, 𝒚𝟐
32
Two Dependent Variables in 𝒚𝟏, 𝒚𝟐
• Remark
33
Decouple using Covariance Matrix
• Given data, how to find Σ𝑦 and major (or minor) axis
(assume 𝜇𝑦 = 0)
34
ො𝑦2
ො𝑦1
ො𝑥2
ො𝑥1
𝜆2
𝜆1
Decouple using Covariance Matrix
35
Decouple using Covariance Matrix
36