ellipse and gaussian distribution - i-systems.github.ioi-systems.github.io/hse545/machine learning...

Ellipse and Gaussian Distribution

Industrial AI Lab.

Coordinates with Basis

• Basis Ƹ𝑒1 Ƹ𝑒2 or basis ො𝑞1 ො𝑞2

2

Coordinate Transformation

3

Coordinate Transformation

• Coordinate change to basis of ො𝑞1 ො𝑞2

4

Equation of an Ellipse

• Unit circle

5

ො𝑥1

ො𝑥2

1

1

Equation of an Ellipse

• Independent ellipse

6

ො𝑥1

ො𝑥2

𝑏

𝑎

Equation of an Ellipse

• Dependent ellipse (Rotated ellipse)– Coordinate changes

• Now we know in basis ො𝑥1, ො𝑥2 = 𝐼

7

ො𝑥1

ො𝑥2

ො𝑦2

ො𝑦1

𝑏

𝑎ො𝑦1

ො𝑦2

𝑏𝑎

𝑥 = 𝑢𝑇𝑦

𝑢 = [ො𝑥1, ො𝑥2]

dependent ellipse

Equation of an Ellipse

• Then, we can find Σ𝑦 such that

8

Equation of an Ellipse (Python)

9

Equation of an Ellipse (Python)

10

Equation of an Ellipse (Python)

11

Equation of an Ellipse (Python)

12

Question (Reverse Problem)

• Given Σ𝑦−1 (or Σ𝑦), how to find 𝑎 (major axis) and 𝑏 (minor axis)

or

• How to find the proper matrix 𝑢

• Eigenvectors of Σ

13

Question (Reverse Problem)

14

ො𝑦1

ො𝑦2ො𝑥1

ො𝑥2𝑎 = 𝜆1

𝑏 = 𝜆2

Question (Reverse Problem)

15

Summary

• Independent ellipse in ො𝑥1, ො𝑥2• Dependent ellipse in ො𝑦1, ො𝑦2• Decouple– diagonalize

– eigen-analysis

16

ො𝑥1

ො𝑥2

ො𝑦2

ො𝑦1

𝑏

𝑎ො𝑦1

ො𝑦2

𝑏𝑎

𝑥 = 𝑢𝑇𝑦

𝑢 = [ො𝑥1, ො𝑥2]

dependent ellipse

Standard Univariate Normal Distribution

17

0 𝑦

𝑃(𝑦)

Standard Univariate Normal Distribution

18

Standard Univariate Normal Distribution

19

Standard Univariate Normal Distribution

20

Univariate Normal distribution

• Gaussian or normal distribution, 1D (mean 𝜇, variance 𝜎2)

21

0 𝑥

𝑃(𝑥)

Univariate Normal distribution

22

Univariate Normal distribution

23

Univariate Normal distribution

24

Multivariate Gaussian Models

• Similar to a univariate case, but in a matrix form

• Multivariate Gaussian models and ellipse– Ellipse shows constant Δ2 value…

25

Two Independent Variables

• In a matrix form

26

Two Independent Variables

• Summary in a matrix form

27

ො𝑥1

ො𝑥2

Two Independent Variables

28

Two Independent Variables

29

Two Dependent Variables in 𝒚𝟏, 𝒚𝟐

• Compute 𝑃𝑌 𝑦 from 𝑃𝑋 𝑥

• Relationship between 𝑦 and 𝑥

30

ො𝑥1

ො𝑥2

ො𝑦2 ො𝑦1

ො𝑦2

ො𝑦1

Two Dependent Variables in 𝒚𝟏, 𝒚𝟐

• Σ𝑥 : covariance matrix of 𝑥

• Σ𝑦 : covariance matrix of 𝑦

• If 𝑢 is an eigenvector matrix of Σ𝑦, then Σ𝑥 is a diagonal matrix

31

Two Dependent Variables in 𝒚𝟏, 𝒚𝟐

32

Two Dependent Variables in 𝒚𝟏, 𝒚𝟐

• Remark

33

Decouple using Covariance Matrix

• Given data, how to find Σ𝑦 and major (or minor) axis

(assume 𝜇𝑦 = 0)

34

ො𝑦2

ො𝑦1

ො𝑥2

ො𝑥1

𝜆2

𝜆1

Decouple using Covariance Matrix

35

Decouple using Covariance Matrix

36