model-based clustering by probabilistic self-organizing maps

Intelligent Database Systems Lab

國立雲林科技大學National Yunlin University of Science and Technology

Model-Based Clustering by Probabilistic Self-Organizing Maps

Presenter： Chien-Hsing Chen

Author:

Shih-Sian Cheng

Hsin-Chia Fu

Hsin-Min Wang

1

2009.IEEE TNN.22

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

2

Outline Motivation Objective Method Experiments Conclusion Comment

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

3

develop a mixture clustering model EM, CEM, DAEM are applied to combine with PbSOM

Background knowledge competition, cooperation in SOM EM (E-step, M-step), K-means ? likelihood multivariate Gaussian distribution when K-means = SOM ?

Motivation

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

4

1. introduce three approaches, and a PbSOM2. combine the three approaches with PbSOM

Objective

PbSOM

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

5

EM

1 5 9

26

488

=p(2; θk=1)= the value is large

p(48; θk=1)= the value is small

expect that each xi can be close to a certain k

5

6

assume t=15need update?

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

6

CEM

1 5 9

2 6488

expect that each k has good quality of data projection

5

4

3

assume t=15need update?

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

7

DAEM

1 5 9

26

488

expect that do not believe f(k|xi ; θ) too much, when t=1 believe f(k|xi ; θ) larger, when t=10

EM:

large

5

4

small

initialization bias, thenlocal optimal, <1 gradual increase to 1

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

8

EM

CEM

DAEM

EM based approaches

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

9

Overall

PbSOM

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

10

Principle concept of PbSOM

3

53

2

23

87

53

98 9

8

1

xi xi

xi

When selecting the winning neuron,PbSOM considers the neighborhood information;in contrast, SOM, does not.

|| 3- 1||

|| 3-8 |||| 3-9 ||

|| 8-1 |||| 8-1 ||

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

11

PbSOM (Probabilistic SOM) xi k, if k=arg mink ||xi - nk||

xi

xi

(energy function to be maximized)

5

4

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

12

Multivariate Gaussian distribution

lx1

x7

x5

x8

x1, x5, x7, x8~iid~N(ul, )

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

13

PbSOM (Probabilistic SOM)

5

4

xi

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

14

PbSOM (Probabilistic SOM)

5

4

xi

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

15

Overall

PbSOM

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

16

EM

CEM

DAEM

EM based approaches

PbSOM

h

h

h h

h

h

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

17

CEM

SOCEM (PbSOM+CEM)

h h

1 5 9

2 6488

conversional SOM update:||xi - nl||||nk - nl||

batch update:xi / N||nk - nl||

3

similar to a batch K-means algorithm with considering h

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

18

EM

SOME (PbSOM+EM)

h

h

1 5 9

26

488

similar to a batch K-means algorithm with considering h

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

19

DAEM

SODAEM (PbSOM+DASOM)

h h

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

20

overall

PbSOM

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

21

Experiment- SOCEM

σ= 0.6

σ= 0.45 σ= 0.3 σ= 0.15

σ in hkl gradually recued 0.6 to 0.15

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

22

Experiment- SOEM

compare to previous page, this result is more global

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

23

Experiment- SODAEM

SODAEM is almost equivalent to SOME and SOCEM, respectively. When uses different β.It is not able to obtain ordered map during learning process if the value of σ is to small.

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

24

Experiment-SOCEM

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

25

Experiment-SOEM

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

26

Experiment-SODAEM

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

27

Experiment

stability

without PbSOM

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

28

Experiment 1/2 distinguishKohonenSOM

SOCEM

SOCEM

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

29

Experiment

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

30

Experiment

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

31

Experiment

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

32

Experiment

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

33

Experiment

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

34

Conclusion

PbSOM

Intelligent Database Systems Lab

N.Y.U.S.T.

I. M.

35

Comment Advantage

a mixture approach, sounds solid, is presented

Drawback less novelty Is it better than conversional SOM?

Application SOM