model-based clustering by probabilistic self-organizing maps

Intelligent Database Systems Lab

國立雲林科技大學National Yunlin University of Science and Technology

Model-Based Clustering by Probabilistic Self-Organizing Maps

Presenter： Chien-Hsing Chen

Author:

Shih-Sian Cheng

Hsin-Chia Fu

Hsin-Min Wang

1

2009.IEEE TNN.22


N.Y.U.S.T.

I. M.

2

Outline Motivation Objective Method Experiments Conclusion Comment


N.Y.U.S.T.

I. M.

3

develop a mixture clustering model EM, CEM, DAEM are applied to combine with PbSOM

Background knowledge competition, cooperation in SOM EM (E-step, M-step), K-means ? likelihood multivariate Gaussian distribution when K-means = SOM ?

Motivation


N.Y.U.S.T.

I. M.

4

1. introduce three approaches, and a PbSOM2. combine the three approaches with PbSOM

Objective

PbSOM


N.Y.U.S.T.

I. M.

5

EM

1 5 9

26

488

=p(2; θk=1)= the value is large

p(48; θk=1)= the value is small

expect that each xi can be close to a certain k

5

6

assume t=15need update?


N.Y.U.S.T.

I. M.

6

CEM

1 5 9

2 6488

expect that each k has good quality of data projection

5

4

3

assume t=15need update?


N.Y.U.S.T.

I. M.

7

DAEM

1 5 9

26

488

expect that do not believe f(k|xi ; θ) too much, when t=1 believe f(k|xi ; θ) larger, when t=10

EM:

large

5

4

small

initialization bias, thenlocal optimal, <1 gradual increase to 1


N.Y.U.S.T.

I. M.

8

EM

CEM

DAEM

EM based approaches


N.Y.U.S.T.

I. M.

9

Overall

PbSOM


N.Y.U.S.T.

I. M.

10

Principle concept of PbSOM

3

53

2

23

87

53

98 9

8

1

xi xi

xi

When selecting the winning neuron,PbSOM considers the neighborhood information;in contrast, SOM, does not.

|| 3- 1||

|| 3-8 |||| 3-9 ||

|| 8-1 |||| 8-1 ||


N.Y.U.S.T.

I. M.

11

PbSOM (Probabilistic SOM) xi k, if k=arg mink ||xi - nk||

xi

xi

(energy function to be maximized)

5

4


N.Y.U.S.T.

I. M.

12

Multivariate Gaussian distribution

lx1

x7

x5

x8

x1, x5, x7, x8~iid~N(ul, )


N.Y.U.S.T.

I. M.

13

PbSOM (Probabilistic SOM)

5

4

xi


N.Y.U.S.T.

I. M.

14

PbSOM (Probabilistic SOM)

5

4

xi


N.Y.U.S.T.

I. M.

15

Overall

PbSOM


N.Y.U.S.T.

I. M.

16

EM

CEM

DAEM

EM based approaches

PbSOM

h

h

h h

h

h


N.Y.U.S.T.

I. M.

17

CEM

SOCEM (PbSOM+CEM)

h h

1 5 9

2 6488

conversional SOM update:||xi - nl||||nk - nl||

batch update:xi / N||nk - nl||

3

similar to a batch K-means algorithm with considering h


N.Y.U.S.T.

I. M.

18

EM

SOME (PbSOM+EM)

h

h

1 5 9

26

488

similar to a batch K-means algorithm with considering h


N.Y.U.S.T.

I. M.

19

DAEM

SODAEM (PbSOM+DASOM)

h h


N.Y.U.S.T.

I. M.

20

overall

PbSOM


N.Y.U.S.T.

I. M.

21

Experiment- SOCEM

σ= 0.6

σ= 0.45 σ= 0.3 σ= 0.15

σ in hkl gradually recued 0.6 to 0.15


N.Y.U.S.T.

I. M.

22

Experiment- SOEM

compare to previous page, this result is more global


N.Y.U.S.T.

I. M.

23

Experiment- SODAEM

SODAEM is almost equivalent to SOME and SOCEM, respectively. When uses different β.It is not able to obtain ordered map during learning process if the value of σ is to small.


N.Y.U.S.T.

I. M.

24

Experiment-SOCEM


N.Y.U.S.T.

I. M.

25

Experiment-SOEM


N.Y.U.S.T.

I. M.

26

Experiment-SODAEM


N.Y.U.S.T.

I. M.

27

Experiment

stability

without PbSOM


N.Y.U.S.T.

I. M.

28

Experiment 1/2 distinguishKohonenSOM

SOCEM

SOCEM


N.Y.U.S.T.

I. M.

29

Experiment


N.Y.U.S.T.

I. M.

30

Experiment


N.Y.U.S.T.

I. M.

31

Experiment


N.Y.U.S.T.

I. M.

32

Experiment


N.Y.U.S.T.

I. M.

33

Experiment


N.Y.U.S.T.

I. M.

34

Conclusion

PbSOM


N.Y.U.S.T.

I. M.

35

Comment Advantage

a mixture approach, sounds solid, is presented

Drawback less novelty Is it better than conversional SOM?

Application SOM

model-based clustering by probabilistic self-organizing maps

Documents