주례발표(20101019)spectralclusteringforclass

8/8/2019 (20101019)SpectralClusteringforClass

1/18

Weekly Seminar

10/14 :

- Co-clustering many-to-many corresponding

block instances using bipartite spectral graphpartitioning

- Graph partitioning with Fiedler vector

- Graph partitioning with graph spectrum

- Co-clustering with SVD( singular valuedecomposition)

- Simple implementation for ideal data set withSVD method


2/18

Graph partitioning with Fiedler vector

Minimise weight of connections between groups

Graph cut theory :

Maximise weight of within-group connections Minimise weight of between-group connections

min cut(A,B)

Optimal cut


3/18


a

b

c

d

1

3

4

3

0

3

4

3

343

000

001

010

d

c

b

a

A

dcba

!

10

0

0

0

000

300

050

004

d

cb

a

D

dcba

!

D: degree matrix; A: adjacency matrix; D-A: Laplacian matrix

Eigenvectors of Q(=D-A) form the Laplacian spectrum


4/18


!

!

!

!

!

Eji

ji

Eji

jiii

jiijii

TT

jiij

T

xx

xaij xxd

xxAxd

AxxDxx

xxQQxx

),(

2

),(

2

2

)(

2

= 4 * C(X, X)

Given a bisection ( X, X ), define a partitioning vector

clearly, x B 1, x { 0 ( 1 = (1, 1, , 1 ), 0 =(0, 0, , 0)), thus

'1

1.s.t),,,( 21

Xi

Xixxxxx in

!! 0

C(X, X)

-Number of edges between

between-group connections-Therefore, we want tominimize

TQxx


5/18


Property of

- Q is symmetric and semi-definite, i.e.

(i)

(ii) all eigenvalues of Q are u 0- The smallest eigenvalue of Q is 0

- According toCourant-Fischerminimax principle:

the 2nd smallest eigenvalue satisfies:

- Fiedlersmethod with relaxation of x as continuous form

TQ xx

0u! jiijT xxQQxx


6/18


OriginalGraph

Sortingaccordingtothe

secondeigenvectords components

Graph Diagonalization

andpartitioning


7/18

Co-clustering is to group two types of objects

into their own clusters simultaneously.

Bipartite graph partitioning (Dhillon and Zha) Use bipartite graph to model the inter-relationship between the t

wo types of objects: the edges are of the same type in the bipartite graph so the graph cut is still easy to define.

It can be proven that the solutions are the singular vectors associ

ated with the second smallest singular value of the normalized inter-relationship matrix 2/1

2

2/1

1

! ADDA

Co-clustering with singular value decomposition


8/18

Co-clustering with non-symmetry matrix A

Simple matrix manipulation for Laplacian matrix

Thus may be written as2/12

2/1

1

! ADDA



9/18

Rewrite the above equations as

Rewrite , as

These are precisely the equations that define thesingular value decomposition (SVD) of thenormalized matrix An

2/1

2

2/1

1

! ADDA



10/18

Computation of the left and right singular vectorscorresponding to the second (largest) singular valueof An,

Just as the second singular vectors contain bimodalinformation, singular vectors u2,u3, . . . uk and v2,v3, . . . vk often contain k-modal information aboutthe data set



11/18



12/18

Simple implementation

P11

P12

P13

P14

P15

P16

P17

P21

P22

P23

P24P25

P26

P27 P28 P29

1:1corresponding :

1:ncorresponding :

m:ncorresponding :

Ideal dataset


13/18


P11

P12

P13

P14

P15

P16

P17

P21

P22

P23

P24

P25

P26

P27 P28 P29

P11 P12 P13 P14 P15 P16 P17

P21 1 0 0 0 0 0 0

P22 0 1 1 0 0 0 0P23 0 0 1 1 0 0 0

P24 0 0 0 0 1 0 0

P25 0 0 0 0 1 0 1

P26 0 0 0 0 0 0 1

P27 0 0 0 0 0 1 0

P28 0 0 0 0 0 1 0

P29 0 0 0 0 0 0 1

Adjacency Matrix


14/18


P11

P12

P13

P14

P15

P16

P17

P21

P22

P23

P24

P25

P26

P27 P28 P29

D1 1 0 0 0 0 0 0 0 00 2 0 0 0 0 0 0 0

0 0 2 0 0 0 0 0 0

0 0 0 1 0 0 0 0 0

0 0 0 0 2 0 0 0 0

0 0 0 0 0 1 0 0 00 0 0 0 0 0 1 0 0

0 0 0 0 0 0 0 1 0

0 0 0 0 0 0 0 0 1

D2 1 0 0 0 0 0 00 1 0 0 0 0 0

0 0 2 0 0 0 0

0 0 0 1 0 0 0

0 0 0 0 2 0 0

0 0 0 0 0 2 0

0 0 0 0 0 0 3


15/18


P11

P12

P13

P14

P15

P16

P17

P21

P22

P23

P24

P25

P26

P27 P28 P29

An = (D1^-0.5)*A*(D2^-0.5)

1 0 0 0 0 0 00 0.707 0.5 0 0 0 0

0 0 0.5 0.707 0 0 0

0 0 0 0 0.707 0 0

0 0 0 0 0.5 0 0.408

0 0 0 0 0 0 0.577

0 0 0 0 0 0.707 0

0 0 0 0 0 0.707 00 0 0 0 0 0 0.577


16/18


P11

P12

P13

P14

P15

P16

P17

P21

P22

P23

P24

P25

P26

P27 P28 P29

P21 0 0 0

P

22 0 0 0.5P23 0 0 0.5

P24 0 -0.447 0

P25 0 -0.447 0

P26 0 -0.447 0

P27 -0.707 0 0

P28 -0.707 0 0

P29 0 -0.447 0P11 0 0 0

P12 0 0 0.5

P13 0 0 0.5

P14 0 0 0.5

P15 0 -0.447 0

P16 -0.707 0 0

P17 0 -0.447 0

Simple Clusteringmethodfor Z with L = 2


17/18

Future works

Weighted adjacency matrix not binary

The similarity function for weighted adjacency

Directed Hausdorff distance Overlapping ratio

Construction of graph itself

Determination of L for clustering data


18/18

Far future works

Conflation for feature class level

How to measure similarities between classes consideri

ng location discrepancy and different representationJiMok_01 JiMok_02 JiMok_03JiMok_04 a

A001 A002 a B001 B002 a E001E002 a

주례발표(20101019)spectralclusteringforclass

Documents