dynamic hierarchical algorithms for document clustering

Intelligent Database Systems Lab

國立雲林科技大學National Yunlin University of Science and Technology

1

Dynamic hierarchical algorithms for document clustering

Presenter : Wei-Hao Huang 　Authors : Reynaldo Gil-García, Aurora Pons-Porrata

PRL, 2010


N.Y.U.S.T.

I. M.

2

Outlines Motivation Objectives Hierarchical clustering Methodology Experiments Conclusions Comments


N.Y.U.S.T.

I. M.

3

Motivation· The World Wide Web and the number of text

documents managed in organizational intranets continue to grow at an amazing speed.

· In dynamic information environments is usually desirable to apply adaptive methods for document organization such as clustering.


N.Y.U.S.T.

I. M.Objectives

4

· Static clustering methods mainly rely on having the whole collection ready before applying the algorithm.

· dynamic algorithms able to update the clustering without perform complete reclustering.

· Independent on the data order.


N.Y.U.S.T.

I. M.Hierarchical clustering· Agglomerative and divisive· Provide data-views at different levels

5


N.Y.U.S.T.

I. M.

6

Methodology

Dynamic hierarchical agglomerative framework

Specific algorithm:

Dynamic hierarchical compact (DHC)

Create disjoint hierarchies of clusters

Dynamic hierarchical star (DHS)

Produce overlapped hierarchies


N.Y.U.S.T.

I. M.

7


i is β-similarity j, if their similarity >= β

β-similarityβ is minimum similarity threshold

i is a β-isolated cluster if its similarity with all clusters < β

i j


N.Y.U.S.T.

I. M.

8



N.Y.U.S.T.

I. M.

9

Updating of the max-S graph


N.Y.U.S.T.

I. M.

10

Dynamic hierarchical compact: Connected component cover


N.Y.U.S.T.

I. M.

11

Dynamic hierarchical star:Star cover updating


N.Y.U.S.T.

I. M.Experiments· Using 15 benchmark text collection.· Clustering quality· Sensitivity to parameters· Balance· Efficiency

12


N.Y.U.S.T.

I. M.Clustering quality- Overall F1 measure

13


N.Y.U.S.T.

I. M.Clustering quality- FCubed measure

14


N.Y.U.S.T.

I. M.Clustering quality- HF1

15


N.Y.U.S.T.

I. M.Sensitivity to parameters

16


N.Y.U.S.T.

I. M.Depth and width of the hierarchies

17


N.Y.U.S.T.

I. M.Efficiency

18


N.Y.U.S.T.

I. M.

19

Conclusions

· Methods are suitable for producing hierarchical

clustering solutions in dynamic environments

effectively and efficiently.

· Better balance between depth and width.

· Offer hierarchies easier to browse than traditional

algorithms.


N.Y.U.S.T.

I. M.

20

Comments· Advantages

─ Deal with dynamic data sets.─ Effectiveness and the efficiency of the clustering.

· Applications─ Hierarchical clustering

dynamic hierarchical algorithms for document clustering

Documents