deep learning vs multidimensional classification in human-guided text mining

Deep Learning

2015/07/04

Marat Zhanikeev maratishe@gmail.com GI研＠天神イムズ

PDF: http://bit.do/150704

in Human-Guided

Text Mining vs

Multidimensional Classification

Deep Learning vs MD Classifiers

• Deep Learning 08 10

◦ Feature-based: image → features → NN◦ Raw/Pixels : image → raw pixels → NN

• Multi-Dimentional Classification 04 05

◦ assigning classes to items in multiple dimensions• Human-Guided Text Mining 02

◦ Folksonomy + BigData◦ learning from empty state with gradually diminishing human feedback

08 A.Nguyen+2 "Deep Neural Networks are Easily Fooled..." IEEE CVPR (2015)

10 G.Goos+2 "Neural Networks: Tricks of the Trade" Springer LNCS vol.7700, 2nd edition (2012)

04 X.Zhu+1 "Introduction to Semi-Supervised Learning" Morgan and Claypool Publishers (2009)

05 D.Koller+1 "Probabilistic Graphical Models: Principles and Techniques" MIT Press (2009)

02 myself+0 "Multidimensional Classification Automation with Human Interface based on Metromaps" 4th AAI (2015)

M.Zhanikeev -- maratishe@gmail.com -- Deep Learning vs Multidimensional Classification in Human-Guided Text Mining -- http://bit.do/150704 2/26...

Deep Learning

Deep Learning (1) Feature-Based• many feature extraction libraries, normally specific to environments/targets

• problem 1: wide range of errors, can be from 50% up to 96%

• problem 2: who decides on the features?

Deep Learning (2) Raw Pixels• just feed the raw pixels to the Neural Network and let it sort it out for itself

Deep Learning (3) Google Faces• a feature-based method, extremely specific, recently acquired by Google

Deep Learning (4) Google Cats

Deep Learning (5) Raw/Pixel Method• a standard process for a pixel-based learning 12

• CSV files are traditional, one image becomes one line

0 1 1 … 0

… …

Handwriting

Black -n-white

Pixel map

Matrix in a CSV file 3

Deep Learning

Training Testing

12 "MNIST Dataset of Handwritten Digits" http://yann.lecun.com/exdb/mnist/ (2015)

Multi-Dimensional Classification

MDC : Binary Relevance (BR) Classes

• single dimension• not practical today, when most things exists in multi-dimensional space 06

Training Tuples x1 x2 Y1 Y2 Y3

1 0.7 0.4 1 1 0 2 0.6 0.2 1 1 0 3 0.1 0.9 0 0 1 4 0.3 0.1 0 0 0

h1: X → Y1 h2: X → Y2 h3: X → Y3

06 J.Ortigosa-Hernandez+3 "A Semi-supervised Approach to Multi-dimensional Classification..." 6th TAMIDA (2010)

MDC : PairWise (PW) Sets

• define classes as pairs of base BR classes 06

• lower complexity, higher error rate

1 0.7 0.4 1 1 0 2 0.6 0.2 1 1 0

0.1 0.9 0 0 1 0.3 0.1 0 0 0

h1: X → Z1 h2: X → Z2

Z1 Z2 1 0 0 1 0 0 0 0

MDC : Label Combination (LC) Method

• a class for all combinations of base BR 06

• very high complexity, still high error rate

1 0.7 0.4 1 1 0 2 0.6 0.2 1 1 0 3 0.1 0.9 0 0 1 4 0.3 0.1 0 0 0

h: X → Z

Z 1 0 0 0

MDC : The CC Method• CC: Classifier Chains method 07 -- literally, a chain of BR classes

• controlled complexity, much better error rate, but the main problem is which order?

1 0.7 0.4 1 1 0 2 0.6 0.2 1 1 0 3 0.1 0.9 0 0 1 4 0.3 0.1 0 0 0

h1: X → Y1 h2: Y1 → Y2 h3: Y2 → Y3

h2 h1 h3

07 J.Read+3 "Classifier chains for multi-label classification" Machine Learning, Springer (2011)

The MetroMap Classifier (MMC)

The Metromap Concept• like a map of a train network 01

• main advantage: e2e paths in (ontology) graphs

01 myself+0 "On Context Management Using Metro Maps" 7th SOCA (2014)

MMC : A Practical Setting

Human judgment

Auto judgement

Folksonomy

MMC : Processing Logic• processing based on human-defined metromap, the function is similar tochaining BR classes, but with higher performance

Metromap Classifier

Check Metromap

Fuzzy?

Cold? Hot?

Robot (Automatic Classification)

No Yes

MDC (MMC) vs DL(pixels)

DL : graphics vs Text

• graphics◦ pixels are already numeric◦ images can be resized to provide same-size input -- DL needs fixed-size input

• text requires complex processing1. tokenize text (words)2. frequency distribution -- variable size

3. sample distribution -- finally, the same/fixed size!

Experimental Setup (1) Humans• 2 main cases: hot + cold = picked but not used, hot - cold = picked and used(blackswans) 03

03 myself+0 "Black Swan Disaster Scenarios" IEICE PRMU研 (2014)

Experimental Setup (2) Process• the text is not numeric by nature, has to be converted into sampledfrequency distribution

• calculations in R, used h2o package 11 for deep learning

0 1 1 … 0

… …

Matrix in a CSV file

Deep Learning

Tokenize Frequency Distribution Sample

Bayes Many (Chains, Metromap , etc.)

Path 1

Path 2

11 "H2O: R Package for Learning Algorithms" http://cran.r-project.org/web/packages/h2o (2015)

Results (1) MMC vs BR

0 20 40 60 80 100 120Time sequence

0102030405060708090

Dumb ClassifierMetromap ClassifierHits on a timeline

title:keywords

0102030405060708090

title:keywords:abstract

0 20 40 60 80 100 120Time sequence

02 myself+0 "Multidimensional Classification Automation with Human Interface based on Metromaps" 4th AAI (2015)

Results (2) DL Results

0 20 40 60 80 100Time sequence

Diagonal/humanDeep learningkeys(title)rule(cold#yes hot#yes)

keys(title:keywords:abstract)rule(cold#yes hot#yes)

keys(title:keywords:abstract)rule(cold#no hot#yes)

keys(title:keywords:abstract)rule(cold#yes hot#yes)

• compared to x = ycase

• DL performs verybadly

• best performswhen abstract isused, even then about25% hits

• same performance forhot + cold and hot- cold cases

That’s all, thank you ...

MDC and Social Robotics Go Together

Social Robotics in Text Mining Context

(careless) Input

Human Human

{structure}

(pinpoint) Select

Browse (or use otherwise)

Some Knowledge

(folksonomies, knowledge bases, databases, indexes, ontologies, etc.)

(metromaps )

deep learning vs multidimensional classification in human-guided text mining

Technology

multidimensional scaling analysis

pobreza multidimensional - argentina

contabilista profissional multidimensional

penerapan model guided discovery dan guided...

coaching multidimensional

escalamiento multidimensional

implementation guided discovery model and guided …

realidad multidimensional

ws6 ultrasound-guided

laser guided misiles

pobreza multidimensional - img.lalr.co

lasser guided missile

mod multidimensional 1p

bienestar multidimensional para política pública ·...

soft guided classroom

immediate implant placement and...

implikasi guided imagery.rtf

perbandingan model pembelajaran guided inquiry dan model...

valoracion geriatrica multidimensional

el espacio multidimensional