Support Vector Machine

YZM 3226 – Makine Öğrenmesi

Outline

◘ Support Vector Machine (Destek Vektör Makineleri)

◘ SVM Tarihçe

◘ Linear Sınıflandırıcılar

◘ SVM Uygulama Alanları

◘ SVM Related Links

Support Vector Machine

◘ Değişkenler arasındaki örüntülerin bilinmediği veri setlerindeki

sınıflama problemleri için önerilmiş bir makine öğrenmesi

yöntemidir»

◘ «Sınıflama, regresyon ve aykırı değer belirleme için kullanılabilen

eğiticili (supervised) öğrenme yöntemidir»

◘ Eğitim verisinde öğrenme yaparak yeni veri üzerinde doğru tahmin

yapmaya ve genelleştirmeye çalışan makine öğrenmesidir»

SVM History

◘ Vapnik and colleagues (1992)—groundwork from Vapnik &

Chervonenkis’ statistical learning theory in 1960s.

V. Vapnik

Support Vector Machines (SVM)

◘ It searches for the linear optimal separating hyperplane (i.e., “decision

boundary”)

◘ SVM finds this hyperplane using support vectors (training tuples) and

margins

Support VectorsSmall Margin Large Margin

SVM—General Philosophy

Support Vectors

Small Margin Large Margin

Support Vector Machines (SVM)

◘ There are infinite lines (hyperplanes)

separating the two classes but we

want to find the best one

(the one that minimizes classification

error on unseen data)

◘ SVM searches for the hyperplane

with the largest margin

October 25, 2016 Data Mining: Concepts and Techniques

SVM—Margins and Support Vectors

SVM—When Data Is Linearly Separable

m

Let data D be (X1, y1), …, (X|D|, y|D|), where Xi is the set of training tuples associated with the class labels yi

There are infinite lines (hyperplanes) separating the two classes but we want to find the best one (the one that minimizes classification error on unseen data)

SVM searches for the hyperplane with the largest margin, i.e., maximum marginal hyperplane (MMH)

Linear Classifiersf x

a

yest

denotes +1

denotes -1

f(x,w,b) = sign(w x + b)

How would you classify this data?

w x + b<0

w x + b>0

Linear Classifiersf x

a

yest

denotes +1

denotes -1

f(x,w,b) = sign(w x + b)

How would you classify this data?

Linear Classifiersf x

a

yest

denotes +1

denotes -1

f(x,w,b) = sign(w x + b)

How would you classify this data?

Linear Classifiersf x

a

yest

denotes +1

denotes -1

f(x,w,b) = sign(w x + b)

Any of these would be fine..

..but which is best?

Linear SVM Mathematically

What we know:

◘ w . x+ + b = +1

◘ w . x- + b = -1

◘ w . (x+-x-) = 2

X-

x+

ww

wxxM

2)(

M=Margin Width

SVM Applications

◘ Used for: classification and numeric prediction

◘ Applications:

– handwritten digit recognition,

– text recognition,

– speech recognition,

– object recognition,

– speaker identification,

– benchmarking time-series prediction tests,

– content-based image retrieval,

– biometrics,

– protein sequence problem

– breast cancer diagnosis

– etc.