classifyimages-2012

7/21/2019 Classifyimages-2012

1/24

Classifying imagesD.A. Forsyth


2/24

Image classification - features

Issue:

category will not produce a single, simple pattern

but it might have components that are distinctive, but move around

Idea:

look for distinctive local patches (visual words)

build a histogram


3/24

Important trick: K-Means

Choose a fixed number of clusters

Choose cluster centers and point-cluster allocations to

minimize error

cant do this by search

there are too many possible allocations.

Algorithm

fix cluster centers; allocate points to closest cluster

fix allocation; compute best cluster centers

x could be any set of features for which we can compute a distance

(careful about scaling)

xj" i

2

j#elements of i'th cluster

$%

&'

(

)*i#clusters

$


4/24

K-means


5/24

Building visual words - I

Learn a dictionary

cluster patch representations with k-means

k will be big (1000s-100,000s)


6/24

Building visual words - II

Encode an image

find all interest points

for each patch around each interest point

map patch to closest cluster center

build histogram of interest points


7/24

Visual words


8/24

Visual words


9/24

Visual words


10/24

Features from visual words

Histogram

good summary of what is in image; quick and efficient

insensitive to spatial reorganization

Spatial pyramid

build histograms of local blocks at various scales

less insensitive to spatial reorganization


11/24

Spatial pyramids


12/24

Some standard tasks

Caltech 101, 256

images of isolated objects in 101, 256 categories

Imagenet

same, 1000s of categories

Pascal image classification

not isolated, fewer categories

Scenes

eg indoor, etc.

Materials


13/24

Evaluation methods

Total error rate percentage of classification attempts that get the wrong answer

Accuracy

percentage of classification attempts that get the right answer

Class confusion matrix

table showing how classes are mixed up

Look at errors


14/24

Accuracy on Caltech


15/24

Material classification


16/24

Spatial pyramids + scenes


17/24

Evaluation

Precision

percentage of items in retrieved set that are relevant

Recall

percentage of relevant items that are retrieved

Precision vs recall

use classifier to label a collection of images

now plot precision against recall for different classifier thresholds

AP average precision

average of precision as a function of recall


18/24


19/24

Precision vs recall


20/24


21/24

Can be hard


22/24


23/24


24/24

classifyimages-2012

Documents