1

Object Recognition by Discriminative Combinations of Line Segments, Ellipses and Appearance FeaturesProfessor: S. J. WangStudent : Y. S. Wang1OutlineBackgroundSystem OverviewShape-TokenCode-Book of Shape-TokenCode-Word CombinationHybrid DetectorExperimental ResultConclusion

2outlinereferenceconclusion2BackgroundContour Based Detection Method

Problem of Contour Fragment:Storage requirement is large for training.Slow matching speed.Not scale invariant.Solution provided is Shape-Token.

3

System Overview4

Shape TokenWhat is Shape-Tokens?Constructing Shape-TokensDescribing Shape-TokensMatching Shape-Tokens

5What is Shape-Tokens?Use the combination of line and ellipse to represent the contour fragments.Line for line.Ellipse for curve.Example:

Why shape-tokens?Several parameters are enough for us to describe the contour fragment.

6

Constructing Shape-TokensExtract Shape Primitives of line segments and ellipses by [16] [17]. Pairing reference primitive to its neighboring primitive. Different type combination: Take ellipse as reference.Same type combination: Consider each as reference in turn.Three types of Shape-Tokens:Line-Line, Ellipse-Line, Ellipse-Ellipse.7Constructing Shape-TokensLine-LineCombine neighboring line which has any point falling in trapezium searching area.Ellipse-Line & Ellipse-EllipseCircular Search Area. Consider primitives has any point within searching area and weakly is connected to reference ellipse.

8

Ellipseneighborpointsearch areaweakly connectedmapLEM(line edge map)LEMpathweak connectivity

Indep. with orientation to avoid missing neighbors when pose of an object changes.

8Describing Shape-Tokens9

Matching Shape-TokensDissimilarity Measure (Shape Distance)10

Matching Shape-Tokens11

Codebook of Shape-TokensExtracting Shape-Tokens inside bounding boxes of training images.Producing Code-wordsClustering by ShapeClustering by Relative PositionsSelecting representative code-words into codebook for specific target object.12K-Medoid MethodSimilar to the k-means method.Procedure:Randomly select k of the n data points as medoids.Associate each data point to the closest medoid.For each medoidm For each non-medoid data pointo Swapmandoand compute the total cost of the configuration.Select the configuration with the lowest cost.Repeat the steps above until there is no change in the medoid.

13K-Medoid MethodFirst two steps14

K-Medoid MethodThird to Fourth step15

Clustering by ShapeMethod:Use k-medoid method to cluster the shape-tokens for each type separately.Repeat the step above until the dissimilarity value for each cluster is lower then a specific threshold. Metric:Dissimilarity Value: average shape distance between the medoid and its members.Threshold: 20% of the maximum of D(.).16

Clustering by relative positions1718Candidate Code-WordsExample: the Weizmann horse dataset.19

19Selecting Candidates into CodebookIntuition: Size of cluster.Problem: Lots of selected candidates belong to background clutter.What kind of candidates we prefer ?Distinctive Shape.Flexible enough to accommodate intra-class variations.Precise Location for its members.20Selecting Candidates into Codebook21Selecting Candidates into Codebook22

Selecting Candidates into CodebookExample: the Weizmann horse dataset.

23

Code-Word CombinationWhy code-word combination ?One can use a single code-word that is matched in test image to predict object location. => Less discriminative and easy to matched in background.Instead, a combination of several code-words can be more discriminative.

24Code-Word CombinationMatching a code-word combinationWay to match code-word combination.Finding all matched code-word combinations in training imagesExhaustive set of code-word combinations.Learning discriminative xCC (x-codeword combination)25Matching a Code-Word Combination 26Matching a Code-Word Combination Example:27

Finding all matched code-word combinations in training images28Finding all matched code-word combinations in training images29

Finding all matched code-word combinations in training images30Finding all matched code-word combinations in training images31

Learning Discriminative xCCWed like to obtain a xCC which satisfies the following three constraint.Shape Constraint : Highly related Code-Book EstablishmentGeometric Constraint: Object Location Agreement.Structural Constraint :Reasonable code-word combination for different poses of object.

32

Learning Discriminative xCCExample:33

Learning Discriminative xCCBinary Tree to represent a xCC.Each node is a decision statement:

34

Learning Discriminative xCCAdaBoost Training Procedure to produce one xCC from each iteration.

The Binary Tree depth k can be obtained by 3-fold cross validation.35

Learning Discriminative xCCExample:36

Learning Discriminative xCCExample:37

Learning Discriminative xCCExample:38

Hybrid Detector xMCCIncorporating SIFT as appearance information to enhance the performance.Procedure: (same as previous section)

39

Hybrid Detector xMCCExample:40

Hybrid Detector xMCCExample:41

Hybrid Detector xMCCExample:42

Experimental ResultContour only result under viewpoint change. (train on side-view only)43

Experimental ResultContour only result for discriminating similar shape object classes.44

Experimental ResultCompare with Shotton [6] on Weizmann Horse test set.

Shotton [6]: Use contour fragment, fixed number of code-words for each combination.45

Experimental ResultWeizmann Horse Test Set.46

Experimental ResultGraz-17 classes.47

Experimental ResultGraz-17 dataset.48

Experimental ResultHybrid-Method result49

ConclusionThis article provide a contour based method that exploits very simple and generic shape primitives of line segments and ellipses for image classification and object detection.Novelty:Shape-Token to reduce the time cost for matching and the need of memory storage.No restriction on the number of shape-tokens for combinations.Allow combination of different feature types.50