quantifying and transferring contextual information in object detection
DESCRIPTION
Quantifying and Transferring Contextual Information in Object Detection. Professor: S. J. Wang Student : Y. S. Wang. Outline. Background Goal Difficulties in Usage of Contextual Information Provided solutions Another method: TAS Experimental Results and Discussion - PowerPoint PPT PresentationTRANSCRIPT
1
Quantifying and Transferring Contextual Information
in Object Detection
Professor: S. J. WangStudent : Y. S. Wang
2
OutlineBackgroundGoalDifficulties in Usage of Contextual
InformationProvided solutionsAnother method: TASExperimental Results and
DiscussionConclusion and Future Direction
3
Background (I)Only the properties of target
object used in the detection task in the past.◦Problem: Intolerable number of false
positive
4
Background (I)Only the properties of target
object used in the detection task in the past.◦Problem: Intolerable number of false
positive
5
Background (II)What else??? Contextual
information!
6
GoalEstablish a model to efficiently
utilize the contextual information to boost the performance of detection accuracy.
7
Difficulties (I)Diversity of Contextual Information
◦There are may different types of context often co-existing with different degrees of relevance to the detection for the target object(s) in different images.
◦Terminology: Things (e.g. cars and people) Stuffs (e.g. roads and sky) Scene (e.g. what happen in the image)
◦Thing-Thing, Thing-Stuff, Stuff-Stuffand Scene-Thing
8
Difficulties (II)Ambiguity of Contextual
Information◦Contextual information can be
ambiguous and unreliable, thus may not always have a positive effect on object detection.
◦Ex: Crowded Scene with constant movement and occlusion among multiple objects.
9
Difficulties (III)Lack of Data for Context Learning
◦Not enough training data : Over-fitting problem Wrong degree of relevance
◦Ex: The contextual information of people on top of sofa can be more useful than people on top of grass.
10
Training Data Preparation & Notation Representation
Base Detector(HOG)
Training Image Candidate windows
Positive sample: Red
Negative sample: Green
11
Provided SolutionsA polar geometric descriptor for
contextual representation.A maximum margin context model
(MMC) for quantifying context.A context transfer learning model for
context learning with limited data.
12
Polar Geometric Descriptor
Instead of traditional annotation based descriptor, here we use polar geometric descriptor to describe two kind of contextual information (Thing-Thing, Thing-Stuff).
r :orientationb+1 :radial binsr*b+1 :patches0.5σ, σ and 2σ :bin lengthFeature :HOGPatch representation:Bag of Words method using K-means with K = 100
13
Provided SolutionsA polar geometric descriptor for
contextual representation.A maximum margin context model
(MMC) for quantifying context.A context transfer learning model for
context learning with limited data.
14
Quantifying Context (I)Risk function:
15
Quantifying Context (II)Goal = Minimize the Risk function
Minimize L equal to fulfill the following constraint
Hard to be solved, could be replaced by
16
Quantifying Context (III)Maximum Margin Context Model
Add some extra variables and constraints
17
Provided SolutionsA polar geometric descriptor for
contextual representation.A maximum margin context model
(MMC) for quantifying context.A context transfer learning model
for context learning with limited data.
18
Context Transfer LearningTwo Cases:
◦Similar contextual information Ex: Cars and motorbikes
◦Little in common in both appearance and context, but similar level of assistance provided by contextual information. Ex: People and bikes
19
TMMC-I: Transferring Discriminant Contextual Information
Similar context provide the assistance on the learning of w.
20
TMMC-I: Transferring Discriminant Contextual Information
New Constraint:
Modified optimization function:
21
TMMC-II: Transferring the Weight of Prior Detection Score
Similar level of assistance, same weight
22
TMMC-II: Transferring the Weight of Prior Detection Score
New Constraint:
Modified optimization function:
23
Another Method: TAS
24
Another Method: TAS (I)Steps:1. Segmenting image
into regions.2. Use base-detector
to get the candidate patches.
3. Establish the relationships between candidate patches and regions.
4. Use the relationships to judge there is a target object in the patch or not.
25
Another Method: TAS (II)Region clusters:
26
Another Method: TAS (III)Examples of experiment:
27
Experimental Result and Discussion
Use four data sets for testing:◦VOC 2005◦VOC 2007◦I-LIDS◦FORECOURT
28
Experimental Result and Discussion
29
Experimental Result and Discussion
30
Experimental Result and DiscussionContext Transfer Learning
Models:
31
Experimental Result and DiscussionContext Transfer Learning
Models:
32
Conclusion and Future Direction
In this paper, the author proposes a contextual information model to quantify and select useful context information to boost the detection performance.
What can we do next?◦HOG feature not suits for stuff (e.g. sky, road)◦Automatic selection between TMMC-I, TMMC-
II◦Automatic selection between target object
category and source category
33
ReferenceWei-Shi Zheng, Member, IEEE,
Shaogang Gong, and Tao Xiang, ”Quantifying and Transferring Contextual Information in Object Detection ”, PAMI accepted.
Geremy Heitz, Daphne Koller, “ Learning Spatial Context: Using Stuff to Find Things”, ECCV 2008.
Youtube Search “Hard-Margin SVM”