toward real-time extraction of pedestrian contexts with stereo camera kei suzuki, kazunori takashio,...
TRANSCRIPT
Toward Real-Time Extraction of Pedestrian Contexts with Stereo Camera
Kei Suzuki, Kazunori Takashio, Hideyuki Tokuda,
Masaki Wada, Yusuke Matsuki, Kazunori Umeda
Graduate School of Media and Governance, Keio University
Department of Precision Mechanics, Faculty of Science and Engineering, Chuo University
Fifth International Conference on Networked Sensing SystemsJune 17 - 19, 2008 // Kanazawa, Japan
Introduction
• Vision-based monitoring systems are becoming important
• Many researches of pedestrian recognition from vision data have been conducted.– Pedestrian Detection, Tracking, Activities
Security camera
Motivation
• Current works are difficult for extracting pedestrian’s high-level contexts.
Pedestrian Detection, Tracking, Activity
Pedestrian high-levelContexts
How suspicious is this situation?
SnatchSkirmish
Goals
• Extract the high-level contexts of pedestrians from vision data in real-time– We aim to extract suspicious individuals, groups
and uneasy atmosphere based on the position, velocity, width, and height of those individuals.
– We developed a prototype system that can find people tumbling, walking in a group.
At a street corner system image
Our Approach
• Use of a stereo camera system that focuses on a moving region– We can make real-time 3D measurement of more
than one pedestrian’s region• center of gravity coordinates, height, width, label number
– Easy to install in a new place
• Infer the pedestrian contexts by using Bayesian Networks– We model pedestrian movement as time-series data– We make Bayesian models and let them learn for
each context
The target contexts table
individual
group
Atmosphereof the place
Prototype system
Not yet
walking in a group
tumble
Project target
WalkingRunningTumble
SnatchSkirmish
CrowdedQuiet
System Overview
• Hardware architecture
At a street corner
Stereo cameraAnalyze vision data
network
Stereo camera system Pedestrian context Infer system
Display result of contexts
Infer the context
System Overview
• The flow of inferring contexts
e.g.A: velocity vector similarityB: the average distanceC: the average vector angle
The thread of event detection
The inference thread using theBayesian Networks
sliding window
Input variables into Bayesian Model
Stereo camera system data
Experiment . Extract contexts with real-data
• Extract two pedestrian contexts– Tumble as a individual context– Walking with friends as a group
context
• Result– Show the effectiveness of extracting
two or more contexts in real-time.– The required time of extracting the
contexts was 58 msec, but it worked in real-time due to the event trigger model.
Conclusion / Future Work
• We extracted the pedestrian contexts with stereo camera system
• We developed the prototype system, and confirmed by the experiments.
• Future work– Further evaluation of accuracy, and compare
its performance with some other methods.
• Thank you!
Experiment1 . Stereo camera’s output test
• Walk 6.5m from the side of a camera, then turn right.
-1
0
1
2
3
4
5
6
7
1 4 7 1013 1619 2225 2831 3437 4043 4649 5255 5861 6467 70
xyz
Stereo camera’s frame:14 frame/sec
Camera
Dis
tanc
e fr
om c
amer
a (m
)
Time (sec)
Tur
n rig
ht
Bayesian Network Model
• The sample of tumble model
Pedestrian Recognition at a street corner
• Many projects of pedestrian recognition from video data have been conducted.– Pedestrian Detection– Pedestrian Tracking– Pedestrian’s activities
Security camera
Characteristics of Pedestrian
• The activities of Individuals– walking, running, tumbling
• The activities of Groups, mobs– Harmless group
• Companion , with the same intention
– Contingent group• Moving to same direction, temporary crowded with people
– Suspicious group• Snatchers, fighting, entering no admittance area
• Atmospheres of a place– Crowded, quiet
Pedestrians have 3 characteristics based on their movement.
Goals
• Extracting from suspicious individuals and uneasy atmosphere by using video cameras – We aim to extract high-level pedestrian
contexts in real-time– The system infers contexts based on
pedestrians’ moving region data
Stereo Camera System that focusing on moving regions
• Calculate Moving region feature– Center of gravity coordinate– Distances from camera– Height and width– Timestamp– Label number
Project abstract (1/2)
• Extracting from suspicious individuals and uneasy atmosphere by using stereo cameras – We aim to extract high-level pedestrian contexts in
real-time– The system infers contexts based on pedestrians’
moving region data from stereo camera
At a street corner
Project abstract (2/2)• Recognizing group and mobs
– harmless crowd• Companion group
– Accidental mobs• spectators
– Suspicious mobs target⇒• snatchers, fighting mobs
Bayesian Network
• Extracting of high-level pedestrian contexts using Bayesian Network.
System Over View
At the street
Stereo cameraVideo data analyze
network
Stereo camera system Pedestrian context Infer system
Display result of contexts
Contextinfer
Hardware architecture
Pedestrians at street corner
Stereo cameraProcessing videodata
network
Stereo camera system Inferring Pedestrian Contexts in real-time
Contexts inferred result
Infering contexts
Prototype System’s target Contexts
target
individuals
group
AtmosphereOf a place
Prototype system’s target contexts
Not yet
Walking with friends
tumble
Project target contexts
Normal walkingRunningTumble
Snatchers,Fighting
CrowdingQuiet
実験 2 .コンテクスト推定部の負荷実験
• コンテクスト推定の最大負荷時
コンテクスト同時推定数 平均推定時間 CPU使用率
2 40msec 100%4 160msec 100%