의미 모델링 elaborating sensor data using temporal and spatial commonsense reasoning + mining...
TRANSCRIPT
의미 모델링
Elaborating Sensor Data using Temporal and Spatial Commonsense Reasoning
+Mining Models of Human Activities from the Web
지능 기반 시스템 응용2006. 11. 민준기
2
Agenda
B. Morgan and P. Singh, “Elaborating Sensor Data using Temporal and Spatial Commonsense Reasoning,” BSN 2006.
The Problem Space
LifeNet : A First-Person Model
The Plug Sensor Network
M. Perkowitz, et al., “Mining Models of Human Activities from the Web,” WWW 2004.
Introduction
Proposed Technique
Evaluation
Summary and Future Work
3
The Problem Space
Two distinct directions for researchHuman-out (This paper)
Telephone
Technology-in (Much sensor network research)Text messaging on cell phones
Three topicsLifeNet probabilistic human modelThe Plug sensor networkAn experimental design for evaluation of the LifeNet learning method
4
LifeNet : A First-Person Model
First-person common-sense inference modelOpenMind Common Sense, ConceptNet, The PlaceLab data, Honda’s indoor common sense data
Attempts to anticipate and predict what humans do in the world
All of the reasoning in LifeNet is based on probabilistic propositional logic
“I am washing my hair” before “my hair is clean”
5
The Plug Sensor Network
Using for both learning common sense and for recognizing and predicting human behavior
Using this sensor network to monitor how individuals interact with their physical environment
Nine sensor modalities: sound, vibration, brightness, current, wall voltage, acceleration
6
Agenda
B. Morgan and P. Singh, “Elaborating Sensor Data using Temporal and Spatial Commonsense Reasoning,” BSN 2006.
The Problem Space
LifeNet : A First-Person Model
The Plug Sensor Network
M. Perkowitz, et al., “Mining Models of Human Activities from the Web,” WWW 2004.
Introduction
Proposed Technique
Evaluation
Summary and Future Work
7
Introduction : Recognize Humans Activities
Applications include activity-based actuationDimming lights when a video is being watchedProviding directions for someone using unfamiliar facilitiesetc.
Ubiquitous, proactive, disappearing computingComputers have to understand people’s needs by observing their physical activities (and to act autonomously)The cost of developing recognition infrastructure is too high
Even small classes of activities is hard to recognize
A broadly applicable system should be general-purpose and easy to use
8
Motivation
Vision based systemsNone have reported detecting more than tens of activities in practiceThe features robustly detectable from vision are coarse
Represent the relationships between “blobs” in the image rather than specific objectsEach activity is expensive to model
Learning of the modelsThe developers define the structure of the possible models
System tunes the parameters of the model based on examples from the user
The user is expected to label the patternsThe variety of activities is quite restricted
Introduction
9
Proposed Technique
RFID (Radio Frequency Identification)Cheap: Postage-stamp sized, forty-centWireless and battery free
Activity modelingDefine an activity in terms of the probability and sequence of the objectsGenerate the models by translating textual definitions
Structured like recipes
Produced automatically by mining appropriate web sitesMining models is part of a larger activity recognition system, PROACT (Proactive Activity Toolkit)
10
Usage Model
Assumes that interesting objects in the environment contain RFID tags (tens ~ hundreds)
Making a database entry mapping the tag ID to a name
Within a few years, many household objects may be RFID-tagged before purchase, thus eliminating the overhead of tagging
Medium-range readers (Tag-detecting Gloves) andLong-range readers (Run robots, Carts, …)
PROACT uses the sequence and timing of object to deduce what activity is happening
Likelihood of various activities, details of those activities, degree of certainty, etc…
Proposed Technique
11
System Overview
PROACT provides an activity viewer for debuggingReal-time view of activities in progressThe sensor data seenChanging of belief in each activity with the data
Inference Engine converts the activity models produced by the mining engine into Dynamic Bayesian Networks
D. Patterson, L. Liao, D. Fox, H. Kautz, “Inferring High-Level Behavior from Low-Level Sensors,” Ubicomp 2003.
Proposed Technique
12
Sensors and Models
SensorsUse two different kinds of RFID readers
Long-range reader (mobile robot): map the location of objectsShort-range reader (glove): determine the objects that are
touched
ModelsEach model (activity) is composed of a sequence (step) s1 ~snEach step si has optional duration ti and object oij involved along with the probability pij
Proposed Technique
13
The Model Extractor
Builds formal models of activities using directions
Directions are written in natural language by humanHow-to (ehow.com), recipes (epicurious.com), training manuals, protocols, etc.
Syntactic structure of directions1. A title t for the activity2. A textual list r1~rm, Each step ri has:
Possibly a special keyword delimiting duration diWhat to do during the step: subset of the objects and duration
Proposed Technique
14
Converting Directions to Activity Models
Key steps1. Labeling
Set label of the mined model to title of the directions
2. Parsing stepsDuration: Gaussian with mean = d, stdev = S(d, i, l ) Object Oi and Probability P
3. Tagged object filtering
FunctionsObject
Object extraction: WordNet ontologyNoun-phrase extraction: QTag tagger
ProbabilityFixed probabilitiesGoogle conditional probabilities (GCP)
Proposed Technique
For example,[“making tea”] has 24,200 matches, and[“making tea” cup] has 7,340 matches, thenconditional probability of a cup being involved inmaking tea is 7340/2400 = 0.3
16
Evaluation
Mined modelsehow.com: 2300 directionsffts.com: 400 recipes epicurious.com: 18,600 recipes
Three strategies to approximate comprehensive evaluationHuman activity-trace recognition
Activities of Daily Living (ADLs)
Inter-corpus consistencyMaking cookies recipes
Intra-corpus distinguish-abilityDistinguish-ability within activity domains
18
Human and inter-corpus trace recognition
ADLs domainMany objects were not tagged, missed, and interleavedModels were not perfect
Cookie domainThe identical recipe can have quite different structureFor some of the recipes, there is no counterpart in the other corpus
Evaluation
19
Impact of techniques on accuracy
ADLsDomain is fairly sparse, with many activities involving only few object
Cookie domainEach activity model involves many more objects
Evaluation