grounding to the real world - architecture for ubiquitous...
TRANSCRIPT
Grounding to the Real WorldGrounding to the Real World-- Architecture for Ubiquitous Architecture for Ubiquitous
Computing Computing --ISMIS 2003ISMIS 2003
Hideyuki NakashimaHideyuki Nakashima中島秀之中島秀之
Cyber Assist Research Center, AISTCyber Assist Research Center, AIST
Background
• Information technologies are advanced to the point where it becomes feasible to assist ordinary people enjoy the full capability of information processing powers.– Although the Internet opened up huge
possibilities for daily use, users still have to learn a lot of special concept to use it.
Background 2
• With the help of various sensing technologies, it is time to design arich, keyboard-less interface and provide an ubiquitous computing environment for ordinary people in everyday life. Ubiquitous agent technology plays essential role here.
IntroductionIntroduction
TICKET
Take the next train.Change to A line at Ueno station.The fare is YY yen.
I want to go to Maebashi station.
Good Old DaysGood Old Days
券売機券売機 券売機
●●●●
××××
△△△
○○○○○
▲▲▲▲
120
560
340
780
900
□□□ 210
※※※※ 400
※●※●※
×○×
△■△■
○□○□
▲×▲×▲
120
560
340
780
900
□◎□◎ 210
■※■ 400
●●●●
××××
△△△
○○○○○
▲▲▲▲
120
560
340
780
900
□□□ 210
※※※※ 400
※●※●※
×○×
△■△■
○□○□
▲×▲×▲
120
560
340
780
900
□◎□◎ 210
■※■ 400
券売機
・・・・・
When she getsolder...
When she getsolder...
券売機 券売機 券売機 券売機
●●●●
××××
△△△
○○○○○
▲▲▲▲
120
560
340
780
900
□□□ 210
※※※※ 400
※●※●※
×○×
△■△■
○□○□
▲×▲×▲
120
560
340
780
900
□◎□◎ 210
■※■ 400
●●●●
××××
△△△
○○○○○
▲▲▲▲
120
560
340
780
900
□□□ 210
※※※※ 400
※●※●※
×○×
△■△■
○□○□
▲×▲×▲
120
560
340
780
900
□◎□◎ 210
■※■ 400
How much does it cost?
What is the best route to Maebashistation?
券売機 券売機 券売機 券売機
●●●●
××××
△△△
○○○○○
▲▲▲▲
120
560
340
780
900
□□□ 210
※※※※ 400
※●※●※
×○×
△■△■
○□○□
▲×▲×▲
120
560
340
780
900
□◎□◎ 210
■※■ 400
●●●●
××××
△△△
○○○○○
▲▲▲▲
120
560
340
780
900
□□□ 210
※※※※ 400
※●※●※
×○×
△■△■
○□○□
▲×▲×▲
120
560
340
780
900
□◎□◎ 210
■※■ 400
あっ！
Please go through..
The next train will leave from platformNo 2 to your right.
TURN RIGHT TURN RIGHT
speaker
Connection to Internet
Time table server
Seat reservation system
My-button
LED display
Station master computer
Station LAN
TokyoRF tag
Unawareable environmentUnawareable environmentIR communicator
Position sensor
Ubiquitous Computing• Omnipresence of computing power
• Human centered services– The opposite direction from virtual reality where the
reality is reconstructed in the digital world
– Computers come out to the real world
• Gadgets are hidden– calm computing– disappearing computer
– oxygen
– unawarable computers
MIT Oxygen: Intelligent Space• Context aware environment: computation “in the air”
Stereo Cameras: to track persons and objects
Stereo Cameras: to track persons and objects
Active Cameras: for video-conferencing and meeting archive support
Active Cameras: for video-conferencing and meeting archive support
Microphone Arrays: beam-forming for untethered spoken interface
Microphone Arrays: beam-forming for untethered spoken interface
Before:People had to learn to use machines
Computers in the Digital WorldComputers in the Digital World
People in the Real WorldPeople in the Real World
Internet schoolInternet schoolPC schoolPC school
After:Grounding is the key technology
Digital WorldDigital World
Real WorldReal World
groundinggrounding
“Grounding”
• Connection between digital data and real things– similar to “symbol grounding problem”
– (but not the same)
• Total grounding is not necessary nor possible– utilization of artificial means or human aid
From WWW to HNM
• Wherever, Whenever, Whoever
→ Here, Now, Me (situatedness)
• At the same time:
Just as much as you want to know
→ intelligent information filter
• And,
Just as much as you want to be known
→ privacy
Location based communicationLocation based communication
Human Centered Infrastructure
ID-based communication
ID-based communication
Location-based communication
Location-based communication
Broadcasting satellite３６０００ｋｍ～７８０ｋｍ
Broadcasting satellite３６０００ｋｍ～７８０ｋｍ
High-altitude aeroplane/ship２０ｋｍ
High-altitude aeroplane/ship２０ｋｍ
broadcastbroadcast
broadcastbroadcast
Sensor networkSensor network
Communication Methods
Local 1-10 meters global
Magnetic cardMagnetic cardICIC--cardcard
NetworksNetworksPhonesPhones
BroadbandHigh-speed
LowLow--powerpowerShortShort--rangerange
LocationLocation--basedbased
IDID--basedbased
Advantages of Location Based Communication
<x, y, z, t>
• Location is one of the essential keys for grounding the digital (logical) world to the real world.– Situated information processing and services
• Interaction without ID– Possibility of PRIVACY protection
– For better human interface
– Application to emergency situations
Various location-based communication(addressing) methods
(cricket)*(mobile phones)*
(cricket)*(mobile phones)* carrier pigeoncarrier pigeon
Logical addressing“passenger at 12C”
Positioning by server
Self-Positioning
Physical addressing“you”
(bluetooth)*DSRC
(bluetooth)*DSRC CoBITCoBIT
*positioning only
Security vs. Privacy vs. Convenience
securitysecurity privacyprivacy
convenienceconvenienceThey basically contradict each other with some exceptions
Needs technology for privacy protection
amountofservice
privacy protection
not zero
Dynamic/Voluntary Control of the Balancing Point is Necessary
• Trade-off between the amount of services and the amount of personal information
• Let the user set the point!
Privacy Security
AgentAgent--based Network based Network ArchitectureArchitecture
Ubiquitous Computing
How do we access and control devices
Intuitive User Interfacethat uses real world propertiessuch as location(point and click)
Why Agents?
• Aids for human cognition– interface
• Resource control– cognitive resources
– devices• multiple users and• multiple devices
Cognitive Resource Control
• SmartKom example:– cognitive resource of human
• when running terse• when walking• when standing verbose
・Maintains who reserved which time slot・Controls remote access・The reserver has the access right
homeVCR
toilet
bathliving
air conditionerthermometer
thermometer
・The access right belongs to who reserved・Access right can be delegated・The device may act as a mediator
homeserver
RFID tagRFID tab
RFID tag
Son
父
×
×
○
！
town
OK Dad, go ahead...
Access Control of Home AppliancesBased on Location and Priority
My son has a reservation. Let’s ask him.
someone is using the air conditioner・change reservation
or, delegate authority
・Deny remote access when someone is near-by
・communication center for the family
CONSORTS
InferenceEngines
Personal Agents
ServiceAgentsUser1 Usern
Communication and censoring
devices
Physical WorldContentServer
Devicewrapper
Agent communicationAPIphysical link
Spatial information database- sensor-based geometrical representation- ontology- location information
Spatio-Temporal Reasoning Agent
Camera, Wireless LANCoBIT, etc.
User modelmanager
Physical Grounding
Web Services
Spatio-temporal Inference Agent in Consorts
spatio-temporal map(Segment Representation)
Inference engine:spatio-temporal relations and their changes (distance, phase, connection etc.)
device wrapper agents
personal agents,service agents
request, reply, inform
time
sensor data
Spatial informationdatabase
interpretation
Sensor information at location (x,y) mapped onto a map
Geometric representation of sensor information
Museum
1st Floor
Entrance
3rd Floor2nd Floor
Room CRoom A Room B
Painting BPainting A Painting C
Hierarchical representation of physical space
Painting D Painting D
Ubiquitous Computing with FIPA-Agents
• Direct connections to sensors are not good
• Agents mediate communication of sensor information
• Agentcities Network• http://www.agentcities.net/
– EU centered worldwide research network based on FIPA standard
– 134 platforms• EPFL, BT, Motorola， etc.
– Tokyo.Agentcities.Net (CARC)• http://www.carc.aist.go.jp/agentcities/
A Snapshot of Our Museum Information Support System
Main Window (Museum Map)
Information Window
Message WindowMessage Window
Human Icon: Current Location
Yellow Area: Service Zones
Blue Line: Users’ Trajectories
UBKit
UBKit: Software construction tool kits
middleware
infrared
SOAP DNS
network
IEEE802.11 BluetoothMicroservers
voice positioning
real world
CONSORTSCONSORTS
Micro servers + Optical Communication forIntegrated Remote Control System
Ultra small low energy micro server(Linux) coin size
Self configuring plug and play network
radio
positioning system
voice recognition
sensor networkinfrared
Lightweight middleware (service directory, SOAP)
sensorgateway
UBKit (Ubiquity Building Toolkit)
Integrated control of all home applianceswith intuitive interface
Connection toConnection tothe Semantic Webthe Semantic Web
Annotation as Grounding
Content(Real World)
SemanticWeb
AnnotationAnnotation
Semantics forComputers
Sharing semanticsbetween computers
and human(Grounding)
Multimodal Content Processing
• Grounding linguistic content to the physical world– annotation with location– interaction through CoBIT
• User modeling– record of movement– record of inquiries
• Summarization and interactive presentation
Cyber Assist ProjectCyber Assist Project
The goal:Grounding the digital world to the real worldfor “here, now, and to me” service
informationinformation
Digital worldDigital world
Real world
goodsgoods
humanhumanSemantic WebSemantic Web
Ubiquitous ComputingUbiquitous Computing
My Button• One button
– Do what I want here, now.
– No button?Activation by
brain wave?
• The ultimate man-machine interface？– Intelligent
personal agent– Sharing
situations
command button
fingerprint recognizer
reflectors photo detector
microphone
speaker
Optical Bi-directional Communication
server
cameraLED
terminal
•Download•voice
•power supply
•Upload•position•direction•motion•(ID)
My Button V1: CoBIT
• Compact Battery-less Information Terminal
• Terminal: solar cell + reflector + earphone
青森産のリンゴです。一個200円です。これ買いますか？
ありがとうございます。
アラスカ産のシャケです。
これを買いますか？
となりにはサンマがあります。
Application to Markets
Environment
The Image of the Target System
User
device
Intelligent Content
Sensors and actuators
Situationrecognition
Query/orders
information
request
Information retrievalInformation processing
Voicerecognition
Semantic query
User modeling
Photo/radio communication
vision
Multiagent technology
Application toEmergency rescue
Tokyo
ＷｉｒｅｌｅｓｓＬＡＮ
Ｍｙ-Ｂｕｔｔｏｎ
i-lidar
camera
intelligent contentsemantic web
voicecommand/query
voiceinformation
ＩＲ
The next train will leave from platformNo 2 to your right.
ＣｏＢＩＴ
The evolution of the communication technologies in Star Trek
CoBITmy buttons
conventional
ApplicationsApplications
Mass Support
• Information support for the whole city
• Examples– Global Car Navigation
• Optimal route planning using the current positions and destinations of all cars in the area
• Coordinated signal control
– Theme Park total coordination
– On-line Government• Direct democracy with the help of discussion and
decision support system
Special project for earthquake disaster mitigation in urban areas(Ministry of Education and Science)
• Ad hoc network for emergency communications
• Multiagent simulation for– evacuation
– rescue
Real-time dynamic ad-hoc network for large-scale disasters (earthquake)– Effective in
congested areas
– Alternative to other communication infrastructures• First 10-30 min.
– Connection to other lines when possible
Grounding the Simulation
KernelGIS
Agents
Viewers
ComponentSimulators
Portable devices Robots
Rescue team
Real-worldsensors
Rescue HQ
Summary
• We have to connect the digital world to the real world→ GROUNDING
• Multiagent concept is the key to ubiquitous computing– Grounding
– The core architecture
– Ubiquitous agents for human support– Many MA applications