Download - 11-4-05 Kishore Ramachandran - Dfuse
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
1/56
DFuse and MediaBroker:System support for sensor-based
distributed computing
Kishore Ramachandran
(http://www.cc.gatech.edu/~rama)
College of Computing
Georgia Tech
Colleagues:Rajnish Kumar, Bikash Agarwalla,Junsuk Shin,
David Hilley, Dave Lillethun, Jin Nakazawa, Bin Liu,Xiang Song, Nova Ahmed, Seth Horrigan,
Matt Wolenetz, Arnab Paul, Sameer Adhikari,Ilya Bagrak, Martin Modahl, Phil Hutto
http://www.cc.gatech.edu/~ramahttp://www.cc.gatech.edu/~rama -
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
2/56
Computing/Communication Continuum
Sensor
Network
High connectivity Low connectivity / Wireless
Cameras, sensor nodes High Performance Computing (HPC) resources
HPC
resources
Ambient Computing Infrastructure
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
3/56
What does this enable?
Application context distributed sensors with varying capabilities
control loop involving sensors, actuators
rapid response time at computational perception speeds
Sample applications Video based surveillance
Transportation
Emergency Response
Collaborative search and rescue Evacuation management
Aware Environments
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
4/56
Application Characteristics
Physically distributed heterogeneous devices Interfacing and integrating with the physical
environment
Diverse stream types (low to high BW)
Diverse computation, communication and powercapabilities (from embedded sensors to clusters)
Stream fusion/transformation, with loadable code
Resource scarcities
Dynamicjoin/leave of application components
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
5/56
Key Requirements
Middleware Programming infrastructure
Distributed data fusion
Stream data management
Network Level Protocol stack
Energy efficient routing
Ambient HPC resources Grid
Stampede
DFuseMediaBroker
SensorStack
Streamline
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
6/56
Stampede (TPDS 2003)
Distributed programming system coveringthe continuum Temporal stream data transport Multilingual (C, C++, Java) program
components sharing data abstractions Multiple platforms (x86-Linux, ARM-Linux,
x86-Windows, x86-Solaris, Alpha-Tru64)
Channel
thread
many to many connections
time sequenced data
correlation of streams
automatic GC
put(ts, item)
get(ts, item)
consume(ts)
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
7/56
Key Requirements
Middleware Programming infrastructure
Distributed data fusion
Stream data management
Network Level Protocol stack
Energy efficient routing
Ambient HPC resources Grid
Stampede
DFuseMediaBroker
SensorStack
Streamline
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
8/56
DFuse (ACM SenSys2003)
Future Sensor Networks Todays handhelds, gateways Ubiquitous high-bw sensors
Challenges Overlaying the application onto the physical network
Programming abstraction for data fusion
Fusion Channel(a Virtual Sensor)
Producers
(sensors orother fusionchannels)
Consumers
(actuators orother fusionchannels)
.
..
.
..f()
SinkFilter
Collage
Cameras
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
9/56
Fusion Module
Placement Module
SinkFilter
Collage
Cameras
Fusion( Arg[ ]){
..
}
Resource Monitor,
Routing Layer Interface
Operating System
Hardware
Fusion Module
Placement Module
Resource Monitor,
Routing Layer Interface
Operating System
Hardware
Cost
Function
DFuse functions:1. Placement of fusion and relay points2. Plumbing as required
3. Dynamic migration of fusion points
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
10/56
Status of DFuse
Fusion and placement modules implementedon top of Stampede A prototype iPAQ farm (simulating future
sensor network) runs DFuse
Stampede and DFuse available for downloads MSSN, a simulator for sensor networks
middleware design guidance (BaseNets04,IJNM05, Wolenetzs thesis) Available for downloads
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
11/56
Key Requirements
Middleware Programming infrastructure
Distributed data fusion
Stream data management
Network Level Protocol stack
Energy efficient routing
Ambient HPC resources Grid
Stampede
DFuseMediaBroker
SensorStack
Streamline
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
12/56
MediaBroker
An architecture for stream management
A clearing house for sensors and actuators in agiven space
Stream registry, discovery, plumbing, sharing,
Dynamic connection of sources (producers) and
sinks (consumers) Dynamic injection, and safe execution of
transformation codeFeature extraction, fusion
Dynamic sharing of transformations andstreams
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
13/56
Elements Type server: stores data types, relationships, and
transformation code Transformation engine: allow safe execution of injected
code on cluster nodes
Scheduler: manages workload, and allows prioritizingtransformation requests
Data brokers: manages connections between producers andconsumers
Type Server
DataBroker
DataBroker
Scheduler
TransformationEngine
TransformationEngine
TransformationEngine
ProducerProducer Consumer Consumer
Data ItemsTransformation RequestsTransformation Code
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
14/56
What it enables
Dynamic instantiations and sharing oftransformations
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
15/56
MediaBroker Status
MediaBroker V.1 A subset of the functionalities
Application example: Family intercom
IEEE PerCom 2004, PMC 2005
MediaBroker++ Currently under development
EventWeb built on top
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
16/56
Key Requirements
Middleware Programming infrastructure
Distributed data fusion
Stream data management
Network Level Protocol stack
Energy efficient routing
Ambient HPC resources Grid
Stampede
DFuseMediaBroker
SensorStack
Streamline
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
17/56
SensorStack
Adaptability Vs. Stackability of protocollayers Adaptability deals with cross-layer data
A must for wireless sensor networks
Stackability deals with cross-layerfunctionalitiesA must for modular design
Principle behind SensorStack
Decouple data from functionalities
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
18/56
SensorStack withoutCross-layering Support
AODV routingFlood routing
MAC
Application
Time SyncService
Fusion layer
Fusion datatransmissionrequirement
Data periodicity,
Size, delay tolerance
NeighborhoodInformation,Topology
Link qualityinformation,Neighborhood statuschange, topology
Datatransmissionrequirement
Time synchronization accuracy,accuracy requirement
Fusion requirement
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
19/56
Information Exchange Service (IES)
Design Goals:
1. Efficient use of limited memory
2. Simplifying information sharing interface
3. Extensibility
4. Asynchronous delivery of the information
5. Complex event notification
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
20/56
IES Design
Data management module Stackability by using standard data interface
Publish/subscribe based shared memory system
Fully-associative cache for performance
Event management module Adaptability by notifying when to adapt
Complex event notification
Reactive memory access
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
21/56
DatePublisher
DataSubscriber
Event
Subscriber
Publisher
List
Subscriber
List
Shared
Memory
CacheDRE DAE
EMMRSE
Set rules
getput
DRE: Data request event RSE: Rule satisfied eventDAE: Data available event EMM: Event management module
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
22/56
SensorStack withCross-layering support
Application
Data Fusion Layer
Data Service Layer
Medium Access Layer
InformationExchangeService
HelperServiceLayer
Radio
Application Logic\
In-stack fusion
Next-hop selection,Logical naming, Packet
scatter/gather
Medium Access, ErrorControl, Radio Control
Attribute-Value publish/
subscribe
Locali-zation,
Synchro-nization
Service
Connection
(A) Stack Lay-out (B) Functionalities
Implemented in TinyOS and iPAQ LinuxInitial results (increasing application lifetime) very promising
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
23/56
Application Lifetime Improvementwith Cross-layer Information
DFuse performance withoutCross-layer information
DFuse performance withCross-layer information forrole-migration
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
24/56
Key Requirements
Middleware Programming infrastructure
Distributed data fusion
Stream data management
Network Level Protocol stack
Energy efficient routing
Ambient HPC resources Grid
Stampede
DFuseMediaBroker
SensorStack
Streamline
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
25/56
C
V
CCameraP/T/Z
Recorder
CCamera Humidity
Pressure
Proximity
Cluster
Cluster
Cluster
Grid Infrastructure
Gateway
Gateway
Gateway
Gateway
MediaBroker
DFuse
Stampede
SensorStack
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
26/56
Streamline (MMNC 2006)
Scheduling problem
Input: Computation and communication requirements of various
stages of a coarse-grain dataflow graph
Application-specified constraints
Current resource (processing and bandwidth) availability Resource specific constraints
Output: Placement of the stages of the pipeline on available HPC
resources Performance criteria:
latency and throughput of the application
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
27/56
Streamline Scheduler
Expects to maximize throughput by assigning best
resource to most needy stage Additional policies concerning resources,
applications, and local schedulers can beincorporated in the cost of a particular assignment
ApplicationPolicies
S0
S1
S2 S3
Resource Filtering
Resource Selection
Stage Prioritization
{S2 S0 S1 S3}
Resource
Policies{S2{R0R2R3}}
{S2R0}
R0
R1
R2
R3
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
28/56
Streamline System Architecture
Results Outperforms condor
by an order ofmagnitude for bothcompute andcommunication bound
kernels,particularlyunder non-uniform loadconditions
Performs close toSimulated Annealing
but at considerablylow scheduling time (bya factor of 1000)
DataSources
ResourceInformation
Service
QoS
MonitoringService
ApplicationInformation
Service
GridBoundary
Streaming Application
HPC
Resource
GRAM
HPCResource
GRAM
HPC
Resource
GRAM
StreamingAppication
Application
Policies
SchedulingService
AuthenticationService
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
29/56
Streaming Grid
Streamline schedulerintegrated into Globustoolkit
Example of a mock trafficmonitoring app as aservice composition usingWeb Services
Blue boxes are theStreaming Grid services
D
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
30/56
Demo
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
31/56
Demo Output (live if stars arealigned!)
http://www.cc.gatech.edu/~bikash/sgrid/trafficapp/http://www.cc.gatech.edu/~bikash/sgrid/trafficapp/
trafficapp.html
What is the takeaway? Several technologies working together Service composition, Streamline scheduling, Web
Services, Globus toolkit to process the video stream andshow the output in the browser Streaming grid instantiates, connect, manages the
streaming app
http://www.cc.gatech.edu/~bikash/sgrid/trafficapp/http://www.cc.gatech.edu/~bikash/sgrid/trafficapp/trafficapp.htmlhttp://www.cc.gatech.edu/~bikash/sgrid/trafficapp/trafficapp.htmlhttp://www.cc.gatech.edu/~bikash/sgrid/trafficapp/trafficapp.htmlhttp://www.cc.gatech.edu/~bikash/sgrid/trafficapp/trafficapp.htmlhttp://www.cc.gatech.edu/~bikash/sgrid/trafficapp/ -
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
32/56
Computing/Communication Continuum
Sensor
Network
High connectivity Low connectivity / Wireless
Cameras, sensor nodes High Performance Computing (HPC) resources
HPCresources
Ambient Computing Infrastructure
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
33/56
Conclusions
MediaBroker stream transformation and typed transport
engine
DFuse data fusion architecture
Stampede distributed programming environment
SensorStack Information Exchange Service for cross-
layer support
Streamline Scheduling support for streaming apps on
grid
MediaBroker
DFuse
Stampede
SensorStack
Streamline
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
34/56
Ongoing Work
Programming tools Adaptive resource management
Marrying grid computing and ubiquitouscomputing
Wireless networking considerations sensorstack
energy efficient protocols
mobility considerations
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
35/56
Web Links
http://www.cc.gatech.edu/~rama http://www.cc.gatech.edu/~rama/stampede
http://www.cc.gatech.edu/~rama/up
http://www.cercs.gatech.edu/
http://www.cc.gatech.edu/~ramahttp://www.cc.gatech.edu/~rama/stampedehttp://www.cc.gatech.edu/~rama/uphttp://www.cercs.gatech.edu/http://www.cercs.gatech.edu/http://www.cc.gatech.edu/~rama/uphttp://www.cc.gatech.edu/~rama/stampedehttp://www.cc.gatech.edu/~rama -
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
36/56
Applause!!!
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
37/56
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
38/56
Sensor Networks
Current Sensor Network Nodes
Limited capabilities (Mica-2) habitat monitoring, vineyard
monitoring
Recent trends iMotes
8x radio, 4x CPU, power++
Telos motes>3x radio, similar CPU & power
Future Sensor Network Nodes Todays handhelds, gateways Ubiquitous high-bw sensors Source: CACM #47-6 The platforms enabling wireless
sensor networks, Hill, Horton, Kling, Krishnamurthy, 2004.
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
39/56
HPC resources
Unix / Linux / XP cluster
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
40/56
Fusion Applications : need hierarchical fusion support
Overlaying the fusion graph
collage
filter
uncompress
F il I t
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
41/56
Family Intercom Clients connected via
MediaBroker
Type attributes includeaudio rate and bufferspecs
A client trackingsystem used for I-combo selection
Colocated clients canperform mixing forN-way conferencing
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
42/56
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
43/56
Fusion Module
Structure Management
Producers Consumers
.
..
.
..
Plumbing Issues
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
44/56
Fusion Module
Computation Management
Dynamic embedding of user-specified fusion function
Correlation and aggregation of input streams
Fusion channel migration
.
..
.
..f()f1()
f2()
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
45/56
Fusion Module
Computation Management
Dynamic embedding of user-specified fusion function
Correlation and aggregation of input streams
Fusion channel migration
Memory Management
.
..
.
..f()f1()
f2()
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
46/56
Fusion Module
Error and Status management
.
..
.
..f()f1()
f2()
Failure / Latency hiding
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
47/56
Placement Module
User inputs the task graph
Sink (Display)Filter
Collage
Sources
S1
S2
S3
And, a cost function.
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
48/56
Simple Solution ?
Why not push the fusion points towards itssources ?
Data sources may be lying all around
Fusion points may cause data expansion
Network is dynamic
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
49/56
DFuse: Placement Modules Algorithm
Three phases:1. Nave role assignment
Deploy task graph into the network
Start app at a designated root node and delegate taskgraph subtree roles to richest neighbors recursively
2.
Optimization Given anticipated application behavior (attributes in thetask graph), perform rapid local decisions to adjust whichnode performs which role.
Decisions guided by application-provided cost-function
3. Maintenance Monitor actual application behavior and perform less
frequent optimizations given application-provided cost-function
E l
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
50/56
Example Cost Function
MT1 (Minimize Transmission Cost-1)
Source
Source
f() Sink
2 Kbps
2 Kbps
2 Kbps
2 Kbps
1 Kbps
n1 n2
CMT1 ( n1, f ) = 6 kbps
CMT1 ( n2, f ) = 9 kbps
DF F
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
51/56
DFuse: Cost Functions MT1 (Minimize Transmission Cost 1)
Used in intuitive illustration slides minput data sources (fan-in) noutput data consumers (fan-out)
MPV (Minimize Power Variance) Attempts to keep the power of
network nodes at similar levels.
MTP (Minimize Ratio of TransmissionCost to Power) Attempts to keep the time for how
long nodes can run a fusion functionsimilar.
MT2 (Minimize Transmission Cost 2) Like MT1, but allows role transfer
when node power is below athreshold.
c(k,f)= cost of node kto perform role ft(x)= transmission rate of data source xhopCount(i,k)= network hops between nodes iand k
power(k)= remaining power at node k
P DF l
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
52/56
Prototype DFuse Implementation
Goal Investigate utility of Fusion Point Migration
Hypothesis Migration will increase application lifetime, for constant
QoS
Implementation Fusion Module: ARM Stampede port Simulated placement module Interface for coupling, transmission monitoring
Simple camera sources, fusion functions and sink
Collage
Filter
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
53/56
Experimental Setup 12 iPAQ 3870s in a familiar 0.6.1 Linux basedD-
Stampede 802.11b farm. Only directly adjacent iPAQsin the figure below are considered mutually reachable in1 hop.
Placement module run as a simulation of the distributedalgorithm coupled to the farm via an extended fusionmodule interface
Power usage is modeled to be linear to the number ofapplication-level bytes transmitted across a farm hop(simple model)
Prototype DFuse Results:
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
54/56
rototyp DFus su tsTransmission Cost over Time
Prototype DFuse Results:
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
55/56
yp FVariance, Role Transfers, Lifetime
MPV vs MT2
4x lessvariance
Migrations++
70% lifetime
MTP
good variance
good lifetime
h d k
-
8/2/2019 11-4-05 Kishore Ramachandran - Dfuse
56/56
Pervasive Computing with MediaBroker
Family Intercom and Sign Post Being developed at Georgia Techs Aware Home
Replacing a legacy hardware mixing system with a much morescalable system
Integration with existing RFID and Vision based tracking
system called Sign Post to enable rapid call dispatching andmobility-aware communication