swarms, coalitional games, trust, and network topologies john s. baras institute for systems...

Swarms, Coalitional Games, Trust, Swarms, Coalitional Games, Trust, and Network Topologies and Network Topologies

John S. Baras Institute for Systems Research,

Department of Electrical and Computer Engineering Department of Computer Science,

Fischell Department of Bioengineering University of Maryland College Park

June 3, 20093rd Workshop on Swarming in Natural and Engineered Systems

Block IslandCopyright © John S. Baras 2008

OutlineOutline

• MRF and Autonomous Swarms

• Collaboration in Networked Systems:

Constrained Coalitional Network Games

• Networks and Trust

• Trust and Distributed Estimation

• Topology Matters

• Conclusions and Future Directions

Copyright © John S. Baras 2008 2

3

• Scenario -- Problem– Autonomous, distributed maneuvering of a vehicle (robots) group to

reach and cover a target area

• Constraints– Desired inter-platform distance– Obstacles avoidance– Geographical constraints (walls)– Threats (stationary or moving) avoidance

• Requirement – Using only local, dynamically changing

information

Autonomous collaborative Autonomous collaborative swarmsswarms

4

Collaborative swarms – Collaborative swarms – potentials in nature … potentials in nature …

In winter afternoons, thousands of birds, attracted by the warmth of the Refinery, start a dance that lasts until dusk. They arrive, converge, take warmth, and fly away. Two magical hours, a time to admire the strange relationship between Nature and men: a synergy that, sometimes is possible. – Massimo Cristaldi (photographer)

Artificial Potential Functions Artificial Potential Functions based solutionbased solution

• Bio-inspired APF based solution– Mimic the bacteria foraging process– Multiple objectives and constraints are encoded as

potential functions– Vehicles’ movements follow gradient flow direction

• Weighted sum potential functions:

– Target (attraction) potential J g

– Neighbor (avoidance) potential J n

– Obstacle (avoidance) potential J o – Potential J s due to stationary threats– Potential J m due to moving threats

Gradient flow:

5

, ,( ) ( ) ( ) ( ) ( ) ( ) g n o s mi t i g i n i t i o i s i m t iJ p J p J p J p J p J p

, ( )( )

i t i

ii

J pp t

p

Related and prior work: Leonard 01, Tanner 03, Rimon 92, Others

Gibbs sampler and Gibbs sampler and Gibbs distributionGibbs distribution

• Gibbs distribution (global description)– The marginal probability is function of local potentials

– Gibbs distribution is function of global potentials

– Hammersley-Clifford theorem: A MRF on a graph is equivalent to a GF

• Gibbs sampler– Gibbs sampler defines a Markov chain on a Gibbs Field– The stationary distribution of the MC is the Gibbs distribution– Using simulated annealing algorithm, final configuration converges to

global minimum with probability 1.

sCc

sNc

sCcsNc

sS

sSs

Txx

Txx

TxU

TxxU

sNsNss

e

e

e

exXxXP

/),(

/),(

/),(

/),(

)()()(

)(

\

\

)|(

( )/

( ) , ( ) ( )U x Te

P X x U x J pZ

6

Animation of sequential Animation of sequential Gibbs sampling algorithmGibbs sampling algorithm

Obstacle

Agents

target

'

01

01 '

s

xT

xTx

T sD

sDsG

SxT

xT

xT sDsDsD ,...,, 21

SxT

xT

xT sDsDsD ,...,, 21

'

01

01 '

s

xT

xTx

T sD

sDsG

7

Parallel Gibbs samplerParallel Gibbs sampler

• Problems in sequential sampling– Global indexing is difficult in practice– Slow convergence time

• Parallel sampling– Don’t need global index– Speed up convergence time– Similar emergent behaviors

• Convergence analysis more difficult– Distributed asynchronous version even harder

8

Simulation: line formationSimulation: line formation

• Potential function

is scaling factor is a penalization for node with no neighbor– mk is the number of neighboring nodes of node k k,k’ is the desired angle of the line segment– dk,k’ /Rs puts more weight on farther neighbors, which

encourages the formations of long lines

0 ,

0 sin1

}':{,ˆ

'

2

',',

'

k

Nk kkks

kk

k

kkkk

mif

mifR

d

m

Nkxx

k

9

Simulation: Line formation

• 200 nodes on 50 by 50 grid =10 , =5

• Rm=2sqrt(2)

• Rs=10sqrt(2), 6sqrt(2), 4sqrt(2)

• T(n)=1/(4log(400+n))

One line

Two lines

Three lines

10

Parallel stochastic path exploration based on MRF – parallel Gibbs sampler

Get around local optima – Gibbs Get around local optima – Gibbs sampler based path planningsampler based path planning

Deterministic algorithm –gets stuck in local optima (paths)

11

Hybrid schemes Hybrid schemes with – without memorywith – without memory

• Hybrid scheme with memoryHybrid scheme

12

OutlineOutline









The Fundamental Trade-offThe Fundamental Trade-off

• The nodes gain from collaborating• But collaboration has costs (e.g. communications)• Trade-off: gain from collaboration vs cost of

collaboration

Vector metrics involved typically

Constrained Coalitional Games


Example 1: Network Formation -- Effects on Topology Example 2: Collaborative robotics, communications Example 3: Web-based social networks and services

● ● ●

Example 4: Groups of cancer tumor or virus cells

GainGain

• Each node potentially offers benefits V per time unit to other nodes: e.g. V is the number of bits per time unit.

• Jackson-Wolingsky connections model, gain of node i

• rij is # of hops in the shortest path between i and j

• is the collaboration depreciation rate

1( ) ijr

ij g

w G V

0 1

if there is no path between and ijr i j


CostCost

• Activating links is costly– Example – cost is the energy consumption for sending

data

– Like wireless propagation model, cost cij of link ij as a function of link length dij :

• P is a parameter depending on the transmission/receiver antenna gain and the system loss not related to propagation

• is path loss exponent -- depends on specific propagation environment.

ij ijc Pd


Pairwise Game and Pairwise Game and ConvergenceConvergence

• Payoff of node i from the network is defined as

• Iterated process

– Node pair ij is selected with probability pij– If link ij is already in the network, the decision is whether to

sever it, and otherwise the decision is whether to activate the link– The nodes act myopically, activating the link if it makes each at

least as well off and one strictly better off, and deleting the link if it makes either player better off

– End: if after some time, no additional links are formed or severed– With random mutations , the game converges to a unique

Pareto equilibrium (underlying Markov chain states )

( ) gain cost ( ) ( )i i iv G w G c G


G

Coalition Formation at the Coalition Formation at the Stable StateStable State

• The cost depends on the physical locations of nodes– Random network where nodes are placed according to a

uniform Poisson point process on the [0,1] x [0,1] square.

• Theorem: The coalition formation at the stable state for n∞

— Given is a

sharp threshold for establishing the grand coalition ( number of coalitions = 1).

— For , the threshold is

less than

2

0

ln,

nV P

n

0 1 2

ln.

nP

nn = 20


Topologies FormedTopologies Formed


OutlineOutline









Networks and Trust Networks and Trust

• Trust and reputation critical for collaboration• Characteristics of trust relations:

– Integrative (Parsons1937) – main source of social order– Reduction of complexity – without it bureaucracy and

transaction complexity increases (Luhmann 1988)– Trust as a lubricant for cooperation (Arrow 1974) –

rational choice theory

• Social Webs, Economic Webs

– MySpace, Facebook, Windows Live Spaces, Flickr, Classmates Online, Orkut, Yahoo! Groups, MSN Groups

– e-commerce, e-XYZ, services and service composition – Reputation and recommender systems


Heterogeneous Dynamic Heterogeneous Dynamic Network Analysis and TrustNetwork Analysis and Trust

• Multiple Interacting Graphs – Nodes: agents, individuals, groups,

organizations– Directed graphs– Links: ties, relationships– Weights on links : value (strength,

significance) of tie– Weights on nodes : importance of

node (agent)• Value directed graphs with weighted nodes• Real-life problems: Dynamic, time varying graphs, relations, weights


Social/Cognitive

Information

Comms

Sijw : S

ii w

: Sjj w

Iklw: I

kk w : Ill w

Cmnw: C

mm w : Cnn w

Organizational needs

Network architecture and operation

Next Generation Trust Analytics

• Trust evaluation, trust and mistrust dynamics– Spin glasses (from statistical physics), phase transitions

• Indirect trust; reputations, profiles; Trust computation via ‘linear’ iterations in ordered semirings

• Direct trust: Iterated pairwise games on graphs with players of many types

ˆ( 1) , ( ) |i ji j is k f J s k j N

2 31 a b

ba

2007 IEEE Leonard Abraham prizeNew Book Draft

24Copyright © John S. Baras 2007

Trust Semiring Properties:Trust Semiring Properties:Partial OrderPartial Order

• Combined along-a-path weight should not increase :

• Combined across-paths weight should not decrease :

2 31

b

a

a b


Computing Indirect TrustComputing Indirect Trust

• Path interpretation

• Linear system interpretation

• Treat as a linear system– We are looking for its steady state.

Indicator vector of pre-trusted nodes

1

i j i k k jUser k

n n

t t w

t W t b


Constrained Coalitional Games: Constrained Coalitional Games: Trust and CollaborationTrust and Collaboration

Two linked dynamics • Trust / Reputation propagation and Game evolution

• Beyond linear algebra and weights, semirings of constraints, constraint programming, soft constraints semirings, policies, agents

• Learning on graphs and network dynamic games: behavior, adversaries• Adversarial models, attacks, constrained shortest paths, …

• Integrating network utility maximization (NUM) with constraint based reasoning and coalitional games


Results of Game Results of Game EvolutionEvolution

● Theorem: , there exists τ0, such that for a reestablishing period τ > τ0

– iterated game converges to Nash equilibrium;– In the Nash equilibrium, all nodes cooperate with all their neighbors.

● Compare games with (without) trust mechanism, strategy update:

and

ii i ijj N

i N x J

Percentage of cooperating pairs vs negative links Average payoffs vs negative links

OutlineOutline









Distributed Kalman Distributed Kalman Filtering and TrackingFiltering and Tracking

• Realistic sensor networks: Normal nodes, faulty or corrupted nodes, malicious nodes

• Hierarchical scheme – provide global trust on a particular context without requiring direct trust on the same context between all agents

• Combine techniques from fusion-centric, collaborative filtering, estimation propagation

• Trusted Core– Trust Particles, higher security, additional sensing capabilities,

broader observation of the system, confidentiality and integrity, multipath comms

– Every sensor can communicate with one or more trust particles at a cost


Trust and HierarchyTrust and Hierarchy


• Distributed Kalman Filter Particles: Sensor nodes exchange estimates in their local neighborhood and trusted measurements from the trusted core

• Hierarchical scheme – provide global trust on a particular context without requiring direct trust on the same context between all agents

Sensor NetworkCommunication Graphfrom disc model Gc(V,Ec)

Trust and Induced GraphsTrust and Induced Graphs


Induced Graph G (V, A)

Weighted Directed Dynamic Trust Graph Gt (V, At )

tcV V

( , ) ( ( , ), ( , )[ ])w i j c i j t i j n

Trust relation

Goals of Trusted SystemGoals of Trusted System

1. All the sensors which abide by the protocols of sensing and message passing, should be able to track the trajectories.

2. This implies that those nodes which have poor sensing capabilities, nodes with corrupted sensors, should be aided by their neighbors in tracking.

3. Those nodes which are malicious and pass false estimates, should be quickly detected by the trust mechanism and their estimates should be discarded.


[ 1] [ ] [ ]

[ ] [ ] [ ] [ ]

[ ] [ ] [ ]i i i

tc tc tc

xn Axn Bwn

z n H n xn v n

z H n xn v n

Trusted DKF and ParticlesTrusted DKF and Particles


• Can use any valid trust system as trust update component

• Can replace DKF with any Distributed Sequential MMSE or other filter

• Trust update mechanism: Linear credit and exponential penalty

Trusted DKF PerformanceTrusted DKF Performance


Open Loop Performance Closed Loop PerformanceTrust System Performance

OutlineOutline









• Distributed algorithms are essential– Group of agents with certain abilities– Agents communicate with neighbors, share/process information– Agents perform local actions– Emergence of global behaviors

• Effectiveness of distributed algorithms– The speed of convergence– Robustness to agent/connection failures– Energy/ communication efficiency

• Group topology affects group performance• Design problem: Find graph topologies with favorable tradeoff between performance

improvement (benefit) vs cost of collaboration• Example: Small Word graphs in consensus problems

36

Distributed Algorithms in Distributed Algorithms in Networked Systems and TopologiesNetworked Systems and Topologies

Copyright © John S. Baras 2008

37

Consensus problems: Consensus problems: Design of information flowDesign of information flow

• Fixed graphs: Geometric convergence with rate equal to Second Largest Eigenvalue Modulus (SLEM)

• How does graph topology affect location of eigenvalues?• How can we design graph topologies which result in

good convergence speed?

Symmetric communication

38

Simple Lattice C(n,k)

Small world: Slight variation adding

Small World graphsSmall World graphs

nk

Adding a small portion of well-chosen links → significant increase in convergence rate

39

Mean field explanation and Mean field explanation and perturbation approachperturbation approach

Initial graph

F matrix

Perturbed

Final graph

0(1 ) TF n F 11

40

Watts-Strogatz Watts-Strogatz Small World networksSmall World networks

• Random graph approach (e.g. Durrett 2007, Tahbaz and Jadbabaie 2007)

• Perturbation approach (Higham 2003 )

– Start from lattice structure G0=C(n,k) F0

– Perturb zero elements in the positive direction by for fixed and

– Perturb the formerly nonzero elements equally, such that the stochastic structure of the F matrix is preserved Fε

– Analyze the SLEM as a function of the perturbation as α varies

n

K

0K .1

41

Analysis of WS modelAnalysis of WS model

• For the ring type structure, in the limit of large n:– For the effect of ε-shortcuts is negligible– For the effect of ε-shortcuts starts (spectral gap gain

perturbation of same order)– For the shortcuts dominantly decrease SLEM

• α =3 can be considered as the onset of small world effect with small world effect happening at α = 2.

3 3

2

Onset of SW

SW dominant01 ( )F

1( )O n 2( )O n 1( )

logO

n n

1( )

logO

n n 1(log )O n 1

( )log log

On n

42

Does switching help?Does switching help?

• i.i.d/Markov switching between a set of graphs

• Can be extended to i.i.d. link losses

• Have obtained conditions on probabilistic convergence

1 2( , ,..., )NG G GG

Long range links even with small weights usefulHow about long range links with small probability?

1 1( , )G p 2 2( , )G p ( , )N NG p

Hierarchical organizationHierarchical organization

• Idea: selecting a few well-connected, well-protected and controlled agents as leaders

• Dividing the agents into groups each with

members• Leaders:

– should be well connected to the members of group– should be able to communicate to the other leaders

• The reduction of the size of each group to results in faster inter-group convergence

• The ease of communication between the leaders upon demand results in fast overall convergence

)( nO)( nO

)( nO


44

Distributed exploration Distributed exploration of the graph structureof the graph structure

• Self-organization for better performance and resiliency• Hierarchical scheme to design a network structure capable

of running distributed algorithms with high convergence speed

• A two stage algorithm:

1- Find the most effective choice of local leaders

2- Provide nodes with information about their location with respect to other nodes and leaders and the choice of groups to form

• Divide N agents into K groups with M members each

, select ‘leaders’

,N K M K M N

45

Distributed self - organizationDistributed self - organization

Goal: design a scheme that gives each node a vector of compact global information

46

Two stage semi-decentralized Two stage semi-decentralized algorithmalgorithm

• Stage 1: Determining K leaders– Each node determines its social degree via local query– Dominant nodes in each neighborhood send their degrees to the

central authority– Central authority computes their social scores

Choice of α determines whether leaders in star-like neighborhoods are preferred

– The central authority selects the K nodes with highest scores as social leaders and gives them an arbitrary order

(2) (3)( ) ( ) (1 ) ( ) SC k SD k SD k

47

Algorithm (cont’d)Algorithm (cont’d)

• Stage 2: Determining the influence vectors– Based on its order each leader takes its influence vector to be the

fixed vector ei

– Regular nodes update their influence vector entries:

• For connected graphs, for t large enough, converges to the influence of leader k on node I

• Upon calculation of influence vectors, each regular node determines its local leader and stops its communication with neighbors who have other leaders

• Graph decomposes into two level hierarchy with efficient communication pattern

( )

1( 1) ( ) ( )

1i

k k ki i j

j N ti

x t x t x tn

kix

OutlineOutline









Expander GraphsExpander Graphs

• Fast synchronization of a network of oscillators • Network where any node is “nearby” any other • Fast ‘diffusion’ of information in a network• Fast convergence of consensus • Decide connectivity with smallest memory • Random walks converge rapidly• Easy to construct, even in a distributed way (ZigZag graph product)• Results on colorability …

• Graph G, Cheeger constant h(G) – All partitions of G to S and Sc ,

h(G)=min (#edges connecting S and Sc ) /

(#nodes in smallest of S and Sc )

• (k , N, ) expander : h(G) > ; sparse but locally well connected (1-SLEM(G) increases as h(G)2) 49

Expander Graphs – Ramanujan Graphs

Control Control vs.vs. CommunicationCommunication

• Many graphs as abstractions • Collaboration graph – or a model of what the system

does (behavior)• Communication graph – or a model of what the

system consist of (structure)• Nodes with attributes – multi-graphs – hypergraphs• Key question 1: Given behavior, what structure

(subject to constraints) gives best performance?• Key question 2: Given structure (and constraints)

how well behavior can be executed?


Lessons Learned --Lessons Learned --Future DirectionsFuture Directions

• Constrained coalitional games – unifying concept• Generalized networks, flows - potentials, duality

and network optimization (monotropic optimization)• Time varying graphs – mixing – statistical physics • Understand autonomy – better to have self-

organized topology capable of supporting (scalable, fast) a rich set of distributed algorithms (small world graphs, expander graphs) than optimized topology

• Given a set of distributed computations is there a small set of simple rules that when given to the nodes they can self-generate such topologies?



Thank you!

[email protected]

301-405-6606

http://www.isr.umd.edu/~baras

Questions?

http://www.isr.umd.edu/~baras

swarms, coalitional games, trust, and network topologies john s. baras institute for systems...

Documents