Scalable and Dynamic Quorum Systems

Moni Naor & Udi WiederThe Weizmann Institute of Science

Slide - 2

Quorum Systems

A quorum system is an intersecting family of sets over some universe.

Formal Definition: U – Universe Hello World Hello World is called a quorum system. A,B are called quorum sets.

Examples Majority – all sets that contain more than half of the elements. Dictatorship – all sets that contain a specific element. Many more…

Slide - 3

Quorum Systems - Applications

The universe is associated with a set of processors.

Mutual exclusion: In order to enter a critical section, a user must get permission from a quorum set.

Intersection property guaranties mutual exclusion. [GB85].

Data replication: Divide the quorum sets into reading sets and writing sets.

Intersection property guaranties effective search.

More….

Slide - 4

Example – The PATHS Quorum System Suggested by [NW94].

Element Pair of dual (crossing) edges.

Quorum Set Left – Right path + Top – Bottom path.

Every top-bottom path in the dual grid intersects every left-right path in the grid.

Data replication: Top Bottom – Writing. Right Left – Reading.

Slide - 5

Load Each quorum set is chosen by the user with some probability. Imposes a probability of accessing a processor. The load measures the access probability of the busiest

processor. Example: In the Majority system, if a random set of size n/2 + 1 is

chosen, the load is asymptotically half (and this is best possible).

Theorem [NW94]: The load of a quorum system is at least The load is at least .(For every system and every access strategy).

The PATHS system has load of .

Measures of Quality - Load

Let c be the size of the smallest quorum set.

Slide - 6

Availability. Assume each processor fails with some fixed probability p. What

is the probability that there still exists a quorum set?

Theorem[PW95] : If then the singleton has the ‘best’ availability.

The PATHS system has a live quorum set with probability , ( for ).

Measures of Quality - Availability

Slide - 7

Measures of Quality – Probe Complexity

A random portion of the processors are down.

How many processors should be queried before a live quorum is found?

Adaptive algorithms – more efficient [HP01 , KXG00, PW96]. Non adaptive algorithms (predefined set of probed elements) – Easy

to implement in parallel.

Given a network topology, what is the communication complexity of accessing these processors [Bazzi96].

Processors from the same quorum should be ‘close’ to one another.

Slide - 8

Our Contributions

Probe Complexity: Lower bound for non-adaptive algorithms. Tight upper bound for PATHS. Tight adaptive upper bound for PATHS.

Introduce Dynamic PATHS (DPATHS). Dynamic P2P style version in which processors may join and

leave. Scales well. Maintains the good properties of PATHS.

Slide - 9

Probe Complexity vs Load

Probe complexity depends on the load. Example: Only a small number of processors participate in

quorum sets Small probe complexity and high load.

Notation: 1/c – the load of the quorum system. X – set of probed elements. Non-adaptive algorithm must choose X

in advance. p – failure probability.

The PATHS system has a non-adaptive probe complexity of

Theorem: If then

If load is then

Slide - 10

Probe Complexity – Upper Bound

A failed edge corresponds to a surviving dual edge that crosses it. An edge fails with probability Dual edge fails with probability

A top – bottom path must circumvent red components of dual graph.

By Menshikov’s Theorem - All red components are of diameter O(log n), therefore a strip of width O(log n) would contain a path w.h.p.

Survived dual edge.

Conclusion: The non-adaptive probe complexity of PATHS is

Slide - 11

Adaptive Probe Complexity

All red components have small diameter.

All red component have small volume.

Smart DFS finds a path of length .

Optimal load implies quorum sets are of size .

Conclusion: PATHS has optimal adaptive probe comp.

Survived dual edge.

Non-adaptive probe complexity is strictly higher than adaptive complexity.

Slide - 12

Paths Quorum System - Summary

Load - . Optimal

Availability – Very high when . Optimal

Probe Complexity – Non-adaptive - . Optimal (for load). Adaptive - . Optimal (for load).

Load after failures - . Optimal.

Network implementation - Supports probing algorithms (think of edges as links).

Slide - 13

Dynamic Quorum Systems.

Motivation: Peer-to-Peer applications. Related Work – dynamic probabilistic quorums[AM03]

Processors may join and leave the quorum system. Objectives:

Low cost of Join/Leave Maintain the intersection property (integrity).

New quorum sets Adjust old quorum sets that are no longer valid.

Scalability Load should reduce when processors join. Availability should increase when processors join. Probe complexity should not grow by much.

Slide - 14

Dynamic Quorums

Main Idea: Assign each processor to a point in a continuous space.

The underlying space is the unit square.

Divide the space into cells using a Voronoi diagram. CAN[01] suggests dividing the space into rectangles.

A cell consists of all the points that are closest to the processor.

Average degree of a cell is always 6.

Adding and removing a point is a local computation.

Slide - 15

Voronoi Diagram – The Delaunay Graph

Slide - 16

Voronoi Diagram – Join\Leave

Slide - 17

Random Voronoi Diagram

If each processor chooses its location uniformly and independently then:

The average degree is 6 (always), the maximum degree is with high probability.

The average area of a cell is . The maximum area of a cell is with high probability.

The average projection on the axis lines is of length .

The maximum projection is with high probability.

Slide - 18

The Dynamic Paths Quorum System

Slide - 19

Integrity

Old Quorum sets may not be valid anymore. Solution – application dependent.

New processor must join the quorum set.

Slide - 20

Dynamic Paths – Measurs of Quality Load – Depends on the size of the cell’s projection on

the axis lines. Might reach .

Availability – Probability a live quorum set exists when converges to 1 (when n ). The exact rate of convergence is unknown.

Probe Complexity – Probably good. A Menshikov style theorem for Voronoi Diagrams is conjectured but not proven.

If all cells are of size then there exists such that for failure probability , the qualities of PATHS follow to DPATHS.

Slide - 21

Open Questions

Load balancing schemes. Devise simple and reliable protocols for maintaining all cells in equal size. Good for many applications (such as building expanders [NW03]). Sample log n locations, and enter the largest cell encountered. Deterministic solutions? Simpler solutions?

Menshikov style theorems for random Voronoi Diagrams - pushing the failure probability all the way to half. (Hard).

Can resilience for worst case faults be achieved without damaging other parameters?