1

Randomized Load Balancing with & without Memory

EE384Y – Packet Switch Architecture – II

Rajan Goyal/Jianying Luo

{rgoyal, jyluo}@stanford.edu

2

Outline

• Introduction

• Motivation

• Simulation Study

• Acknowledgment

3

Part - I

4

Outline

• Introduction

• Motivation

• Simulation Study

• Acknowledgment

5

Super Market Model

1 2 3 n

Allocator

μ1

poisson (nλ), λ < 1

μ2μ3 μn

Symmetric Service Rate: μ1 = μ2 = μ3 = … = μn = 1

Asymmetric Service Rate: Σiμi = n

6

Assignment Policy

• Different policies for assigning load– Join shortest queue, SQ– Join random queue, SQ(1)– Join shortest of d random queues, SQ(d)– Join shortest of (d+m) queues, SQ(d,m)

• where d – fresh random samples, m – shortest of the (d+m) choices in last iteration.

7

Outline

• Introduction

• Motivation

• Simulation Study

• Acknowledgment

8

Motivation

• Given this background, we tried to study following three questions:– How many samples? i.e. d = ?– How much memory? i.e. m = ?– Given a choice of (d,m) and d+m=k, where k is

constant, what is the right order of d and m?

9

Outline

• Introduction

• Motivation

• Simulation Study

• Acknowledgment

10

Simulation Setup

1 2 3 n

(d,m) policy

μ1

poisson (nλ), λ < 1

μkμk+1 μn

Asymmetric Service Rate: Σiμi = 1500, 1 i 1500,

Σiμi = 1000, 1 i 25,

Σiμi = 500, 26 i 1500,

n=1500, k = 25

n

μn-1

System Unstable for λ 0.356

Symmetric Service Rate: μ1 = μ2 = μ3 = … = μn = 1

11

Agenda

– How many samples? i.e. d = ?

– How much memory? i.e. m = ?

– Given a choice of (d,m) and d+m=k, where k is constant, what is the right order of d and m?

12

Effect of ‘d’ on Queue Size

13

Effect of ‘d’ on Variance

14

Effect of ‘d’ (Observations)

• For symmetric service rate, d>=2 gives exponential improvement over (d=1) policy in terms of maximum load.

• (d=2) achieves maximum gain for unit increase in d.

15

Agenda

– How many samples? i.e. d = ?

– How much memory? i.e. m = ?

– Given a choice of (d,m) and d+m=k, where k is constant, what is the right order of d and m?

16

Effect of ‘m’ on Queue Size

17

Effect of ‘m’ on Variance

18

Effect of m (Observation)

• Memory keeps tail of the queue size very low, rather than average queue size. i.e. stops queues from growing.

• Memory reduces the variance of queue sizes among queues.

19

Part - II

20

Effect of memory

• Why memory helps?– Memory sort of gives (indirect) feedback to the

allocator regarding server states, which is missing when just fresh samples are picked.

– Memory is useful if variance of queue sizes (between all queues) is high.

– So, memory is more effective for asymmetric service rate servers.

21

Why memory helps?

22

How much memory?

23

How much memory?

24

How much memory? (Observations)

• Increase of memory decreases the queue size and variance.

• However, when memory is large enough, the improvement due to increase in memory is not appreciable.

25

Agenda

– How many samples? i.e. d = ?

– How much memory? i.e. m = ?

– Given a choice of (d,m) and d+m=k, where k is constant, what is the right order of d and m?

26

Exact Order

• d+m=k, where k is constant, what is right balance of d & m?– One extreme policy is (1,k-1) and other extreme is

(k,0).

• Adv. of extra sample at the expense of memory?• Adv. of extra memory at the expense of fresh

sample?• Simulation study for (d+m = 4, d+m = 8)

27

Exact Order (d+m=4)

28

Exact Order (d+m=4)

29

Exact Order (d+m=4)

30

Observations

• From (1, k-1) to (k, 0), ‘d’ is increasing and ‘m’ is decreasing.

• i.e. effects of ‘d’ and ‘m’ are competing with each other.

• Based on previous results, increase in ‘d’ has more effect as compared to increase in ‘m’.

• Also, (k,0) is unstable.

31

Acknowledgment

• We are indebted to Balaji for suggesting this topic and insight. Thanks to Balaji & Devavrat for discussions.

32

Questions

33

Analysis

• SQ is ideal, but hard to implement when n is large.• For symmetric service rates, it is clear that SQ(d)

is stable. SQ(1) is stable for λ < 1 and P(Q>=i)= λi.• It’s well known1 that:

– For SQ(d), P(Q>=i)= λ(di-1)/(d-1)

Note: Our simulation is consistent with the analytical

model

34

Analysis (contd…)

• For asymmetric service rate, it is known2 that:– SQ(d) is unstable even when d = O(n)†.– SQ(1,1) is stable, if λ < 1.

1 Load Balancing and Density Dependent Jump Markov Processes, Michael Mitzenmacher

2 The use of memory in randomized load balancing, Devavrat Shah & Balaji Prabhakar

† sampling is done with replacement.

35

Simulation Setup

1 2 3 n

(d,m) policy

μ1

poisson (nλ), λ < 1

μ2μ3 μn

Symmetric Service Rate: μ1 = μ2 = μ3 = … = μn = 1

n=1500, λ = 0.99