lower bounds in greedy model sashka davis advised by russell impagliazzo (slides modified by jeff)...
TRANSCRIPT
Lower Bounds in Greedy Model
Sashka DavisAdvised by Russell Impagliazzo
(Slides modified by Jeff)
UC San DiegoOctober 6, 2006
Suppose you have to solve a problem Π…
Is there a Greedy algorithm that solves Π?Is there a Backtracking
algorithm that solves Π?
Is there a Dynamic Programming algorithm
that solves Π?
Eureka! I have a DP Algorithm!No Backtracking agl.
exists? Or I didn’t think of one?
Is my DP algorithm optimal or a better one
exists?
No Greedy alg. exists? Or I didn’t think of one?
Suppose we a have formal model of each algorithmic
paradigmIs there a Greedy
algorithm that solves Π?
No Greedy algorithm can solve Π exactly. Is there a Backtracking
algorithm that solves Π?No Backtracking algorithm
can solve Π exactly.
Is there a Dynamic Programming alg. that
solves Π?
DP helps!
Is my algorithm optimal, or a better DP
algorithm exists?
Yes, it is! Because NO DP alg. can solve Π more
efficiently.
The goal
• To build a formal model of each of the basic algorithmic design paradigms which should capture the strengths of the paradigm.
• To develop lower bound technique, for each formal model, that can prove negative results for all algorithms in the class.
Using the framework we can answer the following questions
1. When solving problems exactly:What algorithmic design paradigm can help?• No algorithm within a given formal model can solve the problem
exactly.• We find an algorithm that fits a given formal model.
2. Is a given algorithm optimal?• Prove a lower bound matching the upper bound for all algorithms
in the class.
3. Solving the problems approximately:• What algorithmic paradigm can help?• Is a given approximation scheme optimal within the formal
model?
Some of our results
ADAPTIVEPRIORITY
FIXED
GreedyGreedy
Backtracking & Simple DP
(tree)
Backtracking & Simple DP
(tree)
DynamicProgramming
DynamicProgramming
pBT
pBP
Online
is a set of data items; is a set of options
Input: instance I={1 ,2 ,…,n }, I
Output: solution S={(i , i) | i= 1,2,…,d}; i
1. Order: Objects arrive in worst case order chosen by adversary.
2. Loop considering i in order.
– Make a irrevocable decision i
On-line algorithms
is a set of data items; is a set of options
Input: instance I={1 ,2 ,…,n }, I
Output: solution S={(i , i) | i= 1,2,…,d}; i
1. Order: Algorithm chooses fixed π : →R+
without looking at I.
2. Loop considering i in order.
– Make a irrevocable decision i
Fixed priority algorithms
is a set of data items; is a set of options
Input: instance I={1 ,2 ,…,n }, I
Output: solution S={(i , i) | i= 1,2,…,d}; i
2. Loop
- Order: Algorithm reorders π : →R+
without looking at rest of I.
- Considering next i in current order.
– Make a irrevocable decision i
Adaptive priority algorithms
is a set of data items; is a set of options
Input: instance I={1 ,2 ,…,n }, I
Output: solution S={(i , i) | i= 1,2,…,d}; i
1. Order: Algorithm chooses π : →R+
without looking at I.
2. Loop considering i in order.
– Make a set of decisions i
(one of which will be the final decision.)
Fixed priority “Back Tracking”
Some of our results
PRIORITY
pBT
pBP
ADAPTIVEPRIORITY
FIXED
Shortest Path in negative graphs no cycles
Bellman-Ford
Shortest Path in no-negative graphs
Dijkstra’s
Online
Maximum Matching in Bipartite graphs
Flow Algorithms
Maximum Matching in Bipartite graphs
Minimum Spanning Tree
Prim’sKruskal’sKruskal’s
Some of our results
PRIORITY
pBT
pBP
ADAPTIVEPRIORITY
FIXED
Dijkstra’s
Shortest Path in no-negative graphs
OnlinePrim’sKruskal’s
Minimum Spanning Tree
Kruskal’s
Kruskal algorithm for MST is a Fixed priority algorithm
Input (G=(V,E), ω: E →R)1. Initialize empty solution T2. L = Sorted list of edges in non-decreasing order
according to their weight3. while (L is not empty)
– e = next edge in L– Add the edge to T, as long as T remains a forest
and remove e from L4. Output T
Prim’s algorithm
Input G=(V,E), w: E →R
1. Initialize an empty tree T ← ; S ← 2. Pick a vertex u; S={u};
3. for (i=1 to |V|-1)– (u,v) = min(u,v)cut(S, V-S)w(u,v)
– S←S {v}; T←T{(u,v)}
4. Output T
Prims algorithm for MST is an adaptive priority algorithm
Dijkstra’s Shortest Paths Alg is an adaptive priority algorithm
• Dijkstra algorithm (G=(V,E), s V)• T← ; S←{s};∅• Until (S≠V)• Find e=(u,x) | e = mineCut(S, V-S){path(s, u)+ω(e)}• T← T{e}; S ← S {x}
Some of our results
PRIORITY
pBT
pBP
ADAPTIVEPRIORITY
FIXED
Dijkstra’s
Shortest Path in no-negative graphs
OnlinePrim’sKruskal’s
Minimum Spanning Tree
Kruskal’s
ShortPath Problem: Given a graph G=(V,E),
ω: E →R+; s, t V. Find a directed tree of edges,
rooted at s, such that the combined weight of the
path from s to t is minimal
• Theorem: No Fixed priority algorithm can achieve any constant approximation ratio for the ShortPath problem
Some of our results
• Data items are edges of the graph• Decision options = {accept, reject}
Fixed priority game
Solver Adversaryγd
γi
γ3
γj γk
γ2γ1
Γ0
S_sol = {(γi2,σi2)}
σi2
S_sol = {(γi2,σi2), (γi4,σi4)}
γi2 γi9,…γi1 γi3 γi4 γi5 γi6 γi7 γi8Γ0Γ1 Γ2
σi4
Γ3
End GameS_adv = {(γi2,σ*
i2), (γi4,σ*i4)}
Solver is awarded(S_sol)
(S_adv)
f
f
=∅
Adversary selects 0
t
b
s
a u(k)
w(k)
x(1)
v(1)
y(1)
z(1)
0 , , , , ,u v w x y z
Solver selects an order on 0
If then the Adversary presents: ( ) ( )y z
t
b
s
a u(k)
w(k)
x(1)
v(1)
y(1)
z(1)
1 , , ,u x y z
Adversary’s strategy
Waits until Solver considers edge y(1)
Solver will consider y(1) before z(1)
Event 1σy=accept
Event 2σy=reject
Event 1: Solver accepts y(1)
t
u(k)
x(1)
y(1)
z(1)
b
a
s
The Solver constructs a path {u,y}The Adversary outputs solution {x,z}
1
2
Alg k
Adv
Event 2: Solver rejects y(1)
The Solver fails to construct a path.The Adversary outputs a solution {u,y}.
t
u(k)
x(1)
y(1)
z(1)
b
a
s
The outcome of the game:
• The Solver either fails to output a solution or achieves an approximation ratio (k+1)/2
• The Adversary can set k arbitrarily large and thus can force the Algorithm to claim arbitrarily large approximation ratio
Some of our results
PRIORITY
pBT
pBP
ADAPTIVEPRIORITY
FIXED
Dijkstra’s
Shortest Path in no-negative graphs
Online
Some of our results
PRIORITY
pBT
pBP
ADAPTIVEPRIORITY
FIXED
Interval Schedulingvalue is width
Factor of 3Online
Factor of 3
Interval scheduling on a single machine
• Instance:
Set of intervals I=(i1, i2,…,in), j ij=[rj, dj]
• Problem: schedule intervals on a single machine
• Solution: S I• Objective function: maximize iS(dj - rj)
A simple solution (LPT)
Longest Processing Time algorithm input I=(i1, i2,…,in)
1. Initialize S ← 2. Sort the intervals in decreasing order (dj – rj)
3. while (I is not empty) Let ik be the next in the sorted order
If ik can be scheduled then S ← S U {ik};
I ← I \ {ik}
4. Output S
LPT is a 3-approximation
• LPT sorts the intervals in decreasing order according to their length
• 3 LPT ≥ OPT
OPT OPT OPT
LPT
ri di
Example lower bound [BNR02]
• Theorem1: No adaptive priority algorithm can achieve an approximation ratio better than 3 for the interval scheduling problem with proportional profit for a single machine configuration
Proof of Theorem 1• Adversary’s move
• Algorithm’s move: Algorithm selects an ordering
• Let i be the interval with highest priority
1
2
3
q
q-1q-11
2
3
e
Adversary’s strategy
• If Algorithm decides not to schedule i • During next round Adversary removes all
remaining intervals and schedules interval i
1
2
3
i
jk1
2
3
i
Alg’s value = 0Adv’s value = i
Adversary’s strategy
• If i = and Algorithm schedules i• During next round the Adversary restricts
the sequence:
i
jk1
2
3
i
i
i+1i-1
Alg’s value = iAdv’s value = (i-1)+3(i/3)+(i+1)=3i
Adversary’s strategy
• If i = and Algorithm schedules i• During next round the Adversary restricts
the sequence:
1
Alg’s value = 1Adv’s value = 3(1/3)+(2)=3
1
2
3
i
jk1
2
31
2
Adversary’s strategy
• If i = and Algorithm schedules i• During next round the Adversary restricts
the sequence:
1
2
3
i
jk1
2
3
q
q
q-1q-1
Alg’s value = qAdv’s value = (q-1)+3(q/3)+(q-1)=3q-1
But q is big
Adversary’s strategy
• If i = and Algorithm schedules i• During next round Adversary restricts the
sequence:
1
2
3
m
jk1
2
3
i
i
m
Alg’s value = iAdv’s value = (3i) =3i
PRIORITY
pBT
pBP
ADAPTIVEPRIORITY
FIXED
Interval Schedulingvalue is width
Factor of 3Online
Factor of 3
The algorithm was missed up beforeit got a chance to reorder things.
?
Some of our results
Some of our results
PRIORITY
pBT
pBP
ADAPTIVEPRIORITY
FIXED
Weighted Vertex Cover
Factor of 2
Online
[Joh74] greedy 2-approximation for WVC
Input: instance G with weights on nodes.
Output: solution S V covers all edges and minimizes weight taken nodes.
Repeat until all edges covered.• Take v minimizing ω(v)/(# uncovered adj edges)
Weighted Vertex Cover
• With Shortest Path,a data item is an edge of the graph – = (<u,v>, ω(<u,v>) )
• With weighted vertex cover,– A data item is a vertex of the graph
= (v, ω(v), adj_list(v))
• (Stronger than having the items be edges,because the alg gets more info from nodes.
Weighted Vertex Cover
Theorem: No Adaptive priority algorithm can
achieve an approximation ration better than 2
Adaptive priority gameSolver Adversary
γ3
γ5
γ6
γ1 γ4
γ7
γ2
S_sol = {(γ7,σ7)}σ4S_sol = {(γ7,σ7), (γ4,σ4)}
Γ3Γ1Γ2
σ7
The Game Ends:1. S_adv = {(γ7,σ*
7), (γ4,σ*4),(γ2,σ*
2)}2. Solver is awarded payoff
f(S_sol)/f(S_adv)
γ8
γ9γ10
γ11
γ12
Γ0
σ2 S_sol = {(γ7,σ7), (γ4,σ4),(γ2,σ2)}
The Adversary chooses instances to be graphs Kn,n
The weight function ω:V→ {1, n2}
n2
1n2
n2
n2
11
1
The game
• Data items– each node appears in o as two separate data
items with weights 1, n2 • Solver moves
– Choses a data item, and commits to a decision
• Adversary move– Removes from the next t the data item,
corresponding to the node just committed and..
Adversary’s strategy is to wait unitl
Event 1: Solver accepts a node of weight n2
Event 2: Solver rejects a node of any weight
Event 3: Solver has committed to all but one nodes on either side of the bipartite
1
1
1
11
Event 1: Solver accepts a node ω(v)=n2
A B
The Adversary chooses part B of the bipartite as a cover, and incurs cost n
The cost of a cover for the Solver is at least n2+n-12lg
2A n
Adv n
1
1
n2
1
1
1
1
Event 2: Solver rejects a node of any weight
A B
The Adversary chooses part A of the bipartite as a cover.The Solver must choose part B of the bipartite as a cover.
2
2
lg 22
A n
Adv n n
n2
n2
Event 3: Solver commits to n-1 nodes w(v)=1, on either side of Kn,n
A B
The Adversary chooses part B of the bipartite as a cover, and incurs cost n
The cost of a cover for the Solver is 2n-1lg 2 1
2A n
Adv n
1
1
1
1
1
1
1
n2
Some of our results
PRIORITY
pBT
pBP
ADAPTIVEPRIORITY
FIXED
Weighted Vertex Cover
Factor of 2
Online
Some of our results
PRIORITY
pBT
pBP
ADAPTIVEPRIORITY
FIXED
Facility Location
Factor of logn
Online
Facility location problem
• Instance is a set of cities and set of facilities– The set of cities is C={1,2,…,n}– Each facility fi has an opening cost cost(fi) and
connection costs for each city: {ci1, ci2,…, cin}
• Problem: open a collection of facilities such that each city is connected to at least one facility
• Objective function: minimize the opening and connection costs min(ΣfScost(fi) + ΣjCmin fiScij )
[AB02] result
• Theorem: No adaptive priority algorithm can achieve an approximation ratio better than log(n) for facility location in arbitrary spaces
Adversary presents the instance:
• Cities: C={1,2,…,n}, where n=2k
• Facilities: – Each facility has opening cost n– City connection costs are 1 or ∞ – Each facility covers exactly n/2 cities– cover(fj) = {i | i C,cji=1}
Cu denotes the set of cities not yet covered by the solution of the Algorithm
Adversary’s strategy
At the beginning of each round t – The Adversary chooses St to consist of
facilities f such that fSt iff |cover(f) ∩ Cu| = n/(2t)
– The number of uncovered cities Cu is n/(2t-1)
Two facilities are complementary if together they cover all cities in C. For any round t St consists of complementary facilities
The game
Uncovered cities Cu
End of the game
• Either Algorithm opened log(n) facilities or failed to produce a valid solution
• Cost of Algorithm’s solution is n.log(n)+n
• Adversary opens two facilities incurs total cost 2n+n
Some of our results
PRIORITY
pBT
pBP
ADAPTIVEPRIORITY
FIXED
Facility Location
Factor of logn
Online