np-completeness and approximation algorithmsb97032/ln11_npc.pdf · ♦ np-complete problems •...

1NTUEEAlgorithms

Algorithms演算法

Professor Chien-Mo James Li 李建模

Graduate Institute of Electronics EngineeringNational Taiwan University

NP-Completeness andApproximation Algorithms

2NTUEEAlgorithms

Outline• NP-completeness, CH34

♦ polynomial time verification

♦ NP-completeness and reducibility

♦ NP-completeness proofs

♦ NP-complete problems

• Approximation Algorithms, CH35

3NTUEEAlgorithms

Stephen Cook (1939~, USA)• Stephen Arthur Cook is a renowned American-

Canadian computer scientist and mathematician who has made major contributions to the fields of complexity theory and proof complexity.

• He is currently a University Professor at the University of Toronto, Department of Computer Science and Department of Mathematics.

• Cook formalized the notion of NP-completeness in his seminal 1971 paper "The Complexity of Theorem Proving Procedures", where he proved the existence of an NP-complete problem by showing that the satisfiability problem is NP-complete

• In 1982, Cook received the prestigious Turing award for his contributions to complexity theory.

4NTUEEAlgorithms

Decision Problems vs. Optimization Problems• Decision problems: find answers to yes/no questions

♦ MST: Given a graph G=(V, E) and a bound K, is there a spanning tree with a cost at most K?

♦ TSP: Given cities, distance between each pair of cities, and a bound B, is there a route that starts and ends at a given city, visits every city exactly once, and has total distance at most B ?

• Optimization problems: find a legal answer to minimize cost

♦ MST: Given a graph G=(V, E), find the cost of a minimum spanning tree of G

♦ TSP: Given a set of cities and that distance between each pair of cities, find the distance of a “minimum route” starts and ends at a given city and visits every city exactly once

• Could apply binary search on decision problems to obtain solutions to optimization problems

♦ Assume a cost C, decide if solution exist

♦ If no, increase cost; if yes, reduce cost

♦ Repeat until we get satisfactory answers

5NTUEEAlgorithms

Hamiltonian Cycles (HC)• Hamiltonian cycle: a simple cycle that contains every vertex in an

undirected graph

• Hamiltonian-cycle problem: given a graph G, does it have a Hamiltonian cycle?

♦ Decision problem

• Example: Fig 34.2 (a) yes (b) no

6NTUEEAlgorithms

Complexity Classes P, NP• Complexity class P: the set of problems that can be solved in

polynomial time in the size of input

♦ O(nk) for some constant k

∗ Polynomial time: O(n2), O(n3), O(1), O(n lg n)

− Problems in P are considered tractable

∗ Not polynomial time: O(2n), O(nn), O(n!)

− considered intractable

♦ Closed under addition, multiplication, composition, complement..

• Complexity class NP (Nondeterministic Polynomial): the set of problems that can be verified in polynomial time in the size of input

♦ we know P ⊆ NP, but P = NP ?

Think of a non-deterministic computer as a computer that magically “guesses” a solution, then has to verify that it is correct. If a solution exists, computer always guesses it.One way to imagine it: a parallel computer that can freely spawnan infinite number of processes.

7NTUEEAlgorithms

HC ∈ NP• Verification algorithm: a 2-argument algorithm A, where one

argument is an input x and the other is y (called a certificate). Averifies x if there exists y s.t. A answers “yes”

• Example: Is HC ∈ NP?

♦ Given a list of vertices (certificate)

♦ Check if the tour visits every vertex exactly once

♦ Check whether each edge along the cycle actually exists

♦ Check if the first vertex = last vertex

♦ All can be done in O(n) time, so HC ∈ NP

8NTUEEAlgorithms

The P = NP Question• P = NP? is one of biggest open questions in computer science

♦ Most theorists believe P ≠ NP

• Complexity class NP-complete (NPC):

♦ The set of problems that can be verified in polynomial time

∗ but not sure if can be solved in polynomial time

♦ NPC is at crux of P = NP question

♦ If any NPC problem can be solved in polynomial time,

∗ then all problems in NP can be solved in polynomial time

• a US$1 million reward to anyone who has a formal proof

P=NP=NPC P NPC

NP

not likely more likely

9NTUEEAlgorithms

Complexity Class co-NP• Is class NP closed under complement?

• Class co-NP problems = whose complement problems are in NP

• Example: HC-Complement

♦ Given a set of vertices and a graph, do these vertices NOT form a Hamiltonian cycle?

♦ HC-Complement ∈ NP, therefore, HC ∈ co-NP

• Fig 34.3 Four possibilities

not likely

10NTUEEAlgorithms







11NTUEEAlgorithms

Polynomial-time Reduction• Let L1 and L2 be two decision problems

♦ Suppose algorithm A2 can solve L2. Can we use A2 to solve L1?

• Polynomial-time reduction from L1 to L2: L1 ≤P L2

♦ Function f reduces input for L1 into an input for L2

∗ s.t. reduced input is a “yes” input for L2 iff the original input is a “yes” input for L1

∗ f is computable in polynomial time

♦ L1 is no more than a polynomial factor harder than L2

yesyes

no no

12NTUEEAlgorithms

Lemma 34.3• (Lemma 34.3) If L1 ≤P L2 then L2 ∈ P implies L1 ∈ P

• Fig 34.5

♦ F is a reduction algorithm that computes the reduction function ffrom L1 to L2 in polynomial time

♦ A2 is a polynomial time algorithm that decides L2

♦ A1 decides whether x∈L1 by using F to transform input x into f(x)

13NTUEEAlgorithms

FFT • it takes more time to compute L1 than L2

• why L1 ≤P L2 ? not L1 ≥P L2 ?

14NTUEEAlgorithms

Example: HC ≤P TSP (step 1)• Define a reduction function f for HC ≤P TSP

• Given an arbitrary HC instance G = (V, E) with n vertices

․ Create a set of n cities labeled with names in V

․ Assign distance between u and v

․ Set bound B = n

․ f can be computed in O(V2) time.

15NTUEEAlgorithms

HC ≤P TSP (Step 2)• G has an HC iff the reduced instance has a TSP with distance ≤ B

♦ If x is a solution to HC , then f(x) must be a solution to TSP

∗ Suppose the HC is x = <v1, v2, …, vn, v1>. Then, x is also a tour in the transformed TSP instance

∗ The distance of the tour h is n = B since there are nconsecutive edges in E, and so has distance 1 in f(x)

∗ Thus, f(x) is a solution to TSP

− f(x) has a TSP tour with distance ≤ B

16NTUEEAlgorithms

HC ≤P TSP (step 2, cont’d)• G has an HC iff the reduced instance has a TSP with distance ≤ B

♦ If f(x) is a solution to TSP, then x must be a solution to HC

∗ Suppose there is a TSP tour with distance ≤ n = B

− Let it be <v1, v2, …, vn, v1>

∗ Since distance of the tour ≤ n and there are n edges in the TSP tour, the tour contains only edges in E

∗ Thus, <v1, v2, …, vn, v1> is a Hamiltonian cycle

17NTUEEAlgorithms

NP-Completeness• A decision problem L is NPC if both conditions are true

1. L ∈ NP

2. L' ≤ P L for every L' ∈ NP

• (Theorem 34.4) Suppose L ∈ NPC

♦ If L ∈ P, then there exists a polynomial-time algorithm for every L' ∈ NP (i.e., P = NP)

♦ If L ∉ P, then there exists no polynomial-time algorithm for any L' ∈ NPC (i.e., P ≠ NP)

• NPC is at crux of deciding whether P = NP

NP-completeproblems

intractabletractable??

A problem is called NP-hard if only condition 2 is true

18NTUEEAlgorithms

Combinational Circuit• Inverter ¬x x’

• AND x∧y xy

• OR x∨y x+y

• XOR x⊕y x’y+xy’

• XNOR x ≡y x↔y xy+x’y’

♦ if and only if

• implication x→y x’+xy

x

19NTUEEAlgorithms

Circuit-SAT• Circuit-Satisfiability Problem (Circuit-SAT):

♦ Given a combinational circuit C of AND, OR, and NOT gates

♦ Is there an assignment of Boolean values to the inputs that makes output of C = 1?

• A circuit is satisfiable if there exists a set of Boolean input values that makes output of the circuit =1

♦ Circuit (a) is satisfiable since x1, x2, x3= 1, 1, 0 makes output =1

♦ how about (b)?

=1?

20NTUEEAlgorithms

Circuit-SAT is NPC• To prove Circuit-SAT is NPC, we need two conditions

1. L ∈ NP

2. L' ≤ P L for every L' ∈ NP

• Condition 1: (Lemma 34.5) Circuit-SAT belongs to NP

♦ given inputs, takes only polynomial time to verify if output is 1

• Condition 2: (Lemma 34.6) Circuit-SAT is NP-hard

♦ Informal proof see textbook

• Based on two lemmas, Circuit Satisfiability is NP-complete

♦ our first NPC problem

♦ will prove other NPC problems based on this one

21NTUEEAlgorithms







22NTUEEAlgorithms

How to Prove NP-Completeness?• Five steps for proving that L is NP-complete:

1. Prove L ∈ NP

2. Select a known NP-complete problem L'

3. Construct a reduction function f that maps every instance of L' to an instance of L

4. Prove that x ∈ L' iff f(x) ∈ L for all input x

5. Prove that f is done in polynomial time

• So far, thousands of problems has been proven NPC

♦ http://en.wikipedia.org/wiki/List_of_NP-complete_problems

23NTUEEAlgorithms

Formula Satisfiability (SAT)• A Boolean formula φ composed of

♦ n Boolean variables x1, x2, x3, ... xn

♦ m Boolean connectives:

∗ AND, OR, NOT, implication, if and only if

♦ parentheses

• A satisfying assignment of φ♦ a set of values for variables that evaluates φ = 1

• Example

♦ satisfying assignment x1=0, x2=0, x3=1, x4=1

243121 )))(()(( xxxxxx ¬∧∨↔¬¬∨→=φ

1

1)01(

1))11(1(

0))1)10(()00((

=∧∨=

∧∨¬∨=¬∧∨↔¬¬∨→=φ

24NTUEEAlgorithms

Prove SAT ∈ NPC (1)• step1: prove SAT ∈ NP

♦ given a satisfying assignment, the formula can be evaluated in polynomial time

• step 2: prove SAT ∈ NP-hard

♦ show that Circuit-SAT ≤ P SAT

♦ Given circuit C , reduction function f maps C to a formula φ∗ AND of output with the conjunction of clauses

− each clause describe the operation of a gate in C

∗ f can be done in polynomial time

444 3444 21clause

98710

769

658

4217

46

215

3410

))((

))((

))((

))((

)(

))((

)(

xxxx

xxx

xxx

xxxx

xx

xxx

xxx

∧∧↔∧∨↔∧∨↔∧

∧∧↔∧¬↔∧

∨↔∧¬↔∧=φ

f

25NTUEEAlgorithms

Prove SAT ∈ NPC (2)• step 2: (cont’d) :

♦ prove if x satisfies C, then f(x) satisfies φ∗ x satisfies C, so every clause =1 in φ∗ the whole formula φ = 1

♦ prove if f(x) satisfies φ, then x satisfies C

∗ f(x) satisfies φ, so every clause =1 in φ∗ circuit output=1, so x satisfies circuit C

• Q.E.D.

444 3444 21clause

98710

769

658

4217

46

215

3410

))((

))((

))((

))((

)(

))((

)(

xxxx

xxx

xxx

xxxx

xx

xxx

xxx

∧∧↔∧∨↔∧∨↔∧

∧∧↔∧¬↔∧

∨↔∧¬↔∧=φ

f

26NTUEEAlgorithms

3-CNF Satisfiability• Conjunctive normal form (CNF) if a Boolean formula is expressed as

♦ AND of clauses

• 3-CNF means each clauses has exactly three distinct literals

♦ literal is an occurrence of a variable or its negation

• Example: 3-CNF, 4 clauses

• Exercise: Prove 3CNF-SAT is NPC

{ { 44 344 21clauseliteralliteral

)xxx()xxx()xxx()xxx( 531431423211 ∨¬∨∧¬∨¬∨¬∧∨∨∧∨¬∨

27NTUEEAlgorithms







28NTUEEAlgorithms

Well-known NPC Problems• Fig 34.13: all proof ultimately follow by reduction from Circuit-SAT

Circuit -SAT

SAT

3-CNF SAT

Clique

Vertex Cover

Subset Sum

Ham. Circuit

TSP (decision)

Set Cover *not in textbook

29NTUEEAlgorithms

Max. Clique Problem• Clique is a subset of vertices such that

♦ each pair of vertices in a clique are connected

♦ clique is a complete subgraph

• Maximum clique = find clique of maximum size in the graph

♦ size of clique = number of vertices in clique

• The maximum clique problem (optimization version)

♦ Given an undirected graph G= (V,E) find maximum clique

• The clique problem (decision version)

♦ Given an undirected graph G= (V,E), and size k

♦ Does G has a clique of size k ?

• Example:

♦ maximum clique is {v0, v1, v2, v4}

♦ size= 4

30NTUEEAlgorithms

• step 1: prove CLIQUE ∈ NP

♦ given a set V’ of k vertices

♦ for each pair of vertices u,v ∈ V’, check if there is an edge (u,v)

∗ can be done in polynomial time

Prove Clique ∈ NPC (1)

31NTUEEAlgorithms

• step 2: prove CLIQUE∈NP-hard by reduction 3CNF-SAT ≤ P CLIQUE

♦ given a 3CNF φ of k clauses: C1∧ C2 ∧ ... ∧ Ck

♦ construct a graph G such that

∗ each clause Cr is represented by a triple of 3 vertices vr1 vr

2 vr3

∗ an edge between vri and vs

j of difference clauses (r ≠ s)

− their literals are consistent

e.g. x3 and x3’ are not consistent

• Fig. 34.14

Prove Clique ∈ NPC (2)

)(

)(

)(

321

321

321

xxx

xxx

xxx

∨∨∧∨∨¬∧

¬∨¬∨=φ

f

fNo edge

32NTUEEAlgorithms

Prove Clique ∈ NPC (3)• step 2: (cont’d) 3CNF-SAT ≤ P CLIQUE

♦ x satisfy φ f(x) is a clique of size k

∗ x satisfy φ so at least on literal =1

∗ correspond to a clique of size k in G

∗ example: x2=0, x3=1

♦ f(x) is a clique of size k x satisfy φ∗ f(x) is a clique so it contains exactly one vertex in every triple

∗ assign 1 to each literal in the clique without conflict

∗ thus, x satisfies φ

)(

)(

)(

321

321

321

xxx

xxx

xxx

∨∨∧∨∨¬∧

¬∨¬∨=φf

33NTUEEAlgorithms

Vertex-cover Problem• A vertex cover of undirected graph G=(V,E) is a subset V’ ⊆ V

♦ if (u,v) ∈ E, then u∈ V’ or v ∈ V’ or both

♦ that means, every edges is covered by at least one vertex

♦ Cover size = number of vertices in V’

• Minimum vertex-cover problem: find a vertex cover of minimum size

• Exercises: prove minimum vertex cover is NPC

♦ show that CLIQUE ≤P VERTEX-COVER

♦ example: Fig 34.15

∗ clique V’ = {u,v,y,x}

∗ Vertex cover of ={w, z}

Gf

G

G

34NTUEEAlgorithms

Subset-sum Problem• Given a finite set of positive integers S, and a target integer t

• Find a subset S’ of S whose elements sum to t

• example:

♦ S={1, 2, 7, 14, 49, 98, 343, 686, 2409, 16808, 17206, 117705, 117993}

♦ t = 138457

♦ S’={1, 2, 7, 98, 343, 686, 2409, 17206, 117705}

• Exercise:

♦ Prove Subset-Sum is NPC

♦ Hint: 3CNF-SAT ≤P SUBSET-SUM

35NTUEEAlgorithms



♦ Approximate Vertex-cover 35.1

♦ Approximate TSP 35.2

♦ Approximate Set-Covering 35.3

36NTUEEAlgorithms

Approximation Algorithms• In practice, near optimal solutions are good enough

♦ approximation algorithms returns near-optimal solutions

♦ often in polynomial time

• Ratio bound, ρ(n)≥ 1

♦ C* is optimal solution; C is our solution

♦ if ρ(n) = constant, we call it ρ(n)-approximation algorithm

• Relative error bound, ε(n) ≤ ρ(n)-1

♦ if ε(n) = constant, we call it (1+ ε) approximation algorithm

)(,max*

*n

C

C

C

C ρ≤⎟⎟⎠

⎞⎜⎜⎝

⎛

)(*

|*|n

C

CC ε≤−

37NTUEEAlgorithms

APPROX-VERTEX-COVER(G)1 C = ∅2 E’= G.E4 while E’ ≠ ∅5 let (u,v) be an arbitrary edge of E’6 C = C∪{u,v}7 remove from E’ every edge incident on either u or v8 return C

Approx. Alg. for Vertex Cover• Complexity O(V+E)

• Example: Fig 35.1

♦ pick edge (b,c)

♦ pick edge (e,f)

♦ pick edge (d,g)

♦ Size = 6

• optimal solution = {b,d,e}

♦ Size = 3

38NTUEEAlgorithms

Ratio Bound = 2• (Theorem 35.1) APPROX-VERTEX-COVER is 2-approximation

algorithm

♦ Proof:

∗ Let A = set of edges picked ; C = set of vertices picked

− |C| = 2|A|

∗ no edges in A share an endpoint, so no two edges in A are covered by the same vertex from C*

− |A|≤|C*|

∗ so |C|≤2|C*|

39NTUEEAlgorithms






40NTUEEAlgorithms

Traveling Salesman Problem• TSP: Given an undirected graph G=(V,E) that has non-negative

integer cost c(u,v) associated with each edge, find a Hamiltonian cycle with minimum cost

• Let A = set of edges chosen

• triangle inequality

∑∈

=Avu

vucAc),(

),()(

),(),(),( wvcvucwuc +≤

41NTUEEAlgorithms

Approx-TSP –Tour with Triangle Inequality

• root= vertex a

• MST

• preorder walk

♦ a b c h d e f g

• approx. solution

♦ cost = 19

• optimal solution

♦ cost = 14.7

G.V

H*H

T W

42NTUEEAlgorithms

Ratio Bound = 2• (Theorem 35.2) APPROX-TSP is a polynomial time 2-approximation

algorithm for TSP with triangle inequality

♦ Proof:

∗ Let T = MST

∗ Let H* be the optimal tour.

∗ H* is formed by some tree plus an edge

− c(T)≤ c(H*)

∗ Let W = a full walk along T

− e.g. a, b, c, b, h, b, a, d, e, f, e, g, e, d, a

∗ since W traverse every edge of T twice

− c(W) ≤ 2 c(T)

∗ H = removed from W all but first visit to each vertex

− e.g. a, b, c, h, d, e, f, g

∗ triangle inequality: removing vertex does not increase cost

− c(H) ≤ c(W)

∗ so, c(H) ≤ 2 c(H*)

W

H

43NTUEEAlgorithms

General TSP without Triangle Inequality (* not in exam)

• (Theorem 35.3) If P≠NP, there is no polynomial time ρ-approximation algorithm for the general TSP problem (ρ≥1)

♦ Proof: by contradiction

∗ Suppose there is a polynomial time ρ-approximation for TSP

∗ Suppose we want to find an HC in graph G =(V,E)

∗ Let G’(V,E’) be a complete graph, where E’ contains all edges

∗ apply the ρ-approximation algorithm for TSP, we are guaranteed to find the HC in polynomial time

− because the other edges are too costly to choose

∗ then we get a polynomial time HC algorithm!

− this is wrong since we already proved HC is NPC

otherwise

),( if

1

1 ),(

Evu

Vvuc

∈

⎩⎨⎧

+=

ρ

V VV 1)-V(1)V( ρρρ >+=++

44NTUEEAlgorithms






45NTUEEAlgorithms

Set-Covering Problem• Given a finite set X, and a family F of subsets of X

• subset S∈ F covers its elements

• Set covering problem: find a minimum number of subset C⊆ F that cover all X

• Optimization problem

♦ minimum size of C

• Decision problem

♦ exist a covering C ≤ size k?

♦ exercise: prove set covering is NPC (Hint: use vertex covering)

• Applications of Set-Covering

♦ find minimum of people (C) that perform all tasks (X)

♦ minimum 2-level combinational circuit design

UFS

S X∈

=

UCS

S X∈

=

46NTUEEAlgorithms

Greedy-Set-Cover• Each iteration chooses the subset that covers most elements

♦ running time O(|X||F| min(|X|,|F|))

• example: |X|=12, |F|=6

♦ greedy solution

∗ {S1, S4, S5, S6} size =4

♦ optimal solution

∗ C={S3, S4, S5} size=3

• (Corollary 35.5) GREEDY-SET-COVER is a polynomial time (ln|X|+1)-approximation algorithm

47NTUEEAlgorithms

SOP Minimization * not in exam

• Two-level AND-OR Combinational Circuit

• Sum of product (SOP)

♦ f = a’c’d + bc’d + bcd’ + acd’

• Disjunctive normal form (DNF)

♦ f=(¬a∧¬c ∧d)∨(b∧¬c ∧d)∨(b∧c ∧ ¬ d)∨(a∧c ∧¬d)

• Very important EDA problem:

♦ Given f, find minimize number of gates

♦ if gates are the same, minimize number of inputs

48NTUEEAlgorithms

Quine-McCluskey Algorithm• Invented in 1956, one of the earliest EDA algorithms

• Two-step process

♦ 1. Generate all Prime Implicants (PI)

∗ PI = product term that cannot be combined with other terms

♦ 2. Select minimum number of PI

∗ set-covering

49NTUEEAlgorithms

• f(abcd)=Σm(4,5,6,8,9,10,13)+d(0,7,15)

♦ at least ONE of the following terms must be covered

∗ a’bc’d’ + a’bc’d + a’bcd’ + ab’c’d’ + ab’c’d + ab’cd’ + abc’d’

∗ called On-set

♦ a’b’c’d’+ab’c’d’+a’b’c’d’ may or may not be covered

∗ called Don’t-care set

♦ None of the others can be covered

∗ called Off-set

Prime Implicant Generation (1/5)

abcd 00 01 11 10

00

01

11

10

0

1

0

1

10

X

0

0

1

1

1

0

X

1

X

a

d

b

c

50NTUEEAlgorithms


zero “1”

one “1”

two “1”

three “1”

four “1”

51NTUEEAlgorithms


√

√√

√

√

√

√

√√

√

- are don’t cares

52NTUEEAlgorithms


√

√

√

√

√

√

√

√√

√

√

√

√

√

√

√

√

7 Prime Implicants:0-00 = a'c'd'100- = ab'c'1-01 = ac'd-1-1 = bd-000 = b'c'd'10-0 = ab'd'01- - = a'b

53NTUEEAlgorithms

Karnaugh Map• PI on Karnaugh map

♦ see logic design textbook

54NTUEEAlgorithms

PI Table

55NTUEEAlgorithms

Essential Rows and Distinguished Column

essential rows

distinguished columns

56NTUEEAlgorithms

Select Essential Rows

57NTUEEAlgorithms

Final Result

58NTUEEAlgorithms

FFT• Can Quine-McCluskey find optimal solution for all cases?

♦ obviously not ... set-covering is NPC

• Give an example where PI table has neither essential rows nor distinguished columns

♦ Quine-McCluskey has trouble finding optimal solution

59NTUEEAlgorithms

Coping with NPC, NP-hard problems• Approximation algorithms:

♦ Guarantee to be a fixed percentage away from the optimum.

• Exponential algorithms/Branch and Bound/Exhaustive search:

♦ Feasible only when the problem size is small.

• Local search:

♦ Simulated annealing (hill climbing), genetic algorithms, etc.

• Randomized algorithms:

♦ Make use of a randomizer (random # generator) for operation.

• Pseudo-polynomial time algorithms:

♦ E.g., dynamic programming for the 0-1 Knapsack problem.

• Probabilistic algorithms:

♦ Assume some probabilistic distribution of the instances.

• Restriction: Work on some special cases of the original problem.

♦ E.g., the maximum independent set problem in circle graphs.

• Heuristics: No formal guarantee of performance.

60NTUEEAlgorithms

Reading• CH 34

• CH35

• REF: Roth, “Fundamentals of Logic Design” Cengage Learning, CH6

np-completeness and approximation algorithmsb97032/ln11_npc.pdf · ♦ np-complete problems •...

Documents