CSC 3130: Automata theory and formal languages

NP-complete problems

Fall 2008MELJUN P. CORTES, MELJUN P. CORTES, MBA,MPA,BSCS,ACSMBA,MPA,BSCS,ACS

MELJUN CORTESMELJUN CORTES

Polynomial-time reductions

• Language L polynomial-time reduces to L’ if

there exists a polynomial-time computable map R that takes an instance x of L into instance y of L’ s.t.

x ∈ L if and only if y ∈ L’

L L’(IS) (CLIQUE)

x y(G, k) (G’, k’)

x ∈ L y ∈ L’(G has IS of size k) (G’ has clique of size k’)

R

The Cook-Levin Theorem

Every L ∈ NP reduces to SAT

SAT = {f: f is a satisfiable Boolean formula}

(x1∨x2 ) ∧ (x2 ∨x3 ∨x4) ∧ (x1)f =

is satisfiablex1 = T x2 = F x3 = T x4 = T

(x1∨x2 ) ∧ (x1) ∧ (x2)f =

is not satisfiable

NP-hardness

• Language L is NP-hard if every L’ in NP reduces to L

• Intuitively, NP-hard means “harder than all of NP”

• The Cook-Levin Theorem says

SAT is NP-hard

P

SAT

NP

NP-complete

• L is NP-complete if L is NP-hard and L ∈ NP

• Intuitively, NP-complete means “hardest in NP”

• Recall that SAT ∈ NP, so SAT is NP-complete

If SAT ∈ P, then P = NP

P

SATNP

Proof of Cook-Levin Theorem

• To prove it, we have to describe a reduction R:

Every L ∈ NP reduces to SAT

w Boolean formula f

w ∈ L f is satisfiable

R

Proof of Cook-Levin Theorem

• All we know about L: It has a poly-time NTM M– Let’s look at computation tableau of M on input w

Mw

w1 w2q0

qacc …

S-th configuration symbol at time T

S

T

Since M is nondeterministic,there may be many possible tableaus

Proof of Cook-Levin Theorem

w1 w2q0

qacc …

S

T

n = length of input w

height of tableau is p(n) for some polynomial p

width is at most p(n)

k possible tableau symbols

xT, S, u = true, if the (T, S) cell of

tableau contains uif notfalse,

u

1 ≤ S ≤ p(n)

1 ≤ T ≤ p(n)

1 ≤ u ≤ k

Proof of Cook-Levin Theorem

• We will design a formula f such that:

w Boolean formula f

w ∈ L f is satisfiable

R

variables of f :

assignment to xT, S, u way to fill up the tableau

satisfying assignment accepting computation tableau

f is satisfiable

xT, S, u

M accepts w

Proof of Cook-Levin Theorem

• We want to construct (in time poly(n)) a formula f :

xT, S, u = true, if the (T, S) cell of

tableau contains uif notfalse,

1 ≤ S ≤ p(n)

1 ≤ T ≤ p(n)

1 ≤ u ≤ k

f(x1, 1, 1, ..., xp(n), p(n), k) = true, if the xs represent a valid

accepting tableauif notfalse,

Proof of Cook-Levin Theorem

w1 w2q0

qacc …

S

T

u

f = fcell ∧ f0 ∧ fmove ∧ facc

fcell: “Every cell contains exactly one symbol”

“The first row is q0w1w2...wk☐...☐”

f0:

fmove: “The moves between rows follow the transitions of M”

facc: “qacc appears somewhere in the last row”

Proof of Cook-Levin Theorem

• Desired meaning

• Implementation:

fcell: “Every cell contains exactly one symbol”

fcell = fcell 1, 1 ... ∧ ∧ fcell p(n), p(n) where

fcell T, S = (xT, S, 1 ... ∨ ∨ xT, S, k) ∧ (xT, S, 1 ∨ xT, S, 2) ∧ (xT, S, 1 ∨ xT, S, 3) ∧ ... (∧ xT, S, k-1 ∨ xT, S, k)

at least one symbol

no two symbols

or: “Exactly one of xS, T, 1 ... ∨ ∨ xS, T, k is true”

Proof of Cook-Levin Theorem

• Desired meaning

• Implementation:

f0: “The first row is q0w1w2...wk☐...☐”facc: “qacc appears somewhere in the last row”

f0 = x1, 1, q0 ∧ x1, 1, w1 ∧ x1, 1, w2 ... ∧ ∧ x1, p(n),☐

facc = xp(n), 1, qacc ∨ xp(n), 2, qacc ... ∨ ∨ xp(n), p(n), qacc

Valid and invalid windows

… 6c3a0t0 …… 0c6a0p0 …0

… 6t3t0u0 …… 0t6t0u0 …0

valid window

invalid window

… 6t3t0u0 …… 0t6t0q3 …0

valid window

… 6t3q3u0 …… 0t6a0q7 …0

valid if δ(q3, u) = (q7, a, R)

… 6q3t0u0 …… 0k6t0q0 …0

invalid window

… 6c3a0t0 …… 0b6a0t0 …0

valid window

Proof of Cook-Levin Theorem

• Desired meaning

• Implementation:

fmove: “The moves between rows follow transitions of M”

a aq0

qacc …fmove = fmove 1, 1 ... ∧ ∧ fmove p(n)-3, p(n)-3

b

q3b a b

b q7c b

fmove 2, 2

fmove T, S =over all

(xT, S, a1 ∧ xT, S+1, a2 ∧ xT, S+2, a3

∧ xT+1, S, a4 ∧ xT+1, S+1, a5 ∧ xT+2, S+1, a6)∨

valid windowsa1 a2 a3a4 a5 a6

Other NP-complete problems

CLIQUE = {(G, k): G is a graph with a clique of k vertices}

IS = {(G, k): G is a graph with an independent set of k vertices}

VC = {(G, k): G is a graph with a vertex cover of k vertices}

CLIQUE, IS and VC are NP-complete

SAT

NP

CLIQUE IS VC

Proving NP-hardness

• To show L is NP-hard, it is enough to reduce from some L’ we already know is NP-hard– For now we can take L’ = SAT

• To show L is NP-complete, we also need to argue that L is in NP– This is usually the easy part

roadmap:

SAT

3SAT

IS

CLIQUE VC

3SAT

SAT = {f: f is a satisfiable Boolean formula}

3SAT = {f: f is a satisfiable Boolean formula in conjunctive normal form with at most 3 distinct literals per clause}

CNF: AND of ORs of literals

literal: xi or xi

(conjunctive normal form)

3CNF: CNF with ≤3 lit/clause

(x1∨x2 ) ∧ (x2 ∨x3 ∨x4) ∧ (x1)

clauseliterals

(x2∨(x1∧x2 ))∧(x1∧(x1∨x2 ))gates

NP-hardness of 3SAT

• Theorem

• Proof: We describe a reduction R from SAT

Boolean formula f 3CNF formula f’

f’ is satisfiable

R

f is satisfiable

3SAT is NP-hard

Reducing SAT to 3SAT

• Example: f = (x2∨(x1∧x2 ))∧(x1∧(x1∨x2 ))

We give extra variables to every gate (“wire”)

x3

x6 x7

x8 x9

x4 x5

x10

AND

OR NOT

AND

NOT OR

AND

NOT

x2 x1 x2 x1 x1 x2

x7 = x4 ∧ x5x4 x5 x7

T T T TT T F FT F T FT F F TF T T FF T F TF F T FF F F T

(x4∨x5∨x7)(x4∨x5∨x7)

(x4∨x5∨x7)

(x4∨x5∨x7)

(x4∨x5∨x7)∧(x4∨x5∨x7)∧(x4∨x5∨x7)∧(x4∨x5∨x7)

Reducing SAT to 3SAT

• Step 1: Add variable xn+j for each gate Gj in f

• Step 2: Write 3CNF fj for each gate Gj, j = {1, ..., t}

• Step 3: Output

Boolean formula f 3CNF formula f’R

f’ = fn+1 ∧ fn+2 ... ∧ ∧ ft ∧ xn+t

requires thatoutput of f is true

Reducing SAT to 3SAT

• Every satisfying assignment of f extends uniquely to a satisfying assignment of f’

• Conversely, every satisfying assignment of f’ must contain a satisfying assignment of f

Boolean formula f 3CNF formula f’R

f’ is satisfiablef is satisfiable

Independent set

• Theorem

✓

IS = {(G, k): G is a graph with an independent set of k vertices}

1 2

3 4

An independent set is a subset of vertices so that no pair is connected

{1, 2}, {1, 3}, {4} are independent sets

IS is NP-hard

SAT

3SAT

IS

CLIQUE VC

Reducing 3SAT to IS

• Proof: We describe a reduction from 3SAT to IS

3SAT = {f: f is a satisfiable Boolean formula in 3CNF}

IS = {(G, k): G is a graph with an independent set of k vertices}

3CNF formula f (G, k)

G has an independentset of size k

R

f is satisfiable

Reducing 3SAT to IS

• Example: f = (x1∨x2 ) ∧ (x2 ∨x3 ∨x4) ∧ (x1)

TT

TF

FT

FF

TTT

TTF

TFT

TFF

FTT

FTF

FFT

FFF

T

F

Put an edge for every inconsistency

all interconnected

x2x3x4

x1x2

x1

etc.

Reducing 3SAT to IS

• Example: f = (x1∨x2 ) ∧ (x2 ∨x3 ∨x4) ∧ (x1)

TT

TF

FF

TTT

TTF

TFT

TFF

FTT

FFT

FFF

T

x2x3x4

x1x2

x1

G

satisfying assignment of f

IS of size 3 in G

x1 = T x2 = F x3 = T x4 = T

edges = inconsistencies

any IS of size 3 in G

satisfying assignment of f

Reducing 3SAT to IS

• G has a vertex for every clause Ci of f and every satisfying assignment of Ci

• G has an edge between any two vertices that represent inconsistent assignments

• k is the number of clauses in f

3CNF formula f (G, k)R

Reducing 3SAT to IS

• Every satisfying assignment of f gives an independent set of size k in G

• Conversely, from every IS of size k in G we can “extract” a consistent and satisfying assignment of f

3CNF formula f (G, k)

G has an IS of size k

R

f is satisfiable

Vertex cover

✓SAT

3SAT

IS

CLIQUE VC

✓✓

• Theorem

VC is NP-hard

A vertex cover is a set of vertices that touches (covers) all edges

{2, 4}, {3, 4}, {1, 2, 3} are vertex covers

1 2

3 4

VC = {(G, k): G is a graph with a vertex cover of size k}

Reducing IS to VC

• Proof: We describe a reduction from IS to VC

• Example

(G’, k’)

G’ has a VC of size k’

R(G, k)

G has an IS of size k

1 2

3 4

∅, {1}, {2}, {3}, {4},{1, 3}, {2, 3}

{2, 4}, {3, 4}, {1, 2, 3}, {1, 2, 4},{1, 3, 4}, {2, 3, 4},{1, 2, 3, 4}

independent setsvertex covers

Reducing IS to VC

• Claim

• Proof

S is an independent set of G if and only if S is a vertex cover of G

S is an independent set of G

no edge has both endpoints in S

every edge has an endpoint in S

S is a vertex cover of G

Reducing IS to VC

• Set G’ = G, k’ = n – k (n = number of vertices)

by previous Claim.

(G’, k’)R(G, k)

G’ has a VC of size k’G has an IS of size k

The ubiquity of NP-complete problems

• We saw a few examples of NP-complete problems, but there are many more

• A surprising fact of life is that most CS problems are either in P or NP-complete

• A 1979 book by Garey and Johnsonlists 100+ NP-complete problems