Synchronous Context-Free Grammars

and Optimal Linear Parsing Strategies

Daniel Gildea Giorgio SattaUniversity of Rochester Università di Padova

Synchronous CFG

Context-free Grammar:

X → A B

Synchronous Context-free Grammar (SCFG)

X →A1

B2

C3

D4 , C

3A

1D

4B

2

C →Powell, 鲍威尔

Synchronous CFG

• Synchronous parsing: find tree from two strings

– used to learn grammar from parallel text

• This talk: parsing strategies for long rules

• Results also apply to translation with n-gram

language model

Context-Free Grammar

A → B C

B

C

A

Binary SCFG

A → B1

C2 , C

2B

1

B

C

A

SCFG with 4 nonterminals

A → B1

C2

D3

E4 , C

2E

4B

1D

3

E

D

C

B

A

Fan-Out

Number of spans in nonterminal.

CFG: fan-out 1 B

C

A

SCFG: fan-out 2 E

D

C

B

A

ϕ(G) = maxN∈G

ϕ(N) (Rambow & Satta, 1999)

Rank

Number of nonterminals on righthand side of rule.

CFG: rank 2 B

C

A

SCFG: rank r E

D

C

B

A

ρ(G) = maxP∈G

ρ(P)

Parsing Strategies

Reduce rankE

D

C

B

A

A → B C D E

C

B

X

D

X

Y

E

Y

A

X → B C Y → X D A → Y E

Parsing Strategies

Reduce rank, may increase fan-out

E

D

C

B

A

C

B

X

Rule Length in Synchronous CFG

• Binary grammar (ITG): parsing is O(n6) (Wu, 1997)

– Works in real MT (Zhang et al. 2006)

• Many rules cannot be binarized without

increasing fan-out (Aho and Ullman, 1972)

• Fan-out affects space and time complexity

Parsing Complexity

Space complexity: O(n2ϕ(A))

Time complexity: O(nϕ(A)+ϕ(B)+ϕ(C))

B

C

A

B

C

A

O(n2) space O(n4) space

O(n3) time O(n6) time

(Seki et al. 1991)

SCFG Parsing Strategies

E

D

C

B

A

C

B

X

naïve strategy: O(n2r+2) time

best strategy: Ω(ncr ) for some c

(Gildea and Štefankovic 2007)

This Talk

• Finding optimal space complexity is

NP-complete

• Finding optimal time complexity ⇒ better algs

for treewidth

Example Rule

B8

B7

B6

B5

B4

B3

B2

B1

A

Optimal Parsing Strategy

n7

n5

B1

n3

B2

n1

B3

B4

n6

B5

n4

B6

n2

B7

B8

B4

B3

n1

Carving Width

2 3 4

1

G

1 2 3 4

tree layout of G

Carving width: max number edges of G routed

through tree layout

Cyclic Permutation Multigraph

B1

B2

B3

B4

B5

B6

B7

B8A

A → B1B

2B

3B

4B

5B

6B

7B

8 ,

B5B

7B

3B

1B

8B

6B

2B

4

Carving Width = Space Complexity

A

n7

n5

n3

n1

n6

n4

n2

B1

B2

B3

B4

B5

B6

B7

B8

Our Reduction

• Carving width instance: (G, k )

• Construct permutation multigraph G′, integer k ′

• Carving width of G ⇔ Carving width of G′⇔

optimal parsing for SCFG

Our Construction

2 3 4

1

G

1 2 3 4

tree layout of G

X1

G1

X2

G2

X3

G3

X4

G4

G1

X1

G2

X2

G3

X3

G4

X4

Space Complexity

Theorem 1: Finding the parsing strategy with optimal

space complexity for an SCFG rule is NP-complete

Treewidth

A C E G I K M

B D F H J L

N

P

R

O

Q

S

CDE DEF EFG FGH GHI HIJ IJK

BCD GHN JK L

ABC HNO K LM

NOP

OPQ PQR QRS

Dependency Graph

x0 x1 x2 x3 x4

y0 y1 y2 y3 y4

x0 x1 x2 x3 x4

A → B C D E S → A1B

2C

3D

4 , B2

D4

A1

C3

Treewidth = Time Complexity

x0 x1 x2 x3 x4

x0x1x2 x0x2x3 x0x3x4

A → B C D E

C

B

X

D

X

Y

E

Y

A

X → B C Y → X D A → Y E

Our Reduction

• Treewidth instance: (G, k )

• Construct dependency graph G′, integer k ′

• Approx of treewidth of G ⇔ Treewidth of G′⇔

optimal time complexity for SCFG

Dependency Graph Construction

Approximation Algorithm for Treewidth

SOL < 8∆(G)(OPT + 1) .

SOL: solution using SCFG parsing strategy

OPT : optimal treewidth of input graph G

∆(G) = degree (max num edges touching one vertex)

Time Complexity

Theorem 2: Finding the parsing strategy with optimal

time complexity for an SCFG rule implies a

∆(G)-factor approximation algorithm for treewidth.

Time Complexity

Theorem 3: If finding the parsing strategy with

optimal time complexity for an SCFG rule is

NP-complete, then treewidth for graphs of degree 6 is

NP-complete.

Conclusion

• Finding parsing strategy with best space

complexity is NP-hard.

• P-time alg for finding parsing strategy with best

time complexity implies better approximation

algs for treewidth

• NP-hardness for time complexity implies

NP-hardness for treewidth of graphs of degree

six