1

Dynamic ProgrammingComputing Binomial Coefficient

and Longest Common Subsequence

Dr. Ying Luylu@cse.unl.edu

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms

2

Giving credit where credit is due:Giving credit where credit is due:• Most of the lecture notes are based on the slides from Most of the lecture notes are based on the slides from

the Textbook’s companion websitethe Textbook’s companion website– http://www.aw-bc.com/info/levitinhttp://www.aw-bc.com/info/levitin

• Some of the notes are based on slides by Dr. Tao Jiang Some of the notes are based on slides by Dr. Tao Jiang at University of California - Riversideat University of California - Riverside

• I have modified many of their slides and added new I have modified many of their slides and added new slidesslides

RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms

3

Dynamic programmingDynamic programming

• Dynamic Programming Dynamic Programming is a general algorithm design is a general algorithm design techniquetechnique

• Invented by American mathematician Richard Bellman in the Invented by American mathematician Richard Bellman in the 1950s to solve optimization problems1950s to solve optimization problems

• “ “Programming” here means “planning”Programming” here means “planning”

• Main idea:Main idea:• set up a recurrence relating a solution to a larger instance set up a recurrence relating a solution to a larger instance

to solutions of some smaller instancesto solutions of some smaller instances• solve smaller instances oncesolve smaller instances once• record solutions in a table record solutions in a table • extract solution to the initial instance from that tableextract solution to the initial instance from that table

4

Example: Fibonacci numbersExample: Fibonacci numbers

• Recall definition of Fibonacci numbers:

f(0) = 0f(1) = 1f(n) = f(n-1) + f(n-2)

• Computing the nth Fibonacci number recursively (top-down):

f(n)

f(n-1) + f(n-2)

f(n-2) + f(n-3) f(n-3) + f(n-4)

...

5

Example: Fibonacci numbersExample: Fibonacci numbers

Computing the nth Fibonacci number using bottom-up iteration:

• f(0) = 0• f(1) = 1• f(2) = 0+1 = 1• f(3) = 1+1 = 2• f(4) = 1+2 = 3• f(5) = 2+3 = 5• • • • • f(n-2) = • f(n-1) = • f(n) = f(n-1) + f(n-2)

6

Examples of dynamic Examples of dynamic programming algorithmsprogramming algorithms

• Coin-row problemCoin-row problem

• Computing binomial coefficients Computing binomial coefficients

• Longest Common Subsequence (LCS) Longest Common Subsequence (LCS)

•Warshall’s algorithm for transitive closureWarshall’s algorithm for transitive closure

• Floyd’s algorithm for all-pairs shortest pathsFloyd’s algorithm for all-pairs shortest paths

•Some instances of difficult discrete optimization problems:Some instances of difficult discrete optimization problems:•0-1 knapsack0-1 knapsack

7

Coin-row problemCoin-row problem

There is a row of There is a row of nn coins whose values are some positive integers coins whose values are some positive integers c₁, c₂,...,cc₁, c₂,...,cnn, , not necessarily distinct. The goal is to pick up the not necessarily distinct. The goal is to pick up the

maximum amount of money subject to the constraint that no two maximum amount of money subject to the constraint that no two coins adjacent in the initial row can be picked up.coins adjacent in the initial row can be picked up.

E.g.: 5, 1, 2, 10, 6, 2, E.g.: 5, 1, 2, 10, 6, 2,

the best selection is 5, 10, 2, giving the maximum amount of 17. the best selection is 5, 10, 2, giving the maximum amount of 17.

8

DP solution to the coin-row problemDP solution to the coin-row problem

Let F(Let F(nn) be the maximum amount that can be picked up from the ) be the maximum amount that can be picked up from the row of n coins. To derive a recurrence for F(row of n coins. To derive a recurrence for F(nn), we partition all ), we partition all the allowed coin selections into two groups:the allowed coin selections into two groups:

those without last coin – the max amount is ?those without last coin – the max amount is ? those with the last coin -- the max amount is ?those with the last coin -- the max amount is ?

9

DP solution to the coin-row problemDP solution to the coin-row problem

Let F(Let F(nn) be the maximum amount that can be picked up from the ) be the maximum amount that can be picked up from the row of n coins. To derive a recurrence for F(row of n coins. To derive a recurrence for F(nn), we partition all ), we partition all the allowed coin selections into two groups:the allowed coin selections into two groups:

those without last coin – the max amount is ?those without last coin – the max amount is ? those with the last coin -- the max amount is ?those with the last coin -- the max amount is ?

Thus we have the following recurrence Thus we have the following recurrence

F(F(nn) = max{c) = max{cn n + F(+ F(nn-2), F(-2), F(nn-1)} for -1)} for n n > 1, > 1,

F(0) = 0, F(1)=c₁F(0) = 0, F(1)=c₁

10

DP solution to the coin-row problem DP solution to the coin-row problem (cont.)(cont.)

indexindex 00 11 22 33 44 55 66

coinscoins ---- 55 11 22 1010 66 22

F( )F( )

F(F(nn) = max{c) = max{cnn + F(+ F(nn-2), F(-2), F(nn-1)} for -1)} for n n > 1, > 1,

F(0) = 0, F(1)=c₁F(0) = 0, F(1)=c₁

Max amount:Coins of optimal solution:Time efficiency:Space efficiency:

Note: All smaller instances were solved.

Computing a binomial coefficient Computing a binomial coefficient by DPby DP

Binomial coefficients are coefficients of the binomial formula:Binomial coefficients are coefficients of the binomial formula:((a + ba + b))nn = = CC((nn,0),0)aannbb0 0 + . . . + + . . . + CC((nn,,kk))aan-kn-kbbkk + . . . + . . . + CC((nn,,nn))aa00bbnn

C(n, k), the number of combinations of k elements from an n-C(n, k), the number of combinations of k elements from an n-element set (0 element set (0 k k n) n)

Recurrence: Recurrence: CC((nn,,kk) = ?) = ?

Computing a binomial coefficient Computing a binomial coefficient by DPby DP

Binomial coefficients are coefficients of the binomial formula:Binomial coefficients are coefficients of the binomial formula:((a + ba + b))nn = = CC((nn,0),0)aannbb0 0 + . . . + + . . . + CC((nn,,kk))aan-kn-kbbkk + . . . + . . . + CC((nn,,nn))aa00bbnn

C(n, k), the number of combinations of k elements from an n-C(n, k), the number of combinations of k elements from an n-element set (0 element set (0 k k n) n)

Recurrence: Recurrence: CC((nn,,kk) = ?) = ?

those without last element – the number of such combinations is ?those without last element – the number of such combinations is ? those with the last element -- the number of such combinations is ?those with the last element -- the number of such combinations is ?

Computing a binomial coefficient Computing a binomial coefficient by DPby DP

Binomial coefficients are coefficients of the binomial formula:Binomial coefficients are coefficients of the binomial formula:((a + ba + b))nn = = CC((nn,0),0)aannbb0 0 + . . . + + . . . + CC((nn,,kk))aan-kn-kbbkk + . . . + . . . + CC((nn,,nn))aa00bbnn

Recurrence: Recurrence: CC((nn,,kk) = ) = CC((n-n-1,1,kk) + ) + CC((nn-1,-1,kk-1) for -1) for n > k n > k > 0> 0 CC((nn,0) = 1, ,0) = 1, CC((nn,,nn) = 1 for ) = 1 for n n 0 0 Value of Value of CC((nn,,kk) can be computed by filling a table:) can be computed by filling a table:

0 1 2 . . . 0 1 2 . . . kk-1 -1 kk 0 10 1 1 1 11 1 1 .. .. .. n-n-1 1 CC((n-n-1,1,kk-1) -1) CC((n-n-1,1,kk) ) nn C C((nn,,kk) )

Computing Computing CC((n,kn,k): pseudocode and ): pseudocode and analysisanalysis

Time efficiency: Time efficiency: ??

Space efficiency: Space efficiency: ??

Computing Computing CC((n,kn,k): pseudocode and ): pseudocode and analysisanalysis

Time efficiency: Time efficiency: ΘΘ((nknk))

Space efficiency: Space efficiency: ΘΘ((nknk))

In-class exerciseIn-class exercise

Compute C(6, 3) by applying the dynamic Compute C(6, 3) by applying the dynamic programming algorithmprogramming algorithm

Design and Analysis of Algorithms - Chapter 8 16

17

In-Class ExerciseIn-Class Exercise

Several coins are placed in cells of an Several coins are placed in cells of an nn××mm board. A robot, board. A robot, located in the upper left cell of the board, needs to collect as located in the upper left cell of the board, needs to collect as many of the coins as possible and bring them to the bottom right many of the coins as possible and bring them to the bottom right cell. On each step, the robot can move either one cell to the right cell. On each step, the robot can move either one cell to the right or one cell down from its current location. or one cell down from its current location.

1 2 3 4 5 6

1

2

3

4

5

Longest Common Subsequence Longest Common Subsequence (LCS)(LCS)

A subsequence of a sequence/string A subsequence of a sequence/string S S is obtained by is obtained by deleting zero or more symbols from deleting zero or more symbols from SS. For example, . For example, the following are the following are somesome subsequences of “president”: subsequences of “president”: pred, sdn, predent. In other words, the letters of a pred, sdn, predent. In other words, the letters of a subsequence of S appear in order insubsequence of S appear in order in S S, but they are not , but they are not required to be consecutive.required to be consecutive.

The longest common subsequence problem is to find a The longest common subsequence problem is to find a maximum length common subsequence between two maximum length common subsequence between two sequences.sequences.

LCSLCS

For instance,For instance,

Sequence 1: presidentSequence 1: president

Sequence 2: providenceSequence 2: providence

Its LCS is priden.Its LCS is priden.

president

providence

LCSLCS

Another example:Another example:

Sequence 1: algorithmSequence 1: algorithm

Sequence 2: alignmentSequence 2: alignment

One of its LCS is algm.One of its LCS is algm.

a l g o r i t h m

a l i g n m e n t

How to compute LCS?How to compute LCS?

Let ALet A=a=a11aa22…a…am m and and B=bB=b11bb22…b…bnn . .

lenlen((i, ji, j): the length of an LCS between ): the length of an LCS between aa11aa22…a…ai i and and bb11bb22…b…bjj

How to compute LCS?How to compute LCS?

Let ALet A=a=a11aa22…a…am m and and B=bB=b11bb22…b…bnn . .

lenlen((i, ji, j): the length of an LCS between ): the length of an LCS between aa11aa22…a…ai i and and bb11bb22…b…bjj

With proper initializations, With proper initializations, lenlen((i, ji, j) can be computed as ) can be computed as followsfollows

,

. and 0, if)),1(),1,(max(

and 0, if1)1,1(

,0or 0 if0

),(

ji

ji

bajijilenjilen

bajijilen

ji

jilen

How to compute LCS?How to compute LCS?

Let ALet A=a=a11aa22…a…am m and and B=bB=b11bb22…b…bnn . .

lenlen((i, ji, j): the length of an LCS between ): the length of an LCS between aa11aa22…a…ai i and and bb11bb22…b…bjj

With proper initializations, With proper initializations, lenlen((i, ji, j) can be computed as ) can be computed as followsfollows

What is the corresponding LCS?What is the corresponding LCS?

,

. and 0, if)),1(),1,(max(

and 0, if1)1,1(

,0or 0 if0

),(

ji

ji

bajijilenjilen

bajijilen

ji

jilen

p r o c e d u r e L C S - L e n g t h ( A , B )

1 . f o r i ← 0 t o m d o l e n ( i , 0 ) = 0

2 . f o r j ← 1 t o n d o l e n ( 0 , j ) = 0

3 . f o r i ← 1 t o m d o

4 . f o r j ← 1 t o n d o

5 . i f ji ba t h e n

" "),(

1)1,1(),(

jiprev

jilenjilen

6 . e l s e i f )1,(),1( jilenjilen

7 . t h e n

" "),(

),1(),(

jiprev

jilenjilen

8 . e l s e

" "),(

)1,(),(

jiprev

jilenjilen

9 . r e t u r n l e n a n d p r e v

i j 0 1 p

2 r

3 o

4 v

5 i

6 d

7 e

8 n

9 c

10 e

0 0 0 0 0 0 0 0 0 0 0 0

1 p 2

0

2 r 0

3 e 0

4 s 0

5 i 0

6 d 0

7 e 0

8 n 0

9 t 0

Running time and memory: O(mn) and O(mn).

i j 0 1 p

2 r

3 o

4 v

5 i

6 d

7 e

8 n

9 c

10 e

0 0 0 0 0 0 0 0 0 0 0 0

1 p 2

0 1 1 1 1 1 1 1 1 1 1

2 r 0 1 2 2 2 2 2 2 2 2 2

3 e 0 1 2 2 2 2 2 3 3 3 3

4 s 0 1 2 2 2 2 2 3 3 3 3

5 i 0 1 2 2 2 3 3 3 3 3 3

6 d 0 1 2 2 2 3 4 4 4 4 4

7 e 0 1 2 2 2 3 4 5 5 5 5

8 n 0 1 2 2 2 3 4 5 6 6 6

9 t 0 1 2 2 2 3 4 5 6 6 6

Running time and memory: O(mn) and O(mn).

p r o c e d u r e O u tp u t - L C S ( A , p r e v , i , j )

1 i f i = 0 o r j = 0 t h e n r e t u r n

2 i f p r e v ( i , j ) = ” “ t h e n

ia

jiprevALCSOutput

print

)1,1,,(

3 e l s e i f p r e v ( i , j ) = ” “ t h e n O u tp u t - L C S ( A , p r e v , i - 1 , j )

4 e l s e O u tp u t - L C S ( A , p r e v , i , j - 1 )

The backtracing The backtracing algorithmalgorithm

i j 0 1 p

2 r

3 o

4 v

5 i

6 d

7 e

8 n

9 c

10 e

0 0 0 0 0 0 0 0 0 0 0 0

1 p 2

0 1 1 1 1 1 1 1 1 1 1

2 r 0 1 2 2 2 2 2 2 2 2 2

3 e 0 1 2 2 2 2 2 3 3 3 3

4 s 0 1 2 2 2 2 2 3 3 3 3

5 i 0 1 2 2 2 3 3 3 3 3 3

6 d 0 1 2 2 2 3 4 4 4 4 4

7 e 0 1 2 2 2 3 4 5 5 5 5

8 n 0 1 2 2 2 3 4 5 6 6 6

9 t 0 1 2 2 2 3 4 5 6 6 6

Output: priden

Design and Analysis of Algorithms - Chapter 8 29

In-class exerciseIn-class exercise