OR-1 2011 1

Chapter 5. The Duality Theorem

Given an LP, we define another LP derived from the same data, but with different structure. This LP is called the dual problem ( 쌍대문제 ).

The main purpose to consider dual is to obtain an upper bound (estimate) on the optimal objective value of the given LP without solving it to optimality. Also dual problem provides optimality conditions of a solution x* for a LP and help to understand the behavior of the simplex method.

Very important concept to understand the properties of the LP and the simplex method.

OR-1 2011 2

Approach to find optimal value of an opt. problem (max form):

Find lower bound and upper bound so that for optimal value , we have

Lower bound : usually obtained from a feasible solution.

If x* is a feasible solution to LP, c’x* provides a lower bound.Upper bound : usually obtained by solving a relaxation of the problem or by finding a feasible solution to a dual problem.

ex) linear programming relaxation of an integer program:

Given IP, max c’x, Ax b, x 0 and integer, consider the linear programming relaxation, max c’x, Ax b, x 0.

Let z* be the optimal value of IP and z’ be the optimal value of LP relaxation. Then z* z’. ( Let x* be the optimal solution of IP. Then x* is a feasible solution of LP relaxation. So there may exist a feasible solution to LP which provides better objective value)

z z*z zzz *

OR-1 2011 3

(continued)

Dual of the LP problem : Let y be a feasible solution to the dual problem. Then the dual objective value provided by y gives an upper bound on the optimal LP objective value. (study in Chapter 5.)

If the lower and upper bound are the same, we know that z* is optimal value. We may need to find the optimal solution additionally, but the optimal value is found.

Although the lower and upper bound may not be the same, from the gap (= upper bound – lower bound), we can estimate the quality of the solution we have.

OR-1 2011 4

Taking nonnegative linear combination of inequality constraints: Consider two constraints and …. (1’)

In vector notation :

If we multiply scalars y1 0 to the 1st constraint and y2 0 to the 2nd

constraint and add the l.h.s. and r.h.s. respectively,

we get ….. (2’)

In vector notation,

Any vector that satisfies (1’) also satisfies (2’), but converse is not true.

Moreover, the coefficient vector in l.h.s. of (2’) is obtained by taking the nonnegative linear combination of the coefficient vectors in (1’)

32 21 xx 42 21 xx

1

2,

2

1,4',3' 2121 aaxaxa where

)43()2()2( 21221121 yyxyyxyy )43()''( 21

22

11 yyxayay

OR-1 2011 5

x1

x2

(0, 0)

x1+2x23

2x1+x24y1(1, 2)+y2(2, 1), y1, y2 0

(1, 2)

(2, 1)

(5/3, 2/3)(y1a1’+y2a2’)x (3y1+4y2)

OR-1 2011 6

ex)

Lower bound : consider feasible solution (0, 0, 1, 0) z* 5

(3, 0, 2, 0) z* 22

Upper bound : consider inequality obtained by multiplying 0 to the 1st , 1 to the 2nd, and 1 to the 3rd constraints and add the l.h.s. and r.h.s. respectively

…… (1)

3532

55835

13..

354max

4321

4321

4321

4321

xxxx

xxxx

xxxxts

xxxx

0,,, 4321 xxxx

583634 4321 xxxx

OR-1 2011 7

Since we multiplied nonnegative numbers, any vector that satisfies the three constraints (including feasible solutions to the LP) also satisfies (1). Hence any feasible solution to LP, which satisfies the three constraints and nonnegativity, also satisfies (1).

Note that all the points satisfying 4x1 + 3x2 + 6x3 + 3x4 58 do not

necessarily satisfy the three constraints in LP.

Further, any feasible solution to the LP must satisfy

( 58)

since any feasible solution must satisfy nonnegativity constraints on the variables and the coefficients in the second expression is greater than or equal to the corresponding coefficients in the first expression.

So 58 is an upper bound on the optimal value of the LP.

(1) ....... 583634 4321 xxxx

3634354 43214321 xxxxxxxx

OR-1 2011 8

Now, we may use nonnegative weights yi for each constraint.

In vector notation,

Objective function of the LP is

321

4321332123211321

355

)583()33()2()5(

yyy

xyyyxyyyxyyyxyyy

)355(532,1(831,5(3,11,1({ 321321 yyyxyyy )}' , , ) , , ) ,

4321 354 xxxx

3532

55835

13..

354max

4321

4321

4321

4321

xxxx

xxxx

xxxxts

xxxx

0,,, 4321 xxxx

OR-1 2011 9

Hence as long as the nonnegative weights yi satisfy

we can use as an upper bound on optimal value. To find more accurate upper bound (smallest upper bound), we want

to solve

Dual problem obtained. Note that the objective value of any feasible solution to the dual problem provides an upper bound on the optimal value of the given LP.

3583,533

12,45

321321

321321

yyyyyy

yyyyyy

321 355 yyy

3583

533

12

45..

355min

321

321

321

321

321

yyy

yyy

yyy

yyyts

yyy

0,, 321 yyy

3532

55835

13..

354max

4321

4321

4321

4321

xxxx

xxxx

xxxxts

xxxx

0,,, 4321 xxxx

OR-1 2011 10

General form

P: primal problem, D: dual problem

maximize

n

jjj xc

1

n

jijij mibxa

1,...,1,subject to

njx j ,...,2,1,0

0

subject to

' maximize

x

bAx

xc

minimize

m

iii yb

1

m

ijiij njcya

1,...,1,subject to

miyi ,...,2,1,0

0

'' subject to

' minimize

y

cAy

yb

(P)

(D)

OR-1 2011 11

Thm: (Weak duality relation)

Suppose (x1, …, xn) is a feasible solution to the primal problem (P) and

(y1, …, ym) is a feasible solution to the dual problem,

then

pf)

Cor: If we can find a feasible x* to (P) and a feasible y* to (D) such that , then x* is an optimal solution to (P) and y* is an optimal solution to (D).

pf) For all feasible solution x to (P), we have

Similarly, for all feasible y to (D), we have

nj

mi iijj ybxc1 1

nj

mi ii

mi

nj ijij

nj

mi jiijjj ybyxaxyaxc1 11 11 1 )()(

nj

mi iijj ybxc1 1

**

nj

mi

nj jjiijj xcybxc1 1 1

**

mi

nj

mi iijjii ybxcyb1 1 1

**

OR-1 2011 12

[Strong Duality Theorem] If (P) has an optimal solution (x1*, …, xn

* ),

then (D) also has an optimal solution, say (y1*, …, ym

* ), and

(i.e., no duality gap, dual optimal value – primal optimal value = 0)

Note that strong duality theorem says that if (P) has an optimal solution, the dual (D) is neither unbounded nor infeasible, but always has an optimal solution.

nj

mi iijj ybxc1 1

**

OR-1 2011 13

Idea of proof: Read the optimal solution of the dual problem from the coefficients of the slack variables in the z-row from the optimal dictionary (tableau).

ex)

3532

55835

13

0354

74321

64321

54321

4321

xxxxx

xxxxx

xxxxx

xxxxz

Note that the dual variables y1, y2, y3 matches naturally with slack variables x5, x6, x7. For example x5 is slack variable for the first constraint and y1 is dual variable for the first constraint, and so on.

OR-1 2011 14

At optimality, the tableau looks

1112195

52

143542

296112

76531

75431

75321

7531

xxxxx

xxxxx

xxxxx

xxxxz

In the z-row of the tableau, the coefficients of the slack variables are –11 for x5, 0 for x6, -6 for x7.

Assigning these values with reversed signs to the corresponding dual variables, we obtain desired optimal solution of the dual:

y1 = 11, y2 = 0, y3 = 6.

OR-1 2011 15

Idea of the proof:

Note that the coefficients of x5, x6, and x7 in the z-row show what

numbers we multiplied to the corresponding equation and add them to the z-row in the elementary row operations (net effect of many row operations)

ex)

Suppose we performed row operations

(row2) 2*(row 1) +(row 2), then ( z-row) 3*(row 2) + (z-row).

The net effect in z-row is adding 6*(row 1) + 3*(row 2) to the z-row and the scalar we multiplied can be read from the coefficients of x5

and x6 in the z-row.

3532

55835

13

0354

74321

64321

54321

4321

xxxxx

xxxxx

xxxxx

xxxxz

OR-1 2011 16

3532

5721417

13

0354

74321

654321

54321

4321

xxxxx

xxxxxx

xxxxx

xxxxz

(ex-continued)

3532

55835

13

0354

74321

64321

54321

4321

xxxxx

xxxxx

xxxxx

xxxxz

(row 1)2 +(row 2)

OR-1 2011 17

3532

5721417

13

17136458225

74321

654321

54321

654321

xxxxx

xxxxxx

xxxxx

xxxxxxz

3532

5721417

13

0354

74321

654321

54321

4321

xxxxx

xxxxxx

xxxxx

xxxxz

(ex-continued)

(row 2) 3 +z-row

OR-1 2011 18

Example – Initial tableau

3532

55835

13

0354

74321

64321

54321

4321

xxxxx

xxxxx

xxxxx

xxxxz

Optimal tableau

1112195

52

143542

296112

76531

75431

75321

7531

xxxxx

xxxxx

xxxxx

xxxxz

OR-1 2011 19

Let yi be the scalar we multiplied to the i-th row and add to the z-row in

the net effect.

Then the coefficient of slack variables in the z-row represent the yi

values we multiplied to the i-th row for i = 1, … ,m.

Also the coefficients of structural variables in the z-row are given as

Now in the optimal tableau, all the coefficients in the z-row are 0, which implies

If we take – yi as a dual solution, they are dual feasible.

njayc mi ijij ,...,1,1

) variablesstructural(for ,...,1,01 njayc mi ijij

ables)slack vari(for ,...,1,0 miyi

njcaymi jiji ,...,1,)(1

miyi ,...,1,0)(

OR-1 2011 20

Also the constant term in the z-row gives the value

So it is the negative of the dual objective value of the dual solution (-yi ), i = 1, ,,, , m

Note that the constant term in the z-row also gives the negative of the objective value of the current primal feasible solution.

So we have found a feasible dual solution which gives the same dual objective value as the current primal feasible solution.

From previous Corollary, the dual solution and the primal solution are optimal to the dual and the primal problem, respectively.

It is the idea of the proof.

mi iiby1

OR-1 2011 21

pf of strong duality theorem)

Suppose we introduce slack variables

and solve the LP by simplex and obtain optimal dictionary with

Let

We claim that yi*, i = 1, … , m is an optimal dual solution, i.e. it

satisfies dual constraints and

mnk

nj jjkkk xczkcxczz 1 1

*** ,0,

micy ini ,...,1,*

mi ii zyb1

**

nj

mnnj jj

nj jjjj xcxczzxcz 1 11

* and Equate

)...,,1(1 mixabx nj jijiin

This equation must be satisfied by any x that satisfies the dictionary (excluding the nonnegativity constraints) since the set of feasible solutions does not change for any dictionary.

nj

nj

mnnj jjjjjj xcxczxc1 1 1

*

OR-1 2011 22

since any feasible solution to dictionary should satisfy this.

above into Substitute mixabx nj jijiin ,...,1,1

nj

nj

mi

nj jijiijjjj xabyxczxc1 1 1 1

** )(

nj

mi

nj

mi jiijjiijj xyacybzxc1 1 1 1

*** )(

nj

nj

mnnj jjjjjj xcxczxc1 1 1

*Have

Now this equation should hold for all feasible solutions to the dictionary. From the initial dictionary, we know that any feasible solution to the dictionary can be obtained by assigning arbitrary values to x1, … , xn and setting

xn+i = bi - j=1n aijxj , ( i = 1, … , m).

Use these solutions. Note that, in the above equation, the variables xn+i do not appear. So it must hold for any choices of xj , j = 1, … , n.

OR-1 2011 23

Equality must hold for all choices of x1, …, xn.

Hence

Hence y* is dual feasible.

Also we have that

Since

yi*, i = 1, … , m is an optimal dual solution and

mi iij

mi jjii njyaccybz 1

*1

** ,...,1,,

mi jiij

mi iijjk cyayackc 1

*1

* 00 n1,...,j,

miyy *ii ,...,1,00*

mi

nj jjii xczyb1 1

***

mi ii

mi

nj jjii yybxcyb 11 1

** duality) (weak feasible dual all for

mi

nj jjii xcyb1 1

**

nj

mi

nj

mi jiijjiijj xyacybzxc1 1 1 1

*** )(