mit and james orlin © 2003 1 integer programming 3 brief review of branch and bound cutting plane...

MIT and James Orlin © 2003

1

Integer Programming 3

Brief Review of Branch and Bound Cutting plane techniques for getting improved

bounds


2

Review from Last Lecture

Investment 1 2 3 4 5 6

Cash Required (1000s)

$5

$7

$4

$3

$4

$6

NPV added (1000s)

$16

$22

$12

$8

$11

$19

Investment budget = $14,000

maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6

subject to 5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6 14

xj binary for j = 1 to 6


3

Branch and Bound

1

2 3

x1 = 0 x1 = 1

44

44 44

4

x2 = 0

42 5

x2 = 1

6

x2 = 0

7

x2 = 1

44 44 44

The incumbent solution has value 43

8

x3 = 0

9

x3 = 1

10

x3 = 0

11

x3 = 1

12

x3 = 0

13

x3 = 1

43 43 43 43 44 -8 9 10 11

-

-

14 15

16 17

44

44

18 19 -38


4

On Bounds from Linear Programming

We found an incumbent integer solution with a value of 43.

The best LP solution value is between 44 and 45.

Is there a way that we could have directly established an upper bound between 43 and 44? Perhaps we could form a “better linear program.”

The closer the LP is to the Integer Program, the better.

Note: not all LP formulations are the same in terms of “closeness”


5

Overview of cutting planes

Try to get better bounds so that more nodes of the branch and bound tree can be fathomed, and the fathoming can be done earlier.

Recall: we eliminate a node j if the LP bound for j is no more than the value of the incumbent.– The lower the bound (for a maximization problem) the

better

This lecture will provide several approaches for obtaining bounds.


6

Goals of this lecture

Illustrate how valuable obtaining better LP’s can be

Illustrate techniques for obtaining better bounds– set packing– capital budgeting– traveling salesman problem– general integer programs


7

What is the maximum number of diamonds that can be packed on a Chinese checkerboard.

Special Thanks to Professor Andreas Schulz for this example.


8

The diamonds are not permitted to overlap, or even to share a single circle. What is the best packing you can find?


9

Is this the best possible?


10

Set Packing Problem

Let D be the collection of diamonds Let O be the pairs of diamonds that

overlap. – (d, d’) O, implies that diamonds d and d’

have a point in common

Decision variables: xd for d D

– xd = 1 if diamond d is selected

– xd = 0 if diamond d is not selected.


11

How many decision variables are there?

There are 88 black circles.

For each black circle, one can hang a yellow diamond from it.

So, there are 88 possible yellow diamonds.

And there are 88 green and red diamonds.

for a total of 264 diamonds.


12

The First Integer Programming Formulation

1

0 1

Maximize

subject to for all ( , ')

and integer for

dd D

d d

d d

x

x x d d O

x x d D

The optimal objective value for this IP is 27.

The optimal objective value for the LP relaxation is 132.

This problem is known as the set packing problem.


13

For many dots, there are 12 diamonds that go through the dot.

5 6

87

1

2

3

4

Heading towards an improved IP formulation.


14

Getting an improved LP

x1 + x2 1

x1 + x3 1

x1 + x4 1

x2 + x3 1

x2 + x4 1

x3 + x4 1

x1, x2, x3, x4 {0,1}

Conclusion: x1 + x2 + x3 + x4 1

We say that S is an overlapping set of diamonds, if every two diamonds in S overlap.

1 dd Sx

for each overlapping set S.


15

An improved integer program

1

0 1

Maximize

subject to for each overlapping set

and integer for

dd D

dd S

d d

x

x S

x x d D

The optimal objective value for this IP is 27

The optimal objective value for the LP relaxation is 27.5

Next: a proof that the at most 27 diamonds can be packed.


16

Node Weight

1

1/2

1/3

1/6

0

The weight of a diamond is the sum of its node weights.


17

Node Weight

1

1/2

1/3

1/6

0

The weight of a diamond is the sum of its node weights.

Each diamond has weight at least 1.


18

Node Weight

1

1/2

1/3

1/6

0

The weight of 28 independent diamonds would be at least 28.

But the total weight of all nodes is 27.5

The weight of 27 independent diamonds is at least 27.

18

1

24

6


19

Node Weight

1

1/2

1/3

1/6

0

No solution has more than 27 diamonds.

The weights were the shadow prices for the LP


20

The Optimal Solution Again


21

Set Packing Problem Putting items into storage

Manufacturing– How many car doors pack into a metal sheet?– How many rolls of paper can be cut from a giant roll?

Closely related to fire station problem: set covering.


22

Review There may be different ways of formulating an IP.

– Each way gives the same IP– The LPs may be very different– Some LPs have different bounds

NEXT: The geometry of Integer Programs and Linear Programs

Using Cutting Planes

Example. Minimize x + 10y subject to x, y are in P x, y integer

P

x

y

Optimum fractional (i.e. infeasible) solution

Optimum (integer) solution



P

x

yIdea: add constraints that eliminate fractional solutions to the LP without eliminating any integer solutions.

add “y x –1”

add “y 1”

These constraints were obtained by inspection. We will develop techniques later.



x

y

Optimum (integer) solution

If we add exactly the right inequalities, then every corner point of the LP will be integer, and the IP can be solved by solving the LP

PWe call this minimal LP, the convex hull of the IP solutions.

For large problems, these constraints are hard to find.


26

More on adding constraints

The tightest possible constraints are very useful, and are called facets.

Suppose that we are maximizing, and zLP is the opt for the LP relaxation, and zIP is the opt for the IP. Then zIP zLP

Ideally, we want zIP to be close to zLP. This is GREAT for branch and bound.

Adding lots of valid inequalities can be very helpful.– It has no effect on zIP.– It can reduce zLP significantly.


27

“Pure” Cutting Plane Technique

Instead of partitioning the feasible region, the (pure) cutting plane technique works with a single LP

It adds cutting planes (valid linear programming inequalities) to this LP iteratively.

At each iteration the feasible region is successively reduced until an integer optimal is found by solving the LP.

In practice, it is also used as part of branch and bound. The essential idea is finding valid “cuts” or inequalities.


28

Where do these cuts come from?

Two approaches– Problem specific• Set Packing• Capital Budgeting (knapsack)• Traveling Salesman Problem

– LP-based approach, that works for general integer programs• Gomory cutting planes


29

Problem Specific

The capital budgeting (knapsack) problem

maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6


xj binary for j = 1 to 6


30

The LP Relaxation

The capital budgeting (knapsack) problem

maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6


0 xj 1 for j = 1 to 6

The optimal solution: x1 = 1, x2 = 3/7, x3 = 0 x4 = 0, x5 = 0, x6 = 1

We say that a subset S is a cover if the sum of its weights is more than 14 (more than the budget)

What are some covers of this capital budgeting problem?


31

Getting constraints from covers

5x1 + 7x2 + 4x3 + 3x4 +4x5 + 6x6 14

Consider the cover S = {1, 2, 3}

We have the constraint 5x1 + 7x2 + 4x3 14 We can obtain the constraint x1 + x2 + x3 2.

– This is a cover constraint– This constraint is “stronger” – It does not eliminate any IP solutions, but it cuts off

fractional LP solutions.

In general, for each cover S,we obtain the constraint

jS xj |S| - 1


32

The LP Relaxation The capital budgeting (knapsack) problem

maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6


0 xj 1 for j = 1 to 6

5x1 + 7x2 + 6x6 14

Excel

The optimal solution: x1 = 1, x2 = 3/7, x3 = 0 x4 = 0, x5 = 0, x6 = 1 z = 44 3/7

x1 + x2 + x6 2

A violated cover is {1, 2, 6}

Our approach: Given the linear solution, try to find a violated cover constraint.


33

After one cutmaximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6


x1 + x2 + x6 2

0 xj 1 for j = 1 to 6The optimal solution: x1 = 0, x2 = 1, x3 = 1/4 x4 = 0, x5 = 0, x6 = 1 z = 44

Exercise: find a violated cover.

Excel


34

After two cuts

maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6


x1 + x2 + x6 2

x2 + x3 + x6 2

0 xj 1 for j = 1 to 6

The optimal solution: x1 = 1/3, x2 = 1, x3 = 1/3 x4 = 0, x5 = 0, x6 = 1 z = 44

Exercise: Find a violated cover:Excel


35

Obtaining a stronger constraint

5x1 + 7x2 + 4x3 + 6x6 14

At most 2 of x1, x2, x3, x6 can be 1.

So x1 + x2 + x3 + x6 2.


36

After “three” cutsmaximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6


x1 + x2 + x6 2

x2 + x3 + x6 2

x1 + x2 + x3 + x6 2

0 xj 1 for j = 1 to 6

ExcelNote: the new cuts dominates the other cuts.


37

We eliminate the redundant constraints

maximize 16x1 + 22x2 + 12x3 + 8x4 +11x5 + 19x6


x1 + x2 + x3 + x6 2

0 xj 1 for j = 1 to 6

Excel

The optimal solution: x1 = 0, x2 = 1, x3 = 0, x4 = 0, x5 = 1/4, x6 = 1 z = 43 3/4

So, z* 43


38

Summary for knapsack problem We could find some simple valid inequalities that

showed that z* 43. This is the optimal objective value.

It took 3 cuts

– Had we been smarter it would have taken 1 cut

We had a simple approach for finding cuts.

– This does not find all of the cuts.

Recall, it took 25 nodes of a branch and bound tree

In fact, researchers have found cutting plane techniques to be necessary to solve large integer programs (usually as a way of getting better bounds.)


39

Traveling Salesman Problem (TSP)

What is a minimum length tour that visits each point?


40

Comments on the TSP Very well studied problem

It’s often the problem for testing out new algorithmic ideas

NP-complete (it is intrinsically difficult in some technical sense)

Large instances have been solved optimally (5000 cities and larger)

Very large instances have been solved approximately (10 million cities to within a couple of percent of optimum.)

We will formulate it by adding constraints that look like cuts


41

The TSP as an IP, almost

Let xe = 1 if arc e is in the tour xe = 0 otherwise

Let A(i) = arcs incident to node i

Minimize e ce xe

subject to eA(i) xe = 2

xe is binary

Are these constraints enough?


42

Subtour: a cycle that passes through a strict subset of cities

7

2

Fact: Any integer solution with exactly two arcs incident to every node is the union of cycles. Why?

9

3

4

Improved IP/LP: for each possible subtour, add a constraints that makes the subtour infeasible for the IP. These are called subtour breaking constraints.


43

A subtour breaking constraint

Let S be any proper subset of nodes, e.g., S = {2, 3, 4, 7, 9}.

This ensures that the set S will not have a subtour going through all five nodes.

7

2

9

3

4

2. A tour for the entire network has at most |S| - 1 arcs with two endpoints in S.

Observations:

1. A subtour that includes all nodes of S has |S| arcs

1,

iji S j S

x S


44

A traveling salesman tour

There are at most 4 arcs of any tour incident to 2, 3, 4, 7, 9

72

9

34


45

Formulation of the TSP as an IP

Minimize e ce xe

subject to e inc to i xe = 2

e in S xe |S| - 1

(subtour breaking constraints)

xe is binaryExponentially many constraints, too many to include in an IP or an LP

In practice: •Include only some of the constraints. Solve the LP. •If a subtour appears as part of the LP solution, add a new subtour elimination constraint to the LP, and solve again.


46

More on the TSP

The IP formulation is a very good formulation good in the following sense: for practical problem the LP bound is usually 1% to 2% from the optimal TSP tour length. (The LP bound is close.)

One can add even more complex constraints to get a better LP formulation, and people do. (and it helps)


47

Gomory Cuts: a method of generating cuts using LP tableaus

Consider the following constraint in an integer program

x1 x2 x4x3

1 3/5 4 1/5 2/53 = 9 4/5

Focus on the fractions.

3/5 x1 + 1/5 x2 +2/5 x4 = Integer + 4/5

3/5 x1 + 1/5 x2 + 2/5 x4 4/5

3 x1 + x2 + 2 x4 4.


48

Gomory Cuts

What did we rely on to obtain the Gomory cut?

x1 x2 x4x3

1 3/5 4 1/5 2/53 = 9 4/5

A single constraint with a fractional right hand side

All coefficients of the constraint were positive.

All variables must be integral


49

What to do if coefficients are negativex1 x2 x4x3

1 3/5 -4 3/5 -2/5-3 = -1 1/5

Rewrite so that the coefficients of the fractional parts are positive.

1 3/5 -5 +2/5 -1 +3/5-3 = -2 +4/5

3/5 x1 + 2/5 x2 +3/5 x4 4/5 .

3 x1 + 2 x2 + 3 x4 4 .

3/5 x1 + 2/5 x2 +3/5 x4 = integer + 4/5 .


50

In general

x1 x2 x4x3

1 3/5 -4 3/5 -2/5-3 = -1 1/5

3/5 2/5 +3/50 +4/5

Let fr(a) be the positive fractional part of a ;

fr(a) = a - a

fr(2 3/5) = 2 3/5 – 2 = 3/5

fr(-2 3/5) = - 2 3/5 – (-3) = 2/5

x1 x2 x4x3

ai1 ai2 ai4 ai3 = bi

fr(ai1) fr(ai2) fr(ai4)fr(ai3) fr(bi)


51

How to generate cutting planes?

In general– After pivoting, find a basic variable that is

fractional.– Write a Gomory cut. (It is an inequality constraint,

leading to a new slack variable).– Note: the only coefficients that are fractional

correspond to non-basic variables (why?)– The Gomory cut makes the previous basic

feasible solution infeasible. (why?)– Resolve the LP with the new constraint, and

iterate.


52

Integer Programming Summary

Dramatically improves the modeling capability– Economic indivisibilities– Logical constraints– Modeling non-linearities

Not as easy to model. Not as easy to solve.


53

IP Solution Techniques Summary

Branch and Bound– very general and adaptive– used in practice (e.g. Excel Solver)

Implicit Enumeration– branch and bound technique for binary IPs

Cutting planes– clever way of improving bounding– active area for research, theoretical and applied


54

Example: Fire company location.

Consider locating fire companies in different districts.

Objective: place fire companies so that each district either has a fire company in it, or one that is adjacent, and so as to minimize cost.


55

Example for the Fire Station Problem

1 2 3

4

5 67

8 9

11

1012

14 15

13

16

Let xj = 1 if a fire station is placed in district j. xj = 0 otherwise.

Let cj = cost of putting a fire station is district j.

With partners, formulate the fire station problem.


56

Set covering problem

set S= {1, …, m} of items to be covered – districts that need to have a fire station or be next

to a district with one

n subsets of S– For each possible location j for a fire station, the

subset is district j plus the list of districts that are adjacent to district j.

– aij = 1 if district i is adjacent to district j or if i = j.

j jjc xMinimize

1 for each iij jja x

is binary for each jjx

subject to


57

Covering constraints are common

Fleet routing for airlines: – Assigning airplanes to flight legs.– Each flight must be included

Assigning crews to airplanes– Each plane must be assigned a crew

Warehouse location– Each retailer needs to be served by some

warehouse


58

Independent Set (set packing)

1 2 3

4

5 67

8 9

11

1012

14 15

13

16

What is the maximum number of districts no two of which have a common border?

With your partner, formulate the independent set problem. List all constraints containing x10.


59

Set Packing constraints arise often in manufacturing and logistics

Cutting shapes out of sheet metal

Manufacturing many items at once which share resources (how much work can be done in parallel?)

mit and james orlin © 2003 1 integer programming 3 brief review of branch and bound cutting plane...

Documents