the simplex algorithm 虞台文 大同大學資工所 智慧型多媒體研究室. content basic...
TRANSCRIPT
The Simplex Algorithm
虞台文大同大學資工所智慧型多媒體研究室
Content Basic Feasible Solutions The Geometry of Linear Programs Moving From Bfs to Bfs Organization of a Tableau
Linear Programming
Basic Feasible Solutions
大同大學資工所智慧型多媒體研究室
The Goal of the Simplex Algorithm
min
0
c x
Ax b
x
1 2( , , , )nx x x x
1 2( , , , )nc c c c
1 2, , ,ij nm nA a A A A
1 2( , , , )mb b b b Any LP can be convertedinto the standard form
m n
Convex Polytope
min
0
c x
Ax b
x
defines a convex set F
1 2( , , , )nx x x x
1 2( , , , )nc c c c 1 2, , ,ij nm n
A a A A A
1 2( , , , )mb b b b m n
F may be1. Bounded2. Unbounded3. Empty
The Basic Idea of Simplex Algorithm
min
0
c x
Ax b
x
basic
feasiblesolution
at a corner
Finding optimum by moving around the corner of the convex polytope in cost descent sense
1 2( , , , )nx x x x
1 2( , , , )nc c c c 1 2, , ,ij nm n
A a A A A
1 2( , , , )mb b b b m n
The Basic Idea of Simplex Algorithm
min
0
c x
Ax b
x
basic
feasiblesolution
at a corner
Finding optimum by moving around the corner of the convex polytope in cost descent sense
How to find an initial feasible solution?
How to find an initial feasible solution?
How to move from corner to corner?How to move from corner to corner?
1 2( , , , )nx x x x
1 2( , , , )nc c c c 1 2, , ,ij nm n
A a A A A
1 2( , , , )mb b b b m n
Assumption 1
min
0
c x
Ax b
x
1 2( , , , )nx x x x
1 2( , , , )nc c c c 1 2, , ,ij nm n
A a A A A
1 2( , , , )mb b b b m n
Assume that A is of rank m.
There is a basis 1, ,
mj jA A B
Independent
ijB A has an inverse.
Basic Solution
min
0
c x
Ax b
x
1 2( , , , )nx x x x
1 2( , , , )nc c c c 1 2, , ,ij nm n
A a A A A
1 2( , , , )mb b b b m n
1,
,m
i
j j
j
A A
B A
B :basis
has inverse
1,
,m
i
j j
j
A A
B A
B :basis
has inverse
1
0
, 1, ,k
j j
thj
x A
x k B b k m
f or
the compone nt of
B1
0
, 1, ,k
j j
thj
x A
x k B b k m
f or
the compone nt of
B
The basis solution corresponding to B is:
The basis solution may be infeasible.The basis solution may be infeasible.
Example
min
0
c x
Ax b
x
1 2( , , , )nx x x x
1 2( , , , )nc c c c 1 2, , ,ij nm n
A a A A A
1 2( , , , )mb b b b m n
1,
,m
i
j j
j
A A
B A
B :basis
has inverse
1,
,m
i
j j
j
A A
B A
B :basis
has inverse
2 4 7
1 2 3 4
1 5
3 6
2 3 7
1 2 3 4 5 6 7
min 2 5
4
2
3
3 6
, , , , , , , 0
x x x
x x x x
x x
x x
x x x
x x x x x x x
1 1 1 1
1 1
1 1
3 1
0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 10
A
x1 x2 x3 x4 x5 x6 x7
Example
min
0
c x
Ax b
x
1 2( , , , )nx x x x
1 2( , , , )nc c c c 1 2, , ,ij nm n
A a A A A
1 2( , , , )mb b b b m n
1,
,m
i
j j
j
A A
B A
B :basis
has inverse
1,
,m
i
j j
j
A A
B A
B :basis
has inverse
2 4 7
1 2 3 4
1 5
3 6
2 3 7
1 2 3 4 5 6 7
min 2 5
4
2
3
3 6
, , , , , , , 0
x x x
x x x x
x x
x x
x x x
x x x x x x x
1 1 1 1
1 1
1 1
3 1
0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 10
A
x1 x2 x3 x4 x5 x6 x7
1
0
, 1, ,k
j j
thj
x A
x k B b k m
f or
the compone nt of
B1
0
, 1, ,k
j j
thj
x A
x k B b k m
f or
the compone nt of
B
4 5 6 7, , ,A A A AB
0 0 0
0 0 0
0 0 0
0 0
1
1
1
10
B
( 4,0 2,3,6),0,0,x
feasible
Example
min
0
c x
Ax b
x
1 2( , , , )nx x x x
1 2( , , , )nc c c c 1 2, , ,ij nm n
A a A A A
1 2( , , , )mb b b b m n
1,
,m
i
j j
j
A A
B A
B :basis
has inverse
1,
,m
i
j j
j
A A
B A
B :basis
has inverse
2 4 7
1 2 3 4
1 5
3 6
2 3 7
1 2 3 4 5 6 7
min 2 5
4
2
3
3 6
, , , , , , , 0
x x x
x x x x
x x
x x
x x x
x x x x x x x
1 1 1 1
1 1
1 1
3 1
0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 10
A
x1 x2 x3 x4 x5 x6 x7
1
0
, 1, ,k
j j
thj
x A
x k B b k m
f or
the compone nt of
B1
0
, 1, ,k
j j
thj
x A
x k B b k m
f or
the compone nt of
B
2 5 6 7, , ,A A A AB
0 0 0
0 0 0
0 0 0
0
1
1
1
3 10
B
0, 0,0( 4, 2,3, 6),x
infeasible
Basic Feasible Solution
min
0
c x
Ax b
x
defines an nonempty convex set F
1 2( , , , )nx x x x
1 2( , , , )nc c c c 1 2, , ,ij nm n
A a A A A
1 2( , , , )mb b b b m n
If a basic solution is in F, it is called a basic feasible solution (bfs).
The Existence of BFS’s
1 2( , , , )nx x x x
1 2( , , , )nc c c c 1 2, , ,ij nm n
A a A A A
1 2( , , , )mb b b b m n
F may be1. Bounded2. Unbounded3. Empty
If F is nonempty, at least one bfs.
min
0
c x
Ax b
x
defines an nonempty convex set F
min
0
c x
Ax b
x
defines an nonempty convex set F
The Existence of BFS’s
1 2( , , , )nx x x x
1 2( , , , )nc c c c 1 2, , ,ij nm n
A a A A A
1 2( , , , )mb b b b m n
F may be1. Bounded2. Unbounded3. Empty
If F is nonempty, at least one bfs.How to find it?How to find it?
Assumptions
1 2( , , , )nx x x x
1 2( , , , )nc c c c 1 2, , ,ij nm n
A a A A A
1 2( , , , )mb b b b m n
1. A is of rank m.2. F is nonempty.3. c’x is bounded below for xF.
1. A is of rank m.2. F is nonempty.3. c’x is bounded below for xF.
min
0
c x
Ax b
x
defines an nonempty convex set F
Assumptions
1 2( , , , )nx x x x
1 2( , , , )nc c c c 1 2, , ,ij nm n
A a A A A
1 2( , , , )mb b b b m n
1. A is of rank m.2. F is nonempty.3. c’x is bounded below for xF.
1. A is of rank m.2. F is nonempty.3. c’x is bounded below for xF.
There exist bfs’s.
min
0
c x
Ax b
x
defines an nonempty convex set F
Ensure there is a bounded solution.
Ensure there is a bounded solution.
Are all bfs’s the vertices of the convex polytope defined by F?
Are all bfs’s the vertices of the convex polytope defined by F?
The Simplex Alogrithm
The Geometry of Linear Programs
大同大學資工所智慧型多媒體研究室
Linear Subspaces of Rd
S Rd is a subspace of Rd if it is closed under vector addition and scalar multiplication.
S defined below is a subspace of Rd.11 1 12 2 1
21 1 22 2 2
1 1 2 2
0
0
0
d d
d dd
m m md d
a x a x a x
a x a x a xS x R
a x a x a x
a set of homogenous linear equations
Dimensions
11 1 12 2 1
21 1 22 2 2
1 1 2 2
0
0
0
d d
d dd
m m md d
a x a x a x
a x a x a xS x R
a x a x a x
dim( ) ijS d rank a dim( ) ijS d rank a d m
Affine subspaces 1 1 , ,0 1 ,d
i id dS x R a x a x i m
A translated linear subspace.
E.g., :A x Su x an affine subspace
1
2
11 1 12 2 1
21 1 22 2 2
1 1 2 2
d d
d dd
m md d mm
a x a x a x
a x a x a xA x R
a x a x a x
b
b
b
a set of nonhomogenous linear equations
an affinesubspace
Dimensions
1 1 , ,0 1 ,di id dS x R a x a x i m
1
2
11 1 12 2 1
21 1 22 2 2
1 1 2 2
d d
d dd
m md d mm
a x a x a x
a x a x a xA x R
a x a x a x
b
b
b
dim( ) dim( )A Sdim( ) dim( )A S d m
an affinesubspace
Subsets of Rd The following subsets are not subspace
or affine subspace.
A line segmentThe first quadrant
A halfspace
Dimensions The following subsets are not subspace
or affine subspace.
A line segmentThe first quadrant
A halfspace
The dimension of any subset of Rd is the smallest dimension of any affine subspace which contains it.
Dim = 1Dim = 2 Dim = 2
The Feasible Spaces of LP
1. A is of rank m.2. F is nonempty.3. c’x is bounded below for xF.
1. A is of rank m.2. F is nonempty.3. c’x is bounded below for xF.
min
0
c x
Ax b
x
defines an nonempty convex set F
dim( )F d m dim( )F d m
Hyperplane/Halfspace
1 1 2 2 d da x a x ba x
An affine subspace of Rd of dimension d1 is called a hyperplane.
A hyperplane defines two closed halfspaces:
1 1 2 2 d da x a x ba x
1 1 2 2 d da x a x ba x
Fact: Halfspaces are convex.
Convex Polytopes
The intersection of a finite number of halfspaces, when it is bounded and nonempty is called a convex polytope, or simply a polytope.
Fact: Halfspaces are convex.
x1
x2
x3
Example
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
More on Polytopes
Geometric views of a polytope– The convex hull of a finite set of points.– The intersection of a finite number of halfspace
s, when it is bounded and nonempty, i.e., defined by
The algebraic view of a polytope1 1 0, 1, ,i in n ia x a x b i m
0
Ax b
x
The feasible space defined by LP (in standard form).
Any relation?
LP Polytopes
0
Ax b
x
Assume
11 12 1,
21 22 2,
1 2 ,
1 0 0
0 1 0
0 0 1
n m
n m
m m m n m
a a a
a a aA
a a a
m columnsn m columns
x1 x2 xnm xnm+1xn
A can always be in this form if rank(A)=m.
LP
LP Polytopes
Assume
11 12 1,
21 22 2,
1 2 ,
1 0 0
0 1 0
0 0 1
n m
n m
m m m n m
a a a
a a aA
a a a
m columnsn m columns
x1 x2 xnm xnm+1xn
Ax b1
, 1, ,n m
i ij jij
x b a x i n m n
0
Ax b
x
LP
LP Polytopes
Ax b1
, 1, ,n m
i ij jij
x b a x i n m n
0
1
0 1, ,
0 1, ,
n m
ij j ij
j
a x b i n m n
x j n m
0
Ax b
x
LP
Polytopes LP11 1 12 2 1, 1
21 1 22 2 2, 2
1 1 2 2 ,
1 2
0
0
0
, , 0
n m n m
n m n m
m m m n m n m m
n m
a x a x a x b
a x a x a x b
a x a x a x b
x x x
m inequalities
11 1 12 2 1, 1 1
21 1 22 2 2, 2 2
1 1 2 2 ,
1 2 1, 2,
, , , 0
n m n m n m
n m n m n m
m m m n m n m n m
n m n m n m
a x a x a x x b
a x a x a x x b
a x a x a x x b
x x x x x
m equalities
m equalities
Polytopes LP11 1 12 2 1, 1
21 1 22 2 2, 2
1 1 2 2 ,
1 2
0
0
0
, , 0
n m n m
n m n m
m m m n m n m m
n m
a x a x a x b
a x a x a x b
a x a x a x b
x x x
m inequalities
11 1 12 2 1, 1 1
21 1 22 2 2, 2 2
1 1 2 2 ,
1 2 1, 2,
, , , 0
n m n m n m
n m n m n m
m m m n m n m n m
n m n m n m
a x a x a x x b
a x a x a x x b
a x a x a x x b
x x x x x
x
x x x
Slack variablesx
Polytopes LP11 1 12 2 1, 1
21 1 22 2 2, 2
1 1 2 2 ,
1 2
0
0
0
, , 0
n m n m
n m n m
m m m n m n m m
n m
a x a x a x b
a x a x a x b
a x a x a x b
x x x
11 1 12 2 1, 1 1
21 1 22 2 2, 2 2
1 1 2 2 ,
1 2 1, 2,
, , , 0
n m n m n m
n m n m n m
m m m n m n m n m
n m n m n m
a x a x a x x b
a x a x a x x b
a x a x a x x b
x x x x x
x
xx x
0
x
x
A b
Slack variablesx
Polytopes and LP11 1 12 2 1, 1
21 1 22 2 2, 2
1 1 2 2 ,
1 2
0
0
0
, , 0
n m n m
n m n m
m m m n m n m m
n m
a x a x a x b
a x a x a x b
a x a x a x b
x x x
11 1 12 2 1, 1 1
21 1 22 2 2, 2 2
1 1 2 2 ,
1 2 1, 2,
, , , 0
n m n m n m
n m n m n m
m m m n m n m n m
n m n m n m
a x a x a x x b
a x a x a x x b
a x a x a x x b
x x x x x
x
xx x
|
0
xH I b
x
x
x
Slack variablesx
H I
H
[ | ]A H I
0
Hx b
x
xx
x
Polytopes and LP
|
0
xH I b
x
x
x
[ | ]A H I
0
Hx b
x
xx
x
x xGiven , =?
xx Given , =?
Hx x b
0x b Hx
0x
b Hxx
xx
x
The answer
Polytopes and LP
|
0
xH I b
x
x
x
[ | ]A H I
0
Hx b
x
xx
x
xx Given , =?
Hx x b
Hx b x b
xx
x
The answer
Polytopes & F of LP
0
Ax b
x
[ | ]A H I
0
Hx b
x
xx
x
defines a polytope
defines a feasible set , 0F x Ax b x
, 0P x Hx b x
Are there any relations?Are there any relations?
Polytopes & F of LP
0
Ax b
x
[ | ]A H I
0
Hx b
x
xx
x
defines a polytope
defines a feasible set , 0F x Ax b x
, 0P x Hx b x
x1
x2
x3
x1
x2
x3
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0) x1
x2
x3
x1
x2
x3
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
Some points in P are vertices.
Some points in F are bfs’s.Are there any relations?Are there any relations?
Theorem 1
, 0x xF A b x
, 0P x Hx b x a convex polytope
a feasible set ofthe corresponding LP
[ | ]A H Ix
xx
x P is a vertex of P
xF
xx
is a bfs.
Theorem 1 , 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
, 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
Pf)
“” See textbook.
Theorem 1 , 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
, 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
Pf) “”
Fact: x is a vertex, then it cannot be the strict combination of points of P, i.e.,
(1 ) , ,0 1x y z y z P withy z x
x1
x2
x3
x1
x2
x3
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0) x1
x2
x3
x1
x2
x3
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0) x1
x2
x3
x1
x2
x3
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
Theorem 1 , 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
, 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
Pf) “”
Fact: x is a vertex, then it cannot be the strict combination of points of P, i.e.,
xF
xx
is a bf s, To show
we must show that x has at most m nonzero elements and …
xF
xx
is a bf s, To show
we must show that x has at most m nonzero elements and …
(1 ) , ,0 1x y z y z P withy z x
Theorem 1 , 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
, 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
Pf) “”
Fact: (1 ) , ,0 1 .x y z y z P y z x with
Define B = {Aj: xj > 0, 1 j n}
1( , , ) .n
xx Fx x
x
Let We want to show at most m nonzero xi’s.
We want to show that vectors in B are linearly independent.
0j
j jA
d A
B
Suppose not. Then, some dj 0.
j
j jA
F x A bx
B
( )j
j j jA
x d A b
B
>0 if is sufficiently small.
Theorem 1 , 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
, 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
Pf) “”
Fact: (1 ) , ,0 1 .x y z y z P y z x with
Define B = {Aj: xj > 0, 1 j n}
1( , , ) .n
xx Fx x
x
Let We want to show at most m nonzero xi’s.
We want to show that vectors in B are linearly independent.
0j
j jA
d A
B
Suppose not. Then, some dj 0.
j
j jA
F x A bx
B
( )j
j j jA
x d A b
B
>0 if is sufficiently small.
j
j jA
y x d A
B
j
j jA
z x d A
B
Define
is sufficiently small.
, , .y z F y x z x and
1 12 2x y z
, , .y z P y x z x and1 12 2x y z
Theorem 1 , 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
, 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
Pf) “”
Fact: (1 ) , ,0 1 .x y z y z P y z x with
Define B = {Aj: xj > 0, 1 j n}
1( , , ) .n
xx Fx x
x
Let We want to show at most m nonzero xi’s.
Aj are linearly independent.
|B| m.
Since rank(A) = m, we can always augment B to include m linearly independent vectors.Using B to form basic columns renders x a bfs.
Discussion , 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
, 0x xF A b x
, 0P x Hx b x a convex polytopea convex polytope
a f easible set ofthe corresponding LPa f easible set ofthe corresponding LP
x P is a vertex of Px P is a vertex of P
xF
xx
is a bf s.x
Fx
x
is a bf s.
Pf) “”
Fact: (1 ) , ,0 1 .x y z y z P y z x with
Define B = {Aj: xj > 0, 1 j n}
1( , , ) .n
xx Fx x
x
Let We want to show at most m nonzero xi’s.
Aj are linearly independent.
|B| m.
Since rank(A) = m, we can always augment B to include m linearly independent vectors.Using B to form basic columns renders x a bfs.
If |B| < m, there may be many ways to augment to m linearly independent vectors.
Two different B and B’ may corresponds to the same bfs.
If |B| < m, there may be many ways to augment to m linearly independent vectors.
Two different B and B’ may corresponds to the same bfs.
Two different , their corresponding B and B’ must be different.
Two different , their corresponding B and B’ must be different.
,x y P
Example
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
Examplex1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
Corresponding F of LPCorresponding F of LP
0
Ax b
x
1
1
1
0 0 0
0 0 0
0 0 0
0 0 0
1 1 1
1 0 0
0 0 1
0 3 1 1
A
1x 2x 3x 4x 5x 6x 7x
1 2 3 6, , ,A A A AB 4 5 7( , , ) (0,0,0)x x x 1
1 2 3 6( , , , ) (2, 2,0,3);x x x x B b
4
2
3
6
b
Examplex1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
Corresponding F of LPCorresponding F of LP
0
Ax b
x
1
1
1
0 0 0
0 0 0
0 0 0
0 0 0
1 1 1
1 0 0
0 0 1
0 3 1 1
A
1x 2x 3x 4x 5x 6x 7x
1 2 3 6, , ,A A A AB 4 5 7( , , ) (0,0,0)x x x 1
1 2 3 6( , , , ) (2, 2,0,3);x x x x B b
4
2
3
6
b
3 5 7( , , ) (0,0,0)x x x 1
1 2 4 6( , , , ) (2,2,0,3);x x x x B b 1 2 4 6, , ,A A A AB
Examplex1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
Corresponding F of LPCorresponding F of LP
0
Ax b
x
1
1
1
0 0 0
0 0 0
0 0 0
0 0 0
1 1 1
1 0 0
0 0 1
0 3 1 1
A
1x 2x 3x 4x 5x 6x 7x
1 2 3 6, , ,A A A AB 4 5 7( , , ) (0,0,0)x x x 1
1 2 3 6( , , , ) (2, 2,0,3);x x x x B b
4
2
3
6
b
3 5 7( , , ) (0,0,0)x x x 1
1 2 4 6( , , , ) (2,2,0,3);x x x x B b 1 2 4 6, , ,A A A AB
B and B’ determine the same bfs, i.e,
x = (2, 2, 0, 0, 0, 3, 0)’.B and B’ determine the same bfs, i.e,
x = (2, 2, 0, 0, 0, 3, 0)’.
Degeneration
A bfs is called degenerate if it co
ntains more than n m zeros.
x
x
Theorem 2
A bfs is called degenerate if it contains more than n m zeros.
If two distinct bases corresponding to the same bfs x, then x is degenerate.
Pf) Let B and B’ be two distinct bases which determine the same bfs x.
B
B’
0 0
00
n
m
m
x
x
Theorem 2
A bfs is called degenerate if it contains more than n m zeros.
Pf) Let B and B’ be two distinct bases which determine the same bfs x.
B
B’
0 0
00
n
m
m0 0x has more than n m zeros and, hence, is degenerate.
If two distinct bases corresponding to the same bfs x, then x is degenerate.
Discussion
A bfs is called degenerate if it contains more than n m zeros.
If two distinct bases corresponding to the same bfs x, then x is degenerate.
Changing basis may keep bfs unchanged.Changing basis may keep bfs unchanged.
More on Theorem 1 and 2
Vertices of polytope Bfs’s of LP
Change vertex Change bfs?A bfs is determined by a chosen basis.
Change vertices Change basis?
Costs
0
x
x
A b
0
Hx b
x
min c x ? ? ? [ | ]A H I
xx
x
0
x
b Hx
:
:
x
x
bf s
vertex of polytope
? ? ?
Costs
0
x
x
A b
0
Hx b
x
min c x min xd
cc
c
c c x c xx
( )c x c b Hx
( )c c H x c b
[ | ]A H I
xx
x
0
x
b Hx
:
:
x
x
bf s
vertex of polytope
d constant
Theorem 3
There is an optimal bfs in any instance of LP.
Furthermore, if q bfs’s are optimal, their co
nvex combinations are also optimal.
Theorem 3There is an optimal bfs in any instance of LP.
Furthermore, if q bfs’s are optimal, their co
nvex combinations are also optimal.
There is an optimal bfs in any instance of LP.
Furthermore, if q bfs’s are optimal, their co
nvex combinations are also optimal.
Let x0 P be the optimal solution, and let x1, …, xN be the vertices of P.
10
N
i ii
xx
with1
1N
ii
Let j be the index corresponding to the vertex with lowest cost.
01
N
i ii
d x d x
1
N
j ii
xd
jxd
xj is the optimum.
Pf)
and 0i
Theorem 3There is an optimal bfs in any instance of LP.
Furthermore, if q bfs’s are optimal, their co
nvex combinations are also optimal.
There is an optimal bfs in any instance of LP.
Furthermore, if q bfs’s are optimal, their co
nvex combinations are also optimal.
Assume y1, …, yq be the optimal vertices.
1
q
i ii
yy
Pf)
Let with1
1N
ii
and 0i
1
q
i ii
d y d y
1
q
i ii
d y
11
q
ii
yd
1yd
Linear Programming
Moving fromBfs to Bfs
大同大學資工所智慧型多媒體研究室
Facts The optimal solution can be found from verti
ces of the corresponding polytope.
The bfs’s of LP and the vertices of polytope are close correlated.
The algorithm to solve LP:– Move from vertex to vertex; or– Move from bfs to bfs
min
0
c x
Ax b
x
The BFS’s
min
0
c x
Ax b
x
( ) : 1, ,B iA i m B
1 nA A A m
jA R
: , 1, ,j jA A j n B B
Basis columns
Nonbasis columns
The bfs is determined from the set of basis columns, i.e., B.
Move from bfs to bfs: ( ) : 1, ,B iA i m B
: , 1, ,j jA A j n B B
Move from Bfs to Bfs
Let x0 be the bfs determined by B.Denote the basic components of x0 as xi0, i = 1, …, m.
0 ( )1
,m
i B ii
x A b
0 0ix
( )1
.m
ij B i ji
x A A
( ) : 1, ,B iA i m B
: , 1, ,j jA A j n B B
For any , we havejA B
0 ( )1
( )m
i ij B i ji
x x A A b
Move from Bfs to Bfs
( ) : 1, ,B iA i m B
: , 1, ,j jA A j n B B
0 ( )1
( )m
i ij B i ji
x x A A b
m + 1 columns are involved.
Choose one Aj from to enter B.
BChoose one AB(i) to leave B.
Who enters?Who leaves?
Move from Bfs to Bfs
( ) : 1, ,B iA i m B
: , 1, ,j jA A j n B B
0 ( )1
( )m
i ij B i ji
x x A A b
m + 1 columns are involved.
Choose one Aj from to enter B.
BChoose one AB(i) to leave B.
Who enters?Who leaves?
must be positive. must be positive.
Make one of them zero, and keep others positive.
The one that we want to make zero must have xij >
0.
The one that we want to make zero must have xij >
0.Why?
Why?
Move from Bfs to Bfs
( ) : 1, ,B iA i m B
: , 1, ,j jA A j n B B
0 ( )1
( )m
i ij B i ji
x x A A b
Suppose that Aj wants to enter B.
Then, we choose
0
0minij
i
xi
ij
x
x
Make one of them zero, and keep others positive.
Move from Bfs to Bfs
( ) : 1, ,B iA i m B
: , 1, ,j jA A j n B B
0 ( )1
( )m
i ij B i ji
x x A A b
Suppose that Aj wants to enter B.
Then, we choose
0
0minij
i
xi
ij
x
x
Make one of them zero, and keep others positive.
1. How about if xi0 = 0 for some i?2. How about if all xij 0?
1. How about if xi0 = 0 for some i?2. How about if all xij 0?
Example
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
Examplex1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
Corresponding F of LPCorresponding F of LP
0
Ax b
x
1
1
1
0 0 0
0 0 0
0 0 0
0 0 0
1 1 1
1 0 0
0 0 1
0 3 1 1
A
1x 2x 3x 4x 5x 6x 7x
1 3 6 7, , ,A A A AB (2,0,2,0,0,1,4)x
4
2
3
6
b
Examplex1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
Corresponding F of LPCorresponding F of LP
0
Ax b
x
1
1
1
0 0 0
0 0 0
0 0 0
0 0 0
1 1 1
1 0 0
0 0 1
0 3 1 1
A
1 3 6 7, , ,A A A AB
4
2
3
6
b
2 2 1 4
1x 2x 3x 4x 5x 6x 7x
(2,0,2,0,0,1,4)x
5A 1 3 6 71 1 1 1A A A A 1 3 6 72 2 1 4A A A A
1 3 6 7(2 ) (2 ) (1 ) (4 )A A A A
Examplex1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
Corresponding F of LPCorresponding F of LP
0
Ax b
x
1
1
1
0 0 0
0 0 0
0 0 0
0 0 0
1 1 1
1 0 0
0 0 1
0 3 1 1
A
1 3 6 7, , ,A A A AB
4
2
3
6
b
2 2 1 4
1x 2x 3x 4x 5x 6x 7x
(2,0,2,0,0,1,4)x
5A 1 3 6 71 1 1 1A A A A 1 3 6 72 2 1 4A A A A
1 3 6 7(2 ) (2 ) (1 ) (4 )A A A A
Choosing =1 makes A6 leaves B.Choosing =1 makes A6 leaves B.
Examplex1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
x1
x2
x3
x1
x2
x3
1 2 3
1
3
2 3
1
2
3
4
2
3
3 6
0
0
0
x x x
x
x
x x
x
x
x
(0, 0, 3)(1, 0, 3)
(2, 0, 2)
(2, 0, 0)
(2, 2, 0)(0, 2, 0)
(0, 1, 3)
(0, 0, 0)
Corresponding F of LPCorresponding F of LP
0
Ax b
x
1
1
1
0 0 0
0 0 0
0 0 0
0 0 0
1 1 1
1 0 0
0 0 1
0 3 1 1
A
1 3 6 7, , ,A A A AB
4
2
3
6
b
2 2 4
1x 2x 3x 4x 5x 6x 7x
(2,0,2,0,0,1,4)x
1 3 6 7(2 ) (2 ) (1 ) (4 )A A A A
Choosing =1 makes A6 leaves B.Choosing =1 makes A6 leaves B.
1 3 5 7, , ,A AAAB (1,0,3,0,1,0,3)x
0 11
3
1
0
3
= 0
= 1
Linear Programming
Organization ofa Tableau
大同大學資工所智慧型多媒體研究室
Example
1 2 3
1 2 3 4
2 3 5
3 2 1
5 3
2 5 4
x x x
x x x x
x x x x
Ans x1 x2 x3 x4 x5
1
3
4
3
5
2
2
1
5
1
1
1
0
1
0
0
0
1
Ans x1 x2 x3 x4 x5
1
2
3
3
2
1
2
1
3
1
0
0
0
1
0
0
0
1
R1
R2R1
R3R1
ElementaryRow operations
Example
1 2 3
1 2 3 4
2 3 5
3 2 1
5 3
2 5 4
x x x
x x x x
x x x x
Ans x1 x2 x3 x4 x5
1
2
3
3
2
1
2
1
3
1
0
0
0
1
0
0
0
1
R1
R2R1
R3R1
B
3 4 51 2 3A A A
Suppose that we want to choose A1 to enter the basis, i.e., choosing A1 as the pivot column.
1 3 4 53 2 1A A A A
3 4 5(1 3 ) (2 2 ) (3 )A A A
1/3
2/2
min
0 (0,0,1,2,3)bfs
Example
1 2 3
1 2 3 4
2 3 5
3 2 1
5 3
2 5 4
x x x
x x x x
x x x x
Ans x1 x2 x3 x4 x5
1
2
3
3
2
1
2
1
3
1
0
0
0
1
0
0
0
1
R1
R2R1
R3R1
B
3 4 51 2 3A A A
1 3 4 53 2 1A A A A
3 4 5(1 3 ) (2 2 ) (3 )A A A
1/3
2/2
min
0 (0,0,1,2,3)bfs
101 41 3 3 3( ,0,0, , )bfs
Example
1 2 3
1 2 3 4
2 3 5
3 2 1
5 3
2 5 4
x x x
x x x x
x x x x
Ans x1 x2 x3 x4 x5
1
2
3
3
2
1
2
1
3
1
0
0
0
1
0
0
0
1
1/3
2/2
min
101 41 3 3 3( ,0,0, , )bfs
Ans x1 x2 x3 x4 x5
1/3
4/3
10/3
1
0
0
2/3
7/3
11/3
1/3
2/3
1/3
0
1
0
0
0
1
113 R
22 13R R
13 13R R