duality theory for composite geometric programming
TRANSCRIPT
Duality Theory for Composite
Geometric Programming
Ya-Ping Wang
Department of Industrial Engineering
University of Pittsburgh
December 14, 2012
DualityTheoryforCompositeGeometricProgramming
Page 1 of 27
Geometric Program (GP)
Primal posynomial program (GP)
(GP) 0inf ( ) . . ( ) 1, : {1,..., }m k
RG s t G k K p
0<tt t
Note that there are
m design variables, 1( , , ) , :{1,..., }mt t J m t 0 ,
each Gk(t) is a sum of terms indexed by the set [k]:
[ ]( ) : ( ), 0,1, , , k ii k
G U k p
t t
Across the p+1 different functions there are a total of
n terms 1, , , {1, , };whereU U I nn
, ( ) : , 0, ija
i i j i ijj Ji I U C t C a R
t ,
These terms are sequentially distributed into the (p+1)
problem functions as follows:
0 1[0] [1] [ ]; , 0< | [ ] |p kI p n n n n n k
The “exponent matrix” is given by ,, 1:[ ]n m
ij i ja A
DualityTheoryforCompositeGeometricProgramming
Page 2 of 27
Equivalent Formulations of (GP)
In the variables = ln , , j jz t j J
ln ( ) ln ln , , lni i ij j i i ij JU C a t c i I c C
it a z
[ ]ˆln ( ) ln exp[ ] : ( ), .i
k i ki kG c g k K
t a z z
A convex formulation of GP:
(GP)z: 0inf ( ) . . ( ) 0, m k
Rg s t g k K
zz z
DualityTheoryforCompositeGeometricProgramming
Page 3 of 27
Equivalent Formulations of (GP) (cont’d)
In the variables =: ix i Iia z,
[ ]( ) ln ( ) ln exp( ) : ( )k k
k k i ii kg G x c geo
z t x c
where [ ]
( ) : ln[ exp ] : knkii k
geo x R R
x is called a
geometric function (also called logexp(x)), [ ][ ]ki i kx x
and [ ][ ]ki i kc c .
A GGP (Generalized Geometric Programming)
formulation of GP: (GP)x
0 0 inf ( ) . . ( ) 0, ,k kgeo s t geo k K x x c x c x P
where | m nR R Az zP is the column space of A.
DualityTheoryforCompositeGeometricProgramming
Page 4 of 27
Dual Posynomial Program (GD)
ˆ [ ]
0
1
[ ]
sup ( ) : (Dual function)
. . 0, , 1 (normality condition)
0, (orthogonality conditions)
where : ,
i
ni k ik K i k
R
i
n
ij ii
k ii k
V C
s t i I
a j J
ˆ .k K
Each dual variable i corresponds to a primal term iU .
Degree of difficulty is defined as ( 1)d n m
The log-dual function
0 [ ]
1 1
( ) : ln ( ) ln /
( ln ) ln
p
i i k ik i k
pn
i i i k ki k
v V C
c
is concave on its domain of definition, and differentiable
in the interior of its domain.
DualityTheoryforCompositeGeometricProgramming
Page 5 of 27
Main Lemma of GP
Lemma: If t is feasible for primal program (GP) and is
feasible for dual program (GD), then
0 ( ) ( )G Vt .
Moreover, under the same conditions 0 ( ) ( )G Vt if, and
only if, one of the following two sets of equivalent
extremality conditions holds:
I. ( ) 1, , (1)
ˆ( ) ( ), [ ], (2)
kk
i k k i
G k K
G U i k k K
t
t t
II. 0( ) ( ), [0] (1)
( ), [ ], (2)i
ik i
U G i
U i k k K
t t
t
in which case t is optimal for primal program (GP) and
is optimal for dual program (GD).
DualityTheoryforCompositeGeometricProgramming
Page 6 of 27
Dual to Primal Conversion at Optimality
When 0 ( ) ( )G Vt ,
we get from condition II on p.5 the equations
( ), [0]
( ), [ ], , and 0
ii
i k k
V iU
i k k K
t
,
which are log-linear:
1
ln( ( )), [0]
ln , [ ], for which 0
mi
ij j ij i k k
V ia z c
i k k K
So, knowing the optimal dual solution an optimal
primal solution can in general be easily recovered by
solving this log-linear system for z and hence for t.
DualityTheoryforCompositeGeometricProgramming
Page 7 of 27
EXAMPLE:
(Maximum Likelihood Estimator of a Bernoulli Parameter)
Suppose that n independent trials, each of which is a
success with probability p, are performed. What is the
maximum likelihood estimator (MLE) of p?
The likelihood function to be maximized is
1, 1( , | ) (1 ) , with , 0 or 1,n n
ns n sn n i if x x p p p s x x i
Example: n=3, m=2: 1
( , ) 0inf s.t. 1n ns s n
p qf p q p q
The exponent matrix and a unique solution from
extremality condition II (2) on p.5: * *
1
1 **2
2 *1
3
=
1
ˆ 1 0 , so
0 1
n n
n nn
n
p q n
s s ns S
s p p Xn n
n s
DualityTheoryforCompositeGeometricProgramming
Page 8 of 27
Prior Extensions to GP
Signomial GP which allow some <0iC non-
convex programs: duality results are not strong
Peterson’s Generalized GP (GGP)More general separable convex programs
Extensions to GP investigated in this thesis
Composite GP (CGP): itself a special case of Peterson’s GGP. It includes as special cases:
Exponential GP (EGP)
Quadratic GP (QGP)
(lpGP)
We start with Exponential GP (EGP) using a motivating example…
DualityTheoryforCompositeGeometricProgramming
Page 9 of 27
A Motivational Example
(Maximum Likelihood Estimator of a Poisson Parameter)
Suppose 1, , nX X are independent Poisson random
variables each having mean λ. Determine the MLE of λ.
The likelihood function to be maximized is
1, 1 1( , | ) / ( ! !), where :n
ns nn n n ii
f x x e x x s x
Equivalently, one can 0 min ns ne . Although this is
not a posynomial, it can be solved as an EGP:
11 1
1
From condition (e) on p.14, we get
1 ˆ / = , so /1
nn n
n
y sy s n S n X
s
This same method also works for Exponential and Normal
parameters…
DualityTheoryforCompositeGeometricProgramming
Page 10 of 27
Exponential Geometric Program (EGP)
In the previous example:
a posynomial term is multiplied by an exponential
factor of another posynomial term.
This gives rise to an EGP problem where some
posynomial term ( )iU t is multiplied by an exponential
factor of another posynomial ( ) : ( )i ll iE V
t t .
Primal EGP problem (EGP): (cf (GP) on p.1)
0inf ( ) . . ( ) 1, ,m kRG s t G k K
0
<tt t
where
[ ]
1
ˆ( ) : ( ) exp ( ) , : {0}
( ) , ,and ( ) ,
{1, , } 1 ,| | 0,
ij lj
k i li k l i
a b
i i j l l jj J j J
i n
G U V k K K
U C t i I V D t l L
L r n i r r r r
t t t
t t
[ ] :ljB b r m
DualityTheoryforCompositeGeometricProgramming
Page 11 of 27
Equivalent formulations of (EGP)
In the variables
A convex form of (EGP): (EGP)z
0inf ( ) . . ( ) 0, m k
Rg s t g k K
zz z , where
[ ]ˆ( ) : ln ( ) ln exp exp , i
k k i li k l ig G c d k K
lz t a z b z +
In the variables : , : , i lx i I l L i la z, b z,
[ ] [ ]( ) ( , ) with exp,k k k kk lg geo h l L z x c ξ d
where
[ ]
[ ]( , ) : ln exp exp( )k k
i li k l igeo x
x ξ
:k
kn rR R R is called an exponential geometric
function, [ ][ ] [ ]
[ ] , [ ] , :irk i i ki k l l i ii k
R r r ξ ξ ξ ,
and [ ]kd is similarly defined.
= ln , , j jz t j J
DualityTheoryforCompositeGeometricProgramming
Page 12 of 27
Equivalent formulations of (EGP) (cont’d)
A GGP formulation of (EGP):
(EGP)x,ξ
0 0 [0] [0]
( )
[ ] [ ]
inf ( , )
. .
( , ) 0, ,( )
n rR R
k k k k
geo
s t
geo k K
x,x c ξ d
x c ξ d x,ξ
P
where | m n rM R R z zP is the column space of the
composite exponent matrix A
BM
.
DualityTheoryforCompositeGeometricProgramming
Page 13 of 27
Dual of EGP program (EGD) (cf (GD) on p.4)
ˆ( ) ( , ) [ ]
0
1 1
sup ( , ) :
. . 0, , 0, , 1,
0, (Orthogonality Conditions)
0
i l
n r
y
i k l i
R R i k l ik K i Ii l
i l
n r
ij i lj li l
i
C D yV e
y
s t y i I l L
a y b j J
y
,
yy
[ ]
0, , (*) p.16
ˆ where : , , : =
l
k i li k l L
l i i I
y k K
Note that ( ), ( )i i l ly U V t t
and degree of difficulty ( 1)d n r m →p.4
The log-dual function (cf p.4)
ˆ [ ]
( , ) ln / ln /i i k i l l i li k l ik K i I
v y C y D y
y
is usc proper concave in( , )y , and differentiable in the
interior of its domain.
DualityTheoryforCompositeGeometricProgramming
Page 14 of 27
Main Lemma of EGP
Lemma If t is feasible for primal program (EGP) and
( , )y is feasible for dual program (EGD), then 0 ( ) ( , )G V y t
Moreover, under the same conditions, 0 ( ) ( , )G V y t if
and only if one of the following two sets of equivalent
extremality conditions hold: (cfI&IIonp.5)
I ' Condition (e) & I ( ) 1, ,
ˆ( ) ( ), [ ],
kk
i k k i
G k K
y G U i k k K
t
t t
II ' Condition (e) & II 0( ) ( ), [0]
( ), [ ],
ii
k i
U G iy
U i k k K
t t
t
where (e) ( ), , l i lyV l i i I t
in which case t is optimal for primal program (EGP) and
( , )y is optimal for dual program (EGD).
p.9 Solve the previous motivational Example.
DualityTheoryforCompositeGeometricProgramming
Page 15 of 27
Composite Geometric Program (CGP)
Primal CGP Program (CGP) (cf (EGP) on p.10)
0inf ( ) . . ( ) 1, m kRG s t G k K
0
<tt t , where
[ ]ˆ ( ) ( ) exp (ln ( )) ,k i l li k l i
G U h V k K
t t t ,
:lh R R is a differentiable and strictly convex function.
Note that:
(QGP): if 212( ) ,lh l L
(lpGP): if ( ) | | / , where 1,lpl l lh p p l L
DualityTheoryforCompositeGeometricProgramming
Page 16 of 27
Composite Geometric Program (CGP) (cont’d)
A Convex Form of (CGP): (CGP)z
0 inf ( ) . . ( ) 0, m kRg s t g k K
zz z , where
[ ]( ) : ln ( ) ln exp i l
k k i l li k l ig G c h d
z t a z b z
A GGP Form of (CGP): (CGP)x,
0 [0] [ ]0
( )inf ( , ) . . ( , ) 0, , ( )
n r
k kk
R Rf s t f k K
ξ ξ ξ
x,x x x, P
with [ ]( , ) ( , ) : ln expk i l li k l i
f geo x h
DualityTheoryforCompositeGeometricProgramming
Page 17 of 27
Dual program (CGD) (cf (EGD) on p.13)
*
ˆ( ) [ ]
0
1 1
sup ( , ) : exp ( / )
. . 0, , 1, (Normality Condition)
0, (Ortho
i
n r
y
i kl l i l l i
R R l ii kk K i Ii
i
n r
ij i lj li l
CV d y h y
y
s t y i I
a y b j J
y,η
y
[ ]
gonality Conditions)
, , (**)
ˆwhere : , .
l i l
k ii k
y J l i i I
y k K
where *
lh , the conjugate of lh , has domain interval lJ .
(**) 0l in (EGD) on p. 13 and that in any (CGD)
(*) 0 0, , i ly l i i I
The log-dual function (cf p.13)
*
ˆ [ ]
( , ) ln / ( / )i i k i l l i l l ii k l ik K i I
v y C y d y h y
y
is usc proper concave in( , )y , and differentiable in the
interior of its domain.
DualityTheoryforCompositeGeometricProgramming
Page 18 of 27
Main Lemma of CGP
Lemma: If t is feasible for primal program (CGP) and
( , )y is feasible for dual program (CGD), then
0 ( ) ( , )G V t y .
Moreover, under the same conditions, 0 ( ) ( , )G V t y if
and only if one of the following two sets of equivalent
extremality conditions hold: (cfI’&II’onp.14)
I '' Condition (c) & I ( ) 1, ,
ˆ( ) ( ), [ ],
kk
i k k i
G k K
y G U i k k K
t
t t
II '' Condition (c) & II 0( ) ( ), [0]
( ), [ ],
ii
k i
U G iy
U i k k K
t t
t
where (c) ' (ln ( )), , ll i ly h V l i i I t
in which case t is optimal for primal program (CGP) and
( , )y is optimal for dual program (CGD).
(lpGP): (c) 1| ln ( ) | sgn(ln ( )), ,lpl i l ly V V l i i I t t
DualityTheoryforCompositeGeometricProgramming
Page 19 of 27
First Duality Theorem of CGP
NOTE: The statements for EGP, QGP, lpGP cases are
almost the same.
Theorem: Suppose that primal program (CGP) is super-
consistent. Then the following three conditions are
equivalent:
1) t’ is a minimal solution to (CGP).
2) There exists a vector ' pR for z′ (where z′= ln t′)
such that ( ', ')z forms a saddle point of ( , )l z .
where ( , )l z is the Lagrangian of (CGP)z.
DualityTheoryforCompositeGeometricProgramming
Page 20 of 27
3) There exists a vector ' pR for z′ (where z′= ln t′)
such that ( ', ')z satisfies the KKT conditions for
(CGP)z:
in which case the set of all such vectors 'λ is a non-empty
compact convex subset of pR , and the dual program
(CGD) also has a maximum solution ( ', ')y such that
0min(CGP) ( ') ( ', ') max(CGD)G V t y
and they satisfy the extremality conditions I ''& II '' on
p.18.
(Perfect Duality)
' '
' '0ˆ
( ) 0, ( ) 0, ( ) 0,
( ) ( , ') ( ) , 1
k k k k
z k kk K
a g g k K
b l g where
0
z' z'
z' z'
DualityTheoryforCompositeGeometricProgramming
Page 21 of 27
Second Duality Theorem of EGP
From linear programming duality theory:
Of the following two linear systems, exactly one has a
solution (where :A
BM n r m
):
(I) Find z with 0 0
A
Bz
(II) Find >0 with T TA B
y
y 0
We say that program (EGD) is canonical if system (II)
has a solution when M is the program’s composite
exponent matrix.
Theorem: Suppose that primal program (EGP) is
consistent. Then the minimum set of program (EGP)z is
non-empty and bounded if, and only if, dual program
(EGD) is canonical, in which case program (EGP) has a
minimum solution t′.
DualityTheoryforCompositeGeometricProgramming
Page 22 of 27
Second Duality Theorem of lpGP*
* This theorem also applies to its special case QGP.
Again, from linear programming duality theory
Of the following two linear systems exactly one has a
solution (where :A
BM n r m
):
(I) Find z with , 0 Az 0 Bz 0
(II) Find with , and T TA B
y
y 0 y 0
We say that program (lpGD) is canonical if system (II)
has a solution when M is the program’s composite
exponent matrix.
Theorem 7.3.2 Suppose that primal program (lpGP) is
consistent. Then the minimum set of program (lpGP)z is
non-empty and bounded if and only if its dual program
(lpGD) is canonical.
DualityTheoryforCompositeGeometricProgramming
Page 23 of 27
SUMMARY OF CONTRIBUTIONS
Extension of traditional GP models to
o to the more general Exponential GP models,
o to the even more general Composite GP models,
which include as important special cases:
(EGP)
(lpGP)
(QGP)
Showing that all of these are special cases of
Peterson’s GGP models (for which he has given a
Main Lemma but not a First or a Second duality
theorem).
DualityTheoryforCompositeGeometricProgramming
Page 24 of 27
CONTRIBUTIONS (cont’d)
For each extension:
o multiple direct proofs of the Main Lemma, with a
2nd set of equivalent Extremality Conditions.
For the First Duality Theorem, proof that
o a superconsistent primal program (CGP) has a
minimal solution 't if, and only if, there exists a
vector ' pR such that ( ', ')z forms a saddle
point of the Lagrangian ( , )l z and if, and only
if, it satisfies the KKT conditions for (CGP)z,
o in which case the set of all such Lagrange
multiplier vectors λ is a non-empty compact
convex subset of pR , whereas the original
theorem is a “If…then” statement and only
showed the existence of λ for the (GP) case only.
DualityTheoryforCompositeGeometricProgramming
Page 25 of 27
For the Second Duality Theorem, proof that
o for the special (and more important) cases of
(QGP), (lpGP) and (EGP),
the minimum set of a consistent primal
program is non-empty and bounded if and
only if its dual program is canonical
the original theorem showed the existence of
a primal solution for the (GP) case only,
when its dual program is canonical.
Sensitivity Analysis: If *z is optimal for (CGP)z:
* * ** * *0 0 0
* ** * * *0 0
( ) ( ) ( ), , ,
( ) ( ), , where ln
i l ki l k
i j l j k kij lj
g g gy
c d b
g gy z z b B
a b
z z z
z z
Dual to primal conversion of optimal solutions
when there is no duality gap: 0 ( ) ( , )G V y t .
DualityTheoryforCompositeGeometricProgramming
Page 26 of 27
Directions for Future Research
1. Development of proofs of a strong version of the
First Duality Theorem of (CGP): If a primal program
(CGP) is superconsistent and has a finite infimum,
then the dual program (CGD) has a maximum
solution ( ', ')y such that
inf(CGP) max(CGD) ( ', ')V y
2. Development of computational algorithms.
DualityTheoryforCompositeGeometricProgramming
Page 27 of 27
QUESTIONS