system optimization (1) liang yu department of biological systems engineering washington state...
TRANSCRIPT
System Optimization(1)
Liang Yu
Department of Biological Systems Engineering
Washington State University
04. 16. 2013
Outline
Background of Engineering Optimization
Application of Optimization
Introduction
Fundamentals of Optimization
Optimization Toolbox in Matlab
Optimization in Process Plants
Engineering applications of optimization
Some typical applications from different engineering disciplines
Design of water resources systems for maximum benefit
Design of pumps, turbines, and heat transfer equipment for maximum efficiency
Optimum design of chemical processing equipment and plants
Selection of a site for an industry
Optimum design of control systems
Application: Metabolic Flux AnalysisFlux Balance Analysis (FBA)
in silico simulation
Linear programming (LP)
Genome-scale
13C-assisted Metabolic Flux Analysis
in vivo search
Nonlinear programming (NLP)
Simplified model
maximize ∑ci ∙vi
s.t. S∙v = 0
lb < v < ub
minimize (MDVexp-MDVsim)2
s.t. S∙v = 0
IDV = f(v, IMM, IDV)
MDV = M∙IDV
lb < v < ub
Metabolic Steady state
Metabolic & isotopic Steady stateIDV: isotopomer distribution vectorMDV: mass distribution vector
Application for optimization of biorefinery configurations
Pham, Viet, and Mahmoud El‐Halwagi. "Process synthesis and optimization of biorefinery configurations." AIChE Journal 58.4 (2012): 1212-1221.
Part of the branching trees for the production of bio-alcohols from lignocellulosic bio-mass
Optimization Tree
IntroductionDefinition
Optimization is the act of obtaining the best result under given circumstances. It can be defined as the process of finding the conditions that give the maximum or minimum value of a function.
Goaleither to minimize the effort required or to maximize the desired benefit.
Optimization problem could be linear or non-linear.
Non –linear optimization is accomplished by numerical ‘Search Methods’.
Search methods are used iteratively before a solution is achieved.
The search procedure is termed as algorithm.
Introduction
Minimum of f (x) is same as maximum of −f (x)
Optimum solution is found while satisfying its constraint (derivative must be zero at optimum).
Introduction
Linear problem – solved by Simplex or Graphical methods.
The solution of the linear problem lies on boundaries of the feasible region.
Non-linear problem solution lies within and on the boundaries of the feasible region.
Solution of linear problem Three dimensional solution of non-linear problem
Introduction
Optimization Programming LanguagesGAMS - General Algebraic Modeling System
LINDO - Widely used in business applications
AMPL - A Mathematical Programming Language
Others: MPL, ILOG
Software with Optimization CapabilitiesExcel – SolverMATLABMathCADMathematicaMapleOthers
Statement of an optimization problem
An optimization or a mathematical programming problem can be stated
x1x2...xn
which minimizes f (X)Find X =
Subject to the constraints
gj (X) ≤ 0, j = 1, 2, . . . ,mlj (X) = 0, j = 1, 2, . . . , p
Fundamentals of Optimization
Single Objective function f(x)Maximization
Minimization
Design Variables, xi , i=0,1,2,3…..
ConstraintsInequality
Equality
Optimal pointsLocal minima/maxima points: A point or Solution x* is at local point if there is no other x in its Neighborhood less than x*
Global minima/maxima points: A point or Solution x** is at global point if there is no other x in entire search space less than x**
Example of design variables and constraints used in optimization.
Maximize X1 + 1.5 X2 Subject to:X1 + X2 ≤ 1500.25 X1 + 0.5 X2 ≤ 50X1 ≥ 50X2 ≥ 25X1 ≥0, X2 ≥0
Fundamentals of Optimization
Global versus local optimization. Local point is equal to global point if the function is convex.
Fundamentals of Optimization
Function f is convex if f(Xa) is less than value of the corresponding point joining f(X1) and f(X2).
Convexity condition – Hessian 2nd order derivative) matrix of function f must be positive semi definite ( Eigen values +ve or zero).
Convex and nonconvex set Convex function
Mathematical Background
Slop or gradient of the objective function f – represent the direction in which the function will decrease/increase most rapidly
Taylor series expansion
Jacobian – matrix of gradient of f with respect to several variables
x
f
x
xfxxf
dx
dfxx
00lim
)()(lim
.......)(!2
1)()( 2
2
2
xdx
fdx
dx
dfxxf
pp xxp
z
g
y
g
x
gz
f
y
f
x
f
J
Mathematical Background
Slope -First order Condition (FOC) – Provides function’s slope information
Hessian – Second derivative of function of several variables, Sign indicates max.(+ve) or min.(-ve)
Second order condition (SOC) Eigen values of H(X*) are all positive
Determinants of all lower order of H(X*) are +ve
2
22
2
2
2
y
f
yx
fxy
f
x
f
H
0*)( Xf
Optimization Algorithm
Deterministic - specific rules to move from one iteration to next , gradient, Hessian
Stochastic – probalistic rules are used for subsequent iteration
Optimal Design – Engineering Design based on optimization algorithm
Lagrangian method – sum of objective function and linear combination of the constraints.
Optimization Methods
DeterministicDirect Search – Use Objective function values to locate minimum
Gradient Based – first or second order of objective function.
Minimization objective function f(x) is used with –ve sign – f(x) for maximization problem.
Single VariableNewton – Raphson is Gradient based technique (FOC)
Golden Search – step size reducing iterative method
Multivariable Techniques ( Make use of Single variable Techniques specially Golden Section)
Unconstrained OptimizationPowell Method – Quadratic (degree 2) objective function polynomial is non-gradient based.
Gradient Based – Steepest Descent (FOC) or Least Square minimum (LMS)
Hessian Based -Conjugate Gradient (FOC) and BFGS (SOC)
Optimization Methods - ConstrainedConstrained Optimization
Indirect approach – by transforming into unconstrained problem.
Exterior Penalty Function (EPF) and Augmented Lagrange Multiplier
Direct Method Sequential Linear Programming (SLP), Sequential Quadratic Programming (SQP) and steepest Generalized Reduced Gradient Method (GRG)
Descent Gradient or LMS
Advanced Optimization Methods
Global Optimization – Stochastic techniques
Simulated Annealing (SA) method – minimum energy principle of cooling metal crystalline structure
Genetic Algorithm (GA) – Survival of the fittest principle based upon evolutionary theory
Optimization Toolbox in Matlab
Key featuresInteractive tools for defining and solving optimization problems and monitoring solution progress
Solvers for nonlinear and multiobjective optimization
Solvers for nonlinear least squares, data fitting, and nonlinear equations
Methods for solving quadratic and linear programming problems
Methods for solving binary integer programming problems
Parallel computing support in selected constrained nonlinear solvers
How to use Optimization Toolbox
Optimization FunctionsFunction files can be directly provided by M File
Syntax: [x,fval] = fminsearch(fun,x0)
Optimization Tool graphical user interface (GUI)Define and modify problems quickly
Use the correct syntax for optimization functions
Import and export from the MATLAB workspace
Generate code containing your configuration for a solver and options
Change parameters of an optimization during the execution of certain Global Optimization Toolbox functions
Function OptimizationOptimization concerns the minimization or maximization of functions
Standard Optimization Problem:
~
~minxf x
~
0jg x
~
0ih x
L Uk k kx x x
Equality ConstraintsSubject to:
Inequality Constraints
Side Constraints
~
f x is the objective function, which measure and evaluate the performance of a system. In a standard problem, we are minimizing the function. For maximization, it is equivalent to minimization of the –ve of the objective function.
Where:
~x is a column vector of design variables, which can
affect the performance of the system.
Function Optimization
Constraints – Limitation to the design space. Can be linear or nonlinear, explicit or implicit functions
~
0ih x
L Uk k kx x x
Equality Constraints
Inequality Constraints
Side Constraints
~
0jg x
Most algorithm require less than
Optimization Toolbox SolversMinimizers
This group of solvers attempts to find a local minimum of the objective function near a starting point x0. They address problems of unconstrained optimization, linear programming, quadratic programming, and general nonlinear programming.
Multiobjective minimizersThis group of solvers attempts to either minimize the maximum value of a set of functions (fminimax), or to find a location where a collection of functions is below some prespecified values (fgoalattain).
Equation solversThis group of solvers attempts to find a solution to a scalar- or vector-valued nonlinear equation f(x) = 0 near a starting point x0. Equation-solving can be considered a form of optimization because it is equivalent to finding the minimum norm of f(x) near x0.
Least-Squares (curve-fitting) solversThis group of solvers attempts to minimize a sum of squares. This type of problem frequently arises in fitting a model to data. The solvers address problems of finding nonnegative solutions, bounded or linearly constrained solutions, and fitting parameterized nonlinear models to data.
Objective Function
Linear
Quadratic
Sum-of-squares (Least squares)
Smooth nonlinear
Nonsmooth
Constraint Type
None (unconstrained)
Bound
Linear (including bound)
General smooth
Discrete (integer)
Select Solvers by Objective and Constraint
Constraint Type
Objective TypeLinear Quadratic Least Squares Smooth nonlinear Nonsmooth
None n/a (f = const, or min = )
quadprog, Theory, Examples
\, lsqcurvefit, lsqnonlin, Theory, Examples
fminsearch, fminunc, Theory, Examples
fminsearch, *
Bound linprog, Theory, Examples
quadprog, Theory, Examples
lsqcurvefit, lsqlin, lsqnonlin, lsqnonneg, Theory, Examples
fminbnd, fmincon, fseminf, Theory, Examples
*
Linear linprog, Theory, Examples
quadprog, Theory, Examples
lsqlin, Theory, Examples
fmincon, fseminf, Theory, Examples
*
General smooth fmincon, Theory, Examples
fmincon, Theory, Examples
fmincon, Theory, Examples
fmincon, fseminf, Theory, Examples
*
Discrete bintprog, Theory, Example
Minimization Algorithm
Minimization Algorithm (Cont.)
Equation Solving Algorithms
Least-Squares Algorithms
Implementing Optimization Toolbox
Most of these optimization routines require the definition of an M-file containing the function, f, to be minimized.
Maximization is achieved by supplying the routines with –f.
Optimization options passed to the routines change optimization parameters.
Default optimization parameters can be changed through an options structure.
Unconstrained Minimization
Consider the problem of finding a set of values [x1 x2]T that solves
Steps:Create an M-file that returns the function value (Objective Function). Call it objfun.m
Then, invoke the unconstrained minimization routine. Use fminunc
1
~
2 21 2 1 2 2
~min 4 2 4 2 1x
xf x e x x x x x
1 2~
Tx x x
Step 1 – Objective Function
function f = objfun(x)
f=exp(x(1))*(4*x(1)^2+2*x(2)^2+4*x(1)*x(2)+2*x(2)+1);
1 2~
Tx x x
Objective function
Step 2 – Invoke Routine
x0 = [-1,1];
options = optimset(‘LargeScale’,’off’);
[xmin,feval,exitflag,output]=
fminunc(‘objfun’,x0,options);
Output arguments Input arguments
Starting with a guess
Optimization parameters settings
Step 3 – Results
xmin =
0.5000 -1.0000
feval =
1.3028e-010
exitflag =
1
output =
iterations: 7
funcCount: 40
stepsize: 1
firstorderopt: 8.1998e-004
algorithm: 'medium-scale: Quasi-Newton line search'
Minimum point of design variables
Objective function value
Exitflag tells if the algorithm is converged.If exitflag > 0, then local minimum is found
Some other information
More on fminunc – Input
[xmin,feval,exitflag,output,grad,hessian]=
fminunc(fun,x0,options,P1,P2,…)
fun : Return a function of objective function.
x0 : Starts with an initial guess. The guess must be a vector
of size of number of design variables.
Option : To set some of the optimization parameters. (More after few slides)
P1,P2,… : To pass additional parameters.
More on fminunc – Output
[xmin,feval,exitflag,output,grad,hessian]=
fminunc(fun,x0,options,P1,P2,…)
• xmin: Vector of the minimum point (optimal point). The size is the number of design variables.
• feval: The objective function value of at the optimal point.
• exitflag: A value shows whether the optimization routine is terminated successfully. (converged if >0)
• Output: This structure gives more details about the optimization
• grad : The gradient value at the optimal point.
• hessian: The hessian value of at the optimal point
Next Class
Please take your laptop and install Matlab