Science it, aalto scip Matlab-course series, 16.4.2019, Heikki Apiola and Juha Kuortti
>> myopts=optimoptions('fminbnd') % leads to error and advice:
'Use OPTIMSET to create options for FMINBND'
This confused me in the end of lecture, it must have changed into vers. 2019a.
Now you can create the default optim-struct like this:
>> myopts=optimset('fminbnd');
Parallel Computing Toolbox User's Guide [login MathWoks (?)]
Optimization Toolbox™ User's Guide [login MathWoks (?)]
Global Optimization Toolbox, User's Guide [login MathWoks (?)]
Optimization Toolbox, help
Choose a Parallel Computing Solution
fmincon
fminunc
fgoalattain
fminimax
fsolve
lsqcurvefit
lsqnonlin
These solvers use parallel gradient estimation under the following conditions:
functíon z=objfun(x)
x1=x(1);x2=x(2);
z = (4 - 2.1*x1^2 + x1^4/3)*x1^2 + x1*x2 + (-4 + 4*x2^2)*x2^2;
Choose an optimization solver. Create an objective function, typically the function you want to minimize. Create constraints, if any. Set options, or use the default options. Call the appropriate solver.For a basic nonlinear optimization example, see Solve a Constrained Nonlinear Problem.
x = fmincon(fun,x0,A,b,Aeq,beq,lb,ub,nonlincon,options)
fun: function to be minimized "objective function"
x0: starting point: vector or n-column matrix
A*x <= b: Linear inequality constraint, A matrix, b vector
Aeq*x = beq: Linear equality constraint Aeq matrix, beq vector
lb,ub : Lower bound, Upper bound: scalars, vectors or matrices
nonlincon: Nonlinear constraints: function
>>options=optimoptions('fmincon') % Creates struct with defaults...
% to be modified according to needs.
Use empty braces [] for missing arguments.
The objective function fun for all optimization routines is of the form
(1) fun=@(x) f(x(1),x(2), ..., x(n)).
Thus fun is a function of
one vector argument. The input x can also be an $m\times n$ matrix.
Therefore the definition can also be given in the vectorized form:
(2) fun=@(x) f(x(:,1),x(:,2),...,x(:,n)).
Both forms are equally good for the optimization routine, but the latter can also be used for plotting, tabulation and experimantation with several starting points etc.
Note:
In the 2 variable case one could also do as follows:
f=@(x,y)g(x,y)
% g(x,y) is a vectorized expression with x's and y's.
objfun=@(x) f(x(1),x(2))
% Convert f into a function of one vector argument.
Thus f could be used with meshgrid and surf, contour, etc. and objfun would
be good for fmincon and others.
Banana-function of Rosenbrock:
$$r(x,y)=100(y -x^2)^2 + (1-x)^2$$
Using the above form (2) we could define:
rosenbrock=@(x)100*(x(:,2) - x(:,1).^2).^2 + (1 - x(:,1)).^2;
For optimization routines (here fmincon) it would be as good to write according
to form (1):
rosenbrock1=@(x)100*(x(2) - x(1)^2)^2 + (1 - x(1))^2;
Constraints: $c(x) \leq 0$ or $ceq(x)=0$.
This example includes only an inequality constraint, so you must pass an empty array [] as the equality constraint function ceq.
function [c,ceq] = unitdisk(x) c = x(1)^2 + x(2)^2 - 1; ceq = [ ];Note: Rosenbrock's function is a standard test function in optimization. It has a unique minimum value of 0 attained at the point [1,1]. Finding the minimum is a challenge for some algorithms because the function has a shallow minimum inside a deeply curved valley. The solution for this problem is not at the point [1,1] because that point does not satisfy the constraint.
options = optimoptions(@fmincon,...
'Display','iter','Algorithm','interior-point');
[x,fval] = fmincon(rosenbrock,[0 0],...
[],[],[],[],[],[],@unitdisk,options)
The six sets of empty brackets represent optional constraints that are not being used in this example. See the fmincon function reference pages for the syntax.
MATLAB outputs a table of iterations and the results of the optimization.
Local minimum found that satisfies the constraints.
Optimization completed because the objective function is non-decreasing in
feasible directions, to within the selected value of the function tolerance,
and constraints are satisfied to within the selected value of the constraint tolerance.
x =
0.7864 0.6177
fval =
0.0457
This example shows how to use two nonlinear optimization solvers and how to set options. The nonlinear solvers that we use in this example are fminunc and fmincon. (Much the same as above with Rosenbrock, but there's more, too.)
All the principles outlined in this example apply to the other nonlinear solvers, such as fgoalattain, fminimax, lsqnonlin, lsqcurvefit, and fsolve.
f = @(x,y) x.*exp(-x.^2-y.^2)+(x.^2+y.^2)/20; fsurf(f,[-2,2],'ShowContours','on')Function def. for optimization:
fun = @(x) f(x(1),x(2));
x0 = [-.5; 0];
options = optimoptions('fminunc','Algorithm','quasi-newton');
options.Display = 'iter';
[x, fval, exitflag, output] = fminunc(fun,x0,options);
>> syms x y
>> f(x,y)
ans =
x*exp(- x^2 - y^2) + x^2/20 + y^2/20
>> gradient(f(x,y),[x,y])
ans =
x/10 + exp(- x^2 - y^2) - 2*x^2*exp(- x^2 - y^2)
y/10 - 2*x*y*exp(- x^2 - y^2)
function f = expensive_objfun(x)
%EXPENSIVE_OBJFUN An expensive objective function used in
% optimparfor example.
% Simulate an expensive function by pausing
pause(0.1)
% Evaluate objective function
f = exp(x(1)) * (4*x(3)^2 + 2*x(4)^2 + 4*x(1)*x(2) + 2*x(2) + 1);
function [c,ceq] = expensive_confun(x)
%EXPENSIVE_CONFUN An expensive constraint function used in optimparfor example.
% Simulate an expensive function by pausing
pause(0.1);
% Evaluate constraints
c = [1.5 + x(1)*x(2)*x(3) - x(1) - x(2) - x(4);
-x(1)*x(2) + x(4) - 10];
% No nonlinear equality constraints:
ceq = [];
startPoint = [-1 1 1 -1];
options = optimoptions('fmincon','Display','iter','Algorithm','interior-point');
startTime = tic;
xsol = fmincon(@expensive_objfun,startPoint,[],[],[],[],[],[],...
@expensive_confun,options);
time_fmincon_sequential = toc(startTime);
Serial FMINCON optimization takes 17.0722 seconds
rng default % for reproducibility
try
gaAvailable = false;
nvar = 4;
gaoptions = optimoptions('ga','MaxGenerations',15,'Display','iter');
startTime = tic;
gasol = ga(@expensive_objfun,nvar,[],[],[],[],[],[],[],gaoptions);
time_ga_sequential = toc(startTime);
fprintf('Serial GA optimization takes %g seconds.\n',time_ga_sequential);
gaAvailable = true;
catch ME
warning(message('optimdemos:optimparfor:gaNotFound'));
end
Optimization terminated: maximum number of generations exceeded.
Serial GA optimization takes 80.2351 seconds
options = optimoptions(options,'UseParallel',true);
startTime = tic;
xsol = fmincon(@expensive_objfun,startPoint,[],[],[],[],[],[],...
@expensive_confun,options);
time_fmincon_parallel = toc(startTime);
fprintf('Parallel FMINCON optimization takes %g seconds.\n',...
time_fmincon_parallel);
Parallel FMINCON optimization takes 8.11945 seconds.
HOW MANY WORKERS?
rng default % to get the same evaluations as the previous run
if gaAvailable
gaoptions = optimoptions(gaoptions,'UseParallel',true);
startTime = tic;
gasol = ga(@expensive_objfun,nvar,[],[],[],[],[],[],[],gaoptions);
time_ga_parallel = toc(startTime);
fprintf('Parallel GA optimization takes %g seconds.\n',time_ga_parallel);
end
Optimization terminated: maximum number of generations exceeded.
Parallel GA optimization takes 15.6984 seconds.
X = [time_fmincon_sequential time_fmincon_parallel];
Y = [time_ga_sequential time_ga_parallel];
t = [0 1];
plot(t,X,'r--',t,Y,'k-')
ylabel('Time in seconds')
legend('fmincon','ga')
ax = gca;
ax.XTick = [0 1];
ax.XTickLabel = {'Serial' 'Parallel'};
axis([0 1 0 ceil(max([X Y]))])
title('Serial Vs. Parallel Times')
Utilizing parallel function evaluation via parfor improved the efficiency of both fmincon and ga. The improvement is typically better for expensive objective and constraint functions.
Compare timings: 1) sequential, 2) seq. with symbolic, 3) parallel (seq/symb)
Most Optimization Toolbox solvers run faster and more accurately when your objective and constraint function files include derivative calculations. Some solvers also benefit from second derivatives, or Hessians. While calculating a derivative is straightforward, it is also quite tedious and error-prone. Calculating second derivatives is even more tedious and fraught with opportunities for error. How can you get your solver to run faster and more accurately without the pain of computing derivatives manually?
This article demonstrates how to ease the calculation and use of gradients using Symbolic Math Toolbox. The techniques described here are applicable to almost any optimization problem where the objective or constraint functions can be defined analytically
Minimize: $$x + y + \cosh(x-1.1y) + \sinh(z/4)$$ over the region defined by the implicit equation $$z^2 = \sin(z-x^2y^2), -1\leq x \leq 1,-1\leq y \leq 1,-1\leq z \leq 1.$$ First write the objective and constraint functions symbolically:
confun=matlabFunction([],w,'vars',{t},'outputs',{'c','ceq'})
... Let's leave this here for a while...
syms x1 x2 real x = [x1;x2]; % column vector of symbolic variables f = log(1 + 3*(x2 - (x1^3 - x1))^2 + (x1 - 4/3)^2) % Symbolic expression
@(in1)deal(log((in1(1,:)-4.0./3.0).^2+(in1(1,:)+in1(2,:)-... in1(1,:).^3).^2.*3.0+1.0),[-(in1(1,:).*-2.0+(in1(1,:).^2.*3.0-1.0).*... (in1(1,:)+in1(2,:)
>> help fmincon
fmincon finds a constrained minimum of a function of several variables.
fmincon attempts to solve problems of the form:
min F(X) subject to: A*X <= B, Aeq*X = Beq (linear constraints)
X C(X) <= 0, Ceq(X) = 0 (nonlinear constraints)
LB <= X <= UB (bounds)
...