gavindiaz/EA_with_Python
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
52067c19732df93dc0d8652dd3838908c30222fe
EA_with_Python/Include/Math/Alglib/optimization.mqh
T

51767 lines
2.1 MiB
Plaintext
Raw Blame History

//+------------------------------------------------------------------+
//| optimization.mqh |
//| Copyright 2003-2022 Sergey Bochkanov (ALGLIB project) |
//| Copyright 2012-2026, MetaQuotes Ltd. |
//| www.mql5.com |
//+------------------------------------------------------------------+
//| Implementation of ALGLIB library in MetaQuotes Language 5 |
//| |
//| The features of the library include: |
//| - Linear algebra (direct algorithms, EVD, SVD) |
//| - Solving systems of linear and non-linear equations |
//| - Interpolation |
//| - Optimization |
//| - FFT (Fast Fourier Transform) |
//| - Numerical integration |
//| - Linear and nonlinear least-squares fitting |
//| - Ordinary differential equations |
//| - Computation of special functions |
//| - Descriptive statistics and hypothesis testing |
//| - Data analysis - classification, regression |
//| - Implementing linear algebra algorithms, interpolation, etc. |
//| in high-precision arithmetic (using MPFR) |
//| |
//| This file is free software; you can redistribute it and/or |
//| modify it under the terms of the GNU General Public License as |
//| published by the Free Software Foundation (www.m_fsf.org); either |
//| version 2 of the License, or (at your option) any later version. |
//| |
//| This program is distributed in the hope that it will be useful, |
//| but WITHOUT ANY WARRANTY; without even the implied warranty of |
//| MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
//| GNU General Public License for more details. |
//+------------------------------------------------------------------+
#include "matrix.mqh"
#include "ap.mqh"
#include "alglibinternal.mqh"
#include "linalg.mqh"
//+------------------------------------------------------------------+
//| This structure is used to store OptGuard report, i.e. report on |
//| the properties of the nonlinear function being optimized with |
//| ALGLIB. |
//| After you tell your optimizer to activate OptGuardthis technology|
//| starts to silently monitor function values and gradients / |
//| Jacobians being passed all around during your optimization |
//| session. Depending on specific set of checks enabled OptGuard may|
//| perform additional function evaluations (say, about 3*N |
//| evaluations if you want to check analytic gradient for errors). |
//| Upon discovering that something strange happens (function values |
//| and/or gradient components change too sharply and/or |
//| unexpectedly) OptGuard sets one of the "suspicion flags" |
//| (without interrupting optimization session). |
//| After optimization is done, you can examine OptGuard report. |
//| Following report fields can be set: |
//| * nonc0suspected |
//| * nonc1suspected |
//| * badgradsuspected |
//| === WHAT CAN BE DETECTED WITH OptGuard INTEGRITY CHECKER ===== |
//| Following types of errors in your target function (constraints) |
//| can be caught: |
//| a) discontinuous functions ("non-C0" part of the report) |
//| b) functions with discontinuous derivative ("non-C1" part of |
//| the report) |
//| c) errors in the analytic gradient provided by user |
//| These types of errors result in optimizer stopping well before |
//| reaching solution (most often - right after encountering |
//| discontinuity). |
//| Type A errors are usually coding errors during implementation |
//| of the target function. Most "normal" problems involve continuous|
//| functions, and anyway you can't reliably optimize discontinuous |
//| function. |
//| Type B errors are either coding errors or (in case code itself is|
//| correct) evidence of the fact that your problem is an "incorrect"|
//| one. Most optimizers (except for ones provided by MINNS |
//| subpackage) do not support nonsmooth problems. |
//| Type C errors are coding errors which often prevent optimizer |
//| from making even one step or result in optimizing stopping too |
//| early, as soon as actual descent direction becomes too different |
//| from one suggested by user-supplied gradient. |
//| === WHAT IS REPORTED =========================================== |
//| Following set of report fields deals with discontinuous target |
//| functions, ones not belonging to C0 continuity class: |
//| * nonc0suspected - is a flag which is set upon discovering some|
//| indication of the discontinuity. If this flag is false, the |
//| rest of "non-C0" fields should be ignored |
//| * nonc0fidx - is an index of the function (0 for target |
//| function, 1 or higher for nonlinear constraints) which is |
//| suspected of being "non-C0" |
//| * nonc0lipshitzc - a Lipchitz constant for a function which was|
//| suspected of being non-continuous. |
//| * nonc0test0positive - set to indicate specific test which |
//| detected continuity violation (test #0) |
//| Following set of report fields deals with discontinuous gradient/|
//| Jacobian, i.e. with functions violating C1 continuity: |
//| * nonc1suspected - is a flag which is set upon discovering some|
//| indication of the discontinuity. If this flag is false, the |
//| rest of "non-C1" fields should be ignored |
//| * nonc1fidx - is an index of the function (0 for target |
//| function, 1 or higher for nonlinear constraints) which is |
//| suspected of being "non-C1" |
//| * nonc1lipshitzc - a Lipchitz constant for a function gradient |
//| which was suspected of being non-smooth. |
//| * nonc1test0positive - set to indicate specific test which |
//| detected continuity violation (test #0) |
//| * nonc1test1positive - set to indicate specific test which |
//| detected continuity violation (test #1) |
//| Following set of report fields deals with errors in the gradient:|
//| * badgradsuspected - is a flad which is set upon discovering an|
//| error in the analytic gradient supplied by user |
//| * badgradfidx - index of the function with bad gradient (0 for|
//| target function, 1 or higher for nonlinear constraints) |
//| * badgradvidx - index of the variable |
//| * badgradxbase - location where Jacobian is tested |
//| * following matrices store user-supplied Jacobian and its |
//| numerical differentiation version (which is assumed to be |
//| free from the coding errors), both of them computed near the |
//| initial point: |
//| * badgraduser, an array[K,N], analytic Jacobian supplied |
//| by user |
//| * badgradnum, an array[K,N], numeric Jacobian computed |
//| by ALGLIB |
//| Here K is a total number of nonlinear functions (target + |
//| nonlinear constraints), N is a variable number. |
//| The element of badgraduser[] with index [badgradfidx,badgradvidx]|
//| is assumed to be wrong. |
//| More detailed error log can be obtained from optimizer by |
//| explicitly requesting reports for tests C0.0, C1.0, C1.1. |
//+------------------------------------------------------------------+
struct COptGuardReport
{
bool m_nonc0suspected;
bool m_nonc0test0positive;
int m_nonc0fidx;
double m_nonc0lipschitzc;
bool m_nonc1suspected;
bool m_nonc1test0positive;
bool m_nonc1test1positive;
int m_nonc1fidx;
double m_nonc1lipschitzc;
bool m_badgradsuspected;
int m_badgradfidx;
int m_badgradvidx;
CRowDouble m_badgradxbase;
CMatrixDouble m_badgraduser;
CMatrixDouble m_badgradnum;
//--- constructor / destructor
COptGuardReport(void);
~COptGuardReport(void) {}
//---
void Copy(const COptGuardReport &obj);
//--- overloading
void operator=(const COptGuardReport &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
COptGuardReport::COptGuardReport(void)
{
m_nonc0suspected=false;
m_nonc0test0positive=false;
m_nonc0fidx=0;
m_nonc0lipschitzc=0;
m_nonc1suspected=false;
m_nonc1test0positive=false;
m_nonc1test1positive=false;
m_nonc1fidx=0;
m_nonc1lipschitzc=0;
m_badgradsuspected=false;
m_badgradfidx=0;
m_badgradvidx=0;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void COptGuardReport::Copy(const COptGuardReport &obj)
{
m_nonc0suspected=obj.m_nonc0suspected;
m_nonc0test0positive=obj.m_nonc0test0positive;
m_nonc0fidx=obj.m_nonc0fidx;
m_nonc0lipschitzc=obj.m_nonc0lipschitzc;
m_nonc1suspected=obj.m_nonc1suspected;
m_nonc1test0positive=obj.m_nonc1test0positive;
m_nonc1test1positive=obj.m_nonc1test1positive;
m_nonc1fidx=obj.m_nonc1fidx;
m_nonc1lipschitzc=obj.m_nonc1lipschitzc;
m_badgradsuspected=obj.m_badgradsuspected;
m_badgradfidx=obj.m_badgradfidx;
m_badgradvidx=obj.m_badgradvidx;
m_badgradxbase=obj.m_badgradxbase;
m_badgraduser=obj.m_badgraduser;
m_badgradnum=obj.m_badgradnum;
}
//+------------------------------------------------------------------+
//| This structure is used for detailed reporting about suspected C0 |
//| continuity violation. |
//| === WHAT IS TESTED ============================================= |
//| C0 test studies function values (not gradient!) obtained |
//| during line searches and monitors estimate of the Lipschitz |
//| constant. Sudden spikes usually indicate that discontinuity was |
//| detected. |
//| === WHAT IS REPORTED =========================================== |
//| Actually, report retrieval function returns TWO report |
//| structures: |
//| * one for most suspicious point found so far (one with highest |
//| change in the function value), so called "strongest" report |
//| * another one for most detailed line search (more function |
//| evaluations = easier to understand what's going on) which |
//| triggered test #0 criteria, so called "longest" report |
//| In both cases following fields are returned: |
//| * positive - is TRUE when test flagged suspicious point; |
//| FALSE if test did not notice anything (in the latter cases |
//| fields below are empty). |
//| * fidx - is an index of the function (0 for target function, 1 |
//| or higher for nonlinear constraints) which is suspected of |
//| being "non-C1" |
//| * x0[], d[] - arrays of length N which store initial point and |
//| direction for line search (d[] can be normalized, but does |
//| not have to) |
//| * stp[], f[] - arrays of length CNT which store step lengths |
//| and function values at these points; f[i] is evaluated in |
//| x0+stp[i]*d. |
//| * stpidxa, stpidxb - we suspect that function violates C1 |
//| continuity between steps #stpidxa and #stpidxb (usually we |
//| have stpidxb=stpidxa+3, with most likely position of the |
//| violation between stpidxa+1 and stpidxa+2. |
//| You can plot function values stored in stp[] and f[] arrays and |
//| study behavior of your function by your own eyes, just to be sure|
//| that test correctly reported C1 violation. |
//+------------------------------------------------------------------+
struct COptGuardNonC0Report
{
int m_stpidxb;
int m_cnt;
int m_stpidxa;
int m_n;
int m_fidx;
bool m_positive;
CRowDouble m_d;
CRowDouble m_x0;
CRowDouble m_stp;
CRowDouble m_f;
//--- constructor / destructor
COptGuardNonC0Report(void);
~COptGuardNonC0Report(void) {}
//---
void Copy(const COptGuardNonC0Report &obj);
//--- overloading
void operator=(const COptGuardNonC0Report &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
COptGuardNonC0Report::COptGuardNonC0Report(void)
{
m_stpidxb=0;
m_cnt=0;
m_stpidxa=0;
m_n=0;
m_fidx=0;
m_positive=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void COptGuardNonC0Report::Copy(const COptGuardNonC0Report &obj)
{
m_stpidxb=obj.m_stpidxb;
m_cnt=obj.m_cnt;
m_stpidxa=obj.m_stpidxa;
m_n=obj.m_n;
m_fidx=obj.m_fidx;
m_positive=obj.m_positive;
m_d=obj.m_d;
m_x0=obj.m_x0;
m_stp=obj.m_stp;
m_f=obj.m_f;
}
//+------------------------------------------------------------------+
//| This structure is used for detailed reporting about suspected C1 |
//| continuity violation as flagged by C1 test #0 (OptGuard has |
//| several tests for C1 continuity, this report is used by #0). |
//| === WHAT IS TESTED ============================================= |
//| C1 test #0 studies function values (not gradient!) obtained |
//| during line searches and monitors behavior of directional |
//| derivative estimate. This test is less powerful than test #1, but|
//| it does not depend on gradient values and thus it is more robust |
//| against artifacts introduced by numerical differentiation. |
//| === WHAT IS REPORTED =========================================== |
//| Actually, report retrieval function returns TWO report |
//| structures: |
//| * one for most suspicious point found so far (one with highest |
//| change in the directional derivative), so called "strongest" |
//| report |
//| * another one for most detailed line search (more function |
//| evaluations = easier to understand what's going on) which |
//| triggered test #0 criteria, so called "longest" report |
//| In both cases following fields are returned: |
//| * positive - is TRUE when test flagged suspicious point; |
//| FALSE if test did not notice anything (in the latter cases |
//| fields below are empty). |
//| * fidx - is an index of the function (0 for target function, |
//| 1 or higher for nonlinear constraints) which is suspected of |
//| being "non-C1" |
//| * x0[], d[] - arrays of length N which store initial point and |
//| direction for line search (d[] can be normalized, but does |
//| not have to) |
//| * stp[], f[] - arrays of length CNT which store step lengths |
//| and function values at these points; f[i] is evaluated in |
//| x0+stp[i]*d. |
//| * stpidxa, stpidxb - we suspect that function violates C1 |
//| continuity between steps #stpidxa and #stpidxb (usually we |
//| have stpidxb=stpidxa+3, with most likely position of the |
//| violation between stpidxa+1 and stpidxa+2. |
//| You can plot function values stored in stp[] and f[] arrays |
//| and study behavior of your function by your own eyes, just to |
//| be sure that test correctly reported C1 violation. |
//+------------------------------------------------------------------+
struct COptGuardNonC1Test0Report
{
int m_cnt;
int m_fidx;
int m_n;
int m_stpidxa;
int m_stpidxb;
bool m_positive;
CRowDouble m_d;
CRowDouble m_f;
CRowDouble m_stp;
CRowDouble m_x0;
//--- constructor / destructor
COptGuardNonC1Test0Report(void);
~COptGuardNonC1Test0Report(void) {}
//---
void Copy(const COptGuardNonC1Test0Report &obj);
//--- overloading
void operator=(const COptGuardNonC1Test0Report &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
COptGuardNonC1Test0Report::COptGuardNonC1Test0Report(void)
{
m_cnt=0;
m_fidx=0;
m_n=0;
m_stpidxa=0;
m_stpidxb=0;
m_positive=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void COptGuardNonC1Test0Report::Copy(const COptGuardNonC1Test0Report &obj)
{
m_cnt=obj.m_cnt;
m_fidx=obj.m_fidx;
m_n=obj.m_n;
m_stpidxa=obj.m_stpidxa;
m_stpidxb=obj.m_stpidxb;
m_positive=obj.m_positive;
m_d=obj.m_d;
m_f=obj.m_f;
m_stp=obj.m_stp;
m_x0=obj.m_x0;
}
//+------------------------------------------------------------------+
//| This structure is used for detailed reporting about suspected C1 |
//| continuity violation as flagged by C1 test #1 (OptGuard has |
//| several tests for C1 continuity, this report is used by #1). |
//| === WHAT IS TESTED ============================================= |
//| C1 test #1 studies individual components of the gradient as |
//| recorded during line searches. Upon discovering discontinuity in |
//| the gradient this test records specific component which was |
//| suspected (or one with highest indication of discontinuity if |
//| multiple components are suspected). |
//| When precise analytic gradient is provided this test is more |
//| powerful than test #0 which works with function values and |
//| ignores user-provided gradient. However, test #0 becomes |
//| more powerful when numerical differentiation is employed (in such|
//| cases test #1 detects higher levels of numerical noise and |
//| becomes too conservative). |
//| This test also tells specific components of the gradient which |
//| violate C1 continuity, which makes it more informative than #0, |
//| which just tells that continuity is violated. |
//| === WHAT IS REPORTED =========================================== |
//| Actually, report retrieval function returns TWO report |
//| structures: |
//| * one for most suspicious point found so far (one with highest |
//| change in the directional derivative), so called "strongest" |
//| report |
//| * another one for most detailed line search (more function |
//| evaluations = easier to understand what's going on) which |
//| triggered test #1 criteria, so called "longest" report |
//| In both cases following fields are returned: |
//| * positive - is TRUE when test flagged suspicious point; |
//| FALSE if test did not notice anything (in the latter cases |
//| fields below are empty). |
//| * fidx - is an index of the function (0 for target function, |
//| 1 or higher for nonlinear constraints) which is suspected of |
//| being "non-C1" |
//| * vidx - is an index of the variable in [0,N) with nonsmooth |
//| derivative |
//| * x0[], d[] - arrays of length N which store initial point and |
//| direction for line search (d[] can be normalized, but does |
//| not have to) |
//| * stp[], g[] - arrays of length CNT which store step lengths |
//| and gradient values at these points; g[i] is evaluated in |
//| x0+stp[i]*d and contains vidx-th component of the gradient. |
//| * stpidxa, stpidxb - we suspect that function violates C1 |
//| continuity between steps #stpidxa and #stpidxb (usually we |
//| have stpidxb=stpidxa+3, with most likely position of the |
//| violation between stpidxa+1 and stpidxa+2. |
//| You can plot function values stored in stp[] and g[] arrays |
//| and study behavior of your function by your own eyes, just to be |
//| sure that test correctly reported C1 violation. |
//+------------------------------------------------------------------+
struct COptGuardNonC1Test1Report
{
int m_cnt;
int m_fidx;
int m_n;
int m_stpidxa;
int m_stpidxb;
int m_vidx;
bool m_positive;
CRowDouble m_d;
CRowDouble m_g;
CRowDouble m_stp;
CRowDouble m_x0;
//--- constructor / destructor
COptGuardNonC1Test1Report(void);
~COptGuardNonC1Test1Report(void) {}
//---
void Copy(const COptGuardNonC1Test1Report &obj);
//--- overloading
void operator=(const COptGuardNonC1Test1Report &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
COptGuardNonC1Test1Report::COptGuardNonC1Test1Report(void)
{
m_cnt=0;
m_fidx=0;
m_n=0;
m_stpidxa=0;
m_stpidxb=0;
m_vidx=0;
m_positive=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void COptGuardNonC1Test1Report::Copy(const COptGuardNonC1Test1Report &obj)
{
m_cnt=obj.m_cnt;
m_fidx=obj.m_fidx;
m_n=obj.m_n;
m_stpidxa=obj.m_stpidxa;
m_stpidxb=obj.m_stpidxb;
m_vidx=obj.m_vidx;
m_positive=obj.m_positive;
m_d=obj.m_d;
m_g=obj.m_g;
m_stp=obj.m_stp;
m_x0=obj.m_x0;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class COptGuardApi
{
public:
static void OptGuardInitInternal(COptGuardReport &rep,int n,int k);
static void OptGuardExportReport(COptGuardReport &srcrep,int n,int k,bool badgradhasxj,COptGuardReport &dstrep);
static void SmoothnessMonitorExportC1Test0Report(COptGuardNonC1Test0Report &srcrep,CRowDouble &s,COptGuardNonC1Test0Report &dstrep);
static void SmoothnessMonitorExportC1Test1Report(COptGuardNonC1Test1Report &srcrep,CRowDouble &s,COptGuardNonC1Test1Report &dstrep);
static bool OptGuardAllClear(COptGuardReport &rep);
};
//+------------------------------------------------------------------+
//| This subroutine initializes "internal" OptGuard report, i.e. one|
//| intended for internal use by optimizers. |
//+------------------------------------------------------------------+
void COptGuardApi::OptGuardInitInternal(COptGuardReport &rep,
int n,
int k)
{
rep.m_nonc0suspected=false;
rep.m_nonc0test0positive=false;
rep.m_nonc0lipschitzc=0;
rep.m_nonc0fidx=-1;
rep.m_nonc1suspected=false;
rep.m_nonc1test0positive=false;
rep.m_nonc1test1positive=false;
rep.m_nonc1lipschitzc=0;
rep.m_nonc1fidx=-1;
rep.m_badgradsuspected=false;
rep.m_badgradfidx=-1;
rep.m_badgradvidx=-1;
}
//+------------------------------------------------------------------+
//| This subroutine exports report to user-readable representation |
//| (all arrays are forced to have exactly same size as needed; |
//| unused arrays are set to zero length). |
//+------------------------------------------------------------------+
void COptGuardApi::OptGuardExportReport(COptGuardReport &srcrep,
int n,
int k,
bool badgradhasxj,
COptGuardReport &dstrep)
{
//--- copy
dstrep.m_nonc0suspected=srcrep.m_nonc0suspected;
dstrep.m_nonc0test0positive=srcrep.m_nonc0test0positive;
if(srcrep.m_nonc0suspected)
{
dstrep.m_nonc0lipschitzc=srcrep.m_nonc0lipschitzc;
dstrep.m_nonc0fidx=srcrep.m_nonc0fidx;
}
else
{
dstrep.m_nonc0lipschitzc=0;
dstrep.m_nonc0fidx=-1;
}
dstrep.m_nonc1suspected=srcrep.m_nonc1suspected;
dstrep.m_nonc1test0positive=srcrep.m_nonc1test0positive;
dstrep.m_nonc1test1positive=srcrep.m_nonc1test1positive;
if(srcrep.m_nonc1suspected)
{
dstrep.m_nonc1lipschitzc=srcrep.m_nonc1lipschitzc;
dstrep.m_nonc1fidx=srcrep.m_nonc1fidx;
}
else
{
dstrep.m_nonc1lipschitzc=0;
dstrep.m_nonc1fidx=-1;
}
dstrep.m_badgradsuspected=srcrep.m_badgradsuspected;
if(srcrep.m_badgradsuspected)
{
dstrep.m_badgradfidx=srcrep.m_badgradfidx;
dstrep.m_badgradvidx=srcrep.m_badgradvidx;
}
else
{
dstrep.m_badgradfidx=-1;
dstrep.m_badgradvidx=-1;
}
if(badgradhasxj)
{
dstrep.m_badgradxbase=srcrep.m_badgradxbase;
dstrep.m_badgraduser=srcrep.m_badgraduser;
dstrep.m_badgradnum=srcrep.m_badgradnum;
}
else
{
dstrep.m_badgradxbase.Resize(0);
dstrep.m_badgraduser.Resize(0,0);
dstrep.m_badgradnum.Resize(0,0);
}
}
//+------------------------------------------------------------------+
//| This subroutine exports report to user-readable representation |
//| (all arrays are forced to have exactly same size as needed; |
//| unused arrays are set to zero length). |
//| NOTE: we assume that SrcRep contains scaled X0[] and D[], i.e. |
//| explicit variable scaling was applied. We need to rescale them |
//| during export, that's why we need S[] parameter. |
//+------------------------------------------------------------------+
void COptGuardApi::SmoothnessMonitorExportC1Test0Report(COptGuardNonC1Test0Report &srcrep,
CRowDouble &s,
COptGuardNonC1Test0Report &dstrep)
{
dstrep.m_positive=srcrep.m_positive;
if(srcrep.m_positive)
{
dstrep.m_stpidxa=srcrep.m_stpidxa;
dstrep.m_stpidxb=srcrep.m_stpidxb;
dstrep.m_fidx=srcrep.m_fidx;
dstrep.m_cnt=srcrep.m_cnt;
dstrep.m_n=srcrep.m_n;
dstrep.m_x0=srcrep.m_x0*s+0;
dstrep.m_d=srcrep.m_d*s+0;
dstrep.m_stp=srcrep.m_stp;
dstrep.m_f=srcrep.m_f;
}
else
{
dstrep.m_stpidxa=-1;
dstrep.m_stpidxb=-1;
dstrep.m_fidx=-1;
dstrep.m_cnt=0;
dstrep.m_n=0;
dstrep.m_x0.Resize(0);
dstrep.m_d.Resize(0);
dstrep.m_stp.Resize(0);
dstrep.m_f.Resize(0);
}
}
//+------------------------------------------------------------------+
//| This subroutine exports report to user-readable representation |
//| (all arrays are forced to have exactly same size as needed; |
//| unused arrays are set to zero length). |
//| NOTE: we assume that SrcRep contains scaled X0[], D[] and G[], |
//| i.e. explicit variable scaling was applied. We need to |
//| rescale them during export, that's why we need S[] |
//| parameter. |
//+------------------------------------------------------------------+
void COptGuardApi::SmoothnessMonitorExportC1Test1Report(COptGuardNonC1Test1Report &srcrep,
CRowDouble &s,
COptGuardNonC1Test1Report &dstrep)
{
dstrep.m_positive=srcrep.m_positive;
if(srcrep.m_positive)
{
//--- check
if(!CAp::Assert(srcrep.m_vidx>=0 && srcrep.m_vidx<srcrep.m_n,__FUNCTION__+": integrity check failed"))
return;
dstrep.m_stpidxa=srcrep.m_stpidxa;
dstrep.m_stpidxb=srcrep.m_stpidxb;
dstrep.m_fidx=srcrep.m_fidx;
dstrep.m_vidx=srcrep.m_vidx;
dstrep.m_cnt=srcrep.m_cnt;
dstrep.m_n=srcrep.m_n;
dstrep.m_x0=srcrep.m_x0*s+0;
dstrep.m_d=srcrep.m_d*s+0;
dstrep.m_stp=srcrep.m_stp;
dstrep.m_g=srcrep.m_g/s[srcrep.m_vidx]+0;
dstrep.m_g.Resize(srcrep.m_cnt);
}
else
{
dstrep.m_stpidxa=-1;
dstrep.m_stpidxb=-1;
dstrep.m_fidx=-1;
dstrep.m_vidx=-1;
dstrep.m_cnt=0;
dstrep.m_n=0;
dstrep.m_x0.Resize(0);
dstrep.m_d.Resize(0);
dstrep.m_stp.Resize(0);
dstrep.m_g.Resize(0);
}
}
//+------------------------------------------------------------------+
//| Returns True when all flags are clear. Intended for easy coding |
//| of unit tests. |
//+------------------------------------------------------------------+
bool COptGuardApi::OptGuardAllClear(COptGuardReport &rep)
{
return (!(rep.m_badgradsuspected || rep.m_nonc0suspected || rep.m_nonc1suspected));
}
//+------------------------------------------------------------------+
//| This structure is used to store temporary buffers for |
//| L-BFGS-based preconditioner. |
//+------------------------------------------------------------------+
struct CPrecBufLBFGS
{
CRowInt m_bufb;
CRowInt m_idx;
CRowDouble m_alpha;
CRowDouble m_bufa;
CRowDouble m_norms;
CRowDouble m_rho;
CMatrixDouble m_yk;
CRowDouble norms;
//--- constructor / destructor
CPrecBufLBFGS(void) {}
~CPrecBufLBFGS(void) {}
void Copy(const CPrecBufLBFGS &obj);
//--- overloading
void operator=(const CPrecBufLBFGS &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CPrecBufLBFGS::Copy(const CPrecBufLBFGS &obj)
{
m_bufb=obj.m_bufb;
m_idx=obj.m_idx;
m_alpha=obj.m_alpha;
m_bufa=obj.m_bufa;
m_norms=obj.m_norms;
m_rho=obj.m_rho;
m_yk=obj.m_yk;
}
//+------------------------------------------------------------------+
//| This structure is used to store temporary buffers for LowRank |
//| preconditioner. |
//+------------------------------------------------------------------+
struct CPrecBufLowRank
{
int m_k;
int m_n;
CRowDouble m_bufc;
CRowDouble m_d;
CRowDouble m_tmp;
CMatrixDouble m_bufw;
CMatrixDouble m_bufz;
CMatrixDouble m_v;
CPrecBufLowRank(void) { m_k=0; m_n=0; }
~CPrecBufLowRank(void) {}
void Copy(const CPrecBufLowRank &obj);
//--- overloading
void operator=(const CPrecBufLowRank &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CPrecBufLowRank::Copy(const CPrecBufLowRank &obj)
{
m_k=obj.m_k;
m_n=obj.m_n;
m_bufc=obj.m_bufc;
m_d=obj.m_d;
m_tmp=obj.m_tmp;
m_bufw=obj.m_bufw;
m_bufz=obj.m_bufz;
m_v=obj.m_v;
}
//+------------------------------------------------------------------+
//| This structure is a smoothness monitor. |
//+------------------------------------------------------------------+
struct CSmoothnessMonitor
{
RCommState m_probingrcomm;
RCommState m_rstateg0;
COptGuardReport m_rep;
COptGuardNonC1Test1Report m_nonc1test1lngrep;
COptGuardNonC1Test1Report m_nonc1test1strrep;
COptGuardNonC1Test0Report m_nonc1test0lngrep;
COptGuardNonC1Test0Report m_nonc1test0strrep;
COptGuardNonC0Report m_nonc0lngrep;
COptGuardNonC0Report m_nonc0strrep;
int m_enqueuedcnt;
int m_k;
int m_n;
int m_probingnstepsstored;
int m_probingnvalues;
int m_sortedcnt;
double m_nonc0currentrating;
double m_nonc0lngrating;
double m_nonc0strrating;
double m_nonc1currentrating;
double m_nonc1test0lngrating;
double m_nonc1test0strrating;
double m_nonc1test1lngrating;
double m_nonc1test1strrating;
double m_probingstepmax;
double m_probingstepscale;
double m_probingstp;
bool m_badgradhasxj;
bool m_checksmoothness;
bool m_linesearchspoiled;
bool m_linesearchstarted;
bool m_needfij;
CRowInt m_bufi;
CRowInt m_sortedidx;
CRowInt m_tmpidx;
CRowDouble m_bufr;
CRowDouble m_dcur;
CRowDouble m_deltax;
CRowDouble m_du;
CRowDouble m_enqueuedfunc;
CRowDouble m_enqueuedstp;
CRowDouble m_enqueuedx;
CRowDouble m_f;
CRowDouble m_f0;
CRowDouble m_fbase;
CRowDouble m_fc;
CRowDouble m_fi;
CRowDouble m_fm;
CRowDouble m_fp;
CRowDouble m_g;
CRowDouble m_jc;
CRowDouble m_jm;
CRowDouble m_jp;
CRowDouble m_probingf;
CRowDouble m_probingsteps;
CRowDouble m_s;
CRowDouble m_sortedstp;
CRowDouble m_stp;
CRowDouble m_x;
CRowDouble m_xbase;
CRowDouble m_xu;
CMatrixDouble m_enqueuedjac;
CMatrixDouble m_j;
CMatrixDouble m_j0;
CMatrixDouble m_jbasenum;
CMatrixDouble m_jbaseusr;
CMatrixDouble m_probingslopes;
CMatrixDouble m_probingvalues;
//--- constructor / destructor
CSmoothnessMonitor(void);
~CSmoothnessMonitor(void) {}
void Copy(const CSmoothnessMonitor &obj);
//--- overloading
void operator=(const CSmoothnessMonitor &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CSmoothnessMonitor::CSmoothnessMonitor(void)
{
m_enqueuedcnt=0;
m_k=0;
m_n=0;
m_probingnstepsstored=0;
m_probingnvalues=0;
m_sortedcnt=0;
m_nonc0currentrating=0;
m_nonc0lngrating=0;
m_nonc0strrating=0;
m_nonc1currentrating=0;
m_nonc1test0lngrating=0;
m_nonc1test0strrating=0;
m_nonc1test1lngrating=0;
m_nonc1test1strrating=0;
m_probingstepmax=0;
m_probingstepscale=0;
m_probingstp=0;
m_badgradhasxj=false;
m_checksmoothness=false;
m_linesearchspoiled=false;
m_linesearchstarted=false;
m_needfij=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CSmoothnessMonitor::Copy(const CSmoothnessMonitor &obj)
{
m_probingrcomm=obj.m_probingrcomm;
m_rstateg0=obj.m_rstateg0;
m_rep=obj.m_rep;
m_nonc1test1lngrep=obj.m_nonc1test1lngrep;
m_nonc1test1strrep=obj.m_nonc1test1strrep;
m_nonc1test0lngrep=obj.m_nonc1test0lngrep;
m_nonc1test0strrep=obj.m_nonc1test0strrep;
m_nonc0lngrep=obj.m_nonc0lngrep;
m_nonc0strrep=obj.m_nonc0strrep;
m_enqueuedcnt=obj.m_enqueuedcnt;
m_k=obj.m_k;
m_n=obj.m_n;
m_probingnstepsstored=obj.m_probingnstepsstored;
m_probingnvalues=obj.m_probingnvalues;
m_sortedcnt=obj.m_sortedcnt;
m_nonc0currentrating=obj.m_nonc0currentrating;
m_nonc0lngrating=obj.m_nonc0lngrating;
m_nonc0strrating=obj.m_nonc0strrating;
m_nonc1currentrating=obj.m_nonc1currentrating;
m_nonc1test0lngrating=obj.m_nonc1test0lngrating;
m_nonc1test0strrating=obj.m_nonc1test0strrating;
m_nonc1test1lngrating=obj.m_nonc1test1lngrating;
m_nonc1test1strrating=obj.m_nonc1test1strrating;
m_probingstepmax=obj.m_probingstepmax;
m_probingstepscale=obj.m_probingstepscale;
m_probingstp=obj.m_probingstp;
m_badgradhasxj=obj.m_badgradhasxj;
m_checksmoothness=obj.m_checksmoothness;
m_linesearchspoiled=obj.m_linesearchspoiled;
m_linesearchstarted=obj.m_linesearchstarted;
m_needfij=obj.m_needfij;
m_bufi=obj.m_bufi;
m_sortedidx=obj.m_sortedidx;
m_tmpidx=obj.m_tmpidx;
m_bufr=obj.m_bufr;
m_dcur=obj.m_dcur;
m_deltax=obj.m_deltax;
m_du=obj.m_du;
m_enqueuedfunc=obj.m_enqueuedfunc;
m_enqueuedstp=obj.m_enqueuedstp;
m_enqueuedx=obj.m_enqueuedx;
m_f=obj.m_f;
m_f0=obj.m_f0;
m_fbase=obj.m_fbase;
m_fc=obj.m_fc;
m_fi=obj.m_fi;
m_fm=obj.m_fm;
m_fp=obj.m_fp;
m_g=obj.m_g;
m_jc=obj.m_jc;
m_jm=obj.m_jm;
m_jp=obj.m_jp;
m_probingf=obj.m_probingf;
m_probingsteps=obj.m_probingsteps;
m_s=obj.m_s;
m_sortedstp=obj.m_sortedstp;
m_stp=obj.m_stp;
m_x=obj.m_x;
m_xbase=obj.m_xbase;
m_xu=obj.m_xu;
m_enqueuedjac=obj.m_enqueuedjac;
m_j=obj.m_j;
m_j0=obj.m_j0;
m_jbasenum=obj.m_jbasenum;
m_jbaseusr=obj.m_jbaseusr;
m_probingslopes=obj.m_probingslopes;
m_probingvalues=obj.m_probingvalues;
}
//+------------------------------------------------------------------+
//| Auxiliary functions for other classes |
//+------------------------------------------------------------------+
class COptServ
{
public:
//--- constants
static const double m_ognoiselevelf;
static const double m_ognoiselevelg;
static const double m_ogminrating0;
static const double m_ogminrating1;
static void CheckBcViolation(bool &HasBndL[],CRowDouble &bndl,bool &HasBndU[],CRowDouble &bndu,CRowDouble &x,int n,CRowDouble &s,bool nonunits,double &bcerr,int &bcidx);
static void CheckLcViolation(CMatrixDouble &cleic,CRowInt &lcsrcidx,int nec,int nic,CRowDouble &x,int n,double &lcerr,int &lcidx);
static void CheckNLcViolation(CRowDouble &fi,int ng,int nh,double &nlcerr,int &nlcidx);
static void UnScaleAndCheckNLcViolation(CRowDouble &fi,CRowDouble &fscales,int ng,int nh,double &nlcerr,int &nlcidx);
static void TrimPrepare(double f,double &threshold);
static void TrimFunction(double &f,double &g[],const int n,const double threshold);
static void TrimFunction(double &f,CRowDouble &g,const int n,const double threshold);
static bool EnforceBoundaryConstraints(CRowDouble &x,CRowDouble &bl,bool &havebl[],CRowDouble &bu,bool &havebu[],int nmain,int nslack);
static void ProjectGradientIntoBC(CRowDouble &x,CRowDouble &g,CRowDouble &bl,bool &havebl[],CRowDouble &bu,bool &havebu[],int nmain,int nslack);
static void CalculateStepBound(CRowDouble &x,CRowDouble &d,double alpha,CRowDouble &bndl,bool &havebndl[],CRowDouble &bndu,bool &havebndu[],int nmain,int nslack,int &variabletofreeze,double &valuetofreeze,double &maxsteplen);
static int PostProcessBoundedStep(CRowDouble &x,CRowDouble &xprev,CRowDouble &bndl,bool &havebndl[],CRowDouble &bndu,bool &havebndu[],int nmain,int nslack,int variabletofreeze,double valuetofreeze,double steptaken,double maxsteplen);
static void FilterDirection(CRowDouble &d,CRowDouble &x,CRowDouble &bndl,bool &havebndl[],CRowDouble &bndu,bool &havebndu[],CRowDouble &s,int nmain,int nslack,double droptol);
static int NumberOfChangedConstraints(CRowDouble &x,CRowDouble &xprev,CRowDouble &bndl,bool &havebndl[],CRowDouble &bndu,bool &havebndu[],int nmain,int nslack);
static bool FindFeasiblePoint(CRowDouble &x,CRowDouble &bndl,bool &havebndl[],CRowDouble &bndu,bool &havebndu[],int nmain,int nslack,CMatrixDouble &ce,int k,double epsi,int &qpits,int &gpaits);
static bool DerivativeCheck(double f0,double df0,double f1,double df1,double f,double df,double width);
static void EstimateParabolicModel(double absasum,double absasum2,double mx,double mb,double md,double d1,double d2,int &d1est,int &d2est);
static void InexactLBFGSPreconditioner(CRowDouble &s,int n,CRowDouble &d,CRowDouble &c,CMatrixDouble &w,int k,CPrecBufLBFGS &buf);
static void PrepareLowRankPreconditioner(CRowDouble &d,CRowDouble &c,CMatrixDouble &w,int n,int k,CPrecBufLowRank &buf);
static void ApplyLowRankPreconditioner(CRowDouble &s,CPrecBufLowRank &buf);
static void SmoothnessMonitorInit(CSmoothnessMonitor &monitor,CRowDouble &s,int n,int k,bool checksmoothness);
static void SmoothnessMonitorStartLineSearch(CSmoothnessMonitor &monitor,CRowDouble &x,CRowDouble &fi,CMatrixDouble &jac);
static void SmoothnessMonitorStartLineSearch1u(CSmoothnessMonitor &monitor,CRowDouble &s,CRowDouble &invs,CRowDouble &x,double f0,CRowDouble &j0);
static void SmoothnessMonitorEnqueuePoint(CSmoothnessMonitor &monitor,CRowDouble &d,double stp,CRowDouble &x,CRowDouble &fi,CMatrixDouble &jac);
static void SmoothnessMonitorEnqueuePoint1u(CSmoothnessMonitor &monitor,CRowDouble &s,CRowDouble &invs,CRowDouble &d,double stp,CRowDouble &x,double f0,CRowDouble &j0);
static void SmoothnessMonitorFinalizeLineSearch(CSmoothnessMonitor &monitor);
static void SmoothnessMonitorStartProbing(CSmoothnessMonitor &monitor,double stpmax,int nvalues,double stepscale);
static bool SmoothnessMonitorProbe(CSmoothnessMonitor &monitor);
static void SmoothnessMonitorTraceProbingResults(CSmoothnessMonitor &monitor);
static void SmoothnessMonitorTraceStatus(CSmoothnessMonitor &monitor,bool callersuggeststrace);
static void SmoothnessMonitorExportReport(CSmoothnessMonitor &monitor,COptGuardReport &rep);
static bool SmoothnessMonitorCheckGradientATX0(CSmoothnessMonitor &monitor,CRowDouble &unscaledx0,CRowDouble &s,CRowDouble &bndl,CRowDouble &bndu,bool hasboxconstraints,double teststep);
private:
static double FeasibilityError(CMatrixDouble &ce,CRowDouble &x,int nmain,int nslack,int k,CRowDouble &m_tmp0);
static void FeasibilityErrorGrad(CMatrixDouble &ce,CRowDouble &x,int nmain,int nslack,int k,double &err,CRowDouble &grad,CRowDouble &m_tmp0);
static void TestC0Continuity(double f0,double f1,double f2,double f3,double noise0,double noise1,double noise2,double noise3,double delta0,double delta1,double delta2,bool applyspecialcorrection,double &rating,double &lipschitz);
static void C1ContinuityTest0(CSmoothnessMonitor &monitor,int funcidx,int stpidx,int sortedcnt);
static void C1ContinuityTest1(CSmoothnessMonitor &monitor,int funcidx,int stpidx,int sortedcnt);
};
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
const double COptServ::m_ognoiselevelf=1.0E2*CMath::m_machineepsilon;
const double COptServ::m_ognoiselevelg=1.0E4*CMath::m_machineepsilon;
const double COptServ::m_ogminrating0=50.0;
const double COptServ::m_ogminrating1=50.0;
//+------------------------------------------------------------------+
//| This subroutine checks violation of the box constraints. On |
//| output it sets bcerr to the maximum scaled violation, bcidx to |
//| the index of the violating constraint. |
//| if bcerr=0 (say, if no constraints are violated) then bcidx=-1.|
//| If nonunits=false then s[] is not referenced at all (assumed |
//| unit). |
//+------------------------------------------------------------------+
void COptServ::CheckBcViolation(bool &HasBndL[],
CRowDouble &bndl,
bool &HasBndU[],
CRowDouble &bndu,
CRowDouble &x,
int n,
CRowDouble &s,
bool nonunits,
double &bcerr,
int &bcidx)
{
//--- create variables
int i=0;
double vs=0;
double ve=0;
//--- initialization
bcerr=0;
bcidx=-1;
for(i=0; i<n; i++)
{
//--- Fetch scale
if(nonunits)
vs=1/s[i];
else
vs=1;
//--- Check lower bound
if(HasBndL[i] && x[i]<bndl[i])
{
ve=(bndl[i]-x[i])*vs;
if(ve>bcerr)
{
bcerr=ve;
bcidx=i;
}
}
//--- Check upper bound
if(HasBndU[i] && x[i]>bndu[i])
{
ve=(x[i]-bndu[i])*vs;
if(ve>bcerr)
{
bcerr=ve;
bcidx=i;
}
}
}
}
//+------------------------------------------------------------------+
//| This subroutine checks violation of the general linear |
//| constraints. |
//| Constraints are assumed to be un-normalized and stored in the |
//| format "NEC equality ones followed by NIC inequality ones". |
//| On output it sets lcerr to the maximum scaled violation, lcidx to|
//| the source index of the most violating constraint (row indexes of|
//| CLEIC are mapped to the indexes of the "original" constraints via|
//| LCSrcIdx[] array. |
//| if lcerr=0 (say, if no constraints are violated) then lcidx=-1.|
//| If nonunits=false then s[] is not referenced at all (assumed |
//| unit). |
//+------------------------------------------------------------------+
void COptServ::CheckLcViolation(CMatrixDouble &cleic,
CRowInt &lcsrcidx,
int nec,
int nic,
CRowDouble &x,
int n,
double &lcerr,
int &lcidx)
{
//--- create variables
int i=0;
int j=0;
double cx=0;
double cnrm=0;
double v=0;
lcerr=0;
lcidx=-1;
for(i=0; i<nec+nic; i++)
{
cx=-cleic.Get(i,n);
cnrm=0;
for(j=0; j<n; j++)
{
v=cleic.Get(i,j);
cx+=v*x[j];
cnrm+=v*v;
}
cnrm=MathSqrt(cnrm);
cx=cx/CApServ::Coalesce(cnrm,1);
if(i<nec)
{
cx=MathAbs(cx);
}
else
{
cx=MathMax(cx,0);
}
if(cx>lcerr)
{
lcerr=cx;
lcidx=lcsrcidx[i];
}
}
}
//+------------------------------------------------------------------+
//| This subroutine checks violation of the nonlinear constraints. |
//| Fi[0] is the target value (ignored), Fi[1:NG+NH] are values of |
//| nonlinear constraints. |
//| On output it sets nlcerr to the scaled violation, nlcidx to the |
//| index of the most violating constraint in [0,NG+NH-1] range. |
//| if nlcerr=0 (say, if no constraints are violated) then |
//| nlcidx=-1. |
//| If nonunits=false then s[] is not referenced at all (assumed |
//| unit). |
//+------------------------------------------------------------------+
void COptServ::CheckNLcViolation(CRowDouble &fi,
int ng,
int nh,
double &nlcerr,
int &nlcidx)
{
//--- create variables
int i=0;
double v=0;
nlcerr=0;
nlcidx=-1;
for(i=0; i<ng+nh; i++)
{
v=fi[i+1];
if(i<ng)
v=MathAbs(v);
else
v=MathMax(v,0);
if(v>nlcerr)
{
nlcerr=v;
nlcidx=i;
}
}
}
//+------------------------------------------------------------------+
//| This subroutine is same as CheckNLCViolation, but is works with |
//| scaled constraints: it assumes that Fi[] were divided by |
//| FScales[] vector BEFORE passing them to this function. |
//| The function checks scaled values, but reports unscaled errors. |
//+------------------------------------------------------------------+
void COptServ::UnScaleAndCheckNLcViolation(CRowDouble &fi,
CRowDouble &fscales,
int ng,
int nh,
double &nlcerr,
int &nlcidx)
{
//--- create variables
int i=0;
double v=0;
nlcerr=0;
nlcidx=-1;
for(i=0; i<ng+nh; i++)
{
//--- check
if(!CAp::Assert(fscales[i+1]>0.0,__FUNCTION__+": integrity check failed"))
return;
v=fi[i+1]*fscales[i+1];
if(i<ng)
v=MathAbs(v);
else
v=MathMax(v,0);
if(v>nlcerr)
{
nlcerr=v;
nlcidx=i;
}
}
}
//+------------------------------------------------------------------+
//| This subroutine is used to prepare threshold value which will be |
//| used for trimming of the target function (see comments on |
//| TrimFunction() for more information). |
//| This function accepts only one parameter: function value at the |
//| starting point. It returns threshold which will be used for |
//| trimming. |
//+------------------------------------------------------------------+
void COptServ::TrimPrepare(double f,double &threshold)
{
threshold=10*(MathAbs(f)+1);
}
//+------------------------------------------------------------------+
//| This subroutine is used to "trim" target function, i.e. to do |
//| following transformation: |
//| { {F,G} if F<Threshold |
//| {F_tr, G_tr} = { |
//| { {Threshold, 0} if F>=Threshold |
//| Such transformation allows us to solve problems with |
//| singularities by redefining function in such way that it becomes |
//| bounded from above. |
//+------------------------------------------------------------------+
void COptServ::TrimFunction(double &f,double &g[],const int n,
const double threshold)
{
//--- check
if(f>=threshold)
{
f=threshold;
ArrayFill(g,0,n,0);
}
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void COptServ::TrimFunction(double &f,CRowDouble &g,const int n,
const double threshold)
{
//--- check
if(f>=threshold)
{
f=threshold;
if(g.Size()<=n)
g=vector<double>::Zeros(n);
else
for(int i=0; i<n; i++)
g.Set(i,0.0);
}
}
//+------------------------------------------------------------------+
//| This function enforces boundary constraints in the X. |
//| This function correctly(although a bit inefficient) handles BL[i]|
//| which are -INF and BU[i] which are +INF. |
//| We have NMain + NSlack dimensional X, with first NMain components|
//| bounded by BL/BU, and next NSlack ones bounded by non-negativity |
//| constraints. |
//| INPUT PARAMETERS: |
//| X - array[NMain + NSlack], point |
//| BL - array[NMain], lower bounds (may contain -INF, when|
//| bound is not present) |
//| HaveBL - array[NMain], if HaveBL[i] is False, then i-th |
//| bound is not present |
//| BU - array[NMain], upper bounds (may contain +INF, when|
//| bound is not present) |
//| HaveBU - array[NMain], if HaveBU[i] is False, then i-th |
//| bound is not present |
//| OUTPUT PARAMETERS: |
//| X - X with all constraints being enforced |
//| It returns True when constraints are consistent, |
//| False - when constraints are inconsistent. |
//+------------------------------------------------------------------+
bool COptServ::EnforceBoundaryConstraints(CRowDouble &x,
CRowDouble &bl,
bool &havebl[],
CRowDouble &bu,
bool &havebu[],
int nmain,
int nslack)
{
for(int i=0; i<nmain; i++)
{
if(havebl[i] && havebu[i] && bl[i]>bu[i])
return(false);
//---
if(havebl[i] && x[i]<bl[i])
x.Set(i,bl[i]);
if(havebu[i] && x[i]>bu[i])
x.Set(i,bu[i]);
}
for(int i=0; i<nslack; i++)
if(x[nmain+i]<0.0)
x.Set(nmain+i,0);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| This function projects gradient into feasible area of boundary |
//| constrained optimization problem. X can be infeasible with |
//| respect to boundary constraints. We have NMain + NSlack |
//| dimensional X, with first NMain components bounded by BL/BU,|
//| and next NSlack ones bounded by non-negativity constraints. |
//| INPUT PARAMETERS: |
//| X - array[NMain + NSlack], point |
//| G - array[NMain + NSlack], gradient |
//| BL - lower bounds(may contain -INF, when bound is not |
//| present) |
//| HaveBL - if HaveBL[i] is False, then i-th bound is not |
//| present |
//| BU - upper bounds(may contain +INF, when bound is not |
//| present) |
//| HaveBU - if HaveBU[i] is False, then i-th bound is not |
//| present |
//| OUTPUT PARAMETERS: |
//| G - projection of G. Components of G which satisfy one |
//| of the following |
//| (1)(X[I] <= BndL[I]) and (G[I] > 0), OR |
//| (2)(X[I] >= BndU[I]) and (G[I] < 0) |
//| are replaced by zeros. |
//| NOTE 1: this function assumes that constraints are feasible. It |
//| throws exception otherwise. |
//| NOTE 2: in fact, projection of ANTI - gradient is calculated, |
//| because this function trims components of - G which |
//| points outside of the feasible area. However, working |
//| with - G is considered confusing, because all |
//| optimization source work with G. |
//+------------------------------------------------------------------+
void COptServ::ProjectGradientIntoBC(CRowDouble &x,
CRowDouble &g,
CRowDouble &bl,
bool &havebl[],
CRowDouble &bu,
bool &havebu[],
int nmain,
int nslack)
{
for(int i=0; i<nmain; i++)
{
//--- check
if(!CAp::Assert(!havebl[i] || !havebu[i] || bl[i]<=bu[i],__FUNCTION__+": internal error (infeasible constraints)"))
return;
if(havebl[i] && x[i]<=bl[i] && g[i]>0.0)
g.Set(i,0);
if(havebu[i] && x[i]>=bu[i] && g[i]<0.0)
g.Set(i,0);
}
for(int i=0; i<nslack; i++)
if(x[nmain+i]<=0.0 && g[nmain+i]>0.0)
g.Set(nmain+i,0);
}
//+------------------------------------------------------------------+
//| Given |
//| a) initial point X0[NMain + NSlack] (feasible with respect |
//| to bound constraints) |
//| b) step vector alpha*D[NMain + NSlack] |
//| c) boundary constraints BndL[NMain], BndU[NMain] |
//| d) implicit non-negativity constraints for slack variables |
//| this function calculates bound on the step length |
//| subject to boundary constraints. |
//| It returns: |
//| * MaxStepLen - such step length that X0 + MaxStepLen*alpha*D |
//| is exactly at the boundary given by constraints |
//| * VariableToFreeze - index of the constraint to be activated, |
//| 0 <= VariableToFreeze < NMain + NSlack |
//| * ValueToFreeze - value of the corresponding constraint. |
//| Notes: |
//| * it is possible that several constraints can be activated by |
//| the step at once. In such cases only one constraint is |
//| returned. It is caller responsibility to check other |
//| constraints. This function makes sure that we activate at |
//| least one constraint, and everything else is the |
//| responsibility of the caller. |
//| * steps smaller than MaxStepLen still can activate constraints |
//| due to numerical errors. Thus purpose of this function is |
//| not to guard against accidental activation of the |
//| constraints - quite the reverse, its purpose is to activate |
//| at least constraint upon performing step which is too long. |
//| * in case there is no constraints to activate, we return |
//| negative VariableToFreeze and zero MaxStepLen and |
//| ValueToFreeze. |
//| * this function assumes that constraints are consistent; it |
//| throws exception otherwise. |
//| INPUT PARAMETERS: |
//| X - array[NMain + NSlack], point. Must be feasible with|
//| respect to bound constraints(exception will be |
//| thrown otherwise) |
//| D - array[NMain + NSlack], step direction |
//| alpha - scalar multiplier before D, alpha<>0 |
//| BndL - lower bounds, array[NMain] (may contain -INF, when|
//| bound is not present) |
//| HaveBndL - array[NMain], if HaveBndL[i] is False, then i-th |
//| bound is not present |
//| BndU - array[NMain], upper bounds (may contain +INF, when|
//| bound is not present) |
//| HaveBndU - array[NMain], if HaveBndU[i] is False, then i th |
//| bound is not present |
//| NMain - number of main variables |
//| NSlack - number of slack variables |
//| OUTPUT PARAMETERS: |
//| VariableToFreeze: |
//| * negative value = step is unbounded, ValueToFreeze = 0,|
//| MaxStepLen = 0. |
//| * Non-Negative value = at least one constraint, given by |
//| this parameter, will be activated |
//| upon performing maximum step. |
//| ValueToFreeze - value of the variable which will be |
//| constrained |
//| MaxStepLen - maximum length of the step. Can be zero when |
//| step vector looks outside of the feasible area|
//+------------------------------------------------------------------+
void COptServ::CalculateStepBound(CRowDouble &x,CRowDouble &d,
double alpha,CRowDouble &bndl,
bool &havebndl[],CRowDouble &bndu,
bool &havebndu[],int nmain,int nslack,
int &variabletofreeze,
double &valuetofreeze,
double &maxsteplen)
{
//--- create variables
double prevmax=0;
double initval=0;
//--- initialization
variabletofreeze=0;
valuetofreeze=0;
maxsteplen=0;
//--- check
if(!CAp::Assert(alpha!=0.0,__FUNCTION__+": zero alpha"))
return;
variabletofreeze=-1;
initval=CMath::m_maxrealnumber;
maxsteplen=initval;
for(int i=0; i<nmain; i++)
{
if(havebndl[i] && (alpha*d[i])<0.0)
{
//--- check
if(!CAp::Assert(x[i]>=bndl[i],__FUNCTION__+": infeasible X"))
return;
prevmax=maxsteplen;
maxsteplen=CApServ::SafeMinPosRV(x[i]-bndl[i],-(alpha*d[i]),maxsteplen);
if(maxsteplen<prevmax)
{
variabletofreeze=i;
valuetofreeze=bndl[i];
}
}
if(havebndu[i] && (alpha*d[i])>0.0)
{
//--- check
if(!CAp::Assert(x[i]<=bndu[i],__FUNCTION__+": infeasible X"))
return;
prevmax=maxsteplen;
maxsteplen=CApServ::SafeMinPosRV(bndu[i]-x[i],alpha*d[i],maxsteplen);
if(maxsteplen<prevmax)
{
variabletofreeze=i;
valuetofreeze=bndu[i];
}
}
}
for(int i=0; i<nslack; i++)
{
if((alpha*d[nmain+i])<0.0)
{
//--- check
if(!CAp::Assert(x[nmain+i]>=0.0,__FUNCTION__+": infeasible X"))
return;
prevmax=maxsteplen;
maxsteplen=CApServ::SafeMinPosRV(x[nmain+i],-(alpha*d[nmain+i]),maxsteplen);
if(maxsteplen<prevmax)
{
variabletofreeze=nmain+i;
valuetofreeze=0;
}
}
}
if(maxsteplen==initval)
{
valuetofreeze=0;
maxsteplen=0;
}
}
//+------------------------------------------------------------------+
//| This function postprocesses bounded step by: |
//| * analysing step length(whether it is equal to MaxStepLen) and |
//| activating constraint given by VariableToFreeze if needed |
//| * checking for additional bound constraints to activate |
//| This function uses final point of the step, quantities calculated|
//| by the CalculateStepBound() function. As result, it returns |
//| point which is exactly feasible with respect to boundary |
//| constraints. |
//| NOTE 1: this function does NOT handle and check linear equality |
//| constraints |
//| NOTE 2: when StepTaken = MaxStepLen we always activate at least |
//| one constraint |
//| INPUT PARAMETERS: |
//| X - array[NMain + NSlack], final point to postprocess |
//| XPrev - array[NMain + NSlack], initial point |
//| BndL - lower bounds, array[NMain] (may contain -INF, when|
//| bound is not present) |
//| HaveBndL - array[NMain], if HaveBndL[i] is False, then i-th |
//| bound is not present |
//| BndU - array[NMain], upper bounds (may contain +INF, when|
//| bound is not present) |
//| HaveBndU - array[NMain], if HaveBndU[i] is False, then i-th |
//| bound is not present |
//| NMain - number of main variables |
//| NSlack - number of slack variables |
//| VariableToFreeze - result of CalculateStepBound() |
//| ValueToFreeze - result of CalculateStepBound() |
//| StepTaken- actual step length (actual step is equal to the |
//| possibly non-unit step direction vector times this |
//| parameter). StepTaken <= MaxStepLen. |
//| MaxStepLen- result of CalculateStepBound() |
//| OUTPUT PARAMETERS: |
//| X - point bounded with respect to constraints. |
//| components corresponding to active constraints are |
//| exactly equal to the boundary values. |
//| RESULT: |
//| number of constraints activated in addition to previously |
//| active ones. |
//| Constraints which were DEACTIVATED are ignored(do not influence |
//| function value). |
//+------------------------------------------------------------------+
int COptServ::PostProcessBoundedStep(CRowDouble &x,
CRowDouble &xprev,
CRowDouble &bndl,
bool &havebndl[],
CRowDouble &bndu,
bool &havebndu[],
int nmain,
int nslack,
int variabletofreeze,
double valuetofreeze,
double steptaken,
double maxsteplen)
{
//--- create variables
int result=0;
int i=0;
bool wasactivated;
//--- check
if(!CAp::Assert(variabletofreeze<0 || steptaken<=maxsteplen))
return(result);
//--- Activate constraints
if(variabletofreeze>=0 && steptaken==maxsteplen)
x.Set(variabletofreeze,valuetofreeze);
for(i=0; i<nmain; i++)
{
if(havebndl[i] && x[i]<bndl[i])
x.Set(i,bndl[i]);
if(havebndu[i] && x[i]>bndu[i])
x.Set(i,bndu[i]);
}
for(i=0; i<nslack; i++)
if(x[nmain+i]<=0.0)
x.Set(nmain+i,0);
//--- Calculate number of constraints being activated
for(i=0; i<nmain; i++)
{
wasactivated=(x[i]!=xprev[i] && ((havebndl[i] && x[i]==bndl[i]) || (havebndu[i] && x[i]==bndu[i])));
wasactivated=wasactivated || variabletofreeze==i;
if(wasactivated)
result++;
}
for(i=0; i<nslack; i++)
{
wasactivated=(x[nmain+i]!=xprev[nmain+i] && x[nmain+i]==0.0);
wasactivated=wasactivated || variabletofreeze==nmain+i;
if(wasactivated)
result++;
}
return(result);
}
//+------------------------------------------------------------------+
//| The purpose of this function is to prevent algorithm from |
//| "unsticking" from the active bound constraints because of |
//| numerical noise in the gradient or Hessian. |
//| It is done by zeroing some components of the search direction D. |
//| D[i] is zeroed when both(a) and (b) are true: |
//| a) corresponding X[i] is exactly at the boundary |
//| b) | D[i]*S[i] | <= DropTol*Sqrt(SUM(D[i] ^ 2 * S[I] ^ 2)) |
//| D can be step direction, antigradient, gradient, or anything |
//| similar. Sign of D does not matter, nor matters step length. |
//| NOTE 1: boundary constraints are expected to be consistent, as |
//| well as X is expected to be feasible. Exception will be |
//| thrown otherwise. |
//| INPUT PARAMETERS: |
//| D - array[NMain + NSlack], direction |
//| X - array[NMain + NSlack], current point |
//| BndL - lower bounds, array[NMain] (may contain -INF, when |
//| bound is not present) |
//| HaveBndL - array[NMain], if HaveBndL[i] is False, then i-th |
//| bound is not present |
//| BndU - array[NMain], upper bounds (may contain +INF, when |
//| bound is not present) |
//| HaveBndU - array[NMain], if HaveBndU[i] is False, then i-th |
//| bound is not present |
//| S - array[NMain + NSlack], scaling of the variables |
//| NMain - number of main variables |
//| NSlack - number of slack variables |
//| DropTol - drop tolerance, >= 0 |
//| OUTPUT PARAMETERS: |
//| X - point bounded with respect to constraints. |
//| components corresponding to active constraints are |
//| exactly equal to the boundary values. |
//+------------------------------------------------------------------+
void COptServ::FilterDirection(CRowDouble &d,
CRowDouble &x,
CRowDouble &bndl,
bool &havebndl[],
CRowDouble &bndu,
bool &havebndu[],
CRowDouble &s,
int nmain,
int nslack,
double droptol)
{
//--- create variables
int i=0;
double scalednorm=0;
bool isactive=false;
for(i=0; i<nmain+nslack; i++)
scalednorm+=CMath::Sqr(d[i]*s[i]);
scalednorm=MathSqrt(scalednorm);
for(i=0; i<nmain; i++)
{
//--- check
if(!CAp::Assert(!havebndl[i] || x[i]>=bndl[i],__FUNCTION__+": infeasible point"))
return;
if(!CAp::Assert(!havebndu[i] || x[i]<=bndu[i],__FUNCTION__+": infeasible point"))
return;
isactive=(havebndl[i] && x[i]==bndl[i]) || (havebndu[i] && x[i]==bndu[i]);
if(isactive && MathAbs(d[i]*s[i])<=(droptol*scalednorm))
d.Set(i,0.0);
}
for(i=0; i<nslack; i++)
{
//--- check
if(!CAp::Assert(x[nmain+i]>=0.0,__FUNCTION__+": infeasible point"))
return;
if(x[nmain+i]==0.0 && MathAbs(d[nmain+i]*s[nmain+i])<=(droptol*scalednorm))
d.Set(nmain+i,0.0);
}
}
//+------------------------------------------------------------------+
//| This function returns number of bound constraints whose State was|
//| changed (either activated or deactivated) when making step from |
//| XPrev to X. |
//| Constraints are considered: |
//| * active - when we are exactly at the boundary |
//| * inactive - when we are not at the boundary |
//| You should note that antigradient direction is NOT taken into |
//| account when we make decions on the constraint status. |
//| INPUT PARAMETERS: |
//| X - array[NMain + NSlack], final point. Must be |
//| feasible with respect to bound constraints. |
//| XPrev - array[NMain + NSlack], initial point. Must be |
//| feasible with respect to bound constraints. |
//| BndL - lower bounds, array[NMain] (may contain -INF, when |
//| bound is not present) |
//| HaveBndL - array[NMain], if HaveBndL[i] is False, then i-th |
//| bound is not present |
//| BndU - array[NMain], upper bounds (may contain +INF, when |
//| bound is not present) |
//| HaveBndU - array[NMain], if HaveBndU[i] is False, then i-th |
//| bound is not present |
//| NMain - number of main variables |
//| NSlack - number of slack variables |
//| RESULT: number of constraints whose State was changed. |
//+------------------------------------------------------------------+
int COptServ::NumberOfChangedConstraints(CRowDouble &x,
CRowDouble &xprev,
CRowDouble &bndl,
bool &havebndl[],
CRowDouble &bndu,
bool &havebndu[],
int nmain,
int nslack)
{
//--- create variables
int result=0;
int i=0;
bool statuschanged;
for(i=0; i<nmain; i++)
if(x[i]!=xprev[i])
{
statuschanged=false;
if(havebndl[i] && (x[i]==bndl[i] || xprev[i]==bndl[i]))
statuschanged=true;
if(havebndu[i] && (x[i]==bndu[i] || xprev[i]==bndu[i]))
statuschanged=true;
if(statuschanged)
result++;
}
for(i=0; i<nslack; i++)
if(x[nmain+i]!=xprev[nmain+i] && (x[nmain+i]==0.0 || xprev[nmain+i]==0.0))
result=result+1;
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function finds feasible point of(NMain + NSlack)-dimensional|
//| problem subject to NMain explicit boundary constraints (some |
//| constraints can be omitted), NSlack implicit non-negativity |
//| constraints, K linear equality constraints. |
//| INPUT PARAMETERS: |
//| X - array[NMain + NSlack], initial point. |
//| BndL - lower bounds, array[NMain] (may contain -INF, when |
//| bound is not present) |
//| HaveBndL - array[NMain], if HaveBndL[i] is False, then i-th |
//| bound is not present |
//| BndU - array[NMain], upper bounds (may contain +INF, when |
//| bound is not present) |
//| HaveBndU - array[NMain], if HaveBndU[i] is False, then i-th |
//| bound is not present |
//| NMain - number of main variables |
//| NSlack - number of slack variables |
//| CE - array[K, NMain + NSlack + 1], equality constraints|
//| CE*x = b. Rows contain constraints, first |
//| NMain + NSlack columns contain coefficients before |
//| X[], last column contain right part. |
//| K - number of linear constraints |
//| EpsI - infeasibility(error in the right part) allowed in |
//| the solution |
//| OUTPUT PARAMETERS: |
//| X - feasible point or best infeasible point found |
//| beforec algorithm termination |
//| QPIts - number of QP iterations(for debug purposes) |
//| GPAIts - number of GPA iterations(for debug purposes) |
//| RESULT: |
//| True in case X is feasible, |
//| False - if it is infeasible. |
//+------------------------------------------------------------------+
bool COptServ::FindFeasiblePoint(CRowDouble &x,
CRowDouble &bndl,
bool &havebndl[],
CRowDouble &bndu,
bool &havebndu[],
int nmain,
int nslack,
CMatrixDouble &CE,
int k,
double epsi,
int &qpits,
int &gpaits)
{
//--- create variables
bool result;
int i=0;
int j=0;
int idx0=0;
int idx1=0;
CRowDouble permx;
CRowDouble xn;
CRowDouble xa;
CRowDouble newtonstep;
CRowDouble g;
CRowDouble pg;
CRowDouble tau;
CRowDouble s;
double armijostep=0;
double armijobeststep=0;
double armijobestfeas=0;
double v=0;
double vv=0;
double mx=0;
double feaserr=0;
double feaserr0=0;
double feaserr1=0;
double feasold=0;
double feasnew=0;
double pgnorm=0;
double vn=0;
double vd=0;
double stp=0;
int vartofreeze=0;
double valtofreeze=0;
double maxsteplen=0;
bool werechangesinconstraints;
bool stage1isover;
bool converged;
CRowDouble activeconstraints;
CRowDouble tmpk;
CRowDouble colnorms;
int nactive=0;
int nfree=0;
CRowInt p1;
CRowInt p2;
CApBuff buf;
int itscount=0;
int itswithintolerance=0;
int maxitswithintolerance=0;
int badits=0;
int maxbadits=0;
int gparuns=0;
int maxarmijoruns=0;
double infeasibilityincreasetolerance=0;
CMatrixDouble permce;
CMatrixDouble q;
int i_=0;
//--- initialization
CMatrixDouble ce=CE;
qpits=0;
gpaits=0;
maxitswithintolerance=3;
maxbadits=3;
maxarmijoruns=5;
qpits=0;
gpaits=0;
//--- Initial enforcement of the feasibility with respect to boundary constraints
//--- NOTE: after this block we assume that boundary constraints are consistent.
if(!EnforceBoundaryConstraints(x,bndl,havebndl,bndu,havebndu,nmain,nslack))
return(false);
//--- No linear constraints, we can exit right now
if(k==0)
return(true);
//--- Scale rows of CE in such way that max(CE[i,0..m_nmain+nslack-1])=1 for any i=0..m_k-1
for(i=0; i<k; i++)
{
v=CAblasF::RMaxAbsR(nmain+nslack,ce,i);
if(v!=0.0)
ce.Row(i,ce[i]/v);
}
//--- Allocate temporaries
xn.Resize(nmain+nslack);
xa.Resize(nmain+nslack);
permx.Resize(nmain+nslack);
g.Resize(nmain+nslack);
pg.Resize(nmain+nslack);
tmpk.Resize(k);
permce.Resize(k,nmain+nslack);
activeconstraints.Resize(nmain+nslack);
newtonstep.Resize(nmain+nslack);
s=vector<double>::Ones(nmain+nslack);
colnorms=(ce*ce+0).Sum(0); //--- squerd summ by columns
//--- K>0, we have linear equality constraints combined with bound constraints.
//--- Try to find feasible point as minimizer of the quadratic function
//--- F(x) = 0.5*||CE*x-b||^2 = 0.5*x'*(CE'*CE)*x - (b'*CE)*x + 0.5*b'*b
//--- subject to boundary constraints given by BL, BU and non-negativity of
//--- the slack variables. BTW, we drop constant term because it does not
//--- actually influences on the solution.
//--- Below we will assume that K>0.
itswithintolerance=0;
badits=0;
itscount=0;
while(true)
{
//--- Dynamically adjust infeasibility error tolerance
infeasibilityincreasetolerance=MathMax(CAblasF::RMaxAbsV(nmain+nslack,x),1)*(1000+nmain)*CMath::m_machineepsilon;
//--- Stage 0: check for exact convergence
converged=true;
feaserr=FeasibilityError(ce,x,nmain,nslack,k,tmpk);
for(i=0; i<k; i++)
{
//--- Calculate MX - maximum term in the left part
//--- Terminate if error in the right part is not greater than 100*Eps*MX.
//--- IMPORTANT: we must perform check for non-strict inequality, i.e. to use <= instead of <.
//--- it will allow us to easily handle situations with zero rows of CE.
//--- NOTE: it is important to calculate feasibility error with dedicated
//--- function. Once we had a situation when usage of "inline" code
//--- resulted in different numerical values calculated at different
//--- parts of program for exactly same X. However, this value is
//--- essential for algorithm's ability to terminate before entering
//--- infinite loop, so reproducibility of numerical results is very
//--- important.
mx=0;
v=-ce.Get(i,nmain+nslack);
for(j=0; j<nmain+nslack; j++)
{
mx=MathMax(mx,MathAbs(ce.Get(i,j)*x[j]));
v+=ce.Get(i,j)*x[j];
}
converged=converged && MathAbs(v)<=(100.0*CMath::m_machineepsilon*mx);
}
feaserr0=feaserr;
if(converged)
return (feaserr<=epsi);
//--- Stage 1: equality constrained quadratic programming
//--- * treat active bound constraints as equality ones (constraint is considered
//--- active when we are at the boundary, independently of the antigradient direction)
//--- * calculate unrestricted Newton step to point XM (which may be infeasible)
//--- calculate MaxStepLen = largest step in direction of XM which retains feasibility.
//--- * perform bounded step from X to XN:
//--- a) XN=XM (if XM is feasible)
//--- b) XN=X-MaxStepLen*(XM-X) (otherwise)
//--- * X := XN
//--- * if XM (Newton step subject to currently active constraints) was feasible, goto Stage 2
//--- * repeat Stage 1
//--- NOTE 1: in order to solve constrained qudratic subproblem we will have to reorder
//--- variables in such way that ones corresponding to inactive constraints will
//--- be first, and active ones will be last in the list. CE and X are now
//--- [ xi ]
//--- separated into two parts: CE = [CEi CEa], x = [ ], where CEi/Xi correspond
//--- [ xa ]
//--- to INACTIVE constraints, and CEa/Xa correspond to the ACTIVE ones.
//--- Now, instead of F=0.5*x'*(CE'*CE)*x - (b'*CE)*x + 0.5*b'*b, we have
//--- F(xi) = 0.5*(CEi*xi,CEi*xi) + (CEa*xa-b,CEi*xi) + (0.5*CEa*xa-b,CEa*xa).
//--- Here xa is considered constant, i.e. we optimize with respect to xi, leaving xa fixed.
//--- We can solve it by performing SVD of CEi and calculating pseudoinverse of the
//--- Hessian matrix. Of course, we do NOT calculate pseudoinverse explicitly - we
//--- just use singular vectors to perform implicit multiplication by it.
while(true)
{
//--- Calculate G - gradient subject to equality constraints,
//--- multiply it by inverse of the Hessian diagonal to obtain initial
//--- step vector.
//--- Bound step subject to constraints which can be activated,
//--- run Armijo search with increasing step size.
//--- Search is terminated when feasibility error stops to decrease.
//--- NOTE: it is important to test for "stops to decrease" instead
//--- of "starts to increase" in order to correctly handle cases with
//--- zero CE.
armijobeststep=0.0;
FeasibilityErrorGrad(ce,x,nmain,nslack,k,armijobestfeas,g,tmpk);
for(i=0; i<nmain; i++)
{
if((havebndl[i] && x[i]==bndl[i]) || (havebndu[i] && x[i]==bndu[i]))
g.Set(i,0.0);
}
for(i=0; i<nslack; i++)
if(x[nmain+i]==0.0)
g.Set(nmain+i,0.0);
v=0.0;
for(i=0; i<nmain+nslack; i++)
{
if(CMath::Sqr(colnorms[i])!=0.0)
newtonstep.Set(i,-(g[i]/CMath::Sqr(colnorms[i])));
else
newtonstep.Set(i,0.0);
v+=CMath::Sqr(newtonstep[i]);
}
if(v==0.0)
{
//--- Constrained gradient is zero, QP iterations are over
break;
}
CalculateStepBound(x,newtonstep,1.0,bndl,havebndl,bndu,havebndu,nmain,nslack,vartofreeze,valtofreeze,maxsteplen);
if(vartofreeze>=0 && maxsteplen==0.0)
{
//--- Can not perform step, QP iterations are over
break;
}
if(vartofreeze>=0)
armijostep=MathMin(1.0,maxsteplen);
else
armijostep=1;
while(true)
{
xa=x.ToVector()+newtonstep*armijostep;
EnforceBoundaryConstraints(xa,bndl,havebndl,bndu,havebndu,nmain,nslack);
feaserr=FeasibilityError(ce,xa,nmain,nslack,k,tmpk);
if(feaserr>=armijobestfeas)
break;
armijobestfeas=feaserr;
armijobeststep=armijostep;
armijostep=2.0*armijostep;
}
x+=newtonstep*armijobeststep+0;
EnforceBoundaryConstraints(x,bndl,havebndl,bndu,havebndu,nmain,nslack);
//--- Determine number of active and free constraints
nactive=0;
for(i=0; i<nmain; i++)
{
activeconstraints.Set(i,0);
if(havebndl[i] && x[i]==bndl[i])
activeconstraints.Set(i,1);
if(havebndu[i] && x[i]==bndu[i])
activeconstraints.Set(i,1);
if(activeconstraints[i]>0.0)
nactive++;
}
for(i=0; i<nslack; i++)
{
activeconstraints.Set(nmain+i,0);
if(x[nmain+i]==0.0)
activeconstraints.Set(nmain+i,1);
if(activeconstraints[nmain+i]>0.0)
nactive++;
}
nfree=nmain+nslack-nactive;
if(nfree==0)
break;
qpits++;
//--- Reorder variables: CE is reordered to PermCE, X is reordered to PermX
CTSort::TagSortBuf(activeconstraints,nmain+nslack,p1,p2,buf);
permce=ce;
permx=x;
for(j=0; j<nmain+nslack; j++)
{
if(p2[j]!=j)
{
idx0=p2[j];
idx1=j;
permce.SwapCols(idx0,idx1);
permx.Swap(idx0,idx1);
}
}
//--- Calculate (unprojected) gradient:
//--- G(xi) = CEi'*(CEi*xi + CEa*xa - b)
for(i=0; i<nfree; i++)
g.Set(i,0);
for(i=0; i<k; i++)
{
v=CAblasF::RDotVR(nmain+nslack,permx,permce,i);
tmpk.Set(i,v-ce.Get(i,nmain+nslack));
}
for(i=0; i<k; i++)
{
v=tmpk[i];
for(i_=0; i_<nfree; i_++)
g.Add(i_,v*permce.Get(i,i_));
}
//--- Calculate Newton step using pseudoinverse PermCE:
//--- F(xi) = 0.5*xi'*H*xi + g'*xi (Taylor decomposition)
//--- XN = -H^(-1)*g (new point, solution of the QP subproblem)
//--- H = CEi'*CEi
//--- H^(-1) can be calculated via QR or LQ decomposition (see below)
//--- step = -H^(-1)*g
//--- NOTE: PermCE is destroyed after this block
newtonstep=vector<double>::Zeros(nmain+nslack);
if(k<=nfree)
{
//--- CEi = L*Q
//--- H = Q'*L'*L*Q
//--- inv(H) = Q'*inv(L)*inv(L')*Q
//--- NOTE: we apply minor regularizing perturbation to diagonal of L,
//--- which is equal to 10*K*Eps
COrtFac::RMatrixLQ(permce,k,nfree,tau);
COrtFac::RMatrixLQUnpackQ(permce,k,nfree,tau,k,q);
vector<double> diag=permce.Diag();
v=(MathAbs(diag)).Max();
if(v!=0.0)
permce.Diag(diag+10.0*k*CMath::m_machineepsilon*v);
CAblas::RMatrixGemVect(k,nfree,1.0,q,0,0,0,g,0,0.0,tmpk,0);
CAblas::RMatrixTrsVect(k,permce,0,0,false,false,1,tmpk,0);
CAblas::RMatrixTrsVect(k,permce,0,0,false,false,0,tmpk,0);
CAblas::RMatrixGemVect(nfree,k,-1.0,q,0,0,1,tmpk,0,0.0,newtonstep,0);
}
else
{
//--- CEi = Q*R
//--- H = R'*R
//--- inv(H) = inv(R)*inv(R')
//--- NOTE: we apply minor regularizing perturbation to diagonal of R,
//--- which is equal to 10*K*Eps
COrtFac::RMatrixQR(permce,k,nfree,tau);
vector<double> diag=permce.Diag();
diag.Resize(nfree);
v=(MathAbs(diag)).Max();
if(v!=0.0)
for(i=0; i<nfree; i++)
{
vv=10*nfree*CMath::m_machineepsilon*v;
if(diag[i]<0.0)
vv=-vv;
permce.Add(i,i,vv);
}
for(i_=0; i_<nfree; i_++)
newtonstep.Set(i_,-g[i_]);
CAblas::RMatrixTrsVect(nfree,permce,0,0,true,false,1,newtonstep,0);
CAblas::RMatrixTrsVect(nfree,permce,0,0,true,false,0,newtonstep,0);
}
//--- Post-reordering of Newton step
for(j=nmain+nslack-1; j>=0; j--)
{
if(p2[j]!=j)
{
idx0=p2[j];
idx1=j;
newtonstep.Swap(idx0,idx1);
}
}
//--- NewtonStep contains Newton step subject to active bound constraints.
//--- Such step leads us to the minimizer of the equality constrained F,
//--- but such minimizer may be infeasible because some constraints which
//--- are inactive at the initial point can be violated at the solution.
//--- Thus, we perform optimization in two stages:
//--- a) perform bounded Newton step, i.e. step in the Newton direction
//--- until activation of the first constraint
//--- b) in case (MaxStepLen>0)and(MaxStepLen<1), perform additional iteration
//--- of the Armijo line search in the rest of the Newton direction.
CalculateStepBound(x,newtonstep,1.0,bndl,havebndl,bndu,havebndu,nmain,nslack,vartofreeze,valtofreeze,maxsteplen);
if(vartofreeze>=0)
{
if(maxsteplen==0.0)
//--- Activation of the constraints prevent us from performing step,
//--- QP iterations are over
break;
v=MathMin(1.0,maxsteplen);
}
else
v=1.0;
xn=x.ToVector()+newtonstep*v;
PostProcessBoundedStep(xn,x,bndl,havebndl,bndu,havebndu,nmain,nslack,vartofreeze,valtofreeze,v,maxsteplen);
if(maxsteplen>0.0 && maxsteplen<1.0)
{
//--- Newton step was restricted by activation of the constraints,
//--- perform Armijo iteration.
//--- Initial estimate for best step is zero step. We try different
//--- step sizes, from the 1-MaxStepLen (residual of the full Newton
//--- step) to progressively smaller and smaller steps.
armijobeststep=0.0;
armijobestfeas=FeasibilityError(ce,xn,nmain,nslack,k,tmpk);
armijostep=1-maxsteplen;
for(j=0; j<maxarmijoruns; j++)
{
xa=xn.ToVector()+newtonstep*armijostep;
EnforceBoundaryConstraints(xa,bndl,havebndl,bndu,havebndu,nmain,nslack);
feaserr=FeasibilityError(ce,xa,nmain,nslack,k,tmpk);
if(feaserr<armijobestfeas)
{
armijobestfeas=feaserr;
armijobeststep=armijostep;
}
armijostep=0.5*armijostep;
}
xa=xn.ToVector()+newtonstep*armijobeststep;
EnforceBoundaryConstraints(xa,bndl,havebndl,bndu,havebndu,nmain,nslack);
}
else
{
//--- Armijo iteration is not performed
xa=xn;
}
stage1isover=(maxsteplen>=1.0 || maxsteplen==0.0);
//--- Calculate feasibility errors for old and new X.
//--- These quantinies are used for debugging purposes only.
//--- However, we can leave them in release code because performance impact is insignificant.
//--- Update X. Exit if needed.
feasold=FeasibilityError(ce,x,nmain,nslack,k,tmpk);
feasnew=FeasibilityError(ce,xa,nmain,nslack,k,tmpk);
if(feasnew>=(feasold+infeasibilityincreasetolerance))
break;
x=xa;
if(stage1isover)
break;
}
//--- Stage 2: gradient projection algorithm (GPA)
//--- * calculate feasibility error (with respect to linear equality constraints)
//--- * calculate gradient G of F, project it into feasible area (G => PG)
//--- * exit if norm(PG) is exactly zero or feasibility error is smaller than EpsC
//--- * let XM be exact minimum of F along -PG (XM may be infeasible).
//--- calculate MaxStepLen = largest step in direction of -PG which retains feasibility.
//--- * perform bounded step from X to XN:
//--- a) XN=XM (if XM is feasible)
//--- b) XN=X-MaxStepLen*PG (otherwise)
//--- * X := XN
//--- * stop after specified number of iterations or when no new constraints was activated
//--- NOTES:
//--- * grad(F) = (CE'*CE)*x - (b'*CE)^T
//--- * CE[i] denotes I-th row of CE
//--- * XM = X+stp*(-PG) where stp=(grad(F(X)),PG)/(CE*PG,CE*PG).
//--- Here PG is a projected gradient, but in fact it can be arbitrary non-zero
//--- direction vector - formula for minimum of F along PG still will be correct.
werechangesinconstraints=false;
for(gparuns=1; gparuns<=k; gparuns++)
{
//--- calculate feasibility error and G
FeasibilityErrorGrad(ce,x,nmain,nslack,k,feaserr,g,tmpk);
//--- project G, filter it (strip numerical noise)
pg=g;
ProjectGradientIntoBC(x,pg,bndl,havebndl,bndu,havebndu,nmain,nslack);
FilterDirection(pg,x,bndl,havebndl,bndu,havebndu,s,nmain,nslack,1.0E-9);
for(i=0; i<nmain+nslack; i++)
{
if(CMath::Sqr(colnorms[i])!=0.0)
pg.Mul(i,1.0/CMath::Sqr(colnorms[i]));
else
pg.Set(i,0.0);
}
//--- Check GNorm and feasibility.
//--- Exit when GNorm is exactly zero.
pgnorm=MathSqrt(pg.Dot(pg));
if(pgnorm==0.0)
{
result=(feaserr<=epsi);
return(result);
}
//--- calculate planned step length
vn=g.Dot(pg);
CAblas::RMatrixGemVect(k,nmain+nslack,1.0,ce,0,0,0,pg,0,0.0,tmpk,0);
vd=tmpk.Dot(tmpk);
stp=vn/vd;
//--- Calculate step bound.
//--- Perform bounded step and post-process it
CalculateStepBound(x,pg,-1.0,bndl,havebndl,bndu,havebndu,nmain,nslack,vartofreeze,valtofreeze,maxsteplen);
if(vartofreeze>=0 && maxsteplen==0.0)
{
result=false;
return(result);
}
if(vartofreeze>=0)
v=MathMin(stp,maxsteplen);
else
v=stp;
xn=x.ToVector()-pg*v;
PostProcessBoundedStep(xn,x,bndl,havebndl,bndu,havebndu,nmain,nslack,vartofreeze,valtofreeze,v,maxsteplen);
//--- update X
//--- check stopping criteria
werechangesinconstraints=(werechangesinconstraints || NumberOfChangedConstraints(xn,x,bndl,havebndl,bndu,havebndu,nmain,nslack)>0);
x=xn;
gpaits++;
if(!werechangesinconstraints)
break;
}
//--- Stage 3: decide to stop algorithm or not to stop
//--- 1. we can stop when last GPA run did NOT changed constraints status.
//--- It means that we've found final set of the active constraints even
//--- before GPA made its run. And it means that Newton step moved us to
//--- the minimum subject to the present constraints.
//--- Depending on feasibility error, True or False is returned.
feaserr=FeasibilityError(ce,x,nmain,nslack,k,tmpk);
feaserr1=feaserr;
if(feaserr1>=(feaserr0-infeasibilityincreasetolerance))
badits++;
else
badits=0;
if(feaserr<=epsi)
itswithintolerance++;
else
itswithintolerance=0;
if((!werechangesinconstraints || itswithintolerance>=maxitswithintolerance) || badits>=maxbadits)
{
result=(feaserr<=epsi);
return(result);
}
itscount++;
}
return(true);
}
//+------------------------------------------------------------------+
//| This function checks that input derivatives are right. First it |
//| scales parameters DF0 and DF1 from segment [A; B] to [0; 1]. Then|
//| it builds Hermite spline and derivative of it in 0.5. Search |
//| scale as Max(DF0, DF1, | F0 - F1 |). Right derivative has |
//| to satisfy condition: |
//| | H - F | / S <= 0, 001, | H'-F' | / S <= 0, 001. |
//| INPUT PARAMETERS: |
//| F0 - function's value in X-TestStep point; |
//| DF0 - derivative's value in X-TestStep point; |
//| F1 - function's value in X+TestStep point; |
//| DF1 - derivative's value in X+TestStep point; |
//| F - testing function's value; |
//| DF - testing derivative's value; |
//| Width - width of verification segment. |
//| RESULT: |
//| If input derivatives is right then function returns true, else |
//| function returns false. |
//+------------------------------------------------------------------+
bool COptServ::DerivativeCheck(double f0,double df0,double f1,
double df1,double f,double df,
double width)
{
//--- create variables
double s=0;
double h=0;
double dh=0;
//--- Rescale input data to [0,1]
df=width*df;
df0=width*df0;
df1=width*df1;
//--- Compute error scale, two sources are used:
//--- * magnitudes of derivatives and secants
//--- * magnitudes of input data times sqrt(machine_epsilon)
s=0.0;
s=MathMax(s,MathAbs(df0));
s=MathMax(s,MathAbs(df1));
s=MathMax(s,MathAbs(f1-f0));
s=MathMax(s,MathSqrt(CMath::m_machineepsilon)*MathAbs(f0));
s=MathMax(s,MathSqrt(CMath::m_machineepsilon)*MathAbs(f1));
//--- Compute H and dH/dX at the middle of interval
h=0.5*(f0+f1)+0.125*(df0-df1);
dh=1.5*(f1-f0)-0.250*(df0+df1);
//--- Check
if(s!=0.0)
{
if((MathAbs(h-f)/s)>0.001 || (MathAbs(dh-df)/s)>0.001)
return(false);
}
else
{
if((double)(h-f)!=0.0 || (double)(dh-df)!=0.0)
return(false);
}
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Having quadratic target function |
//| f(x)=0.5*x'*A*x+b'*x+penaltyfactor*0.5*(C*x-b)'*(C*x-b) |
//| and its parabolic model along direction D |
//| F(x0 + alpha*D) = D2 * alpha ^ 2 + D1 * alpha |
//| this function estimates numerical errors in the coefficients of |
//| the model. |
//| It is important that this function does NOT calculate D1/D2 - it |
//| only estimates numerical errors introduced during evaluation and |
//| compares their magnitudes against magnitudes of numerical errors.|
//| As result, one of three outcomes is returned for each |
//| coefficient: |
//| *"true" coefficient is almost surely positive |
//| *"true" coefficient is almost surely negative |
//| * numerical errors in coefficient are so large that it can |
//| not be reliably distinguished from zero |
//| INPUT PARAMETERS: |
//| AbsASum - SUM( | A.Get(i,j) |) |
//| AbsASum2 - SUM(A.Get(i,j) ^ 2) |
//| MB - max( | B |) |
//| MX - max( | X |) |
//| MD - max( | D |) |
//| D1 - linear coefficient |
//| D2 - quadratic coefficient |
//| OUTPUT PARAMETERS: |
//| D1Est - estimate of D1 sign, accounting for possible |
//| numerical errors: |
//| *>0 means "almost surely positive"(D1 > 0 |
//| and large) |
//| *<0 means "almost surely negative"(D1 < 0 |
//| and large) |
//| *=0 means "pessimistic estimate of numerical|
//| errors in D1 is larger than magnitude of|
//| D1 itself; it is impossible to reliably |
//| distinguish D1 from zero". |
//| D2Est - estimate of D2 sign, accounting for possible |
//| numerical errors: |
//| *>0 means "almost surely positive"(D2 > 0 |
//| and large) |
//| *<0 means "almost surely negative"(D2 < 0 |
//| and large) |
//| *=0 means "pessimistic estimate of numerical|
//| errors in D2 is larger than magnitude of|
//| D2 itself; it is impossible to reliably |
//| distinguish D2 from zero". |
//+------------------------------------------------------------------+
void COptServ::EstimateParabolicModel(double absasum,double absasum2,
double mx,double mb,
double md,double d1,
double d2,int &d1est,
int &d2est)
{
//--- create variables
double d1esterror=0;
double d2esterror=0;
double eps=0;
double e1=0;
double e2=0;
d1est=0;
d2est=0;
//--- Error estimates:
//--- * error in D1=d'*(A*x+b) is estimated as
//--- ED1 = eps*MAX_ABS(D)*(MAX_ABS(X)*ENORM(A)+MAX_ABS(B))
//--- * error in D2=0.5*d'*A*d is estimated as
//--- ED2 = eps*MAX_ABS(D)^2*ENORM(A)
//--- Here ENORM(A) is some pseudo-norm which reflects the way numerical
//--- error accumulates during addition. Two ways of accumulation are
//--- possible - worst case (errors always increase) and mean-case (errors
//--- may cancel each other). We calculate geometrical average of both:
//--- * ENORM_WORST(A) = SUM(|A[i,j]|) error in N-term sum grows as O(N)
//--- * ENORM_MEAN(A) = SQRT(SUM(A[i,j]^2)) error in N-term sum grows as O(sqrt(N))
//--- * ENORM(A) = SQRT(ENORM_WORST(A),ENORM_MEAN(A))
eps=4*CMath::m_machineepsilon;
e1=eps*md*(mx*absasum+mb);
e2=eps*md*(mx*MathSqrt(absasum2)+mb);
d1esterror=MathSqrt(e1*e2);
if(MathAbs(d1)<=d1esterror)
d1est=0;
else
d1est=(int)MathSign(d1);
e1=eps*md*md*absasum;
e2=eps*md*md*MathSqrt(absasum2);
d2esterror=MathSqrt(e1*e2);
if(MathAbs(d2)<=d2esterror)
d2est=0;
else
d2est=(int)MathSign(d2);
}
//+------------------------------------------------------------------+
//| This function calculates inexact rank - K preconditioner for |
//| Hessian matrix H = D + W'*C*W, where: |
//| * H is a Hessian matrix, which is approximated by D / W / C |
//| * D is a diagonal matrix with positive entries |
//| * W is a rank - K correction |
//| * C is a diagonal factor of rank - K correction |
//| This preconditioner is inexact but fast - it requires O(N*K) time|
//| to be applied. Its main purpose - to be used in barrier / penalty|
//| / AUL methods, where ill - conditioning is created by combination|
//| of two factors: |
//| * simple bounds on variables => ill - conditioned D |
//| * general barrier / penalty => correction W with large |
//| coefficient C(makes problem ill - conditioned) but W itself |
//| is well conditioned. |
//| Preconditioner P is calculated by artificially constructing a set|
//| of BFGS updates which tries to reproduce behavior of H: |
//| * Sk = Wk(k - th row of W) |
//| * Yk = (D + Wk'*Ck*Wk)*Sk |
//| * Yk / Sk are reordered by ascending of C[k] * norm(Wk) ^ 2 |
//| Here we assume that rows of Wk are orthogonal or nearly |
//| orthogonal, which allows us to have O(N * K + K^2) update instead|
//| of O(N * K ^ 2) one. Reordering of updates is essential for |
//| having good performance on non-orthogonal problems (updates which|
//| do not add much of curvature are added first, and updates which |
//| add very large eigenvalues are added last and override effect of |
//| the first updates). |
//| On input this function takes direction S and components of H. |
//| On output it returns inv(H) * S |
//+------------------------------------------------------------------+
void COptServ::InexactLBFGSPreconditioner(CRowDouble &s,
int n,
CRowDouble &d,
CRowDouble &c,
CMatrixDouble &w,
int k,
CPrecBufLBFGS &buf)
{
//--- create variables
int idx=0;
int i=0;
int j=0;
double v=0;
double v0=0;
double v1=0;
double vx=0;
double vy=0;
int i_=0;
//--- allocate
buf.m_norms.Resize(k);
buf.m_alpha.Resize(k);
buf.m_rho.Resize(k);
buf.m_yk.Resize(k,n);
buf.m_idx.Resize(k);
//--- Check inputs
if(!CAp::Assert(d.Min()>0.0,"InexactLBFGSPreconditioner: D[]<=0"))
return;
if(!CAp::Assert(c.Min()>=0.0,"InexactLBFGSPreconditioner: C[]<0"))
return;
//--- Reorder linear terms according to increase of second derivative.
//--- Fill Norms[] array.
vector<double> temp;
for(idx=0; idx<k; idx++)
{
temp=w[idx];
temp.Resize(n);
v=temp.Dot(temp);
buf.m_norms.Set(idx,v*c[idx]);
buf.m_idx.Set(idx,idx);
}
CTSort::TagSortFastI(buf.m_norms,buf.m_idx,buf.m_bufa,buf.m_bufb,k);
//--- Apply updates
for(idx=0; idx<k; idx++)
{
//--- Select update to perform (ordered by ascending of second derivative)
i=buf.m_idx[idx];
//--- Calculate YK and Rho
temp=w[i];
temp.Resize(n);
v0=temp.Dot(temp);
v=v0*c[i];
buf.m_yk.Row(i,(d+v)*temp);
v=temp.Dot(buf.m_yk[i]+0);
v1=CAblasF::RDotRR(n,buf.m_yk,i,buf.m_yk,i);
if(v>0.0 && (v0*v1)>0.0 && (v/MathSqrt(v0*v1))>(n*10.0*CMath::m_machineepsilon))
buf.m_rho.Set(i,1/v);
else
buf.m_rho.Set(i,0.0);
}
for(idx=k-1; idx>=0; idx--)
{
//--- Select update to perform (ordered by ascending of second derivative)
i=buf.m_idx[idx];
temp=w[i];
temp.Resize(n);
//--- Calculate Alpha[] according to L-BFGS algorithm
//--- and update S[]
v=s.Dot(temp);
v*=buf.m_rho[i];
buf.m_alpha.Set(i,v);
s-=buf.m_yk[i]*v;
}
s/=d;
for(idx=0; idx<k; idx++)
{
//--- Select update to perform (ordered by ascending of second derivative)
i=buf.m_idx[idx];
temp=w[i];
temp.Resize(n);
//--- Calculate Beta according to L-BFGS algorithm
//--- and update S[]
v=s.Dot(buf.m_yk[i]+0);
v=buf.m_alpha[i]-buf.m_rho[i]*v;
s+=temp*v;
}
}
//+------------------------------------------------------------------+
//| This function prepares exact low-rank preconditioner for Hessian |
//| matrix H = D + W'*C*W, where: |
//| * H is a Hessian matrix, which is approximated by D / W / C |
//| * D is a diagonal matrix with positive entries |
//| * W is a rank - K correction |
//| * C is a diagonal factor of rank - K correction, positive |
//| semidefinite |
//| This preconditioner is exact but relatively slow - it requires |
//| O(N * K ^ 2) time to be prepared and O(N*K) time to be applied. |
//| It is calculated with the help of Woodbury matrix identity. |
//| It should be used as follows: |
//| * PrepareLowRankPreconditioner() call PREPARES data structure |
//| * subsequent calls to ApplyLowRankPreconditioner() APPLY |
//| preconditioner to user - specified search direction. |
//+------------------------------------------------------------------+
void COptServ::PrepareLowRankPreconditioner(CRowDouble &d,
CRowDouble &c,
CMatrixDouble &w,
int n,
int k,
CPrecBufLowRank &buf)
{
//--- create variables
int i=0;
int j=0;
double v=0;
bool b;
//--- Check inputs
if(!CAp::Assert(n>0,__FUNCTION__+": N<=0"))
return;
if(!CAp::Assert(k>=0,__FUNCTION__+": N<=0"))
return;
if(!CAp::Assert(d.Min()>0.0,__FUNCTION__+": D[]<=0"))
return;
if(!CAp::Assert(c.Min()>=0.0,__FUNCTION__+": C[]<0"))
return;
//--- Prepare buffer structure; skip zero entries of update.
CApServ::RVectorSetLengthAtLeast(buf.m_d,n);
CApServ::RMatrixSetLengthAtLeast(buf.m_v,k,n);
CApServ::RVectorSetLengthAtLeast(buf.m_bufc,k);
CApServ::RMatrixSetLengthAtLeast(buf.m_bufw,k+1,n);
buf.m_n=n;
buf.m_k=0;
for(i=0; i<k; i++)
{
//--- Estimate magnitude of update row; skip zero rows (either W or C are zero)
vector<double> temp=w[i];
v=temp.Dot(temp)*c[i];
if(v==0.0)
continue;
//--- check
if(!CAp::Assert(v>0.0,__FUNCTION__+": internal error"))
return;
//--- Copy non-zero update to buffer
buf.m_bufc.Set(buf.m_k,c[i]);
buf.m_v.Row(buf.m_k,temp);
buf.m_bufw.Row(buf.m_k,temp);
buf.m_k++;
}
//--- Reset K (for convenience)
k=buf.m_k;
//--- Prepare diagonal factor; quick exit for K=0
buf.m_d=d.Pow(-1.0)+0;
if(k==0)
return;
//--- Use Woodbury matrix identity
buf.m_bufz=matrix<double>::Zeros(k,k);
buf.m_bufc.Resize(k);
buf.m_bufz.Diag(buf.m_bufc.Pow(-1.0)+0);
buf.m_bufw.Row(k,d.Pow(-0.5)+0);
for(i=0; i<k; i++)
for(j=0; j<n; j++)
buf.m_bufw.Mul(i,j,buf.m_bufw.Get(k,j));
CAblas::RMatrixGemm(k,k,n,1.0,buf.m_bufw,0,0,0,buf.m_bufw,0,0,1,1.0,buf.m_bufz,0,0);
b=CTrFac::SPDMatrixCholeskyRec(buf.m_bufz,0,k,true,buf.m_tmp);
//--- check
if(!CAp::Assert(b,__FUNCTION__+": internal error (Cholesky failure)"))
return;
CAblas::RMatrixLeftTrsM(k,n,buf.m_bufz,0,0,true,false,1,buf.m_v,0,0);
for(i=0; i<k; i++)
for(j=0; j<n; j++)
buf.m_v.Mul(i,j,buf.m_d[j]);
}
//+------------------------------------------------------------------+
//| This function apply exact low - rank preconditioner prepared by |
//| PrepareLowRankPreconditioner function (see its comments for more |
//| information). |
//+------------------------------------------------------------------+
void COptServ::ApplyLowRankPreconditioner(CRowDouble &s,
CPrecBufLowRank &buf)
{
//--- create variables
int i=0;
int j=0;
double v=0;
int n=buf.m_n;
int k=buf.m_k;
buf.m_tmp=vector<double>::Zeros(n);
for(i=0; i<n; i++)
buf.m_tmp.Set(i,buf.m_d[i]*s[i]);
for(i=0; i<k; i++)
{
v=CAblasF::RDotVR(n,s,buf.m_v,i);
for(j=0; j<n; j++)
buf.m_tmp.Add(j,-buf.m_v.Get(i,j)*v);
}
for(i=0; i<n; i++)
s.Set(i,buf.m_tmp[i]);
}
//+------------------------------------------------------------------+
//| This subroutine initializes smoothness monitor at the beginning |
//| of the optimization session. It requires variable scales to be |
//| passed. |
//| It is possible to perform "dummy" initialization with N = K = 0. |
//+------------------------------------------------------------------+
void COptServ::SmoothnessMonitorInit(CSmoothnessMonitor &monitor,
CRowDouble &s,
int n,
int k,
bool checksmoothness)
{
monitor.m_n=n;
monitor.m_k=k;
monitor.m_checksmoothness=checksmoothness;
monitor.m_linesearchspoiled=false;
monitor.m_linesearchstarted=false;
monitor.m_enqueuedcnt=0;
monitor.m_sortedcnt=0;
monitor.m_s=s;
monitor.m_nonc0currentrating=0.0;
monitor.m_nonc1currentrating=0.0;
COptGuardApi::OptGuardInitInternal(monitor.m_rep,n,k);
monitor.m_nonc0strrating=0.0;
monitor.m_nonc0lngrating=-CMath::m_maxrealnumber;
monitor.m_nonc0strrep.m_positive=false;
monitor.m_nonc0lngrep.m_positive=false;
monitor.m_nonc1test0strrating=0.0;
monitor.m_nonc1test0lngrating=-CMath::m_maxrealnumber;
monitor.m_nonc1test0strrep.m_positive=false;
monitor.m_nonc1test0lngrep.m_positive=false;
monitor.m_nonc1test1strrating=0.0;
monitor.m_nonc1test1lngrating=-CMath::m_maxrealnumber;
monitor.m_nonc1test1strrep.m_positive=false;
monitor.m_nonc1test1lngrep.m_positive=false;
monitor.m_badgradhasxj=false;
monitor.m_rstateg0.ia.Resize(5);
monitor.m_rstateg0.ra.Resize(4);
monitor.m_rstateg0.stage=-1;
}
//+------------------------------------------------------------------+
//| This subroutine starts line search |
//+------------------------------------------------------------------+
void COptServ::SmoothnessMonitorStartLineSearch(CSmoothnessMonitor &monitor,
CRowDouble &x,
CRowDouble &fi,
CMatrixDouble &jac)
{
//--- create variables
int i=0;
int j=0;
double v=0;
int n=monitor.m_n;
int k=monitor.m_k;
//--- Skip if inactive or spoiled by NAN
if(!monitor.m_checksmoothness)
return;
v=0;
for(i=0; i<n; i++)
v=0.5*v+x[i];
for(i=0; i<k; i++)
v=0.5*v+fi[i];
for(i=0; i<k; i++)
for(j=0; j<n; j++)
v=0.5*v+jac.Get(i,j);
if(!MathIsValidNumber(v))
{
monitor.m_linesearchspoiled=true;
return;
}
//--- Finalize previous line search
if(monitor.m_enqueuedcnt>0)
SmoothnessMonitorFinalizeLineSearch(monitor);
//--- Store initial point
monitor.m_linesearchstarted=true;
monitor.m_enqueuedcnt=1;
monitor.m_enqueuedx=x;
monitor.m_enqueuedfunc=fi;
monitor.m_enqueuedjac=jac;
monitor.m_enqueuedstp.Resize(monitor.m_enqueuedcnt);
monitor.m_enqueuedx.Resize(monitor.m_enqueuedcnt*n);
monitor.m_enqueuedfunc.Resize(monitor.m_enqueuedcnt*k);
monitor.m_enqueuedjac.Resize(monitor.m_enqueuedcnt*k,n);
monitor.m_enqueuedstp.Set(0,0.0);
//--- Initialize sorted representation
monitor.m_sortedstp.Resize(1);
monitor.m_sortedidx.Resize(1);
monitor.m_sortedstp.Set(0,0.0);
monitor.m_sortedidx.Set(0,0);
monitor.m_sortedcnt=1;
}
//+------------------------------------------------------------------+
//| This subroutine starts line search for a scalar function - |
//| convenience wrapper for ....StartLineSearch() with unscaled |
//| variables. |
//+------------------------------------------------------------------+
void COptServ::SmoothnessMonitorStartLineSearch1u(CSmoothnessMonitor &monitor,
CRowDouble &s,
CRowDouble &invs,
CRowDouble &x,
double f0,
CRowDouble &j0)
{
//--- create variables
int n=monitor.m_n;
int k=monitor.m_k;
if(!monitor.m_checksmoothness)
return;
//--- check
if(!CAp::Assert(k==1,__FUNCTION__+": K<>1"))
return;
monitor.m_xu=x*invs+0;
monitor.m_xu.Resize(n);
monitor.m_f0.Resize(1);
monitor.m_j0.Resize(1,n);
monitor.m_j0.Row(0,j0*s+0);
monitor.m_f0.Set(0,f0);
SmoothnessMonitorStartLineSearch(monitor,monitor.m_xu,monitor.m_f0,monitor.m_j0);
}
//+------------------------------------------------------------------+
//| This subroutine enqueues one more trial point |
//+------------------------------------------------------------------+
void COptServ::SmoothnessMonitorEnqueuePoint(CSmoothnessMonitor &monitor,
CRowDouble &d,
double stp,
CRowDouble &x,
CRowDouble &fi,
CMatrixDouble &jac)
{
//--- create variables
int i=0;
int j=0;
double v=0;
int enqueuedcnt=0;
int sortedcnt=0;
bool hasduplicates;
int funcidx=0;
int stpidx=0;
double f0=0;
double f1=0;
double f2=0;
double f3=0;
double f4=0;
double noise0=0;
double noise1=0;
double noise2=0;
double noise3=0;
double rating=0;
double lipschitz=0;
double nrm=0;
double lengthrating=0;
int n=monitor.m_n;
int k=monitor.m_k;
//--- Skip if inactive or spoiled by NAN
if(!monitor.m_checksmoothness || monitor.m_linesearchspoiled || !monitor.m_linesearchstarted)
return;
v=stp;
for(i=0; i<n; i++)
v=0.5*v+x[i];
for(i=0; i<n; i++)
v=0.5*v+d[i];
for(i=0; i<k; i++)
v=0.5*v+fi[i];
for(i=0; i<k; i++)
for(j=0; j<n; j++)
v=0.5*v+jac.Get(i,j);
if(!MathIsValidNumber(v))
{
monitor.m_linesearchspoiled=true;
return;
}
//--- Enqueue
monitor.m_enqueuedcnt++;
enqueuedcnt=monitor.m_enqueuedcnt;
monitor.m_dcur=d;
monitor.m_dcur.Resize(n);
monitor.m_enqueuedstp.Resize(enqueuedcnt);
monitor.m_enqueuedx.Resize(enqueuedcnt*n);
monitor.m_enqueuedfunc.Resize(enqueuedcnt*k);
monitor.m_enqueuedjac.Resize(enqueuedcnt*k,n);
monitor.m_enqueuedstp.Set(enqueuedcnt-1,stp);
for(j=0; j<n; j++)
monitor.m_enqueuedx.Set((enqueuedcnt-1)*n+j,x[j]);
for(i=0; i<k; i++)
monitor.m_enqueuedfunc.Set((enqueuedcnt-1)*k+i,fi[i]);
for(i=0; i<k; i++)
for(j=0; j<n; j++)
monitor.m_enqueuedjac.Set((enqueuedcnt-1)*k+i,j,jac.Get(i,j));
//--- Update sorted representation: insert to the end, reorder
sortedcnt=monitor.m_sortedcnt;
hasduplicates=false;
for(i=0; i<sortedcnt; i++)
hasduplicates=(hasduplicates || monitor.m_sortedstp[i]==stp);
if(!hasduplicates)
{
monitor.m_sortedcnt++;
sortedcnt=monitor.m_sortedcnt;
monitor.m_sortedstp.Resize(sortedcnt);
monitor.m_sortedidx.Resize(sortedcnt);
monitor.m_sortedstp.Set(sortedcnt-1,stp);
monitor.m_sortedidx.Set(sortedcnt-1,enqueuedcnt-1);
for(i=sortedcnt-2; i>=0; i--)
{
if(monitor.m_sortedstp[i]<=monitor.m_sortedstp[i+1])
break;
monitor.m_sortedstp.Swap(i,i+1);
monitor.m_sortedidx.Swap(i,i+1);
}
}
//--- Scan sorted representation, check for C0 and C1 continuity
//--- violations.
monitor.m_f.Resize(sortedcnt);
monitor.m_g.Resize(sortedcnt*n);
for(funcidx=0; funcidx<k; funcidx++)
{
//--- Fetch current function and its gradient to the contiguous storage
for(i=0; i<sortedcnt; i++)
{
monitor.m_f.Set(i,monitor.m_enqueuedfunc[monitor.m_sortedidx[i]*k+funcidx]);
for(j=0; j<n; j++)
monitor.m_g.Set(i*n+j,monitor.m_enqueuedjac.Get(monitor.m_sortedidx[i]*k+funcidx,j));
}
//--- Check C0 continuity.
//--- The basis approach is that we find appropriate candidate point
//--- (either a local minimum along the line - for target; or an interval
//--- where function sign is changed - for constraints), calculate left
//--- and right estimates of the Lipschitz constant (slopes between points
//--- #0 and #1, #2 and #3), and then calculate slope between points #1 and
//--- #2 and compare it with left/right estimates.
//--- The actual approach is a bit more complex to account for different
//--- sources of numerical noise and different false positive scenarios.
if(funcidx==0)
{
for(stpidx=0; stpidx<sortedcnt-3; stpidx++)
{
f0=monitor.m_f[stpidx+0];
f1=monitor.m_f[stpidx+1];
f2=monitor.m_f[stpidx+2];
f3=monitor.m_f[stpidx+3];
noise0=m_ognoiselevelf*MathMax(MathAbs(f0),1.0);
noise1=m_ognoiselevelf*MathMax(MathAbs(f1),1.0);
noise2=m_ognoiselevelf*MathMax(MathAbs(f2),1.0);
noise3=m_ognoiselevelf*MathMax(MathAbs(f3),1.0);
if(!(f1<f0+(noise0+noise1) && f1<f2))
continue;
TestC0Continuity(f0,f1,f2,f3,noise0,noise1,noise2,noise3,monitor.m_sortedstp[stpidx+1]-monitor.m_sortedstp[stpidx+0],monitor.m_sortedstp[stpidx+2]-monitor.m_sortedstp[stpidx+1],monitor.m_sortedstp[stpidx+3]-monitor.m_sortedstp[stpidx+2],false,rating,lipschitz);
if(rating>m_ogminrating0)
{
//--- Store to total report
monitor.m_rep.m_nonc0suspected=true;
monitor.m_rep.m_nonc0test0positive=true;
if(rating>monitor.m_nonc0currentrating)
{
monitor.m_nonc0currentrating=rating;
monitor.m_rep.m_nonc0lipschitzc=lipschitz;
monitor.m_rep.m_nonc0fidx=funcidx;
}
//--- Store to "strongest" report
if(rating>monitor.m_nonc0strrating)
{
monitor.m_nonc0strrating=rating;
monitor.m_nonc0strrep.m_positive=true;
monitor.m_nonc0strrep.m_fidx=funcidx;
monitor.m_nonc0strrep.m_n=n;
monitor.m_nonc0strrep.m_cnt=sortedcnt;
monitor.m_nonc0strrep.m_stpidxa=stpidx+0;
monitor.m_nonc0strrep.m_stpidxb=stpidx+3;
monitor.m_nonc0strrep.m_x0.Resize(n);
monitor.m_nonc0strrep.m_d.Resize(n);
for(i=0; i<n; i++)
{
monitor.m_nonc0strrep.m_x0.Set(i,monitor.m_enqueuedx[monitor.m_sortedidx[0]*n+i]);
monitor.m_nonc0strrep.m_d.Set(i,monitor.m_dcur[i]);
}
monitor.m_nonc0strrep.m_stp=monitor.m_sortedstp;
monitor.m_nonc0strrep.m_f=monitor.m_f;
monitor.m_nonc0strrep.m_stp.Resize(sortedcnt);
monitor.m_nonc0strrep.m_f.Resize(sortedcnt);
}
//--- Store to "longest" report
nrm=0;
for(i=0; i<n; i++)
{
nrm+=CMath::Sqr(monitor.m_enqueuedx[monitor.m_sortedidx[0]*n+i] -
monitor.m_enqueuedx[monitor.m_sortedidx[sortedcnt-1]*n+i]);
}
nrm=MathSqrt(nrm);
nrm=MathMin(nrm,1.0);
nrm=CApServ::Coalesce(nrm,CMath::m_machineepsilon);
lengthrating=sortedcnt+MathLog(nrm)/MathLog(100);
if(lengthrating>monitor.m_nonc0lngrating)
{
monitor.m_nonc0lngrating=lengthrating;
monitor.m_nonc0lngrep.m_positive=true;
monitor.m_nonc0lngrep.m_fidx=funcidx;
monitor.m_nonc0lngrep.m_n=n;
monitor.m_nonc0lngrep.m_cnt=sortedcnt;
monitor.m_nonc0lngrep.m_stpidxa=stpidx+0;
monitor.m_nonc0lngrep.m_stpidxb=stpidx+3;
monitor.m_nonc0lngrep.m_x0.Resize(n);
monitor.m_nonc0lngrep.m_d.Resize(n);
for(i=0; i<n; i++)
{
monitor.m_nonc0lngrep.m_x0.Set(i,monitor.m_enqueuedx[monitor.m_sortedidx[0]*n+i]);
monitor.m_nonc0lngrep.m_d.Set(i,monitor.m_dcur[i]);
}
monitor.m_nonc0lngrep.m_stp=monitor.m_sortedstp;
monitor.m_nonc0lngrep.m_f=monitor.m_f;
monitor.m_nonc0lngrep.m_stp.Resize(sortedcnt);
monitor.m_nonc0lngrep.m_f.Resize(sortedcnt);
}
}
}
}
//--- C1 continuity test #0
for(stpidx=0; stpidx<sortedcnt-6; stpidx++)
{
//--- Fetch function values
f2=monitor.m_f[stpidx+2];
f3=monitor.m_f[stpidx+3];
f4=monitor.m_f[stpidx+4];
noise2=m_ognoiselevelf*MathMax(MathAbs(f2),1.0);
noise3=m_ognoiselevelf*MathMax(MathAbs(f3),1.0);
//--- Decide whether we want to test this interval or not; for target
//--- function we test intervals around minimum, for constraints we
//--- test intervals of sign change.
if(funcidx==0)
{
//--- Skip if not minimum
if(!(f3<f2+(noise2+noise3) && f3<f4))
continue;
}
else
{
//--- Skip if sign does not change
if(MathSign(f2*f4)>0)
continue;
}
C1ContinuityTest0(monitor,funcidx,stpidx+0,sortedcnt);
C1ContinuityTest0(monitor,funcidx,stpidx+1,sortedcnt);
}
//--- C1 continuity test #1
for(stpidx=0; stpidx<sortedcnt-3; stpidx++)
{
//--- Fetch function values from the interval being tested
f0=monitor.m_f[stpidx+0];
f1=monitor.m_f[stpidx+1];
f2=monitor.m_f[stpidx+2];
f3=monitor.m_f[stpidx+3];
noise0=m_ognoiselevelf*MathMax(MathAbs(f0),1.0);
noise1=m_ognoiselevelf*MathMax(MathAbs(f1),1.0);
noise2=m_ognoiselevelf*MathMax(MathAbs(f2),1.0);
noise3=m_ognoiselevelf*MathMax(MathAbs(f3),1.0);
//--- Decide whether we want to test this interval or not; for target
//--- function we test intervals around minimum, for constraints we
//--- test intervals of sign change.
if(funcidx==0)
{
//--- Skip if not minimum
if(!(f1<f0+(noise0+noise1) && f2<f3+noise2+noise3))
continue;
}
else
{
//--- Skip if sign does not change
if(MathSign(f0*f3)>0)
continue;
}
C1ContinuityTest1(monitor,funcidx,stpidx,sortedcnt);
}
}
}
//+------------------------------------------------------------------+
//| This subroutine enqueues one more trial point for a task with |
//| scalar function with unscaled variables - a convenience wrapper |
//| for more general EnqueuePoint() |
//+------------------------------------------------------------------+
void COptServ::SmoothnessMonitorEnqueuePoint1u(CSmoothnessMonitor &monitor,
CRowDouble &s,
CRowDouble &invs,
CRowDouble &d,
double stp,
CRowDouble &x,
double f0,
CRowDouble &j0)
{
//--- create variables
int n=monitor.m_n;
int k=monitor.m_k;
if(!monitor.m_checksmoothness)
return;
//--- check
if(!CAp::Assert(k==1,__FUNCTION__+": K<>1"))
return;
monitor.m_xu=x*invs+0;
monitor.m_du=d*invs+0;
monitor.m_xu.Resize(n);
monitor.m_du.Resize(n);
monitor.m_f0.Resize(1);
monitor.m_j0.Resize(1,n);
monitor.m_j0.Row(0,j0*s+0);
monitor.m_f0.Set(0,f0);
SmoothnessMonitorEnqueuePoint(monitor,monitor.m_du,stp,monitor.m_xu,monitor.m_f0,monitor.m_j0);
}
//+------------------------------------------------------------------+
//| This subroutine finalizes line search |
//+------------------------------------------------------------------+
void COptServ::SmoothnessMonitorFinalizeLineSearch(CSmoothnessMonitor &monitor)
{
//--- As for now - nothing to be done.
}
//+------------------------------------------------------------------+
//| This function starts aggressive probing for a range of step |
//| lengths [0, StpMax]. |
//| This function stores NValues values per step, with the first one |
//| (index 0) value being "primary" one (target function / merit |
//| function) and the rest being supplementary ones. |
//+------------------------------------------------------------------+
void COptServ::SmoothnessMonitorStartProbing(CSmoothnessMonitor &monitor,
double stpmax,
int nvalues,
double stepscale)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(stpmax) && stpmax>0.0,__FUNCTION__+": StpMax<=0"))
return;
if(!CAp::Assert(nvalues>=1,__FUNCTION__+": NValues<1"))
return;
if(!CAp::Assert(MathIsValidNumber(stepscale) && (double)(stepscale)>=0.0,__FUNCTION__+": StepScale<0"))
return;
monitor.m_probingnvalues=nvalues;
monitor.m_probingnstepsstored=0;
monitor.m_probingstepmax=stpmax;
monitor.m_probingstepscale=stepscale;
monitor.m_probingf.Resize(nvalues);
monitor.m_probingrcomm.ia.Resize(2+1);
monitor.m_probingrcomm.ra.Resize(3+1);
monitor.m_probingrcomm.stage=-1;
}
//+------------------------------------------------------------------+
//| This function performs aggressive probing. |
//| After each call it returns step to evaluate in Monitor.ProbingStp|
//| Load values being probed into Monitor.ProbingF and continue |
//| iteration. |
//| Monitor.ProbingF[0] is a special value which is used to guide |
//| probing process towards discontinuities and nonsmooth points. |
//+------------------------------------------------------------------+
bool COptServ::SmoothnessMonitorProbe(CSmoothnessMonitor &monitor)
{
//--- create variables
int i=0;
int j=0;
int idx=0;
double vlargest=0;
double v=0;
double v0=0;
double v1=0;
//--- Reverse communication preparations
//--- I know it looks ugly, but it works the same way
//--- anywhere from C++ to Python.
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(monitor.m_probingrcomm.stage>=0)
{
i=monitor.m_probingrcomm.ia[0];
j=monitor.m_probingrcomm.ia[1];
idx=monitor.m_probingrcomm.ia[2];
vlargest=monitor.m_probingrcomm.ra[0];
v=monitor.m_probingrcomm.ra[1];
v0=monitor.m_probingrcomm.ra[2];
v1=monitor.m_probingrcomm.ra[3];
}
else
{
i=359;
j=-58;
idx=-919;
vlargest=-909;
v=81;
v0=255;
v1=74;
}
if(monitor.m_probingrcomm.stage==0)
{
monitor.m_probingvalues.Row(monitor.m_probingnstepsstored,monitor.m_probingf);
monitor.m_probingslopes.Row(monitor.m_probingnstepsstored,vector<double>::Zeros(monitor.m_probingnvalues));
monitor.m_probingnstepsstored++;
//--- Resort
for(j=monitor.m_probingnstepsstored-1; j>=1; j--)
{
if(monitor.m_probingsteps[j-1]<=monitor.m_probingsteps[j])
break;
monitor.m_probingsteps.Swap(j-1,j);
monitor.m_probingvalues.SwapRows(j-1,j);
}
i++;
}
else
//--- Routine body
i=0;
if(i>40)
return(false);
//--- Increase storage size
monitor.m_probingsteps.Resize(monitor.m_probingnstepsstored+1);
monitor.m_probingvalues.Resize(monitor.m_probingnstepsstored+1,monitor.m_probingnvalues);
monitor.m_probingslopes.Resize(monitor.m_probingnstepsstored+1,monitor.m_probingnvalues);
//--- Determine probing step length, save step to the end of the storage
if(i<=10)
{
//--- First 11 steps are performed over equidistant grid
monitor.m_probingstp=(double)i/10.0*monitor.m_probingstepmax;
}
else
{
//--- Subsequent steps target either points with maximum change in F[0]
//--- (search for discontinuity) or maximum change in slope of F[0] (search
//--- for nonsmoothness)
//--- check
if(!CAp::Assert(monitor.m_probingnstepsstored>=3,__FUNCTION__+": critical integrity check failed"))
return(false);
if(i%2==0)
{
//--- Target interval with maximum change in F[0]
idx=-1;
vlargest=0;
for(j=0; j<=monitor.m_probingnstepsstored-2; j++)
{
v=MathAbs(monitor.m_probingvalues.Get(j+1,0)-monitor.m_probingvalues.Get(j,0));
if(idx<0 || v>vlargest)
{
idx=j;
vlargest=v;
}
}
monitor.m_probingstp=0.5*(monitor.m_probingsteps[idx]+monitor.m_probingsteps[idx+1]);
}
else
{
//--- Target interval [J,J+2] with maximum change in slope of F[0], select
//--- subinterval [J,J+1] or [J+1,J+2] with maximum length.
idx=-1;
vlargest=0;
for(j=0; j<=monitor.m_probingnstepsstored-3; j++)
{
v0=(monitor.m_probingvalues.Get(j+1,0)-monitor.m_probingvalues.Get(j+0,0))/(monitor.m_probingsteps[j+1]-monitor.m_probingsteps[j+0]+CMath::m_machineepsilon);
v1=(monitor.m_probingvalues.Get(j+2,0)-monitor.m_probingvalues.Get(j+1,0))/(monitor.m_probingsteps[j+2]-monitor.m_probingsteps[j+1]+CMath::m_machineepsilon);
v=MathAbs(v0-v1);
if(idx<0 || v>vlargest)
{
idx=j;
vlargest=v;
}
}
if((double)(monitor.m_probingsteps[idx+2]-monitor.m_probingsteps[idx+1])>(double)(monitor.m_probingsteps[idx+1]-monitor.m_probingsteps[idx+0]))
{
monitor.m_probingstp=0.5*(monitor.m_probingsteps[idx+2]+monitor.m_probingsteps[idx+1]);
}
else
{
monitor.m_probingstp=0.5*(monitor.m_probingsteps[idx+1]+monitor.m_probingsteps[idx+0]);
}
}
}
monitor.m_probingsteps.Set(monitor.m_probingnstepsstored,monitor.m_probingstp);
//--- Retrieve user values
monitor.m_probingrcomm.stage=0;
//--- Saving State
monitor.m_probingrcomm.ia.Set(0,i);
monitor.m_probingrcomm.ia.Set(1,j);
monitor.m_probingrcomm.ia.Set(2,idx);
monitor.m_probingrcomm.ra.Set(0,vlargest);
monitor.m_probingrcomm.ra.Set(1,v);
monitor.m_probingrcomm.ra.Set(2,v0);
monitor.m_probingrcomm.ra.Set(3,v1);
return(true);
}
//+------------------------------------------------------------------+
//| This function prints probing results to trace log. |
//| Tracing is performed using fixed width for all columns, so you |
//| may print a header before printing trace - and reasonably expect |
//| that its width will match that of the trace. This function |
//| promises that it wont change trace output format without |
//| introducing breaking changes into its signature. |
//| NOTE: this function ALWAYS tries to print results; it is caller's|
//| responsibility to decide whether he needs tracing or not. |
//+------------------------------------------------------------------+
void COptServ::SmoothnessMonitorTraceProbingResults(CSmoothnessMonitor &monitor)
{
//--- create variables
int i=0;
int j=0;
double steplen=0;
//--- Compute slopes
for(i=0; i<=monitor.m_probingnstepsstored-2; i++)
{
for(j=0; j<=monitor.m_probingnvalues-1; j++)
{
steplen=(monitor.m_probingsteps[i+1]-monitor.m_probingsteps[i]+100.0*CMath::m_machineepsilon)*(monitor.m_probingstepscale+CMath::m_machineepsilon);
monitor.m_probingslopes.Set(i,j,(monitor.m_probingvalues.Get(i+1,j)-monitor.m_probingvalues.Get(i,j))/steplen);
}
}
if(monitor.m_probingnstepsstored>=1)
for(j=0; j<monitor.m_probingnvalues; j++)
monitor.m_probingslopes.Set(monitor.m_probingnstepsstored-1,j,monitor.m_probingslopes.Get(MathMax(monitor.m_probingnstepsstored-2,0),j));
//--- Print to trace log
CAp::Trace("*** ----------");
for(j=0; j<monitor.m_probingnvalues; j++)
CAp::Trace("-------------------------");
CAp::Trace("\n");
for(i=0; i<monitor.m_probingnstepsstored; i++)
{
CAp::Trace(StringFormat("*** | %0.4f |",monitor.m_probingsteps[i]));
for(j=0; j<monitor.m_probingnvalues; j++)
CAp::Trace(StringFormat(" %11.3E %10.2E |",monitor.m_probingvalues.Get(i,j) - monitor.m_probingvalues.Get(0,j),monitor.m_probingslopes.Get(i,j)));
CAp::Trace("\n");
}
CAp::Trace("*** ----------");
for(j=0; j<monitor.m_probingnvalues; j++)
CAp::Trace("-------------------------");
CAp::Trace("\n");
}
//+------------------------------------------------------------------+
//| This subroutine tells monitor to output trace Info. |
//| If CallerSuggestsTrace = True, monitor ALWAYS prints trace, |
//| even if no suspicions were raised during optimization. If |
//| CallerSuggestsTrace = False, the monitor will print trace only |
//| if: |
//| * trace was requested by trace tag 'OPTGUARD' AND suspicious |
//| points were found during optimization |
//| * trace was requested by trace tag 'OPTGUARD.ALWAYS' - always |
//+------------------------------------------------------------------+
void COptServ::SmoothnessMonitorTraceStatus(CSmoothnessMonitor &monitor,
bool callersuggeststrace)
{
//--- create variables
bool needreport;
bool needxdreport;
bool suspicionsraised;
int i=0;
double slope=0;
//--- Do we need trace report?
suspicionsraised=(monitor.m_rep.m_nonc0suspected || monitor.m_rep.m_nonc1suspected || monitor.m_rep.m_badgradsuspected);
needreport=false;
needreport=needreport || callersuggeststrace;
needreport=needreport || CAp::IsTraceEnabled("OPTGUARD.ALWAYS");
needreport=needreport || (CAp::IsTraceEnabled("OPTGUARD") && suspicionsraised);
if(!needreport)
return;
needxdreport=needreport && CAp::IsTraceEnabled("OPTIMIZERS.X");
CAp::Trace("\n");
CAp::Trace("////////////////////////////////////////////////////////////////////////////////////////////////////\n");
CAp::Trace("//--- OPTGUARD INTEGRITY CHECKER REPORT //\n");
CAp::Trace("////////////////////////////////////////////////////////////////////////////////////////////////////\n");
if(!suspicionsraised)
{
CAp::Trace("> no discontinuity/nonsmoothness/bad-gradient suspicions were raised during optimization\n");
return;
}
if(monitor.m_rep.m_nonc0suspected)
CAp::Trace("> [WARNING] suspected discontinuity (aka C0-discontinuity)\n");
if(monitor.m_rep.m_nonc1suspected)
CAp::Trace("> [WARNING] suspected nonsmoothness (aka C1-discontinuity)\n");
CAp::Trace("> printing out test reports...\n");
if(monitor.m_rep.m_nonc0suspected && monitor.m_rep.m_nonc0test0positive)
{
CAp::Trace("> printing out discontinuity test #0 report:\n");
CAp::Trace("*** -------------------------------------------------------\n");
CAp::Trace("*** | Test #0 for discontinuity was triggered (this test |\n");
CAp::Trace("*** | analyzes changes in function values). See below for |\n");
CAp::Trace("*** | detailed Info: |\n");
CAp::Trace(StringFormat("*** | * function index: %10.m_d",monitor.m_nonc0lngrep.m_fidx));
if(monitor.m_nonc0lngrep.m_fidx==0)
CAp::Trace(" (target) |\n");
else
CAp::Trace(" (constraint) |\n");
CAp::Trace(StringFormat("*** | * F() Lipschitz const: %10.2E |\n",monitor.m_rep.m_nonc0lipschitzc));
CAp::Trace("*** | Printing out log of suspicious line search XK+Stp*D |\n");
CAp::Trace("*** | Look for abrupt changes in slope. |\n");
if(!needxdreport)
{
CAp::Trace("*** | NOTE: XK and D are not printed by default. If you |\n");
CAp::Trace("*** | need them,add trace tag OPTIMIZERS.X |\n");
}
CAp::Trace("*** -------------------------------------------------------\n");
CAp::Trace("*** | step along D | delta F | slope |\n");
CAp::Trace("*** ------------------------------------------------------|\n");
for(i=0; i<=monitor.m_nonc0lngrep.m_cnt-1; i++)
{
slope=monitor.m_nonc0lngrep.m_f[MathMin(i+1,monitor.m_nonc0lngrep.m_cnt-1)]-monitor.m_nonc0lngrep.m_f[i];
slope=slope/(1.0e-15+monitor.m_nonc0lngrep.m_stp[MathMin(i+1,monitor.m_nonc0lngrep.m_cnt-1)]-monitor.m_nonc0lngrep.m_stp[i]);
CAp::Trace(StringFormat("*** | %13.5E | %13.5E | %13.5E |",monitor.m_nonc0lngrep.m_stp[i],monitor.m_nonc0lngrep.m_f[i] - monitor.m_nonc0lngrep.m_f[0],slope));
if(i>=monitor.m_nonc0lngrep.m_stpidxa && i<=monitor.m_nonc0lngrep.m_stpidxb)
{
CAp::Trace(" <---");
}
CAp::Trace("\n");
}
CAp::Trace("*** ------------------------------------------------------|\n");
if(needxdreport)
{
CAp::Trace("*** > printing raw variables\n");
CAp::Trace("*** XK = ");
CApServ::TraceVectorUnscaledUnshiftedAutopRec(monitor.m_nonc0lngrep.m_x0,monitor.m_n,monitor.m_s,true,monitor.m_s,false);
CAp::Trace("\n");
CAp::Trace("*** D = ");
CApServ::TraceVectorUnscaledUnshiftedAutopRec(monitor.m_nonc0lngrep.m_d,monitor.m_n,monitor.m_s,true,monitor.m_s,false);
CAp::Trace("\n");
CAp::Trace("*** > printing scaled variables (values are divided by user-specified scales)\n");
CAp::Trace("*** XK = ");
CApServ::TraceVectorAutopRec(monitor.m_nonc0lngrep.m_x0,0,monitor.m_n);
CAp::Trace("\n");
CAp::Trace("*** D = ");
CApServ::TraceVectorAutopRec(monitor.m_nonc0lngrep.m_d,0,monitor.m_n);
CAp::Trace("\n");
}
}
if(monitor.m_rep.m_nonc1suspected && monitor.m_rep.m_nonc1test0positive)
{
CAp::Trace("> printing out nonsmoothness test #0 report:\n");
CAp::Trace("*** -------------------------------------------------------\n");
CAp::Trace("*** | Test #0 for nonsmoothness was triggered (this test |\n");
CAp::Trace("*** | analyzes changes in function values and ignores |\n");
CAp::Trace("*** | gradient Info). See below for detailed Info: |\n");
CAp::Trace(StringFormat("*** | * function index: %10.m_d",monitor.m_nonc1test0lngrep.m_fidx));
if(monitor.m_nonc1test0lngrep.m_fidx==0)
{
CAp::Trace(" (target) |\n");
}
else
{
CAp::Trace(" (constraint) |\n");
}
CAp::Trace(StringFormat("*** | * dF/dX Lipschitz const: %10.2E |\n",monitor.m_rep.m_nonc1lipschitzc));
CAp::Trace("*** | Printing out log of suspicious line search XK+Stp*D |\n");
CAp::Trace("*** | Look for abrupt changes in slope. |\n");
if(!needxdreport)
{
CAp::Trace("*** | NOTE: XK and D are not printed by default. If you |\n");
CAp::Trace("*** | need them,add trace tag OPTIMIZERS.X |\n");
}
CAp::Trace("*** -------------------------------------------------------\n");
CAp::Trace("*** | step along D | delta F | slope |\n");
CAp::Trace("*** ------------------------------------------------------|\n");
for(i=0; i<=monitor.m_nonc1test0lngrep.m_cnt-1; i++)
{
slope=monitor.m_nonc1test0lngrep.m_f[MathMin(i+1,monitor.m_nonc1test0lngrep.m_cnt-1)]-monitor.m_nonc1test0lngrep.m_f[i];
slope=slope/(1.0e-15+monitor.m_nonc1test0lngrep.m_stp[MathMin(i+1,monitor.m_nonc1test0lngrep.m_cnt-1)]-monitor.m_nonc1test0lngrep.m_stp[i]);
CAp::Trace(StringFormat("*** | %13.5E | %13.5E | %13.5E |",monitor.m_nonc1test0lngrep.m_stp[i],monitor.m_nonc1test0lngrep.m_f[i] - monitor.m_nonc1test0lngrep.m_f[0],slope));
if(i>=monitor.m_nonc1test0lngrep.m_stpidxa && i<=monitor.m_nonc1test0lngrep.m_stpidxb)
{
CAp::Trace(" <---");
}
CAp::Trace("\n");
}
CAp::Trace("*** ------------------------------------------------------|\n");
if(needxdreport)
{
CAp::Trace("*** > printing raw variables\n");
CAp::Trace("*** XK = ");
CApServ::TraceVectorUnscaledUnshiftedAutopRec(monitor.m_nonc1test0lngrep.m_x0,monitor.m_n,monitor.m_s,true,monitor.m_s,false);
CAp::Trace("\n");
CAp::Trace("*** D = ");
CApServ::TraceVectorUnscaledUnshiftedAutopRec(monitor.m_nonc1test0lngrep.m_d,monitor.m_n,monitor.m_s,true,monitor.m_s,false);
CAp::Trace("\n");
CAp::Trace("*** > printing scaled variables (values are divided by user-specified scales)\n");
CAp::Trace("*** XK = ");
CApServ::TraceVectorAutopRec(monitor.m_nonc1test0lngrep.m_x0,0,monitor.m_n);
CAp::Trace("\n");
CAp::Trace("*** D = ");
CApServ::TraceVectorAutopRec(monitor.m_nonc1test0lngrep.m_d,0,monitor.m_n);
CAp::Trace("\n");
}
}
if(monitor.m_rep.m_nonc1suspected && monitor.m_rep.m_nonc1test1positive)
{
CAp::Trace("> printing out nonsmoothness test #1 report:\n");
CAp::Trace("*** -------------------------------------------------------\n");
CAp::Trace("*** | Test #1 for nonsmoothness was triggered (this test |\n");
CAp::Trace("*** | analyzes changes in gradient components). See below |\n");
CAp::Trace("*** | for detailed Info: |\n");
CAp::Trace(StringFormat("*** | * function index: %10.m_d",monitor.m_nonc1test1lngrep.m_fidx));
if(monitor.m_nonc1test1lngrep.m_fidx==0)
{
CAp::Trace(" (target) |\n");
}
else
{
CAp::Trace(" (constraint) |\n");
}
CAp::Trace(StringFormat("*** | * variable index I: %10.m_d |\n",monitor.m_nonc1test1lngrep.m_vidx));
CAp::Trace(StringFormat("*** | * dF/dX Lipschitz const: %10.2E |\n",monitor.m_rep.m_nonc1lipschitzc));
CAp::Trace("*** | Printing out log of suspicious line search XK+Stp*D |\n");
CAp::Trace("*** | Look for abrupt changes in slope. |\n");
if(!needxdreport)
{
CAp::Trace("*** | NOTE: XK and D are not printed by default. If you |\n");
CAp::Trace("*** | need them,add trace tag OPTIMIZERS.X |\n");
}
CAp::Trace("*** -------------------------------------------------------\n");
CAp::Trace("*** | step along D | delta Gi | slope |\n");
CAp::Trace("*** ------------------------------------------------------|\n");
for(i=0; i<monitor.m_nonc1test1lngrep.m_cnt; i++)
{
slope=monitor.m_nonc1test1lngrep.m_g[MathMin(i+1,monitor.m_nonc1test1lngrep.m_cnt-1)]-monitor.m_nonc1test1lngrep.m_g[i];
slope=slope/(1.0e-15+monitor.m_nonc1test1lngrep.m_stp[MathMin(i+1,monitor.m_nonc1test1lngrep.m_cnt-1)]-monitor.m_nonc1test1lngrep.m_stp[i]);
CAp::Trace(StringFormat("*** | %13.5E | %13.5E | %13.5E |",monitor.m_nonc1test1lngrep.m_stp[i],monitor.m_nonc1test1lngrep.m_g[i] - monitor.m_nonc1test1lngrep.m_g[0],slope));
if(i>=monitor.m_nonc1test1lngrep.m_stpidxa && i<=monitor.m_nonc1test1lngrep.m_stpidxb)
{
CAp::Trace(" <---");
}
CAp::Trace("\n");
}
CAp::Trace("*** ------------------------------------------------------|\n");
if(needxdreport)
{
CAp::Trace("*** > printing raw variables\n");
CAp::Trace("*** XK = ");
CApServ::TraceVectorUnscaledUnshiftedAutopRec(monitor.m_nonc1test1lngrep.m_x0,monitor.m_n,monitor.m_s,true,monitor.m_s,false);
CAp::Trace("\n");
CAp::Trace("*** D = ");
CApServ::TraceVectorUnscaledUnshiftedAutopRec(monitor.m_nonc1test1lngrep.m_d,monitor.m_n,monitor.m_s,true,monitor.m_s,false);
CAp::Trace("\n");
CAp::Trace("*** > printing scaled variables (values are divided by user-specified scales)\n");
CAp::Trace("*** XK = ");
CApServ::TraceVectorAutopRec(monitor.m_nonc1test1lngrep.m_x0,0,monitor.m_n);
CAp::Trace("\n");
CAp::Trace("*** D = ");
CApServ::TraceVectorAutopRec(monitor.m_nonc1test1lngrep.m_d,0,monitor.m_n);
CAp::Trace("\n");
}
}
}
//+------------------------------------------------------------------+
//| This subroutine exports report to user - readable representation |
//| (all arrays are forced to have exactly same size as needed; |
//| unused arrays are set to zero length). |
//+------------------------------------------------------------------+
void COptServ::SmoothnessMonitorExportReport(CSmoothnessMonitor &monitor,
COptGuardReport &rep)
{
//--- Finalize last line search, just to be sure
if(monitor.m_enqueuedcnt>0)
SmoothnessMonitorFinalizeLineSearch(monitor);
//--- Export report
COptGuardApi::OptGuardExportReport(monitor.m_rep,monitor.m_n,monitor.m_k,monitor.m_badgradhasxj,rep);
}
//+------------------------------------------------------------------+
//| Check numerical gradient at point X0 (unscaled variables!), with |
//| optional box constraints [BndL, BndU](if HasBoxConstraints=True) |
//| and with scale vector S[]. |
//| Step S[i] * TestStep is performed along I-th variable. |
//| NeedFiJ rcomm protocol is used to request derivative information.|
//| Box constraints BndL / BndU are expected to be feasible. It is |
//| possible to have BndL = BndU. |
//+------------------------------------------------------------------+
bool COptServ::SmoothnessMonitorCheckGradientATX0(CSmoothnessMonitor &monitor,
CRowDouble &unscaledx0,
CRowDouble &s,
CRowDouble &bndl,
CRowDouble &bndu,
bool hasboxconstraints,
double teststep)
{
//--- create variables
int n=0;
int k=0;
int i=0;
int j=0;
int varidx=0;
double v=0;
double vp=0;
double vm=0;
double vc=0;
int label=-1;
//--- Reverse communication preparations
//--- I know it looks ugly, but it works the same way
//--- anywhere from C++ to Python.
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(monitor.m_rstateg0.stage>=0)
{
n=monitor.m_rstateg0.ia[0];
k=monitor.m_rstateg0.ia[1];
i=monitor.m_rstateg0.ia[2];
j=monitor.m_rstateg0.ia[3];
varidx=monitor.m_rstateg0.ia[4];
v=monitor.m_rstateg0.ra[0];
vp=monitor.m_rstateg0.ra[1];
vm=monitor.m_rstateg0.ra[2];
vc=monitor.m_rstateg0.ra[3];
}
else
{
n=-788;
k=809;
i=205;
j=-838;
varidx=939;
v=-526;
vp=763;
vm=-541;
vc=-698;
}
//--- select
switch(monitor.m_rstateg0.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
case 3:
label=3;
break;
//--- Routine body
default:
n=monitor.m_n;
k=monitor.m_k;
monitor.m_needfij=false;
//--- Quick exit
if(n<=0 || k<=0 || !MathIsValidNumber(teststep) || teststep==0.0)
return(false);
teststep=MathAbs(teststep);
//--- Allocate storage
CApServ::RVectorSetLengthAtLeast(monitor.m_x,n);
CApServ::RVectorSetLengthAtLeast(monitor.m_fi,k);
CApServ::RMatrixSetLengthAtLeast(monitor.m_j,k,n);
CApServ::RVectorSetLengthAtLeast(monitor.m_xbase,n);
CApServ::RVectorSetLengthAtLeast(monitor.m_fbase,k);
CApServ::RVectorSetLengthAtLeast(monitor.m_fm,k);
CApServ::RVectorSetLengthAtLeast(monitor.m_fc,k);
CApServ::RVectorSetLengthAtLeast(monitor.m_fp,k);
CApServ::RVectorSetLengthAtLeast(monitor.m_jm,k);
CApServ::RVectorSetLengthAtLeast(monitor.m_jc,k);
CApServ::RVectorSetLengthAtLeast(monitor.m_jp,k);
CApServ::RMatrixSetLengthAtLeast(monitor.m_jbaseusr,k,n);
CApServ::RMatrixSetLengthAtLeast(monitor.m_jbasenum,k,n);
CApServ::RVectorSetLengthAtLeast(monitor.m_rep.m_badgradxbase,n);
CApServ::RMatrixSetLengthAtLeast(monitor.m_rep.m_badgraduser,k,n);
CApServ::RMatrixSetLengthAtLeast(monitor.m_rep.m_badgradnum,k,n);
//--- Set XBase/Jacobian presence flag
monitor.m_badgradhasxj=true;
//--- Determine reference point, compute function vector and user-supplied Jacobian
for(i=0; i<n; i++)
{
v=unscaledx0[i];
if((hasboxconstraints && MathIsValidNumber(bndl[i])) && v<bndl[i])
v=bndl[i];
if((hasboxconstraints && MathIsValidNumber(bndu[i])) && v>bndu[i])
v=bndu[i];
monitor.m_xbase.Set(i,v);
monitor.m_rep.m_badgradxbase.Set(i,v);
monitor.m_x.Set(i,v);
}
monitor.m_needfij=true;
monitor.m_rstateg0.stage=0;
label=-1;
break;
}
//--- main loop
while(label>=0)
switch(label)
{
case 0:
monitor.m_needfij=false;
monitor.m_fbase=monitor.m_fi;
monitor.m_jbaseusr=monitor.m_j;
monitor.m_rep.m_badgraduser=monitor.m_j;
//--- Check Jacobian column by column
varidx=0;
case 4:
if(varidx>n-1)
{
label=6;
break;
}
//--- Determine test location.
v=monitor.m_xbase[varidx];
vm=v-s[varidx]*teststep;
vp=v+s[varidx]*teststep;
if((hasboxconstraints && MathIsValidNumber(bndl[varidx])) && vm<bndl[varidx])
vm=bndl[varidx];
if((hasboxconstraints && MathIsValidNumber(bndu[varidx])) && vp>bndu[varidx])
vp=bndu[varidx];
vc=vm+(vp-vm)/2;
//--- Quickly skip fixed variables
if(vm==vp || vc==vm || vc==vp)
{
monitor.m_rep.m_badgradnum.Col(varidx,vector<double>::Zeros(k));
label=5;
break;
}
//--- Compute F/J at three trial points
monitor.m_x=monitor.m_xbase;
monitor.m_x.Set(varidx,vm);
monitor.m_needfij=true;
monitor.m_rstateg0.stage=1;
label=-1;
break;
case 1:
monitor.m_needfij=false;
monitor.m_fm=monitor.m_fi;
monitor.m_jm=monitor.m_j.Col(varidx)+0;
monitor.m_x=monitor.m_xbase;
monitor.m_x.Set(varidx,vc);
monitor.m_needfij=true;
monitor.m_rstateg0.stage=2;
label=-1;
break;
case 2:
monitor.m_needfij=false;
monitor.m_fc=monitor.m_fi;
monitor.m_jc=monitor.m_j.Col(varidx)+0;
monitor.m_x=monitor.m_xbase;
monitor.m_x.Set(varidx,vp);
monitor.m_needfij=true;
monitor.m_rstateg0.stage=3;
label=-1;
break;
case 3:
monitor.m_needfij=false;
monitor.m_fp=monitor.m_fi;
monitor.m_jp=monitor.m_j.Col(varidx)+0;
//--- Check derivative
for(i=0; i<k; i++)
{
monitor.m_rep.m_badgradnum.Set(i,varidx,(monitor.m_fp[i]-monitor.m_fm[i])/(vp-vm));
if(!DerivativeCheck(monitor.m_fm[i],monitor.m_jm[i]*s[varidx],monitor.m_fp[i],monitor.m_jp[i]*s[varidx],monitor.m_fc[i],monitor.m_jc[i]*s[varidx],(vp-vm)/s[varidx]))
{
monitor.m_rep.m_badgradsuspected=true;
monitor.m_rep.m_badgradfidx=i;
monitor.m_rep.m_badgradvidx=varidx;
}
}
case 5:
varidx=varidx+1;
label=4;
break;
case 6:
return(false);
}
//--- Saving State
monitor.m_rstateg0.ia.Set(0,n);
monitor.m_rstateg0.ia.Set(1,k);
monitor.m_rstateg0.ia.Set(2,i);
monitor.m_rstateg0.ia.Set(3,j);
monitor.m_rstateg0.ia.Set(4,varidx);
monitor.m_rstateg0.ra.Set(0,v);
monitor.m_rstateg0.ra.Set(1,vp);
monitor.m_rstateg0.ra.Set(2,vm);
monitor.m_rstateg0.ra.Set(3,vc);
return(true);
}
//+------------------------------------------------------------------+
//| This function calculates feasibility error(square root of sum of |
//| squared errors) for a Kx(NMain + NSlack) system of linear |
//| equalities. |
//| INPUT PARAMETERS: |
//| CE - set of K equality constraints, |
//| array[K, NMain + NSlack + 1] |
//| X - candidate point, array [NMain + NSlack] |
//| NMain - number of primary variables |
//| NSlack - number of slack variables |
//| K - number of constraints |
//| Tmp0 - possible preallocated buffer, automatically resized|
//| RESULT: Sqrt(SUM(Err ^ 2)) |
//+------------------------------------------------------------------+
double COptServ::FeasibilityError(CMatrixDouble &ce,
CRowDouble &x,
int nmain,
int nslack,
int k,
CRowDouble &tmp0)
{
double result=0;
tmp0=ce.Col(nmain+nslack)*(-1.0);
CAblas::RMatrixGemVect(k,nmain+nslack,1.0,ce,0,0,0,x,0,1.0,tmp0,0);
result=tmp0.Dot(tmp0);
result=MathSqrt(result);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function calculates feasibility error(square root of sum of |
//| squared errors) for a Kx(NMain + NSlack) system of linear |
//| equalities and error gradient(with respect to x) |
//| INPUT PARAMETERS: |
//| CE - set of K equality constraints, |
//| array[K, NMain + NSlack + 1] |
//| X - candidate point, array [NMain + NSlack] |
//| NMain - number of primary variables |
//| NSlack - number of slack variables |
//| K - number of constraints |
//| Grad - preallocated array[NMain + NSlack] |
//| Tmp0 - possible preallocated buffer, automatically resized|
//| RESULT: |
//| Err - Sqrt(SUM(Err ^ 2)) |
//| Grad - error gradient with respect to X, |
//| array[NMain + NSlack] |
//+------------------------------------------------------------------+
void COptServ::FeasibilityErrorGrad(CMatrixDouble &ce,
CRowDouble &x,
int nmain,
int nslack,
int k,
double &err,
CRowDouble &grad,
CRowDouble &tmp0)
{
err=0;
//--- check
if(!CAp::Assert(grad.Size()>=nmain+nslack,__FUNCTION__+": integrity check failed"))
return;
tmp0.Resize(k);
CAblas::RMatrixGemVect(k,nmain+nslack,1.0,ce,0,0,0,x,0,0.0,tmp0,0);
tmp0-=ce.Col(nmain+nslack)+0;
err=tmp0.Dot(tmp0);
err=MathSqrt(err);
CAblas::RMatrixGemVect(nmain+nslack,k,1.0,ce,0,0,1,tmp0,0,0.0,grad,0);
}
//+------------------------------------------------------------------+
//| This subroutine checks C0 continuity and returns continuity |
//| rating (normalized value, with values above 50 - 500 being good |
//| indication of the discontinuity) and Lipschitz constant. |
//| An interval between F1 and F2 is tested for(dis) continuity. |
//| Per - point noise estimates are provided. Delta[i] is a step from|
//| F[i] to F[i + 1]. |
//| ApplySpecialCorrection parameter should be set to True if you use|
//| this function to estimate continuity of the model around minimum;|
//| it adds special correction which helps to detect "max(0,1/x)" - |
//| like discontinuities. Without this correction algorithm will |
//| still work, but will be a bit less powerful. Do not use this |
//| correction for situations when you want to estimate continuity |
//| around some non - extremal point - it may result in spurious |
//| discontinuities being reported. |
//+------------------------------------------------------------------+
void COptServ::TestC0Continuity(double f0,double f1,double f2,
double f3,double noise0,double noise1,
double noise2,double noise3,double delta0,
double delta1,double delta2,
bool applyspecialcorrection,
double &rating,double &lipschitz)
{
//--- create variables
double lipschitz01=0;
double lipschitz12=0;
double lipschitz23=0;
rating=0;
lipschitz=0;
//--- Compute Lipschitz constant for the interval [0,1],
//--- add noise correction in order to get increased estimate (makes
//--- comparison below more conservative).
lipschitz01=(MathAbs(f1-f0)+(noise0+noise1))/delta0;
//--- Compute Lipschitz constant for the interval [StpIdx+1,StpIdx+2],
//--- SUBTRACT noise correction in order to get decreased estimate (makes
//--- comparison below more conservative).
lipschitz12=MathMax(MathAbs(f2-f1)-(noise1+noise2),0.0)/delta1;
//--- Compute Lipschitz constant for the interval [StpIdx+2,StpIdx+3]
//--- using special algorithm:
//--- a) if F3<F2-Noise23, Lipschitz constant is assumed to be zero
//--- b) otherwise, we compute Lipschitz constant as usual,
//--- with noise correction term being added
//--- We need case (a) because some kinds of discontinuities
//--- (like one introduced by max(1/x,0)) manifest themselves
//--- in a quite special way.
if(applyspecialcorrection && f3<f2-(noise2+noise3))
lipschitz23=0;
else
lipschitz23=(MathAbs(f3-f2)+(noise2+noise3))/delta2;
//--- Compute rating (ratio of two Lipschitz constants)
//--- check
if(!CAp::Assert(MathMax(lipschitz01,lipschitz23)>0,__FUNCTION__+": integrity check failed"))
return;
rating=lipschitz12/MathMax(lipschitz01,lipschitz23);
lipschitz=lipschitz12;
}
//+------------------------------------------------------------------+
//| This subroutine checks C1 continuity using test #0(function |
//| values from the line search log are studied, gradient is not |
//| used). |
//| An interval between F[StpIdx + 0] and F[StpIdx + 5] is tested for|
//| continuity. An normalized error metric (Lipschitz constant growth|
//| for the derivative) for the interval in question is calculated. |
//| Values above 50 are a good indication of the discontinuity. |
//| A six - point algorithm is used for testing, so we expect that |
//| Monitor.F and Monitor.Stp have enough points for this test. |
//+------------------------------------------------------------------+
void COptServ::C1ContinuityTest0(CSmoothnessMonitor &monitor,
int funcidx,
int stpidx,
int sortedcnt)
{
//--- create variables
double f0=0;
double f1=0;
double f2=0;
double f3=0;
double f4=0;
double f5=0;
double noise0=0;
double noise1=0;
double noise2=0;
double noise3=0;
double noise4=0;
double noise5=0;
double delta0=0;
double delta1=0;
double delta2=0;
double delta3=0;
double delta4=0;
double d0=0;
double d1=0;
double d2=0;
double d3=0;
double newnoise0=0;
double newnoise1=0;
double newnoise2=0;
double newnoise3=0;
double newdelta0=0;
double newdelta1=0;
double newdelta2=0;
double rating=0;
double lipschitz=0;
double lengthrating=0;
int i=0;
int n=monitor.m_n;
double nrm=0;
//--- check
if(!CAp::Assert(stpidx+5<sortedcnt,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(monitor.m_sortedstp[0]==0.0,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(monitor.m_sortedstp[sortedcnt-1]>0.0,__FUNCTION__+": integrity check failed"))
return;
//--- Fetch F, noise, Delta's
f0=monitor.m_f[stpidx+0];
f1=monitor.m_f[stpidx+1];
f2=monitor.m_f[stpidx+2];
f3=monitor.m_f[stpidx+3];
f4=monitor.m_f[stpidx+4];
f5=monitor.m_f[stpidx+5];
noise0=m_ognoiselevelf*MathMax(MathAbs(f0),1.0);
noise1=m_ognoiselevelf*MathMax(MathAbs(f1),1.0);
noise2=m_ognoiselevelf*MathMax(MathAbs(f2),1.0);
noise3=m_ognoiselevelf*MathMax(MathAbs(f3),1.0);
noise4=m_ognoiselevelf*MathMax(MathAbs(f4),1.0);
noise5=m_ognoiselevelf*MathMax(MathAbs(f5),1.0);
delta0=monitor.m_sortedstp[stpidx+1]-monitor.m_sortedstp[stpidx+0];
delta1=monitor.m_sortedstp[stpidx+2]-monitor.m_sortedstp[stpidx+1];
delta2=monitor.m_sortedstp[stpidx+3]-monitor.m_sortedstp[stpidx+2];
delta3=monitor.m_sortedstp[stpidx+4]-monitor.m_sortedstp[stpidx+3];
delta4=monitor.m_sortedstp[stpidx+5]-monitor.m_sortedstp[stpidx+4];
//--- Differentiate functions, get derivative values and noise
//--- estimates at points (0+1)/2, (1+2)/2, (3+4)/2, (3+4)/2,
//--- (4+5)/2. Compute new step values NewDelta[i] and new
//--- noise estimates.
d0=(f1-f0)/delta0;
d1=(f2-f1)/delta1;
d2=(f4-f3)/delta3;
d3=(f5-f4)/delta4;
newnoise0=(noise0+noise1)/delta0;
newnoise1=(noise1+noise2)/delta1;
newnoise2=(noise3+noise4)/delta3;
newnoise3=(noise4+noise5)/delta4;
newdelta0=0.5*(delta0+delta1);
newdelta1=0.5*delta1+delta2+0.5*delta3;
newdelta2=0.5*(delta3+delta4);
//--- Test with C0 continuity tester. "Special correction" is
//--- turned off for this test.
TestC0Continuity(d0,d1,d2,d3,newnoise0,newnoise1,newnoise2,newnoise3,newdelta0,newdelta1,newdelta2,false,rating,lipschitz);
//--- Store results
if(rating>m_ogminrating1)
{
//--- Store to total report
monitor.m_rep.m_nonc1test0positive=true;
if(rating>monitor.m_nonc1currentrating)
{
monitor.m_nonc1currentrating=rating;
monitor.m_rep.m_nonc1suspected=true;
monitor.m_rep.m_nonc1lipschitzc=lipschitz;
monitor.m_rep.m_nonc1fidx=funcidx;
}
//--- Store to "strongest" report
if(rating>monitor.m_nonc1test0strrating)
{
monitor.m_nonc1test0strrating=rating;
monitor.m_nonc1test0strrep.m_positive=true;
monitor.m_nonc1test0strrep.m_fidx=funcidx;
monitor.m_nonc1test0strrep.m_n=n;
monitor.m_nonc1test0strrep.m_cnt=sortedcnt;
monitor.m_nonc1test0strrep.m_stpidxa=stpidx+1;
monitor.m_nonc1test0strrep.m_stpidxb=stpidx+4;
monitor.m_nonc1test0strrep.m_d=monitor.m_dcur;
monitor.m_nonc1test0strrep.m_x0.Resize(n);
monitor.m_nonc1test0strrep.m_d.Resize(n);
for(i=0; i<n; i++)
monitor.m_nonc1test0strrep.m_x0.Set(i,monitor.m_enqueuedx[monitor.m_sortedidx[0]*n+i]);
monitor.m_nonc1test0strrep.m_stp=monitor.m_sortedstp;
monitor.m_nonc1test0strrep.m_f=monitor.m_f;
monitor.m_nonc1test0strrep.m_stp.Resize(sortedcnt);
monitor.m_nonc1test0strrep.m_f.Resize(sortedcnt);
}
//--- Store to "longest" report
nrm=0;
for(i=0; i<n; i++)
nrm+=CMath::Sqr(monitor.m_enqueuedx[monitor.m_sortedidx[0]*n+i]-monitor.m_enqueuedx[monitor.m_sortedidx[sortedcnt-1]*n+i]);
nrm=MathSqrt(nrm);
nrm=MathMin(nrm,1.0);
nrm=CApServ::Coalesce(nrm,CMath::m_machineepsilon);
lengthrating=sortedcnt+MathLog(nrm)/MathLog(100);
if(lengthrating>monitor.m_nonc1test0lngrating)
{
monitor.m_nonc1test0lngrating=lengthrating;
monitor.m_nonc1test0lngrep.m_positive=true;
monitor.m_nonc1test0lngrep.m_fidx=funcidx;
monitor.m_nonc1test0lngrep.m_n=n;
monitor.m_nonc1test0lngrep.m_cnt=sortedcnt;
monitor.m_nonc1test0lngrep.m_stpidxa=stpidx+1;
monitor.m_nonc1test0lngrep.m_stpidxb=stpidx+4;
monitor.m_nonc1test0lngrep.m_d=monitor.m_dcur;
monitor.m_nonc1test0lngrep.m_stp=monitor.m_sortedstp;
monitor.m_nonc1test0lngrep.m_f=monitor.m_f;
monitor.m_nonc1test0lngrep.m_x0.Resize(n);
monitor.m_nonc1test0lngrep.m_d.Resize(n);
monitor.m_nonc1test0lngrep.m_stp.Resize(sortedcnt);
monitor.m_nonc1test0lngrep.m_f.Resize(sortedcnt);
for(i=0; i<n; i++)
monitor.m_nonc1test0lngrep.m_x0.Set(i,monitor.m_enqueuedx[monitor.m_sortedidx[0]*n+i]);
}
}
}
//+------------------------------------------------------------------+
//| This subroutine checks C1 continuity using test #1(individual |
//| gradient components from the line search log are studied for |
//| continuity). |
//| An interval between F[StpIdx + 0] and F[StpIdx + 3]is tested for|
//| continuity. An normalized error metric(Lipschitz constant growth |
//| for the derivative) for the interval in question is calculated. |
//| Values above 50 are a good indication of the discontinuity. |
//+------------------------------------------------------------------+
void COptServ::C1ContinuityTest1(CSmoothnessMonitor &monitor,
int funcidx,
int stpidx,
int sortedcnt)
{
//--- create variables
int n=monitor.m_n;
int i=0;
int varidx=0;
double f0=0;
double f1=0;
double f2=0;
double f3=0;
double noise0=0;
double noise1=0;
double noise2=0;
double noise3=0;
double nrm=0;
double rating=0;
double lengthrating=0;
double lipschitz=0;
//--- check
if(!CAp::Assert(stpidx+3<sortedcnt,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(monitor.m_sortedstp[0]==0.0,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(monitor.m_sortedstp[sortedcnt-1]>0.0,__FUNCTION__+": integrity check failed"))
return;
//--- Study each component of the gradient in the interval in question
for(varidx=0; varidx<n; varidx++)
{
f0=monitor.m_g[(stpidx+0)*n+varidx];
f1=monitor.m_g[(stpidx+1)*n+varidx];
f2=monitor.m_g[(stpidx+2)*n+varidx];
f3=monitor.m_g[(stpidx+3)*n+varidx];
noise0=m_ognoiselevelg*MathMax(MathAbs(f0),1.0);
noise1=m_ognoiselevelg*MathMax(MathAbs(f1),1.0);
noise2=m_ognoiselevelg*MathMax(MathAbs(f2),1.0);
noise3=m_ognoiselevelg*MathMax(MathAbs(f3),1.0);
TestC0Continuity(f0,f1,f2,f3,noise0,noise1,noise2,noise3,monitor.m_sortedstp[stpidx+1]-monitor.m_sortedstp[stpidx+0],monitor.m_sortedstp[stpidx+2]-monitor.m_sortedstp[stpidx+1],monitor.m_sortedstp[stpidx+3]-monitor.m_sortedstp[stpidx+2],false,rating,lipschitz);
//--- Store results
if(rating>m_ogminrating1)
{
//--- Store to total report
monitor.m_rep.m_nonc1test1positive=true;
if(rating>monitor.m_nonc1currentrating)
{
monitor.m_nonc1currentrating=rating;
monitor.m_rep.m_nonc1suspected=true;
monitor.m_rep.m_nonc1lipschitzc=lipschitz;
monitor.m_rep.m_nonc1fidx=funcidx;
}
//--- Store to "strongest" report
if(rating>monitor.m_nonc1test1strrating)
{
monitor.m_nonc1test1strrating=rating;
monitor.m_nonc1test1strrep.m_positive=true;
monitor.m_nonc1test1strrep.m_fidx=funcidx;
monitor.m_nonc1test1strrep.m_vidx=varidx;
monitor.m_nonc1test1strrep.m_n=n;
monitor.m_nonc1test1strrep.m_cnt=sortedcnt;
monitor.m_nonc1test1strrep.m_stpidxa=stpidx+0;
monitor.m_nonc1test1strrep.m_stpidxb=stpidx+3;
monitor.m_nonc1test1strrep.m_d=monitor.m_dcur;
monitor.m_nonc1test1strrep.m_stp=monitor.m_sortedstp;
monitor.m_nonc1test1strrep.m_x0.Resize(n);
monitor.m_nonc1test1strrep.m_d.Resize(n);
monitor.m_nonc1test1strrep.m_stp.Resize(sortedcnt);
monitor.m_nonc1test1strrep.m_g.Resize(sortedcnt);
for(i=0; i<n; i++)
monitor.m_nonc1test1strrep.m_x0.Set(i,monitor.m_enqueuedx[monitor.m_sortedidx[0]*n+i]);
for(i=0; i<sortedcnt ; i++)
monitor.m_nonc1test1strrep.m_g.Set(i,monitor.m_g[i*n+varidx]);
}
//--- Store to "longest" report
nrm=0;
for(i=0; i<n; i++)
nrm+=CMath::Sqr(monitor.m_enqueuedx[monitor.m_sortedidx[0]*n+i]-monitor.m_enqueuedx[monitor.m_sortedidx[sortedcnt-1]*n+i]);
nrm=MathSqrt(nrm);
nrm=MathMin(nrm,1.0);
nrm=CApServ::Coalesce(nrm,CMath::m_machineepsilon);
lengthrating=sortedcnt+MathLog(nrm)/MathLog(100);
if(lengthrating>monitor.m_nonc1test1lngrating)
{
monitor.m_nonc1test1lngrating=lengthrating;
monitor.m_nonc1test1lngrep.m_positive=true;
monitor.m_nonc1test1lngrep.m_fidx=funcidx;
monitor.m_nonc1test1lngrep.m_vidx=varidx;
monitor.m_nonc1test1lngrep.m_n=n;
monitor.m_nonc1test1lngrep.m_cnt=sortedcnt;
monitor.m_nonc1test1lngrep.m_stpidxa=stpidx+0;
monitor.m_nonc1test1lngrep.m_stpidxb=stpidx+3;
monitor.m_nonc1test1lngrep.m_d=monitor.m_dcur;
monitor.m_nonc1test1lngrep.m_stp=monitor.m_sortedstp;
monitor.m_nonc1test1lngrep.m_x0.Resize(n);
monitor.m_nonc1test1lngrep.m_d.Resize(n);
monitor.m_nonc1test1lngrep.m_stp.Resize(sortedcnt);
monitor.m_nonc1test1lngrep.m_g.Resize(sortedcnt);
for(i=0; i<n; i++)
monitor.m_nonc1test1lngrep.m_x0.Set(i,monitor.m_enqueuedx[monitor.m_sortedidx[0]*n+i]);
for(i=0; i<sortedcnt; i++)
monitor.m_nonc1test1lngrep.m_g.Set(i,monitor.m_g[i*n+varidx]);
}
}
}
}
//+------------------------------------------------------------------+
//| This object stores State of the nonlinear CG optimizer. |
//| You should use ALGLIB functions to work with this object. |
//+------------------------------------------------------------------+
class CMinCGState
{
public:
//--- variables
int m_cgtype;
int m_debugrestartscount;
int m_k;
int m_maxits;
int m_mcinfo;
int m_mcstage;
int m_n;
int m_nfev;
int m_prectype;
int m_repiterationscount;
int m_repnfev;
int m_repterminationtype;
int m_rstimer;
int m_smoothnessguardlevel;
int m_vcnt;
double m_betady;
double m_betahs;
double m_curstpmax;
double m_diffstep;
double m_epsf;
double m_epsg;
double m_epsx;
double m_f;
double m_fbase;
double m_fm1;
double m_fm2;
double m_fold;
double m_fp1;
double m_fp2;
double m_lastgoodstep;
double m_lastscaledstep;
double m_stp;
double m_stpmax;
double m_suggestedstep;
double m_teststep;
double m_trimthreshold;
bool m_algpowerup;
bool m_drep;
bool m_innerresetneeded;
bool m_lsend;
bool m_lsstart;
bool m_needf;
bool m_needfg;
bool m_terminationneeded;
bool m_userterminationneeded;
bool m_xrep;
bool m_xupdated;
//--- objects
RCommState m_rstate;
CSmoothnessMonitor m_smonitor;
CLinMinState m_lstate;
//--- arrays
CRowDouble m_d;
CRowDouble m_diagh;
CRowDouble m_diaghl2;
CRowDouble m_dk;
CRowDouble m_dn;
CRowDouble m_g;
CRowDouble m_invs;
CRowDouble m_lastscaleused;
CRowDouble m_s;
CRowDouble m_work0;
CRowDouble m_work1;
CRowDouble m_x;
CRowDouble m_xbase;
CRowDouble m_xk;
CRowDouble m_xn;
CRowDouble m_yk;
//--- matrix
CMatrixDouble m_vcorr;
//--- constructor, destructor
CMinCGState(void);
~CMinCGState(void) {}
//--- copy
void Copy(const CMinCGState &obj);
//--- overloading
void operator=(const CMinCGState &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinCGState::CMinCGState(void)
{
m_cgtype=0;
m_debugrestartscount=0;
m_k=0;
m_maxits=0;
m_mcinfo=0;
m_mcstage=0;
m_n=0;
m_nfev=0;
m_prectype=0;
m_repiterationscount=0;
m_repnfev=0;
m_repterminationtype=0;
m_rstimer=0;
m_smoothnessguardlevel=0;
m_vcnt=0;
m_betady=0;
m_betahs=0;
m_curstpmax=0;
m_diffstep=0;
m_epsf=0;
m_epsg=0;
m_epsx=0;
m_f=0;
m_fbase=0;
m_fm1=0;
m_fm2=0;
m_fold=0;
m_fp1=0;
m_fp2=0;
m_lastgoodstep=0;
m_lastscaledstep=0;
m_stp=0;
m_stpmax=0;
m_suggestedstep=0;
m_teststep=0;
m_trimthreshold=0;
m_algpowerup=false;
m_drep=false;
m_innerresetneeded=false;
m_lsend=false;
m_lsstart=false;
m_needf=false;
m_needfg=false;
m_terminationneeded=false;
m_userterminationneeded=false;
m_xrep=false;
m_xupdated=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinCGState::Copy(const CMinCGState &obj)
{
m_cgtype=obj.m_cgtype;
m_debugrestartscount=obj.m_debugrestartscount;
m_k=obj.m_k;
m_maxits=obj.m_maxits;
m_mcinfo=obj.m_mcinfo;
m_mcstage=obj.m_mcstage;
m_n=obj.m_n;
m_nfev=obj.m_nfev;
m_prectype=obj.m_prectype;
m_repiterationscount=obj.m_repiterationscount;
m_repnfev=obj.m_repnfev;
m_repterminationtype=obj.m_repterminationtype;
m_rstimer=obj.m_rstimer;
m_smoothnessguardlevel=obj.m_smoothnessguardlevel;
m_vcnt=obj.m_vcnt;
m_betady=obj.m_betady;
m_betahs=obj.m_betahs;
m_curstpmax=obj.m_curstpmax;
m_diffstep=obj.m_diffstep;
m_epsf=obj.m_epsf;
m_epsg=obj.m_epsg;
m_epsx=obj.m_epsx;
m_f=obj.m_f;
m_fbase=obj.m_fbase;
m_fm1=obj.m_fm1;
m_fm2=obj.m_fm2;
m_fold=obj.m_fold;
m_fp1=obj.m_fp1;
m_fp2=obj.m_fp2;
m_lastgoodstep=obj.m_lastgoodstep;
m_lastscaledstep=obj.m_lastscaledstep;
m_stp=obj.m_stp;
m_stpmax=obj.m_stpmax;
m_suggestedstep=obj.m_suggestedstep;
m_teststep=obj.m_teststep;
m_trimthreshold=obj.m_trimthreshold;
m_algpowerup=obj.m_algpowerup;
m_drep=obj.m_drep;
m_innerresetneeded=obj.m_innerresetneeded;
m_lsend=obj.m_lsend;
m_lsstart=obj.m_lsstart;
m_needf=obj.m_needf;
m_needfg=obj.m_needfg;
m_terminationneeded=obj.m_terminationneeded;
m_userterminationneeded=obj.m_userterminationneeded;
m_xrep=obj.m_xrep;
m_xupdated=obj.m_xupdated;
m_rstate=obj.m_rstate;
m_smonitor=obj.m_smonitor;
m_d=obj.m_d;
m_diagh=obj.m_diagh;
m_diaghl2=obj.m_diaghl2;
m_dk=obj.m_dk;
m_dn=obj.m_dn;
m_g=obj.m_g;
m_invs=obj.m_invs;
m_lastscaleused=obj.m_lastscaleused;
m_s=obj.m_s;
m_work0=obj.m_work0;
m_work1=obj.m_work1;
m_x=obj.m_x;
m_xbase=obj.m_xbase;
m_xk=obj.m_xk;
m_xn=obj.m_xn;
m_yk=obj.m_yk;
m_vcorr=obj.m_vcorr;
m_lstate=obj.m_lstate;
}
//+------------------------------------------------------------------+
//| This object stores State of the nonlinear CG optimizer. |
//| You should use ALGLIB functions to work with this object. |
//+------------------------------------------------------------------+
class CMinCGStateShell
{
private:
CMinCGState m_innerobj;
public:
//--- constructors, destructor
CMinCGStateShell(void) {}
CMinCGStateShell(CMinCGState &obj) { m_innerobj.Copy(obj); }
~CMinCGStateShell(void) {}
//--- methods
bool GetNeedF(void);
void SetNeedF(const bool b);
bool GetNeedFG(void);
void SetNeedFG(const bool b);
bool GetXUpdated(void);
void SetXUpdated(const bool b);
double GetF(void);
void SetF(const double d);
CMinCGState *GetInnerObj(void);
};
//+------------------------------------------------------------------+
//| Returns the value of the variable needf |
//+------------------------------------------------------------------+
bool CMinCGStateShell::GetNeedF(void)
{
return(m_innerobj.m_needf);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable needf |
//+------------------------------------------------------------------+
void CMinCGStateShell::SetNeedF(const bool b)
{
m_innerobj.m_needf=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable needfg |
//+------------------------------------------------------------------+
bool CMinCGStateShell::GetNeedFG(void)
{
return(m_innerobj.m_needfg);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable needfg |
//+------------------------------------------------------------------+
void CMinCGStateShell::SetNeedFG(const bool b)
{
m_innerobj.m_needfg=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable xupdated |
//+------------------------------------------------------------------+
bool CMinCGStateShell::GetXUpdated(void)
{
return(m_innerobj.m_xupdated);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable xupdated |
//+------------------------------------------------------------------+
void CMinCGStateShell::SetXUpdated(const bool b)
{
m_innerobj.m_xupdated=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable f |
//+------------------------------------------------------------------+
double CMinCGStateShell::GetF(void)
{
return(m_innerobj.m_f);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable f |
//+------------------------------------------------------------------+
void CMinCGStateShell::SetF(const double d)
{
m_innerobj.m_f=d;
}
//+------------------------------------------------------------------+
//| Return object of class |
//+------------------------------------------------------------------+
CMinCGState *CMinCGStateShell::GetInnerObj(void)
{
return(GetPointer(m_innerobj));
}
//+------------------------------------------------------------------+
//| Auxiliary class for CMinCG |
//+------------------------------------------------------------------+
class CMinCGReport
{
public:
int m_iterationscount;
int m_nfev;
int m_terminationtype;
//--- constructor, destructor
CMinCGReport(void) { ZeroMemory(this); }
~CMinCGReport(void) {}
//--- copy
void Copy(const CMinCGReport &obj);
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinCGReport::Copy(const CMinCGReport &obj)
{
//--- copy variables
m_iterationscount=obj.m_iterationscount;
m_nfev=obj.m_nfev;
m_terminationtype=obj.m_terminationtype;
}
//+------------------------------------------------------------------+
//| This class is a shell for class CMinCGReport |
//+------------------------------------------------------------------+
class CMinCGReportShell
{
private:
CMinCGReport m_innerobj;
public:
//--- constructors, destructor
CMinCGReportShell(void) {}
CMinCGReportShell(CMinCGReport &obj) { m_innerobj.Copy(obj); }
~CMinCGReportShell(void) {}
//--- methods
int GetIterationsCount(void);
void SetIterationsCount(const int i);
int GetNFev(void);
void SetNFev(const int i);
int GetTerminationType(void);
void SetTerminationType(const int i);
CMinCGReport *GetInnerObj(void);
};
//+------------------------------------------------------------------+
//| Returns the value of the variable iterationscount |
//+------------------------------------------------------------------+
int CMinCGReportShell::GetIterationsCount(void)
{
return(m_innerobj.m_iterationscount);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable iterationscount |
//+------------------------------------------------------------------+
void CMinCGReportShell::SetIterationsCount(const int i)
{
m_innerobj.m_iterationscount=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable nfev |
//+------------------------------------------------------------------+
int CMinCGReportShell::GetNFev(void)
{
return(m_innerobj.m_nfev);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable nfev |
//+------------------------------------------------------------------+
void CMinCGReportShell::SetNFev(const int i)
{
m_innerobj.m_nfev=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable terminationtype |
//+------------------------------------------------------------------+
int CMinCGReportShell::GetTerminationType(void)
{
return(m_innerobj.m_terminationtype);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable terminationtype |
//+------------------------------------------------------------------+
void CMinCGReportShell::SetTerminationType(const int i)
{
m_innerobj.m_terminationtype=i;
}
//+------------------------------------------------------------------+
//| Return object of class |
//+------------------------------------------------------------------+
CMinCGReport *CMinCGReportShell::GetInnerObj(void)
{
return(GetPointer(m_innerobj));
}
//+------------------------------------------------------------------+
//| Conjugate gradient optimizer |
//+------------------------------------------------------------------+
class CMinCG
{
public:
//--- class constants
static const int m_rscountdownlen;
static const double m_gtol;
//--- public methods
static void MinCGCreate(const int n,double &x[],CMinCGState &State);
static void MinCGCreate(const int n,CRowDouble &x,CMinCGState &State);
static void MinCGCreateF(const int n,double &x[],const double diffstep,CMinCGState &State);
static void MinCGCreateF(const int n,CRowDouble &x,const double diffstep,CMinCGState &State);
static void MinCGSetCond(CMinCGState &State,const double epsg,const double epsf,double epsx,const int m_maxits);
static void MinCGSetScale(CMinCGState &State,double &s[]);
static void MinCGSetScale(CMinCGState &State,CRowDouble &s);
static void MinCGSetXRep(CMinCGState &State,const bool needxrep);
static void MinCGSetDRep(CMinCGState &State,const bool needdrep);
static void MinCGSetCGType(CMinCGState &State,int cgtype);
static void MinCGSetStpMax(CMinCGState &State,const double stpmax);
static void MinCGSuggestStep(CMinCGState &State,const double stp);
static double MinCGLastGoodStep(CMinCGState &State);
static void MinCGSetPrecDefault(CMinCGState &State);
static void MinCGSetPrecDiag(CMinCGState &State,double &d[]);
static void MinCGSetPrecDiag(CMinCGState &State,CRowDouble &d);
static void MinCGSetPrecScale(CMinCGState &State);
static void MinCGOptGuardGradient(CMinCGState &State,double teststep);
static void MinCGOptGuardSmoothness(CMinCGState &State,int level);
static void MinCGOptGuardResults(CMinCGState &State,COptGuardReport &rep);
static void MinCGOptGuardNonC1Test0Results(CMinCGState &State,COptGuardNonC1Test0Report &strrep,COptGuardNonC1Test0Report &lngrep);
static void MinCGOptGuardNonC1Test1Results(CMinCGState &State,COptGuardNonC1Test1Report &strrep,COptGuardNonC1Test1Report &lngrep);
static void MinCGResults(CMinCGState &State,double &x[],CMinCGReport &rep);
static void MinCGResults(CMinCGState &State,CRowDouble &x,CMinCGReport &rep);
static void MinCGResultsBuf(CMinCGState &State,double &x[],CMinCGReport &rep);
static void MinCGResultsBuf(CMinCGState &State,CRowDouble &x,CMinCGReport &rep);
static void MinCGRestartFrom(CMinCGState &State,double &x[]);
static void MinCGRestartFrom(CMinCGState &State,CRowDouble &x);
static void MinCGRequestTermination(CMinCGState &State);
static void MinCGSetPrecDiagFast(CMinCGState &State,double &d[]);
static void MinCGSetPrecDiagFast(CMinCGState &State,CRowDouble &d);
static void MinCGSetPrecLowRankFast(CMinCGState &State,double &d1[],double &c[],CMatrixDouble &v,const int vcnt);
static void MinCGSetPrecLowRankFast(CMinCGState &State,CRowDouble &d1,CRowDouble &c,CMatrixDouble &v,const int vcnt);
static void MinCGSetPrecVarPart(CMinCGState &State,double &d2[]);
static void MinCGSetPrecVarPart(CMinCGState &State,CRowDouble &d2);
static bool MinCGIteration(CMinCGState &State);
private:
static void ClearRequestFields(CMinCGState &State);
static void PreconditionedMultiply(CMinCGState &State,CRowDouble &x,CRowDouble &work0,CRowDouble &work1);
static double PreconditionedMultiply2(CMinCGState &State,CRowDouble &x,CRowDouble &y,CRowDouble &work0,CRowDouble &work1);
static void MinCGInitInternal(const int n,const double diffstep,CMinCGState &State);
};
//+------------------------------------------------------------------+
//| Initialize constants |
//+------------------------------------------------------------------+
const int CMinCG::m_rscountdownlen=10;
const double CMinCG::m_gtol=0.3;
//+------------------------------------------------------------------+
//| NONLINEAR CONJUGATE GRADIENT METHOD |
//| DESCRIPTION: |
//| The subroutine minimizes function F(x) of N arguments by using |
//| one of the nonlinear conjugate gradient methods. |
//| These CG methods are globally convergent (even on non-convex |
//| functions) as long as grad(f) is Lipschitz continuous in a some |
//| neighborhood of the L = { x : f(x)<=f(x0) }. |
//| REQUIREMENTS: |
//| Algorithm will request following information during its |
//| operation: |
//| * function value F and its gradient G (simultaneously) at given |
//| point X |
//| USAGE: |
//| 1. User initializes algorithm State with MinCGCreate() call |
//| 2. User tunes m_solver parameters with MinCGSetCond(), |
//| MinCGSetStpMax() and other functions |
//| 3. User calls MinCGOptimize() function which takes algorithm |
//| State and pointer (delegate, etc.) to callback function which |
//| calculates F/G. |
//| 4. User calls MinCGResults() to get solution |
//| 5. Optionally, user may call MinCGRestartFrom() to solve another |
//| problem with same N but another starting point and/or another |
//| function. MinCGRestartFrom() allows to reuse already |
//| initialized structure. |
//| INPUT PARAMETERS: |
//| N - problem dimension, N>0: |
//| * if given, only leading N elements of X are used|
//| * if not given, automatically determined from |
//| size of X |
//| X - starting point, array[0..m_n-1]. |
//| OUTPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//+------------------------------------------------------------------+
void CMinCG::MinCGCreate(const int n,double &x[],CMinCGState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N too small!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- function call
MinCGInitInternal(n,0.0,State);
//--- function call
MinCGRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinCG::MinCGCreate(const int n,CRowDouble &x,CMinCGState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N too small!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- function call
MinCGInitInternal(n,0.0,State);
//--- function call
MinCGRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| The subroutine is finite difference variant of MinCGCreate(). |
//| It uses finite differences in order to differentiate target |
//| function. |
//| Description below contains information which is specific to this |
//| function only. We recommend to read comments on MinCGCreate() in |
//| order to get more information about creation of CG optimizer. |
//| INPUT PARAMETERS: |
//| N - problem dimension, N>0: |
//| * if given, only leading N elements of X are |
//| used |
//| * if not given, automatically determined from |
//| size of X |
//| X - starting point, array[0..m_n-1]. |
//| DiffStep- differentiation step, >0 |
//| OUTPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NOTES: |
//| 1. algorithm uses 4-point central formula for differentiation. |
//| 2. differentiation step along I-th axis is equal to |
//| DiffStep*S[I] where S[] is scaling vector which can be set by |
//| MinCGSetScale() call. |
//| 3. we recommend you to use moderate values of differentiation |
//| step. Too large step will result in too large truncation |
//| errors, while too small step will result in too large |
//| numerical errors. 1.0E-6 can be good value to start with. |
//| 4. Numerical differentiation is very inefficient - one gradient |
//| calculation needs 4*N function evaluations. This function will|
//| work for any N - either small (1...10), moderate (10...100) or|
//| large (100...). However, performance penalty will be too |
//| severe for any N's except for small ones. |
//| We should also say that code which relies on numerical |
//| differentiation is less robust and precise. L-BFGS needs |
//| exact gradient values. Imprecise gradient may slow down |
//| convergence, especially on highly nonlinear problems. |
//| Thus we recommend to use this function for fast prototyping |
//| on small- dimensional problems only, and to implement |
//| analytical gradient as soon as possible. |
//+------------------------------------------------------------------+
void CMinCG::MinCGCreateF(const int n,double &x[],const double diffstep,
CMinCGState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N too small!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(diffstep),__FUNCTION__+": DiffStep is infinite or NaN!"))
return;
//--- check
if(!CAp::Assert(diffstep>0.0,__FUNCTION__+": DiffStep is non-positive!"))
return;
//--- function call
MinCGInitInternal(n,diffstep,State);
//--- function call
MinCGRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinCG::MinCGCreateF(const int n,CRowDouble &x,const double diffstep,
CMinCGState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N too small!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(diffstep),__FUNCTION__+": DiffStep is infinite or NaN!"))
return;
//--- check
if(!CAp::Assert(diffstep>0.0,__FUNCTION__+": DiffStep is non-positive!"))
return;
//--- function call
MinCGInitInternal(n,diffstep,State);
//--- function call
MinCGRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| This function sets stopping conditions for CG optimization |
//| algorithm. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| EpsG - >=0 |
//| The subroutine finishes its work if the condition|
//| |v|<EpsG is satisfied, where: |
//| * |.| means Euclidian norm |
//| * v - scaled gradient vector, v[i]=g[i]*s[i] |
//| * g - gradient |
//| * s - scaling coefficients set by MinCGSetScale()|
//| EpsF - >=0 |
//| The subroutine finishes its work if on k+1-th |
//| iteration the condition |F(k+1)-F(k)| <= |
//| <= EpsF*max{|F(k)|,|F(k+1)|,1} is satisfied. |
//| EpsX - >=0 |
//| The subroutine finishes its work if on k+1-th |
//| iteration the condition |v|<=EpsX is fulfilled, |
//| where: |
//| * |.| means Euclidian norm |
//| * v - scaled step vector, v[i]=dx[i]/s[i] |
//| * dx - ste pvector, dx=X(k+1)-X(k) |
//| * s - scaling coefficients set by MinCGSetScale()|
//| MaxIts - maximum number of iterations. If MaxIts=0, the |
//| number of iterations is unlimited. |
//| Passing EpsG=0, EpsF=0, EpsX=0 and MaxIts=0 (simultaneously) will|
//| lead to automatic stopping criterion selection (small EpsX). |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetCond(CMinCGState &State,const double epsg,
const double epsf,double epsx,const int m_maxits)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(epsg),__FUNCTION__+": EpsG is not finite number!"))
return;
//--- check
if(!CAp::Assert(epsg>=0.0,__FUNCTION__+": negative EpsG!"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(epsf),__FUNCTION__+": EpsF is not finite number!"))
return;
//--- check
if(!CAp::Assert(epsf>=0.0,__FUNCTION__+": negative EpsF!"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(epsx),__FUNCTION__+": EpsX is not finite number!"))
return;
//--- check
if(!CAp::Assert(epsx>=0.0,__FUNCTION__+": negative EpsX!"))
return;
//--- check
if(!CAp::Assert(m_maxits>=0,__FUNCTION__+": negative MaxIts!"))
return;
//--- check
if(epsg==0.0 && epsf==0.0 && epsx==0.0 && m_maxits==0)
epsx=1.0E-6;
//--- change values
State.m_epsg=epsg;
State.m_epsf=epsf;
State.m_epsx=epsx;
State.m_maxits=m_maxits;
}
//+------------------------------------------------------------------+
//| This function sets scaling coefficients for CG optimizer. |
//| ALGLIB optimizers use scaling matrices to test stopping |
//| conditions (step size and gradient are scaled before comparison |
//| with tolerances). Scale of the I-th variable is a translation |
//| invariant measure of: |
//| a) "how large" the variable is |
//| b) how large the step should be to make significant changes in |
//| the function |
//| Scaling is also used by finite difference variant of CG |
//| optimizer - step along I-th axis is equal to DiffStep*S[I]. |
//| In most optimizers (and in the CG too) scaling is NOT a form of |
//| preconditioning. It just affects stopping conditions. You should |
//| set preconditioner by separate call to one of the |
//| MinCGSetPrec...() functions. |
//| There is special preconditioning mode, however, which uses |
//| scaling coefficients to form diagonal preconditioning matrix. |
//| You can turn this mode on, if you want. But you should understand|
//| that scaling is not the same thing as preconditioning - these are|
//| two different, although related forms of tuning m_solver. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| S - array[N], non-zero scaling coefficients |
//| S[i] may be negative, sign doesn't matter. |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetScale(CMinCGState &State,double &s[])
{
//--- check
if(!CAp::Assert(CAp::Len(s)>=State.m_n,__FUNCTION__+": Length(S)<N"))
return;
for(int i=0; i<State.m_n; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(s[i]),__FUNCTION__+": S contains infinite or NAN elements"))
return;
//--- check
if(!CAp::Assert(s[i]!=0.0,__FUNCTION__+": S contains zero elements"))
return;
State.m_s.Set(i,MathAbs(s[i]));
}
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetScale(CMinCGState &State,CRowDouble &s)
{
//--- check
if(!CAp::Assert(CAp::Len(s)>=State.m_n,__FUNCTION__+": Length(S)<N"))
return;
for(int i=0; i<State.m_n; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(s[i]),__FUNCTION__+": S contains infinite or NAN elements"))
return;
//--- check
if(!CAp::Assert(s[i]!=0.0,__FUNCTION__+": S contains zero elements"))
return;
}
State.m_s=s.Abs()+0;
}
//+------------------------------------------------------------------+
//| This function turns on/off reporting. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NeedXRep- whether iteration reports are needed or not |
//| If NeedXRep is True, algorithm will call rep() callback function |
//| if it is provided to MinCGOptimize(). |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetXRep(CMinCGState &State,const bool needxrep)
{
State.m_xrep=needxrep;
}
//+------------------------------------------------------------------+
//| This function turns on/off line search reports. |
//| These reports are described in more details in developer-only |
//| comments on CMinCGState &object. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NeedDRep- whether line search reports are needed or not |
//| This function is intended for private use only. Turning it on |
//| artificially may cause program failure. |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetDRep(CMinCGState &State,const bool needdrep)
{
State.m_drep=needdrep;
}
//+------------------------------------------------------------------+
//| This function sets CG algorithm. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| CGType - algorithm type: |
//| * -1 automatic selection of the best |
//| algorithm |
//| * 0 DY (Dai and Yuan) algorithm |
//| * 1 Hybrid DY-HS algorithm |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetCGType(CMinCGState &State,int cgtype)
{
//--- check
if(!CAp::Assert(cgtype>=-1 && cgtype<=1,__FUNCTION__+": incorrect CGType!"))
return;
//--- check
if(cgtype==-1)
cgtype=1;
//--- change value
State.m_cgtype=cgtype;
}
//+------------------------------------------------------------------+
//| This function sets maximum step length |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| StpMax - maximum step length, >=0. Set StpMax to 0.0, if |
//| you don't want to limit step length. |
//| Use this subroutine when you optimize target function which |
//| contains exp() or other fast growing functions, and optimization |
//| algorithm makes too large steps which leads to overflow. This |
//| function allows us to reject steps that are too large (and |
//| therefore expose us to the possible overflow) without actually |
//| calculating function value at the x+stp*d. |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetStpMax(CMinCGState &State,const double stpmax)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(stpmax),__FUNCTION__+": StpMax is not finite!"))
return;
//--- check
if(!CAp::Assert(stpmax>=0.0,__FUNCTION__+": StpMax<0!"))
return;
//--- change value
State.m_stpmax=stpmax;
}
//+------------------------------------------------------------------+
//| This function allows to suggest initial step length to the CG |
//| algorithm. |
//| Suggested step length is used as starting point for the line |
//| search. It can be useful when you have badly scaled problem, i.e.|
//| when ||grad|| (which is used as initial estimate for the first |
//| step) is many orders of magnitude different from the desired |
//| step. |
//| Line search may fail on such problems without good estimate of |
//| initial step length. Imagine, for example, problem with |
//| ||grad||=10^50 and desired step equal to 0.1 Line search |
//| function will use 10^50 as initial step, then it will decrease |
//| step length by 2 (up to 20 attempts) and will get 10^44, which is|
//| still too large. |
//| This function allows us to tell than line search should be |
//| started from some moderate step length, like 1.0, so algorithm |
//| will be able to detect desired step length in a several searches.|
//| Default behavior (when no step is suggested) is to use |
//| preconditioner, if it is available, to generate initial estimate |
//| of step length. |
//| This function influences only first iteration of algorithm. It |
//| should be called between MinCGCreate/MinCGRestartFrom() call and |
//| MinCGOptimize call. Suggested step is ignored if you have |
//| preconditioner. |
//| INPUT PARAMETERS: |
//| State - structure used to store algorithm State. |
//| Stp - initial estimate of the step length. |
//| Can be zero (no estimate). |
//+------------------------------------------------------------------+
void CMinCG::MinCGSuggestStep(CMinCGState &State,const double stp)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(stp),__FUNCTION__+": Stp is infinite or NAN"))
return;
//--- check
if(!CAp::Assert(stp>=0.0,__FUNCTION__+": Stp<0"))
return;
//--- change value
State.m_suggestedstep=stp;
}
//+------------------------------------------------------------------+
//| This developer-only function allows to retrieve unscaled (!) |
//| length of last good step (i.e. step which resulted in sufficient |
//| decrease of target function). |
//| It can be used in for solution of sequential optimization |
//| subproblems, where MinCGSuggestStep() is called with length of |
//| previous step as parameter. |
//| INPUT PARAMETERS: |
//| State - structure used to store algorithm State. |
//| RESULT: |
//| length of last good step being accepted |
//| NOTE: result of this function is undefined if you called it |
//| before |
//+------------------------------------------------------------------+
double CMinCG::MinCGLastGoodStep(CMinCGState &State)
{
return(State.m_lastgoodstep);
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: preconditioning is turned |
//| off. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NOTE: you can change preconditioner "on the fly", during |
//| algorithm iterations. |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetPrecDefault(CMinCGState &State)
{
//--- change values
State.m_prectype=0;
State.m_innerresetneeded=true;
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: diagonal of approximate |
//| Hessian is used. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| D - diagonal of the approximate Hessian, |
//| array[0..m_n-1], (if larger, only leading N |
//| elements are used). |
//| NOTE: you can change preconditioner "on the fly", during |
//| algorithm iterations. |
//| NOTE 2: D[i] should be positive. Exception will be thrown |
//| otherwise. |
//| NOTE 3: you should pass diagonal of approximate Hessian - NOT |
//| ITS INVERSE. |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetPrecDiag(CMinCGState &State,double &d[])
{
//--- check
if(!CAp::Assert(CAp::Len(d)>=State.m_n,__FUNCTION__+": D is too short"))
return;
for(int i=0; i<State.m_n; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(d[i]),__FUNCTION__+": D contains infinite or NAN elements"))
return;
//--- check
if(!CAp::Assert(d[i]>0.0,__FUNCTION__+": D contains non-positive elements"))
return;
}
//--- function call
MinCGSetPrecDiagFast(State,d);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetPrecDiag(CMinCGState &State,CRowDouble &d)
{
//--- check
if(!CAp::Assert(CAp::Len(d)>=State.m_n,__FUNCTION__+": D is too short"))
return;
for(int i=0; i<State.m_n; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(d[i]),__FUNCTION__+": D contains infinite or NAN elements"))
return;
//--- check
if(!CAp::Assert(d[i]>0.0,__FUNCTION__+": D contains non-positive elements"))
return;
}
//--- function call
MinCGSetPrecDiagFast(State,d);
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: scale-based diagonal |
//| preconditioning. |
//| This preconditioning mode can be useful when you don't have |
//| approximate diagonal of Hessian, but you know that your variables|
//| are badly scaled (for example, one variable is in [1,10], and |
//| another in [1000,100000]), and most part of the ill-conditioning |
//| comes from different scales of vars. |
//| In this case simple scale-based preconditioner, |
//| with H.Set(i, 1/(s[i]^2), can greatly improve convergence. |
//| IMPRTANT: you should set scale of your variables with |
//| MinCGSetScale() call (before or after MinCGSetPrecScale() call). |
//| Without knowledge of the scale of your variables scale-based |
//| preconditioner will be just unit matrix. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NOTE: you can change preconditioner "on the fly", during |
//| algorithm iterations. |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetPrecScale(CMinCGState &State)
{
//--- change values
State.m_prectype=3;
State.m_innerresetneeded=true;
}
//+------------------------------------------------------------------+
//| This function activates/deactivates verification of the |
//| user-supplied analytic gradient. |
//| Upon activation of this option OptGuard integrity checker |
//| performs numerical differentiation of your target function at |
//| the initial point (note: future versions may also perform check|
//| at the final point) and compares numerical gradient with analytic|
//| one provided by you. |
//| If difference is too large, an error flag is set and optimization|
//| session continues. After optimization session is over, you can |
//| retrieve the report which stores both gradients and specific |
//| components highlighted as suspicious by the OptGuard. |
//| The primary OptGuard report can be retrieved with |
//| MinCGOptGuardResults(). |
//| IMPORTANT: gradient check is a high-overhead option which will |
//| cost you about 3*N additional function evaluations. In|
//| many cases it may cost as much as the rest of the |
//| optimization session. |
//| YOU SHOULD NOT USE IT IN THE PRODUCTION CODE UNLESS YOU WANT TO |
//| CHECK DERIVATIVES PROVIDED BY SOME THIRD PARTY. |
//| NOTE: unlike previous incarnation of the gradient checking code, |
//| OptGuard does NOT interrupt optimization even if it |
//| discovers bad gradient. |
//| INPUT PARAMETERS: |
//| State - structure used to store algorithm State |
//| TestStep - verification step used for numerical |
//| differentiation: |
//| * TestStep=0 turns verification off |
//| * TestStep>0 activates verification |
//| You should carefully choose TestStep. Value which |
//| is too large (so large that function behavior is|
//| non-cubic at this scale) will lead to false alarms.|
//| Too short step will result in rounding errors |
//| dominating numerical derivative. |
//| You may use different step for different parameters|
//| by means of setting scale with MinCGSetScale(). |
//| === EXPLANATION ================================================ |
//| In order to verify gradient algorithm performs following steps: |
//| * two trial steps are made to X[i]-TestStep*S[i] and |
//| X[i]+TestStep*S[i], where X[i] is i-th component of the |
//| initial point and S[i] is a scale of i-th parameter |
//| * F(X) is evaluated at these trial points |
//| * we perform one more evaluation in the middle point of the |
//| interval |
//| * we build cubic model using function values and derivatives|
//| at trial points and we compare its prediction with actual |
//| value in the middle point |
//+------------------------------------------------------------------+
void CMinCG::MinCGOptGuardGradient(CMinCGState &State,
double teststep)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(teststep),__FUNCTION__+": TestStep contains NaN or INF"))
return;
if(!CAp::Assert(teststep>=0.0,__FUNCTION__+": invalid argument TestStep(TestStep<0)"))
return;
State.m_teststep=teststep;
}
//+------------------------------------------------------------------+
//| This function activates/deactivates nonsmoothness monitoring |
//| option of the OptGuard integrity checker. Smoothness monitor |
//| silently observes solution process and tries to detect ill-posed |
//| problems, i.e. ones with: |
//| a) discontinuous target function(non - C0) |
//| b) nonsmooth target function(non - C1) |
//| Smoothness monitoring does NOT interrupt optimization even if it|
//| suspects that your problem is nonsmooth. It just sets |
//| corresponding flags in the OptGuard report which can be retrieved|
//| after optimization is over. |
//| Smoothness monitoring is a moderate overhead option which often |
//| adds less than 1 % to the optimizer running time. Thus, you can |
//| use it even for large scale problems. |
//| NOTE: OptGuard does NOT guarantee that it will always detect |
//| C0 / C1 continuity violations. |
//| First, minor errors are hard to catch-say, a 0.0001 difference in|
//| the model values at two sides of the gap may be due to |
//| discontinuity of the model - or simply because the model has |
//| changed. |
//| Second, C1 - violations are especially difficult to detect |
//| in a noninvasive way. The optimizer usually performs very short |
//| steps near the nonsmoothness, and differentiation usually |
//| introduces a lot of numerical noise. It is hard to tell whether |
//| some tiny discontinuity in the slope is due to real nonsmoothness|
//| or just due to numerical noise alone. |
//| Our top priority was to avoid false positives, so in some rare |
//| cases minor errors may went unnoticed(however, in most cases they|
//| can be spotted with restart from different initial point). |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| Level - monitoring level: |
//| * 0 - monitoring is disabled |
//| * 1 - noninvasive low - overhead monitoring; |
//| function values and / or gradients are |
//| recorded, but OptGuard does not try to |
//| perform additional evaluations in order |
//| to get more information about suspicious |
//| locations. |
//| === EXPLANATION ================================================ |
//| One major source of headache during optimization is the |
//| possibility of the coding errors in the target function / |
//| constraints (or their gradients). Such errors most often manifest|
//| themselves as discontinuity or nonsmoothness of the target / |
//| constraints. |
//| Another frequent situation is when you try to optimize something |
//| involving lots of min() and max() operations, i.e. nonsmooth |
//| target. Although not a coding error, it is nonsmoothness anyway -|
//| and smooth optimizers usually stop right after encountering |
//| nonsmoothness, well before reaching solution. |
//| OptGuard integrity checker helps you to catch such situations: |
//| it monitors function values / gradients being passed to the |
//| optimizer and tries to errors. Upon discovering suspicious |
//| pair of points it raises appropriate flag (and allows you to |
//| continue optimization). When optimization is done, you can study |
//| OptGuard result. |
//+------------------------------------------------------------------+
void CMinCG::MinCGOptGuardSmoothness(CMinCGState &State,int level)
{
//--- check
if(!CAp::Assert(level==0 || level==1,__FUNCTION__+": unexpected value of level parameter"))
return;
State.m_smoothnessguardlevel=level;
}
//+------------------------------------------------------------------+
//| Results of OptGuard integrity check, should be called after |
//| optimization session is over. |
//| === PRIMARY REPORT ============================================= |
//| OptGuard performs several checks which are intended to catch |
//| common errors in the implementation of nonlinear function / |
//| gradient: |
//| * incorrect analytic gradient |
//| * discontinuous(non - C0) target functions (constraints) |
//| * nonsmooth(non - C1) target functions (constraints) |
//| Each of these checks is activated with appropriate function: |
//| * MinCGOptGuardGradient() for gradient verification |
//| * MinCGOptGuardSmoothness() for C0 / C1 checks |
//| Following flags are set when these errors are suspected: |
//| * rep.badgradsuspected, and additionally: |
//| * rep.badgradvidx for specific variable (gradient element) |
//| suspected |
//| * rep.badgradxbase, a point where gradient is tested |
//| * rep.badgraduser, user - provided gradient (stored as 2D |
//| matrix with single row in order to make |
//| report structure compatible with more|
//| complex optimizers like MinNLC or MinLM) |
//| * rep.badgradnum, reference gradient obtained via numerical|
//| differentiation(stored as 2D matrix with|
//| single row in order to make report |
//| structure compatible with more complex |
//| optimizers like MinNLC or MinLM) |
//| * rep.nonc0suspected |
//| * rep.nonc1suspected |
//| === ADDITIONAL REPORTS / LOGS ================================== |
//| Several different tests are performed to catch C0 / C1 errors, |
//| you can find out specific test signaled error by looking to: |
//| * rep.nonc0test0positive, for non - C0 test #0 |
//| * rep.nonc1test0positive, for non - C1 test #0 |
//| * rep.nonc1test1positive, for non - C1 test #1 |
//| Additional information (including line search logs) can be |
//| obtained by means of: |
//| * MinCGOptGuardNonC1Test0Results() |
//| * MinCGOptGuardNonC1Test1Results() |
//| which return detailed error reports, specific points where |
//| discontinuities were found, and so on. |
//| ================================================================ |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| Rep - generic OptGuard report; more detailed reports |
//| can be retrieved with other functions. |
//| NOTE: false negatives(nonsmooth problems are not identified as |
//| nonsmooth ones) are possible although unlikely. |
//| The reason is that you need to make several evaluations |
//| around nonsmoothness in order to accumulate enough information |
//| about function curvature. Say, if you start right from the |
//| nonsmooth point, optimizer simply won't get enough data to |
//| understand what is going wrong before it terminates due to abrupt|
//| changes in the derivative. It is also possible that "unlucky"|
//| step will move us to the termination too quickly. |
//| Our current approach is to have less than 0.1 % false negatives|
//| in our test examples (measured with multiple restarts from random|
//| points), and to have exactly 0 % false positives. |
//+------------------------------------------------------------------+
void CMinCG::MinCGOptGuardResults(CMinCGState &State,COptGuardReport &rep)
{
COptServ::SmoothnessMonitorExportReport(State.m_smonitor,rep);
}
//+------------------------------------------------------------------+
//| Detailed results of the OptGuard integrity check for |
//| nonsmoothness test #0 |
//| Nonsmoothness (non-C1) test #0 studies function values (not |
//| gradient!) obtained during line searches and monitors behavior |
//| of the directional derivative estimate. |
//| This test is less powerful than test #1, but it does not depend|
//| on the gradient values and thus it is more robust against |
//| artifacts introduced by numerical differentiation. |
//| Two reports are returned: |
//| *a "strongest" one, corresponding to line search which had |
//| highest value of the nonsmoothness indicator |
//| *a "longest" one, corresponding to line search which had more |
//| function evaluations, and thus is more detailed |
//| In both cases following fields are returned: |
//| * positive - is TRUE when test flagged suspicious point; |
//| FALSE if test did not notice anything (in the |
//| latter cases fields below are empty). |
//| * x0[], d[] - arrays of length N which store initial point and |
//| direction for line search(d[] can be normalized, |
//| but does not have to) |
//| * stp[], f[]- arrays of length CNT which store step lengths |
//| and function values at these points; f[i] is |
//| evaluated in x0 + stp[i]*d. |
//| * stpidxa, stpidxb - we suspect that function violates C1 |
//| continuity between steps #stpidxa and #stpidxb |
//| (usually we have stpidxb = stpidxa + 3, with |
//| most likely position of the violation between |
//| stpidxa + 1 and stpidxa + 2. |
//| ================================================================ |
//| = SHORTLY SPEAKING: build a 2D plot of(stp, f) and look at it - |
//| = you will see where C1 continuity is violated.|
//| ================================================================ |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| StrRep - C1 test #0 "strong" report |
//| LngRep - C1 test #0 "long" report |
//+------------------------------------------------------------------+
void CMinCG::MinCGOptGuardNonC1Test0Results(CMinCGState &State,
COptGuardNonC1Test0Report &strrep,
COptGuardNonC1Test0Report &lngrep)
{
COptGuardApi::SmoothnessMonitorExportC1Test0Report(State.m_smonitor.m_nonc1test0strrep,State.m_lastscaleused,strrep);
COptGuardApi::SmoothnessMonitorExportC1Test0Report(State.m_smonitor.m_nonc1test0lngrep,State.m_lastscaleused,lngrep);
}
//+------------------------------------------------------------------+
//| Detailed results of the OptGuard integrity check for |
//| nonsmoothness test #1 |
//| Nonsmoothness (non-C1) test #0 studies function values (not |
//| gradient!) obtained during line searches and monitors behavior |
//| of the directional derivative estimate. |
//| This test is less powerful than test #1, but it does not depend|
//| on the gradient values and thus it is more robust against |
//| artifacts introduced by numerical differentiation. |
//| Two reports are returned: |
//| *a "strongest" one, corresponding to line search which had |
//| highest value of the nonsmoothness indicator |
//| *a "longest" one, corresponding to line search which had more |
//| function evaluations, and thus is more detailed |
//| In both cases following fields are returned: |
//| * positive - is TRUE when test flagged suspicious point; |
//| FALSE if test did not notice anything (in the |
//| latter cases fields below are empty). |
//| * x0[], d[] - arrays of length N which store initial point and |
//| direction for line search(d[] can be normalized, |
//| but does not have to) |
//| * stp[], f[]- arrays of length CNT which store step lengths |
//| and function values at these points; f[i] is |
//| evaluated in x0 + stp[i]*d. |
//| * stpidxa, stpidxb - we suspect that function violates C1 |
//| continuity between steps #stpidxa and #stpidxb |
//| (usually we have stpidxb = stpidxa + 3, with |
//| most likely position of the violation between |
//| stpidxa + 1 and stpidxa + 2. |
//| ================================================================ |
//| = SHORTLY SPEAKING: build a 2D plot of(stp, f) and look at it - |
//| = you will see where C1 continuity is violated.|
//| ================================================================ |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| StrRep - C1 test #1 "strong" report |
//| LngRep - C1 test #1 "long" report |
//+------------------------------------------------------------------+
void CMinCG::MinCGOptGuardNonC1Test1Results(CMinCGState &State,
COptGuardNonC1Test1Report &strrep,
COptGuardNonC1Test1Report &lngrep)
{
COptGuardApi::SmoothnessMonitorExportC1Test1Report(State.m_smonitor.m_nonc1test1strrep,State.m_lastscaleused,strrep);
COptGuardApi::SmoothnessMonitorExportC1Test1Report(State.m_smonitor.m_nonc1test1lngrep,State.m_lastscaleused,lngrep);
}
//+------------------------------------------------------------------+
//| Conjugate gradient results |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| X - array[0..m_n-1], solution |
//| Rep - optimization report: |
//| * Rep.TerminationType completetion code: |
//| * 1 relative function improvement is no |
//| more than EpsF. |
//| * 2 relative step is no more than EpsX. |
//| * 4 gradient norm is no more than EpsG |
//| * 5 MaxIts steps was taken |
//| * 7 stopping conditions are too |
//| stringent, further improvement is |
//| impossible, we return best X found |
//| so far |
//| * 8 terminated by user |
//| * Rep.IterationsCount contains iterations count |
//| * NFEV countains number of function calculations |
//+------------------------------------------------------------------+
void CMinCG::MinCGResults(CMinCGState &State,double &x[],CMinCGReport &rep)
{
//--- reset memory
ArrayResize(x,0);
//--- function call
MinCGResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinCG::MinCGResults(CMinCGState &State,CRowDouble &x,CMinCGReport &rep)
{
//--- reset memory
x.Resize(0);
//--- function call
MinCGResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| Conjugate gradient results |
//| Buffered implementation of MinCGResults(), which uses |
//| pre-allocated buffer to store X[]. If buffer size is too small, |
//| it resizes buffer.It is intended to be used in the inner cycles |
//| of performance critical algorithms where array reallocation |
//| penalty is too large to be ignored. |
//+------------------------------------------------------------------+
void CMinCG::MinCGResultsBuf(CMinCGState &State,double &x[],CMinCGReport &rep)
{
//--- create a variable
int i_=0;
//--- copy
State.m_xn.ToArray(x);
//--- change values
rep.m_iterationscount=State.m_repiterationscount;
rep.m_nfev=State.m_repnfev;
rep.m_terminationtype=State.m_repterminationtype;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinCG::MinCGResultsBuf(CMinCGState &State,CRowDouble &x,CMinCGReport &rep)
{
//--- copy
x=State.m_xn;
//--- change values
rep.m_iterationscount=State.m_repiterationscount;
rep.m_nfev=State.m_repnfev;
rep.m_terminationtype=State.m_repterminationtype;
}
//+------------------------------------------------------------------+
//| This subroutine restarts CG algorithm from new point. All |
//| optimization parameters are left unchanged. |
//| This function allows to solve multiple optimization problems |
//| (which must have same number of dimensions) without object |
//| reallocation penalty. |
//| INPUT PARAMETERS: |
//| State - structure used to store algorithm State. |
//| X - new starting point. |
//+------------------------------------------------------------------+
void CMinCG::MinCGRestartFrom(CMinCGState &State,double &x[])
{
//--- check
if(!CAp::Assert(CAp::Len(x)>=State.m_n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,State.m_n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- copy
State.m_xbase=x;
State.m_xbase.Resize(State.m_n);
//--- function call
MinCGSuggestStep(State,0.0);
//--- allocation
State.m_rstate.ia.Resize(2);
State.m_rstate.ra.Resize(3);
//--- change value
State.m_rstate.stage=-1;
//--- function call
ClearRequestFields(State);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinCG::MinCGRestartFrom(CMinCGState &State,CRowDouble &x)
{
//--- check
if(!CAp::Assert(CAp::Len(x)>=State.m_n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,State.m_n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- copy
State.m_xbase=x;
State.m_xbase.Resize(State.m_n);
//--- function call
MinCGSuggestStep(State,0.0);
//--- allocation
State.m_rstate.ia.Resize(2);
State.m_rstate.ra.Resize(3);
//--- change value
State.m_rstate.stage=-1;
//--- function call
ClearRequestFields(State);
}
//+------------------------------------------------------------------+
//| This subroutine submits request for termination of running |
//| optimizer. It should be called from user-supplied callback when |
//| user decides that it is time to "smoothly" terminate |
//| optimization process. As result, optimizer stops at point which |
//| was "current accepted" when termination request was submitted and|
//| returns error code 8 (successful termination). |
//| INPUT PARAMETERS: |
//| State - optimizer structure |
//| NOTE: after request for termination optimizer may perform |
//| several additional calls to user-supplied callbacks. It |
//| does NOT guarantee to stop immediately - it just guarantees|
//| that these additional calls will be discarded later. |
//| NOTE: calling this function on optimizer which is NOT running |
//| will have no effect. |
//| NOTE: multiple calls to this function are possible. First call is|
//| counted, subsequent calls are silently ignored. |
//+------------------------------------------------------------------+
void CMinCG::MinCGRequestTermination(CMinCGState &State)
{
State.m_userterminationneeded=true;
}
//+------------------------------------------------------------------+
//| Faster version of MinCGSetPrecDiag(), for time-critical parts of |
//| code, without safety checks. |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetPrecDiagFast(CMinCGState &State,double &d[])
{
//--- change values
State.m_prectype=2;
State.m_vcnt=0;
State.m_innerresetneeded=true;
//--- copy
State.m_diagh=d;
State.m_diaghl2=vector<double>::Zeros(State.m_n);
State.m_diagh.Resize(State.m_n);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetPrecDiagFast(CMinCGState &State,CRowDouble &d)
{
//--- change values
State.m_prectype=2;
State.m_vcnt=0;
State.m_innerresetneeded=true;
//--- copy
State.m_diagh=d;
State.m_diaghl2=vector<double>::Zeros(State.m_n);
State.m_diagh.Resize(State.m_n);
}
//+------------------------------------------------------------------+
//| This function sets low-rank preconditioner for Hessian matrix |
//| H=D+V'*C*V, where: |
//| * H is a Hessian matrix, which is approximated by D/V/C |
//| * D=D1+D2 is a diagonal matrix, which includes two positive |
//| definite terms: |
//| * constant term D1 (is not updated or infrequently updated) |
//| * variable term D2 (can be cheaply updated from iteration to |
//| iteration) |
//| * V is a low-rank correction |
//| * C is a diagonal factor of low-rank correction |
//| Preconditioner P is calculated using approximate Woodburry |
//| formula: |
//| P = D^(-1) - D^(-1)*V'*(C^(-1)+V*D1^(-1)*V')^(-1)*V*D^(-1) |
//| = D^(-1) - D^(-1)*VC'*VC*D^(-1), |
//| where |
//| VC = sqrt(B)*V |
//| B = (C^(-1)+V*D1^(-1)*V')^(-1) |
//| Note that B is calculated using constant term (D1) only, which |
//| allows us to update D2 without recalculation of B or VC. Such |
//| preconditioner is exact when D2 is zero. When D2 is non-zero, it |
//| is only approximation, but very good and cheap one. |
//| This function accepts D1, V, C. |
//| D2 is set to zero by default. |
//| Cost of this update is O(N*VCnt*VCnt), but D2 can be updated in |
//| just O(N) by MinCGSetPrecVarPart. |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetPrecLowRankFast(CMinCGState &State,double &d1[],
double &c[],CMatrixDouble &v,
const int vcnt)
{
CRowDouble D=d1;
CRowDouble C=c;
MinCGSetPrecLowRankFast(State,D,C,v,vcnt);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetPrecLowRankFast(CMinCGState &State,CRowDouble &d1,
CRowDouble &c,CMatrixDouble &v,
const int vcnt)
{
//--- create variables
int i=0;
int i_=0;
int j=0;
int k=0;
int n=0;
double t=0;
//--- create matrix
CMatrixDouble b;
//--- check
if(vcnt==0)
{
//--- function call
MinCGSetPrecDiagFast(State,d1);
return;
}
//--- initialization
n=State.m_n;
b=matrix<double>::Zeros(vcnt,vcnt);
//--- function call
CApServ::RMatrixSetLengthAtLeast(State.m_vcorr,vcnt,n);
State.m_prectype=2;
State.m_vcnt=vcnt;
State.m_innerresetneeded=true;
//--- copy
State.m_diagh=d1;
State.m_diaghl2=vector<double>::Zeros(n);
State.m_diagh.Resize(n);
//--- calculation
for(i=0; i<vcnt; i++)
{
vector<double> temp=v[i]/d1.ToVector();
for(j=i; j<vcnt; j++)
{
t=temp.Dot(v[j]+0);
b.Set(i,j,t);
}
b.Set(i,i,b.Get(i,i)+1.0/c[i]);
}
//--- check
if(!CTrFac::SPDMatrixCholeskyRec(b,0,vcnt,true,State.m_work0))
{
State.m_vcnt=0;
return;
}
//--- calculation
State.m_vcorr=v;
for(i=0; i<vcnt; i++)
{
//--- change values
for(j=0; j<i; j++)
{
t=b.Get(j,i);
State.m_vcorr.Row(i,State.m_vcorr[i]-State.m_vcorr[j]*t);
}
t=1.0/b.Get(i,i);
//--- change values
State.m_vcorr.Row(i,State.m_vcorr[i]*t);
}
}
//+------------------------------------------------------------------+
//| This function updates variable part (diagonal matrix D2) |
//| of low-rank preconditioner. |
//| This update is very cheap and takes just O(N) time. |
//| It has no effect with default preconditioner. |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetPrecVarPart(CMinCGState &State,double &d2[])
{
//--- copy
State.m_diaghl2=d2;
State.m_diaghl2.Resize(State.m_n);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinCG::MinCGSetPrecVarPart(CMinCGState &State,CRowDouble &d2)
{
//--- copy
State.m_diaghl2=d2;
State.m_diaghl2.Resize(State.m_n);
}
//+------------------------------------------------------------------+
//| Clears request fileds (to be sure that we don't forgot to clear |
//| something) |
//+------------------------------------------------------------------+
void CMinCG::ClearRequestFields(CMinCGState &State)
{
//--- change values
State.m_needf=false;
State.m_needfg=false;
State.m_xupdated=false;
State.m_lsstart=false;
State.m_lsend=false;
State.m_algpowerup=false;
}
//+------------------------------------------------------------------+
//| This function calculates preconditioned product H^(-1)*x and |
//| stores result back into X. Work0[] and Work1[] are used as |
//| temporaries (size must be at least N; this function doesn't |
//| allocate arrays). |
//+------------------------------------------------------------------+
void CMinCG::PreconditionedMultiply(CMinCGState &State,CRowDouble &x,
CRowDouble &work0,CRowDouble &work1)
{
//--- create variables
int i=0;
int n=0;
int vcnt=0;
double v=0;
int i_=0;
//--- initialization
n=State.m_n;
vcnt=State.m_vcnt;
//--- check
if(State.m_prectype==0)
return;
//--- check
if(State.m_prectype==3)
{
x*=State.m_s.Pow(2)+0;
//--- exit the function
return;
}
//--- check
if(!CAp::Assert(State.m_prectype==2,__FUNCTION__+": internal error (unexpected PrecType)"))
return;
//--- handle part common for VCnt=0 and VCnt<>0
x/=State.m_diagh.ToVector()+State.m_diaghl2.ToVector();
//--- if VCnt>0
if(vcnt>0)
{
//--- calculation work0
for(i=0; i<vcnt; i++)
work0.Set(i,CAblasF::RDotVR(n,x,State.m_vcorr,i));
//--- calculation work1
State.m_work1=vector<double>::Zeros(n);
for(i=0; i<vcnt; i++)
{
v=work0[i];
for(i_=0; i_<=n-1; i_++)
State.m_work1.Add(i_,v*State.m_vcorr.Get(i,i_));
}
//--- change x
x-=State.m_work1.ToVector()/(State.m_diagh+State.m_diaghl2);
}
}
//+------------------------------------------------------------------+
//| This function calculates preconditioned product x'*H^(-1)*y. |
//| Work0[] and Work1[] are used as temporaries (size must be at |
//| least N; this function doesn't allocate arrays). |
//+------------------------------------------------------------------+
double CMinCG::PreconditionedMultiply2(CMinCGState &State,CRowDouble &x,
CRowDouble &y,CRowDouble &work0,
CRowDouble &work1)
{
//--- create variables
double result=0;
int i=0;
int n=0;
int vcnt=0;
double v0=0;
double v1=0;
int i_=0;
//--- initialization
n=State.m_n;
vcnt=State.m_vcnt;
//--- no preconditioning
if(State.m_prectype==0)
{
result=x.Dot(y);
//--- return result
return(result);
}
//--- check
if(State.m_prectype==3)
{
result=x.Dot(y.ToVector()*State.m_s.Pow(2.0));
//--- return result
return(result);
}
//--- check
if(!CAp::Assert(State.m_prectype==2,__FUNCTION__+": internal error (unexpected PrecType)"))
return(EMPTY_VALUE);
//--- low rank preconditioning
vector<double> temp=State.m_diagh+State.m_diaghl2;
result=x.Dot(y.ToVector()/temp);
//--- check
if(vcnt>0)
{
//--- prepare arrays
work0=x.ToVector()/temp;
work1=y.ToVector()/temp;
for(i=0; i<=vcnt-1; i++)
{
//--- calculation
v0=CAblasF::RDotVR(n,work0,State.m_vcorr,i);
v1=CAblasF::RDotVR(n,work1,State.m_vcorr,i);
//--- get result
result-=v0*v1;
}
}
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Internal initialization subroutine |
//+------------------------------------------------------------------+
void CMinCG::MinCGInitInternal(const int n,const double diffstep,
CMinCGState &State)
{
//--- initialization
State.m_teststep=0;
State.m_smoothnessguardlevel=0;
COptServ::SmoothnessMonitorInit(State.m_smonitor,State.m_s,0,0,false);
State.m_n=n;
State.m_diffstep=diffstep;
State.m_lastgoodstep=0;
//--- function call
MinCGSetCond(State,0,0,0,0);
//--- function call
MinCGSetXRep(State,false);
//--- function call
MinCGSetDRep(State,false);
//--- function call
MinCGSetStpMax(State,0);
//--- function call
MinCGSetCGType(State,-1);
//--- function call
MinCGSetPrecDefault(State);
//--- allocation
State.m_xk.Resize(n);
State.m_dk.Resize(n);
State.m_xn.Resize(n);
State.m_dn.Resize(n);
State.m_x.Resize(n);
State.m_d.Resize(n);
State.m_g.Resize(n);
State.m_work0.Resize(n);
State.m_work1.Resize(n);
State.m_yk.Resize(n);
State.m_xbase.Resize(n);
State.m_s=vector<double>::Ones(n);
State.m_invs=vector<double>::Ones(n);
State.m_lastscaleused=vector<double>::Ones(n);
}
//+------------------------------------------------------------------+
//| NOTES: |
//| 1. This function has two different implementations: one which |
//| uses exact (analytical) user-supplied gradient, and one which|
//| uses function value only and numerically differentiates |
//| function in order to obtain gradient. |
//| Depending on the specific function used to create optimizer |
//| object (either MinCGCreate() for analytical gradient or |
//| MinCGCreateF() for numerical differentiation) you should |
//| choose appropriate variant of MinCGOptimize() - one which |
//| accepts function AND gradient or one which accepts function |
//| ONLY. |
//| Be careful to choose variant of MinCGOptimize() which |
//| corresponds to your optimization scheme! Table below lists |
//| different combinations of callback (function/gradient) passed |
//| to MinCGOptimize() and specific function used to create |
//| optimizer. |
//| | USER PASSED TO MinCGOptimize() |
//| CREATED WITH | function only | function and gradient |
//| ------------------------------------------------------------ |
//| MinCGCreateF() | work FAIL |
//| MinCGCreate() | FAIL work |
//| Here "FAIL" denotes inappropriate combinations of optimizer |
//| creation function and MinCGOptimize() version. Attemps to use |
//| such combination (for example, to create optimizer with |
//| MinCGCreateF() and to pass gradient information to |
//| MinCGOptimize()) will lead to exception being thrown. Either |
//| you did not pass gradient when it WAS needed or you passed |
//| gradient when it was NOT needed. |
//+------------------------------------------------------------------+
bool CMinCG::MinCGIteration(CMinCGState &State)
{
//--- create variables
int n=0;
int i=0;
double betak=0;
double v=0;
double vv=0;
int i_=0;
int label=-1;
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
//
if(State.m_rstate.stage>=0)
{
n=State.m_rstate.ia[0];
i=State.m_rstate.ia[1];
betak=State.m_rstate.ra[0];
v=State.m_rstate.ra[1];
vv=State.m_rstate.ra[2];
}
else
{
n=359;
i=-58;
betak=-919;
v=-909;
vv=81;
}
switch(State.m_rstate.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
case 3:
label=3;
break;
case 4:
label=4;
break;
case 5:
label=5;
break;
case 6:
label=6;
break;
case 7:
label=7;
break;
case 8:
label=8;
break;
case 9:
label=9;
break;
case 10:
label=10;
break;
case 11:
label=11;
break;
case 12:
label=12;
break;
case 13:
label=13;
break;
case 14:
label=14;
break;
case 15:
label=15;
break;
case 16:
label=16;
break;
case 17:
label=17;
break;
default:
//--- Routine body
//--- Prepare
n=State.m_n;
State.m_terminationneeded=false;
State.m_userterminationneeded=false;
State.m_repterminationtype=0;
State.m_repiterationscount=0;
State.m_repnfev=0;
State.m_debugrestartscount=0;
COptServ::SmoothnessMonitorInit(State.m_smonitor,State.m_s,n,1,State.m_smoothnessguardlevel>0);
State.m_lastscaleused=State.m_s;
State.m_invs=State.m_s.Pow(-1.0)+0;
//--- Check, that transferred derivative value is right
ClearRequestFields(State);
if(!(State.m_diffstep==0.0 && State.m_teststep>0.0))
label=18;
else
label=20;
break;
}
//--- Main loop
while(label>=0)
switch(label)
{
case 20:
if(!COptServ::SmoothnessMonitorCheckGradientATX0(State.m_smonitor,State.m_xbase,State.m_s,State.m_s,State.m_s,false,State.m_teststep))
{
label=21;
break;
}
State.m_x=State.m_smonitor.m_x;
State.m_needfg=true;
State.m_rstate.stage=0;
label=-1;
break;
case 0:
State.m_needfg=false;
State.m_smonitor.m_fi.Set(0,State.m_f);
State.m_smonitor.m_j.Row(0,State.m_g);
label=20;
break;
case 21:
case 18:
//--- Preparations continue:
//--- * set XK
//--- * calculate F/G
//--- * set DK to -G
//--- * powerup algo (it may change preconditioner)
//--- * apply preconditioner to DK
//--- * report update of X
//--- * check stopping conditions for G
State.m_x=State.m_xbase;
State.m_xk=State.m_x;
ClearRequestFields(State);
if(State.m_diffstep!=0.0)
{
label=22;
break;
}
State.m_needfg=true;
State.m_rstate.stage=1;
label=-1;
break;
case 1:
State.m_needfg=false;
label=23;
break;
case 22:
State.m_needf=true;
State.m_rstate.stage=2;
label=-1;
break;
case 2:
State.m_fbase=State.m_f;
i=0;
case 24:
if(i>n-1)
{
label=26;
break;
}
v=State.m_x[i];
State.m_x.Set(i,v-State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=3;
label=-1;
break;
case 3:
State.m_fm2=State.m_f;
State.m_x.Set(i,v-0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=4;
label=-1;
break;
case 4:
State.m_fm1=State.m_f;
State.m_x.Set(i,v+0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=5;
label=-1;
break;
case 5:
State.m_fp1=State.m_f;
State.m_x.Set(i,v+State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=6;
label=-1;
break;
case 6:
State.m_fp2=State.m_f;
State.m_x.Set(i,v);
State.m_g.Set(i,(8*(State.m_fp1-State.m_fm1)-(State.m_fp2-State.m_fm2))/(6*State.m_diffstep*State.m_s[i]));
i++;
label=24;
break;
case 26:
State.m_f=State.m_fbase;
State.m_needf=false;
case 23:
if(!State.m_drep)
{
label=27;
break;
}
//--- Report algorithm powerup (if needed)
ClearRequestFields(State);
State.m_algpowerup=true;
State.m_rstate.stage=7;
label=-1;
break;
case 7:
State.m_algpowerup=false;
case 27:
COptServ::TrimPrepare(State.m_f,State.m_trimthreshold);
State.m_dk=State.m_g.ToVector()*(-1.0);
PreconditionedMultiply(State,State.m_dk,State.m_work0,State.m_work1);
if(!State.m_xrep)
{
label=29;
break;
}
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=8;
label=-1;
break;
case 8:
State.m_xupdated=false;
case 29:
if(State.m_terminationneeded || State.m_userterminationneeded)
{
//--- Combined termination point for "internal" termination by TerminationNeeded flag
//--- and for "user" termination by MinCGRequestTermination() (UserTerminationNeeded flag).
//--- In this location rules for both of methods are same, thus only one exit point is needed.
State.m_xn=State.m_xk;
State.m_repterminationtype=8;
return(false);
}
v=MathPow(State.m_g*State.m_s+0,2.0).Sum();
if(MathSqrt(v)<=State.m_epsg)
{
State.m_xn=State.m_xk;
State.m_repterminationtype=4;
return(false);
}
State.m_repnfev=1;
State.m_k=0;
State.m_fold=State.m_f;
//--- Choose initial step.
//--- Apply preconditioner, if we have something other than default.
if(State.m_prectype==2 || State.m_prectype==3)
{
//--- because we use preconditioner, step length must be equal
//--- to the norm of DK
v=State.m_dk.Dot(State.m_dk);
State.m_lastgoodstep=MathSqrt(v);
}
else
{
//--- No preconditioner is used, we try to use suggested step
if(State.m_suggestedstep>0.0)
State.m_lastgoodstep=State.m_suggestedstep;
else
State.m_lastgoodstep=1.0;
}
//--- Main cycle
State.m_rstimer=m_rscountdownlen;
case 31:
//--- * clear reset flag
//--- * clear termination flag
//--- * store G[k] for later calculation of Y[k]
//--- * prepare starting point and direction and step length for line search
State.m_innerresetneeded=false;
State.m_terminationneeded=false;
State.m_yk=State.m_g*(-1.0)+0;
State.m_d=State.m_dk;
State.m_x=State.m_xk;
State.m_mcstage=0;
State.m_stp=1.0;
CLinMin::LinMinNormalized(State.m_d,State.m_stp,n);
if(State.m_lastgoodstep!=0.0)
State.m_stp=State.m_lastgoodstep;
State.m_curstpmax=State.m_stpmax;
//--- Report beginning of line search (if needed)
//--- Terminate algorithm, if user request was detected
if(!State.m_drep)
{
label=33;
break;
}
ClearRequestFields(State);
State.m_lsstart=true;
State.m_rstate.stage=9;
label=-1;
break;
case 9:
State.m_lsstart=false;
case 33:
if(State.m_terminationneeded)
{
State.m_xn=State.m_x;
State.m_repterminationtype=8;
return(false);
}
//--- Minimization along D
COptServ::SmoothnessMonitorStartLineSearch1u(State.m_smonitor,State.m_s,State.m_invs,State.m_x,State.m_f,State.m_g);
CLinMin::MCSrch(n,State.m_x,State.m_f,State.m_g,State.m_d,State.m_stp,State.m_curstpmax,m_gtol,State.m_mcinfo,State.m_nfev,State.m_work0,State.m_lstate,State.m_mcstage);
case 35:
if(State.m_mcstage==0)
{
label=36;
break;
}
//--- Calculate function/gradient using either
//--- analytical gradient supplied by user
//--- or finite difference approximation.
//--- "Trim" function in order to handle near-singularity points.
ClearRequestFields(State);
if(State.m_diffstep!=0.0)
{
label=37;
break;
}
State.m_needfg=true;
State.m_rstate.stage=10;
label=-1;
break;
case 10:
State.m_needfg=false;
label=38;
break;
case 37:
State.m_needf=true;
State.m_rstate.stage=11;
label=-1;
break;
case 11:
State.m_fbase=State.m_f;
i=0;
case 39:
if(i>n-1)
{
label=41;
break;
}
v=State.m_x[i];
State.m_x.Set(i,v-State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=12;
label=-1;
break;
case 12:
State.m_fm2=State.m_f;
State.m_x.Set(i,v-0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=13;
label=-1;
break;
case 13:
State.m_fm1=State.m_f;
State.m_x.Set(i,v+0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=14;
label=-1;
break;
case 14:
State.m_fp1=State.m_f;
State.m_x.Set(i,v+State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=15;
label=-1;
break;
case 15:
State.m_fp2=State.m_f;
State.m_x.Set(i,v);
State.m_g.Set(i,(8*(State.m_fp1-State.m_fm1)-(State.m_fp2-State.m_fm2))/(6*State.m_diffstep*State.m_s[i]));
i++;
label=39;
break;
case 41:
State.m_f=State.m_fbase;
State.m_needf=false;
case 38:
COptServ::SmoothnessMonitorEnqueuePoint1u(State.m_smonitor,State.m_s,State.m_invs,State.m_d,State.m_stp,State.m_x,State.m_f,State.m_g);
COptServ::TrimFunction(State.m_f,State.m_g,n,State.m_trimthreshold);
//--- Call MCSRCH again
CLinMin::MCSrch(n,State.m_x,State.m_f,State.m_g,State.m_d,State.m_stp,State.m_curstpmax,m_gtol,State.m_mcinfo,State.m_nfev,State.m_work0,State.m_lstate,State.m_mcstage);
label=35;
break;
case 36:
COptServ::SmoothnessMonitorFinalizeLineSearch(State.m_smonitor);
//---*terminate algorithm if "user" request for detected
//--- * report end of line search
//--- * store current point to XN
//--- * report iteration
//---*terminate algorithm if "internal" request was detected
if(State.m_userterminationneeded)
{
State.m_xn=State.m_xk;
State.m_repterminationtype=8;
return(false);
}
if(!State.m_drep)
{
label=42;
break;
}
//--- Report end of line search (if needed)
ClearRequestFields(State);
State.m_lsend=true;
State.m_rstate.stage=16;
label=-1;
break;
case 16:
State.m_lsend=false;
case 42:
State.m_xn=State.m_x;
if(!State.m_xrep)
{
label=44;
break;
}
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=17;
label=-1;
break;
case 17:
State.m_xupdated=false;
case 44:
if(State.m_terminationneeded)
{
State.m_xn=State.m_x;
State.m_repterminationtype=8;
return(false);
}
//--- Line search is finished.
//--- * calculate BetaK
//--- * calculate DN
//--- * update timers
//--- * calculate step length:
//--- * LastScaledStep is ALWAYS calculated because it is used in the stopping criteria
//--- * LastGoodStep is updated only when MCINFO is equal to 1 (Wolfe conditions hold).
//--- See below for more explanation.
if(State.m_mcinfo==1 && !State.m_innerresetneeded)
{
//--- Standard Wolfe conditions hold
//--- Calculate Y[K] and D[K]'*Y[K]
State.m_yk+=State.m_g;
vv=State.m_yk.Dot(State.m_dk);
//--- Calculate BetaK according to DY formula
v=PreconditionedMultiply2(State,State.m_g,State.m_g,State.m_work0,State.m_work1);
State.m_betady=v/vv;
//--- Calculate BetaK according to HS formula
v=PreconditionedMultiply2(State,State.m_g,State.m_yk,State.m_work0,State.m_work1);
State.m_betahs=v/vv;
//--- Choose BetaK
if(State.m_cgtype==0)
betak=State.m_betady;
if(State.m_cgtype==1)
betak=MathMax(0,MathMin(State.m_betady,State.m_betahs));
}
else
{
//--- Something is wrong (may be function is too wild or too flat)
//--- or we just have to restart algo.
//--- We'll set BetaK=0, which will restart CG algorithm.
//--- We can stop later (during normal checks) if stopping conditions are met.
//
betak=0;
State.m_debugrestartscount++;
}
if(State.m_repiterationscount>0 && State.m_repiterationscount%(3+n)==0)
{
//--- clear Beta every N iterations
betak=0;
}
if(State.m_mcinfo==1 || State.m_mcinfo==5)
State.m_rstimer=m_rscountdownlen;
else
State.m_rstimer--;
State.m_dn=State.m_g*(-1.0)+0;
PreconditionedMultiply(State,State.m_dn,State.m_work0,State.m_work1);
State.m_dn+=State.m_dk.ToVector()*betak;
State.m_lastscaledstep=MathPow(State.m_d.ToVector()/State.m_s.ToVector(),2.0).Sum();
State.m_lastscaledstep=State.m_stp*MathSqrt(State.m_lastscaledstep);
if(State.m_mcinfo==1)
{
//--- Step is good (Wolfe conditions hold), update LastGoodStep.
//--- This check for MCINFO=1 is essential because sometimes in the
//--- constrained optimization setting we may take very short steps
//--- (like 1E-15) because we were very close to boundary of the
//--- feasible area. Such short step does not mean that we've converged
//--- to the solution - it was so short because we were close to the
//--- boundary and there was a limit on step length.
//--- So having such short step is quite normal situation. However, we
//--- should NOT start next iteration from step whose initial length is
//--- estimated as 1E-15 because it may lead to the failure of the
//--- linear minimizer (step is too short, function does not changes,
//--- line search stagnates).
State.m_lastgoodstep=CAblasF::RDotV2(n,State.m_d);
State.m_lastgoodstep=State.m_stp*MathSqrt(State.m_lastgoodstep);
}
//--- Update information.
//--- Check stopping conditions.
v=MathPow(State.m_g.ToVector()*State.m_s.ToVector(),2.0).Sum();
if(!MathIsValidNumber(v) || !MathIsValidNumber(State.m_f))
{
//--- Abnormal termination - infinities in function/gradient
State.m_repterminationtype=-8;
return(false);
}
State.m_repnfev+=State.m_nfev;
State.m_repiterationscount++;
if(State.m_repiterationscount>=State.m_maxits && State.m_maxits>0)
{
//--- Too many iterations
State.m_repterminationtype=5;
return(false);
}
if(MathSqrt(v)<=State.m_epsg)
{
//--- Gradient is small enough
State.m_repterminationtype=4;
return(false);
}
if(!State.m_innerresetneeded)
{
//--- These conditions are checked only when no inner reset was requested by user
if((double)(State.m_fold-State.m_f)<=(double)(State.m_epsf*MathMax(MathAbs(State.m_fold),MathMax(MathAbs(State.m_f),1.0))))
{
//--- F(k+1)-F(k) is small enough
State.m_repterminationtype=1;
return(false);
}
if(State.m_lastscaledstep<=State.m_epsx)
{
//--- X(k+1)-X(k) is small enough
State.m_repterminationtype=2;
return(false);
}
}
if(State.m_rstimer<=0)
{
//--- Too many subsequent restarts
State.m_repterminationtype=7;
return(false);
}
//--- Shift Xk/Dk, update other information
State.m_xk=State.m_xn;
State.m_dk=State.m_dn;
State.m_fold=State.m_f;
State.m_k++;
label=31;
break;
case 32:
return(false);
}
//--- Saving State
State.m_rstate.ia.Set(0,n);
State.m_rstate.ia.Set(1,i);
State.m_rstate.ra.Set(0,betak);
State.m_rstate.ra.Set(1,v);
State.m_rstate.ra.Set(2,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| This structure is a SNNLS(Specialized Non-Negative Least Squares)|
//| m_solver. |
//| It solves problems of the form | A*x - b | ^ 2 => min subject to |
//| non-negativity constraints on SOME components of x, with |
//| structured A(first NS columns are just unit matrix, next ND |
//| columns store dense part). |
//| This m_solver is suited for solution of many sequential NNLS |
//| subproblems - it keeps track of previously allocated memory and |
//| reuses it as much as possible. |
//+------------------------------------------------------------------+
struct CSNNLSSolver
{
int m_debugmaxinnerits;
int m_nd;
int m_nr;
int m_ns;
double m_debugflops;
bool m_nnc[];
CRowInt m_rdtmprowmap;
CRowDouble m_b;
CRowDouble m_cb;
CRowDouble m_cborg;
CRowDouble m_crb;
CRowDouble m_cx;
CRowDouble m_d;
CRowDouble m_diagaa;
CRowDouble m_dx;
CRowDouble m_g;
CRowDouble m_r;
CRowDouble m_regdiag;
CRowDouble m_tmp0;
CRowDouble m_tmp1;
CRowDouble m_tmp2;
CRowDouble m_tmpcholesky;
CRowDouble m_trdd;
CRowDouble m_xn;
CRowDouble m_xp;
CMatrixDouble m_densea;
CMatrixDouble m_tmpca;
CMatrixDouble m_tmplq;
CMatrixDouble m_trda;
CSNNLSSolver(void);
~CSNNLSSolver(void) {}
void Copy(const CSNNLSSolver &obj);
//--- overloading
void operator=(const CSNNLSSolver &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CSNNLSSolver::CSNNLSSolver(void)
{
m_debugmaxinnerits=0;
m_nd=0;
m_nr=0;
m_ns=0;
m_debugflops=0;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CSNNLSSolver::Copy(const CSNNLSSolver &obj)
{
m_debugmaxinnerits=obj.m_debugmaxinnerits;
m_nd=obj.m_nd;
m_nr=obj.m_nr;
m_ns=obj.m_ns;
m_debugflops=obj.m_debugflops;
ArrayCopy(m_nnc,obj.m_nnc);
m_rdtmprowmap=obj.m_rdtmprowmap;
m_b=obj.m_b;
m_cb=obj.m_cb;
m_cborg=obj.m_cborg;
m_crb=obj.m_crb;
m_cx=obj.m_cx;
m_d=obj.m_d;
m_diagaa=obj.m_diagaa;
m_dx=obj.m_dx;
m_g=obj.m_g;
m_r=obj.m_r;
m_regdiag=obj.m_regdiag;
m_tmp0=obj.m_tmp0;
m_tmp1=obj.m_tmp1;
m_tmp2=obj.m_tmp2;
m_tmpcholesky=obj.m_tmpcholesky;
m_trdd=obj.m_trdd;
m_xn=obj.m_xn;
m_xp=obj.m_xp;
m_densea=obj.m_densea;
m_tmpca=obj.m_tmpca;
m_tmplq=obj.m_tmplq;
m_trda=obj.m_trda;
}
//+------------------------------------------------------------------+
//| Specialized Non-Negative Least Squares |
//+------------------------------------------------------------------+
class CSNNLS
{
public:
static void SNNLSInit(int nsmax,int ndmax,int nrmax,CSNNLSSolver &s);
static void SNNLSSetProblem(CSNNLSSolver &s,CMatrixDouble &a,CRowDouble &b,int ns,int nd,int nr);
static void SNNLSDropNNC(CSNNLSSolver &s,int idx);
static void SNNLSSolve(CSNNLSSolver &s,CRowDouble &x);
private:
static void FuncGradU(CSNNLSSolver &s,CRowDouble &x,CRowDouble &r,CRowDouble &g,double &f);
static void Func(CSNNLSSolver &s,CRowDouble &x,double &f);
static void TRDPrepare(CSNNLSSolver &s,CRowDouble &x,CRowDouble &diag,double lambdav,CRowDouble &trdd,CMatrixDouble &trda,CRowDouble &m_tmp0,CRowDouble &m_tmp1,CRowDouble &tmp2,CMatrixDouble &tmplq);
static void TRDSolve(CRowDouble &trdd,CMatrixDouble &trda,int ns,int nd,CRowDouble &d);
static void TRDFixVariable(CRowDouble &trdd,CMatrixDouble &trda,int ns,int nd,int idx,CRowDouble &tmp);
};
//+------------------------------------------------------------------+
//| This subroutine is used to initialize SNNLS m_solver. |
//| By default, empty NNLS problem is produced, but we allocated |
//| enough space to store problems with NSMax + NDMax columns and |
//| NRMax rows. It is good place to provide algorithm with initial |
//| estimate of the space requirements, although you may |
//| underestimate problem size or even pass zero estimates - in this |
//| case buffer variables will be resized automatically when you set |
//| NNLS problem. |
//| Previously allocated buffer variables are reused as much as |
//| possible. This function does not clear structure completely, it |
//| tries to preserve as much dynamically allocated memory as |
//| possible. |
//+------------------------------------------------------------------+
void CSNNLS::SNNLSInit(int nsmax,
int ndmax,
int nrmax,
CSNNLSSolver &s)
{
s.m_ns=0;
s.m_nd=0;
s.m_nr=0;
s.m_debugflops=0.0;
s.m_debugmaxinnerits=0;
s.m_densea.Resize(nrmax,ndmax);
s.m_tmpca.Resize(nrmax,ndmax);
s.m_b.Resize(nrmax);
ArrayResize(s.m_nnc,nsmax+ndmax);
}
//+------------------------------------------------------------------+
//| This subroutine is used to set NNLS problem: |
//| ([ 1 | ] [ ] [ ]) ^ 2 |
//| ([ 1 | ] [ ] [ ]) |
//| min([ 1 | Ad ] * [ x ] - [ b ]) s.m_t. x >= 0 |
//| ([ | ] [ ] [ ]) |
//| ([ | ] [ ] [ ]) |
//| where: |
//| * identity matrix has NS*NS size(NS <= NR, NS can be zero) |
//| * dense matrix Ad has NR*ND size |
//| * b is NR * 1 vector |
//| * x is(NS + ND) * 1 vector |
//| * all elements of x are Non - Negative(this constraint can |
//| be removed later by calling SNNLSDropNNC() function) |
//| Previously allocated buffer variables are reused as much as |
//| possible. After you set problem, you can solve it with |
//| SNNLSSolve(). |
//| INPUT PARAMETERS: |
//| S - SNNLS m_solver, must be initialized with SNNLSInit() |
//| call |
//| A - array[NR, ND], dense part of the system |
//| B - array[NR], right part |
//| NS - size of the sparse part of the system, 0<=NS<=NR |
//| ND - size of the dense part of the system, ND >= 0 |
//| NR - rows count, NR > 0 |
//| NOTE 1: You can have NS + ND = 0, m_solver will correctly accept |
//| such combination and return empty array as problem |
//| solution. |
//+------------------------------------------------------------------+
void CSNNLS::SNNLSSetProblem(CSNNLSSolver &s,
CMatrixDouble &a,
CRowDouble &b,
int ns,
int nd,
int nr)
{
//--- check
if(!CAp::Assert(nd>=0,__FUNCTION__+": ND<0"))
return;
if(!CAp::Assert(ns>=0,__FUNCTION__+": NS<0"))
return;
if(!CAp::Assert(nr>0,__FUNCTION__+": NR<=0"))
return;
if(!CAp::Assert(ns<=nr,__FUNCTION__+": NS>NR"))
return;
if(!CAp::Assert(a.Rows()>=nr || nd==0,__FUNCTION__+": rows(A)<NR"))
return;
if(!CAp::Assert(a.Cols()>=nd,__FUNCTION__+": cols(A)<ND"))
return;
if(!CAp::Assert(b.Size()>=nr,__FUNCTION__+": length(B)<NR"))
return;
if(!CAp::Assert(CApServ::IsFiniteMatrix(a,nr,nd),__FUNCTION__+": A contains INF/NAN"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(b,nr),__FUNCTION__+": B contains INF/NAN"))
return;
//--- Copy problem
s.m_ns=ns;
s.m_nd=nd;
s.m_nr=nr;
if(nd>0)
{
s.m_densea=a;
s.m_densea.Resize(nr,nd);
}
s.m_b=b;
s.m_b.Resize(nr);
ArrayResize(s.m_nnc,ns+nd);
ArrayInitialize(s.m_nnc,true);
}
//+------------------------------------------------------------------+
//| This subroutine drops non-negativity constraint from the problem|
//| set by SNNLSSetProblem() call. This function must be called AFTER|
//| problem is set, because each SetProblem() call resets constraints|
//| to their default State (all constraints are present). |
//| INPUT PARAMETERS: |
//| S - SNNLS m_solver, must be initialized with SNNLSInit() |
//| call, problem must be set with SNNLSSetProblem() |
//| call. |
//| Idx - constraint index, 0 <= IDX < NS + ND |
//+------------------------------------------------------------------+
void CSNNLS::SNNLSDropNNC(CSNNLSSolver &s,int idx)
{
//--- check
if(!CAp::Assert(idx>=0,__FUNCTION__+": Idx<0"))
return;
if(!CAp::Assert(idx<s.m_ns+s.m_nd,__FUNCTION__+": Idx>=NS+ND"))
return;
s.m_nnc[idx]=false;
}
//+------------------------------------------------------------------+
//| This subroutine is used to solve NNLS problem. |
//| INPUT PARAMETERS: |
//| S - SNNLS m_solver, must be initialized with SNNLSInit() |
//| call and problem must be set up with |
//| SNNLSSetProblem() call. |
//| X - possibly preallocated buffer, automatically resized|
//| if needed |
//| OUTPUT PARAMETERS: |
//| X - array[NS + ND], solution |
//| NOTE: |
//| 1. You can have NS + ND = 0, m_solver will correctly accept such |
//| combination and return empty array as problem solution. |
//| 2. Internal field S.DebugFLOPS contains rough estimate of FLOPs|
//| used to solve problem. It can be used for debugging |
//| purposes. This field is real - valued. |
//+------------------------------------------------------------------+
void CSNNLS::SNNLSSolve(CSNNLSSolver &s,CRowDouble &x)
{
//--- create variables
int i=0;
int ns=s.m_ns;
int nd=s.m_nd;
int nr=s.m_nr;
bool wasactivation=false;
double lambdav=0;
double v0=0;
double v1=0;
double v=0;
int outerits=0;
int innerits=0;
int maxouterits=0;
double xtol=0;
double kicklength=0;
bool kickneeded=false;
double f0=0;
double f1=0;
double dnrm=0;
int actidx=0;
double stp=0;
double stpmax=0;
//--- Prepare
s.m_debugflops=0.0;
//--- Handle special cases:
//--- * NS+ND=0
//--- * ND=0
if(ns+nd==0)
return;
if(nd==0)
{
x=vector<double>::Zeros(ns);
for(i=0; i<ns; i++)
{
if(s.m_nnc[i])
x.Set(i,MathMax(s.m_b[i],0.0));
else
x.Set(i,s.m_b[i]);
}
return;
}
//--- Main cycle of BLEIC-SNNLS algorithm.
//--- Below we assume that ND>0.
x=vector<double>::Zeros(ns+nd);
s.m_xn.Resize(ns+nd);
s.m_xp.Resize(ns+nd);
s.m_g.Resize(ns+nd);
s.m_d.Resize(ns+nd);
s.m_r.Resize(nr);
s.m_diagaa.Resize(nd);
s.m_regdiag=vector<double>::Ones(ns+nd);
s.m_dx.Resize(ns+nd);
lambdav=1.0E6*CMath::m_machineepsilon;
maxouterits=10;
outerits=0;
innerits=0;
xtol=1.0E3*CMath::m_machineepsilon;
kicklength=MathSqrt(CMath::m_minrealnumber);
while(true)
{
//--- Initial check for correctness of X
for(i=0; i<ns+nd; i++)
{
//--- check
if(!CAp::Assert(!s.m_nnc[i] || x[i]>=0.0,__FUNCTION__+": integrity check failed"))
return;
}
//--- Calculate gradient G and constrained descent direction D
FuncGradU(s,x,s.m_r,s.m_g,f0);
for(i=0; i<ns+nd; i++)
{
if(s.m_nnc[i] && x[i]==0.0 && s.m_g[i]>0.0)
s.m_d.Set(i,0.0);
else
s.m_d.Set(i,-s.m_g[i]);
}
//--- Decide whether we need "kick" stage: special stage
//--- that moves us away from boundary constraints which are
//--- not strictly active (i.e. such constraints that x[i]=0.0 and d[i]>0).
//--- If we need kick stage, we make a kick - and restart iteration.
//--- If not, after this block we can rely on the fact that
//--- for all x[i]=0.0 we have d[i]=0.0
//--- NOTE: we do not increase outer iterations counter here
kickneeded=false;
for(i=0; i<ns+nd; i++)
{
if(s.m_nnc[i] && x[i]==0.0 && s.m_d[i]>0.0)
kickneeded=true;
}
if(kickneeded)
{
//--- Perform kick.
//--- Restart.
//--- Do not increase iterations counter.
for(i=0; i<ns+nd; i++)
{
if(x[i]==0.0 && s.m_d[i]>0.0)
x.Add(i,kicklength);
}
continue;
}
//--- Newton phase
//--- Reduce problem to constrained triangular form and perform Newton
//--- steps with quick activation of constrants (triangular form is
//--- updated in order to handle changed constraints).
s.m_xp=x;
TRDPrepare(s,x,s.m_regdiag,lambdav,s.m_trdd,s.m_trda,s.m_tmp0,s.m_tmp1,s.m_tmp2,s.m_tmplq);
while(true)
{
//--- Skip if debug limit on inner iterations count is turned on.
if(s.m_debugmaxinnerits>0 && innerits>=s.m_debugmaxinnerits)
break;
//--- Prepare step vector.
FuncGradU(s,x,s.m_r,s.m_g,f0);
for(i=0; i<ns+nd; i++)
{
if(s.m_nnc[i] && x[i]==0.0)
s.m_d.Set(i,0.0);
else
s.m_d.Set(i,-s.m_g[i]);
}
TRDSolve(s.m_trdd,s.m_trda,ns,nd,s.m_d);
//--- Perform unconstrained trial step and compare function values.
for(i=0; i<ns+nd; i++)
{
s.m_xn.Set(i,x[i]+s.m_d[i]);
}
Func(s,s.m_xn,f1);
if(f1>=f0)
break;
//--- Calculate length of D, maximum step and component which is
//--- activated by this step. Break if D is exactly zero.
dnrm=CAblasF::RDotV2(ns+nd,s.m_d);
dnrm=MathSqrt(dnrm);
actidx=-1;
stpmax=1.0E50;
for(i=0; i<ns+nd; i++)
{
if(s.m_nnc[i] && s.m_d[i]<0.0)
{
v=stpmax;
stpmax=CApServ::SafeMinPosRV(x[i],-s.m_d[i],stpmax);
if(stpmax<v)
actidx=i;
}
}
if(dnrm==0.0)
break;
//--- Perform constrained step and update X
//--- and triangular model.
stp=MathMin(1.0,stpmax);
for(i=0; i<ns+nd; i++)
{
v=x[i]+stp*s.m_d[i];
if(s.m_nnc[i])
v=MathMax(v,0.0);
s.m_xn.Set(i,v);
}
if(stp==stpmax && actidx>=0)
s.m_xn.Set(actidx,0.0);
wasactivation=false;
for(i=0; i<ns+nd; i++)
{
if(s.m_xn[i]==0.0 && x[i]!=0.0)
{
wasactivation=true;
TRDFixVariable(s.m_trdd,s.m_trda,ns,nd,i,s.m_tmpcholesky);
}
}
for(i=0; i<ns+nd; i++)
x.Set(i,s.m_xn[i]);
//--- Increment iterations counter.
//--- Terminate if no constraint was activated.
innerits++;
if(!wasactivation)
break;
}
//--- Update outer iterations counter.
//--- Break if necessary:
//--- * maximum number of outer iterations performed
//--- * relative change in X is small enough
outerits++;
if(outerits>=maxouterits)
break;
v=0;
for(i=0; i<ns+nd; i++)
{
v0=MathAbs(s.m_xp[i]);
v1=MathAbs(x[i]);
if(v0!=0.0 || v1!=0.0)
v=MathMax(v,MathAbs(x[i]-s.m_xp[i])/MathMax(v0,v1));
}
if(v<=xtol)
break;
}
}
//+------------------------------------------------------------------+
//| This function calculates: |
//| * residual vector R = A * x - b |
//| * unconstrained gradient vector G |
//| * function value F = 0.5 * | R | ^ 2 |
//| R and G must have at least N elements. |
//+------------------------------------------------------------------+
void CSNNLS::FuncGradU(CSNNLSSolver &s,
CRowDouble &x,
CRowDouble &r,
CRowDouble &g,
double &f)
{
//--- create variables
int i=0;
int nr=s.m_nr;
int nd=s.m_nd;
int ns=s.m_ns;
double v=0;
int i_=0;
int i1_=0;
//--- initialization
f=0;
for(i=0; i<nr; i++)
{
i1_=ns;
v=0.0;
for(i_=0; i_<nd; i_++)
v+=s.m_densea.Get(i,i_)*x[i_+i1_];
if(i<ns)
v+= x[i];
v-=s.m_b[i];
r.Set(i,v);
f+=0.5*v*v;
}
for(i=0; i<ns; i++)
g.Set(i,r[i]);
for(i=ns; i<ns+nd; i++)
g.Set(i,0.0);
for(i=0; i<nr; i++)
{
v=r[i];
i1_=-ns;
for(i_=ns; i_<ns+nd; i_++)
g.Add(i_,v*s.m_densea.Get(i,i_+i1_));
}
}
//+------------------------------------------------------------------+
//| This function calculates function value F = 0.5 * | R | ^ 2 at X.|
//+------------------------------------------------------------------+
void CSNNLS::Func(CSNNLSSolver &s,CRowDouble &x,double &f)
{
//--- create variables
int i=0;
int nr=s.m_nr;
int nd=s.m_nd;
int ns=s.m_ns;
double v=0;
int i_=0;
int i1_=0;
//--- initialization
f=0;
for(i=0; i<nr; i++)
{
i1_=ns;
v=0.0;
for(i_=0; i_<nd; i_++)
v+=s.m_densea.Get(i,i_)*x[i_+i1_];
if(i<ns)
v+= x[i];
v-=s.m_b[i];
f+=0.5*v*v;
}
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CSNNLS::TRDPrepare(CSNNLSSolver &s,CRowDouble &x,
CRowDouble &diag,double lambdav,
CRowDouble &trdd,CMatrixDouble &trda,
CRowDouble &tmp0,CRowDouble &tmp1,
CRowDouble &tmp2,CMatrixDouble &tmplq)
{
//--- create variables
int i=0;
int j=0;
int nr=s.m_nr;
int nd=s.m_nd;
int ns=s.m_ns;
double v=0;
double cs=0;
double sn=0;
double r=0;
//--- Triangular reduction
trdd.Resize(ns);
trda.Resize(ns+nd,nd);
tmplq.Resize(nd,nr+nd);
for(i=0; i<ns; i++)
{
//--- Apply rotation to I-th row and corresponding row of
//--- regularizer. Here V is diagonal element of I-th row,
//--- which is set to 1.0 or 0.0 depending on variable
//--- status (constrained or not).
v=1.0;
if(s.m_nnc[i] && x[i]==0.0)
v=0.0;
CRotations::GenerateRotation(v,lambdav,cs,sn,r);
trdd.Set(i,cs*v+sn*lambdav);
for(j=0; j<nd; j++)
{
v=s.m_densea.Get(i,j);
trda.Set(i,j,cs*v);
tmplq.Set(j,i,-(sn*v));
}
}
for(j=0; j<nd; j++)
for(i=ns; i<nr; i++)
tmplq.Set(j,i,s.m_densea.Get(i,j));
for(j=0; j<nd; j++)
{
if(s.m_nnc[ns+j] && x[ns+j]==0.0)
{
//--- Variable is constrained, entire row is set to zero.
for(i=0; i<nr; i++)
tmplq.Set(j,i,0.0);
for(i=0; i<ns; i++)
trda.Set(i,j,0.0);
}
}
for(i=0; i<nd; i++)
{
for(j=0; j<nd; j++)
tmplq.Set(j,nr+i,0.0);
tmplq.Set(i,nr+i,lambdav*diag[i]);
}
tmp0=vector<double>::Zeros(nr+nd+1);
tmp1=vector<double>::Zeros(nr+nd+1);
tmp2=vector<double>::Zeros(nr+nd+1);
COrtFac::RMatrixLQBaseCase(tmplq,nd,nr+nd,tmp0,tmp1,tmp2);
for(i=0; i<nd; i++)
{
if(tmplq.Get(i,i)<0.0)
for(j=i; j<nd; j++)
tmplq.Mul(j,i,-1.0);
}
for(i=0; i<nd; i++)
{
for(j=0; j<=i; j++)
trda.Set(ns+j,i,tmplq.Get(i,j));
}
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CSNNLS::TRDSolve(CRowDouble &trdd,CMatrixDouble &trda,
int ns,int nd,CRowDouble &d)
{
//--- create variables
int i=0;
int j=0;
double v=0;
//--- Solve U'*y=d first.
//--- This section includes two parts:
//--- * solve diagonal part of U'
//--- * solve dense part of U'
for(i=0; i<ns; i++)
{
d.Mul(i,1.0/trdd[i]);
v=d[i];
for(j=0; j<nd; j++)
d.Add(ns+j,- v*trda.Get(i,j));
}
for(i=0; i<nd; i++)
{
d.Mul(ns+i,1.0/trda.Get(ns+i,i));
v=d[ns+i];
for(j=i+1; j<nd; j++)
d.Add(ns+j,- v*trda.Get(ns+i,j));
}
//--- Solve U*x=y then.
//--- This section includes two parts:
//--- * solve trailing triangular part of U
//--- * solve combination of diagonal and dense parts of U
for(i=nd-1; i>=0; i--)
{
v=0.0;
for(j=i+1; j<nd; j++)
v+= trda.Get(ns+i,j)*d[ns+j];
d.Set(ns+i,(d[ns+i]-v)/trda.Get(ns+i,i));
}
for(i=ns-1; i>=0; i--)
{
v=0.0;
for(j=0; j<nd; j++)
v+= trda.Get(i,j)*d[ns+j];
d.Set(i,(d[i]-v)/trdd[i]);
}
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CSNNLS::TRDFixVariable(CRowDouble &trdd,CMatrixDouble &trda,
int ns,int nd,int idx,CRowDouble &tmp)
{
//--- create variables
int i=0;
int j=0;
int k=0;
double cs=0;
double sn=0;
double r=0;
double v=0;
double vv=0;
//--- check
if(!CAp::Assert(ns>=0,__FUNCTION__+": integrity error"))
return;
if(!CAp::Assert(nd>=0,__FUNCTION__+": integrity error"))
return;
if(!CAp::Assert(ns+nd>0,__FUNCTION__+": integrity error"))
return;
if(!CAp::Assert(idx>=0,__FUNCTION__+": integrity error"))
return;
if(!CAp::Assert(idx<ns+nd,__FUNCTION__+": integrity error"))
return;
tmp.Resize(nd);
//--- Depending on variable index, two situations are possible
if(idx<ns)
{
//--- We fix variable in the diagonal part of the model. It means
//--- that prior to fixing we have:
//--- ( | )
//--- ( D | )
//--- ( | )
//--- (-----| A )
//--- ( |0 )
//--- ( |00 )
//--- ( |000 )
//--- ( |0000 )
//--- ( |00000)
//--- then we replace idx-th column by zeros:
//--- (D 0 | )
//--- ( 0 | )
//--- ( 0 D| )
//--- (-----| A )
//--- ( | )
//--- ( | )
//--- ( | )
//--- and append row with unit element to bottom, in order to
//--- regularize problem
//--- (D 0 | )
//--- ( 0 | )
//--- ( 0 D| )
//--- (-----| A )
//--- ( | )
//--- ( | )
//--- ( | )
//--- (00100|00000) <- appended
//--- and then we nullify this row by applying rotations:
//--- (D 0 | )
//--- ( 0 | ) <- first rotation is applied here
//--- ( 0 D| )
//--- (-----| A ) <- subsequent rotations are applied to this row and rows below
//--- ( | )
//--- ( | )
//--- ( | )
//--- ( 0 | 0 ) <- as result, row becomes zero
//--- and triangular structure is preserved
if(nd==0)
{
//--- Quick exit for empty dense part
trdd.Set(idx,1.0);
return;
}
for(j=0; j<nd; j++)
{
//--- Apply first rotation
tmp.Set(j,trda.Get(idx,j));
trda.Set(idx,j,0.0);
}
trdd.Set(idx,1.0);
for(i=0; i<nd; i++)
{
if(tmp[i]!=0.0)
{
//--- Apply subsequent rotations with bottom triangular part of A
CRotations::GenerateRotation(trda.Get(ns+i,i),tmp[i],cs,sn,r);
for(j=i; j<nd; j++)
{
v=trda.Get(ns+i,j);
vv=tmp[j];
trda.Set(ns+i,j,v*cs+vv*sn);
tmp.Set(j,vv*cs-v*sn);
}
}
}
}
else
{
//--- We fix variable in the dense part of the model. It means
//--- that prior to fixing we have:
//--- ( | )
//--- ( D | )
//--- ( | )
//--- (-----| A )
//--- ( |0 )
//--- ( |00 )
//--- ( |000 )
//--- ( |0000 )
//--- ( |00000)
//--- then we replace idx-th column by zeros:
//--- ( | 0 )
//--- ( D | 0 )
//--- ( | 0 )
//--- (-----|A 0 A)
//--- ( | 0 )
//--- ( | 0 )
//--- ( | 0 )
//--- and append row with unit element to bottom, in order to
//--- regularize problem
//--- ( | 0 )
//--- ( D | 0 )
//--- ( | 0 )
//--- (-----|A 0 A)
//--- ( | 0 )
//--- ( | 0 )
//--- ( | 0 )
//--- (00000|00100) <- appended
//--- and then we nullify this row by applying rotations:
//--- (D 0 | )
//--- ( 0 | )
//--- ( 0 D| )
//--- (-----| A )
//--- ( | )
//--- ( | ) <- first rotation is applied here
//--- ( | ) <- subsequent rotations are applied to rows below
//--- ( 0 | 0 ) <- as result, row becomes zero
//--- and triangular structure is preserved.
k=idx-ns;
for(i=0; i<ns+nd; i++)
trda.Set(i,k,0.0);
for(j=k+1; j<nd; j++)
{
//--- Apply first rotation
tmp.Set(j,trda.Get(idx,j));
trda.Set(idx,j,0.0);
}
trda.Set(idx,k,1.0);
for(i=k+1; i<nd; i++)
{
if(tmp[i]!=0.0)
{
//--- Apply subsequent rotations with bottom triangular part of A
CRotations::GenerateRotation(trda.Get(ns+i,i),tmp[i],cs,sn,r);
for(j=i; j<nd; j++)
{
v=trda.Get(ns+i,j);
vv=tmp[j];
trda.Set(ns+i,j,v*cs+vv*sn);
tmp.Set(j,vv*cs-v*sn);
}
}
}
}
}
//+------------------------------------------------------------------+
//| This structure describes set of linear constraints (boundary and |
//| general ones) which can be active and inactive. It also has |
//| functionality to work with current point and current gradient |
//| (determine active constraints, move current point, project |
//| gradient into constrained subspace, perform constrained |
//| preconditioning and so on. |
//| This structure is intended to be used by constrained optimizers |
//| for management of all constraint - related functionality. |
//| External code may access following internal fields of the |
//| structure: |
//| XC - stores current point, array[N]. can be accessed |
//| only in optimization mode |
//| CStatus - active set, array[N + NEC + NIC]: |
//| * CStatus[I]>0 I-Th constraint is in the active set|
//| * CStatus[I]=0 I-Th constraint is at the boundary, |
//| but inactive |
//| * CStatus[I]<0 I-Th constraint is far from the |
//| boundary (and inactive) |
//| * elements from 0 to N-1 correspond to boundary |
//| constraints |
//| * elements from N to N + NEC + NIC - 1 correspond |
//| to linear constraints |
//| * elements from N to N + NEC - 1 are always + 1 |
//| FeasInitPt - SASStartOptimization() sets this flag to True if, |
//| after enforcement of box constraints, initial point|
//| was feasible w.m_r.m_t. general linear constraints. |
//| This field can be used by unit test to validate |
//| some details of initial point calculation algorithm|
//| SparseBatch - indices of box constraints which are NOT included|
//| into dense batch(were activated at the beginning of|
//| the orthogonalization); SparseBatchSize elements |
//| are stored. |
//| SparseBatchSize - size of sparse batcj |
//| PDenseBatch, |
//| IDenseBatch, |
//| SDenseBatch - after call to SASRebuildBasis() these matrices |
//| store active constraints(except for ones included |
//| into sparse batch), reorthogonalized with respect |
//| to some inner product: |
//| a) for "P" batch - one given by preconditioner |
//| matrix (inverse Hessian) |
//| b) for "S" one - one given by square of the |
//| scale matrix |
//| c) for "I" one - traditional dot product |
//| array[DenseBatchSize, N + 1] |
//| All three matrices are linearly equivalent to each other. |
//| IMPORTANT: you have to call SASRebuildBasis() before accessing |
//| these arrays in order to make sure that they are up to|
//| date. |
//| DenseBatchSize - dense batch size(PBasis / SBasis / IBasis) |
//+------------------------------------------------------------------+
struct CSActiveSet
{
int m_algostate;
int m_basisage;
int m_densebatchsize;
int m_n;
int m_nec;
int m_nic;
int m_sparsebatchsize;
bool m_basisisready;
bool m_constraintschanged;
bool m_feasinitpt;
bool m_HasBndL[];
bool m_HasBndU[];
bool m_hasxc;
bool m_mtnew[];
bool m_rctmpisequality[];
CSNNLSSolver m_solver;
CRowInt m_cstatus;
CRowInt m_mtas;
CRowInt m_rctmpconstraintidx;
CRowInt m_sparsebatch;
CRowDouble m_bndl;
CRowDouble m_bndu;
CRowDouble m_cdtmp;
CRowDouble m_corrtmp;
CRowDouble m_h;
CRowDouble m_mtx;
CRowDouble m_rctmpg;
CRowDouble m_rctmplambdas;
CRowDouble m_rctmprightpart;
CRowDouble m_rctmps;
CRowDouble m_s;
CRowDouble m_scntmp;
CRowDouble m_tmp0;
CRowDouble m_tmpci;
CRowDouble m_tmpcp;
CRowDouble m_tmpcs;
CRowDouble m_tmpfeas;
CRowDouble m_tmpnormestimates;
CRowDouble m_tmpprodp;
CRowDouble m_tmpprods;
CRowDouble m_tmpreciph;
CRowDouble m_unitdiagonal;
CRowDouble m_xc;
CMatrixDouble m_cleic;
CMatrixDouble m_idensebatch;
CMatrixDouble m_pdensebatch;
CMatrixDouble m_rctmpdense0;
CMatrixDouble m_rctmpdense1;
CMatrixDouble m_sdensebatch;
CMatrixDouble m_tmpbasis;
CMatrixDouble m_tmpm0;
//--- constructor / destructor
CSActiveSet(void);
~CSActiveSet(void) {}
void Copy(const CSActiveSet &obj);
//--- overloading
void operator=(const CSActiveSet &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
CSActiveSet::CSActiveSet(void)
{
m_algostate=0;
m_basisage=0;
m_densebatchsize=0;
m_n=0;
m_nec=0;
m_nic=0;
m_sparsebatchsize=0;
m_basisisready=false;
m_constraintschanged=false;
m_feasinitpt=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CSActiveSet::Copy(const CSActiveSet &obj)
{
m_algostate=obj.m_algostate;
m_basisage=obj.m_basisage;
m_densebatchsize=obj.m_densebatchsize;
m_n=obj.m_n;
m_nec=obj.m_nec;
m_nic=obj.m_nic;
m_sparsebatchsize=obj.m_sparsebatchsize;
m_basisisready=obj.m_basisisready;
m_constraintschanged=obj.m_constraintschanged;
m_feasinitpt=obj.m_feasinitpt;
ArrayCopy(m_HasBndL,obj.m_HasBndL);
ArrayCopy(m_HasBndU,obj.m_HasBndU);
m_hasxc=obj.m_hasxc;
ArrayCopy(m_mtnew,obj.m_mtnew);
ArrayCopy(m_rctmpisequality,obj.m_rctmpisequality);
m_solver=obj.m_solver;
m_cstatus=obj.m_cstatus;
m_mtas=obj.m_mtas;
m_rctmpconstraintidx=obj.m_rctmpconstraintidx;
m_sparsebatch=obj.m_sparsebatch;
m_bndl=obj.m_bndl;
m_bndu=obj.m_bndu;
m_cdtmp=obj.m_cdtmp;
m_corrtmp=obj.m_corrtmp;
m_h=obj.m_h;
m_mtx=obj.m_mtx;
m_rctmpg=obj.m_rctmpg;
m_rctmplambdas=obj.m_rctmplambdas;
m_rctmprightpart=obj.m_rctmprightpart;
m_rctmps=obj.m_rctmps;
m_s=obj.m_s;
m_scntmp=obj.m_scntmp;
m_tmp0=obj.m_tmp0;
m_tmpci=obj.m_tmpci;
m_tmpcp=obj.m_tmpcp;
m_tmpcs=obj.m_tmpcs;
m_tmpfeas=obj.m_tmpfeas;
m_tmpnormestimates=obj.m_tmpnormestimates;
m_tmpprodp=obj.m_tmpprodp;
m_tmpprods=obj.m_tmpprods;
m_tmpreciph=obj.m_tmpreciph;
m_unitdiagonal=obj.m_unitdiagonal;
m_xc=obj.m_xc;
m_cleic=obj.m_cleic;
m_idensebatch=obj.m_idensebatch;
m_pdensebatch=obj.m_pdensebatch;
m_rctmpdense0=obj.m_rctmpdense0;
m_rctmpdense1=obj.m_rctmpdense1;
m_sdensebatch=obj.m_sdensebatch;
m_tmpbasis=obj.m_tmpbasis;
m_tmpm0=obj.m_tmpm0;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CSActiveSets
{
public:
//--- constants
static const int m_maxbasisage ;
static const double m_maxbasisdecay;
static const double m_minnormseparation;
static void SASInit(int n,CSActiveSet &s);
static void SASSetScale(CSActiveSet &State,CRowDouble &s);
static void SASSetPrecDiag(CSActiveSet &State,CRowDouble &d);
static void SASSetBC(CSActiveSet &State,CRowDouble &bndl,CRowDouble &bndu);
static void SASSetLC(CSActiveSet &State,CMatrixDouble &c,CRowInt &ct,int k);
static void SASSetLCX(CSActiveSet &State,CMatrixDouble &cleic,int nec,int nic);
static bool SASStartOptimization(CSActiveSet &State,CRowDouble &x);
static void SASExploreDirection(CSActiveSet &State,CRowDouble &d,double &stpmax,int &cidx,double &vval);
static int SASMoveTo(CSActiveSet &State,CRowDouble &xn,bool needact,int cidx,double cval);
static void SASImmediateActivation(CSActiveSet &State,int cidx,double cval);
static void SASConstrainedDescent(CSActiveSet &State,CRowDouble &g,CRowDouble &d);
static void SASConstrainedDescentPrec(CSActiveSet &State,CRowDouble &g,CRowDouble &d);
static void SASConstrainedDirection(CSActiveSet &State,CRowDouble &d);
static void SASConstrainedDirectionPrec(CSActiveSet &State,CRowDouble &d);
static void SASCorrection(CSActiveSet &State,CRowDouble &x,double &penalty);
static double SASActiveLCPenalty1(CSActiveSet &State,CRowDouble &x);
static double SASScaledConstrainedNorm(CSActiveSet &State,CRowDouble &d);
static void SASStopOptimization(CSActiveSet &State);
static void SASReactivateConstraints(CSActiveSet &State,CRowDouble &gc);
static void SASReactivateConstraintsPrec(CSActiveSet &State,CRowDouble &gc);
static void SASRebuildBasis(CSActiveSet &State);
static void SASAppendToBasis(CSActiveSet &State,bool &newentries[]);
private:
static void ConstrainedDescent(CSActiveSet &State,CRowDouble &g,CRowDouble &h,CMatrixDouble &ha,bool normalize,CRowDouble &d);
static void ReactivateConstraints(CSActiveSet &State,CRowDouble &gc,CRowDouble &h);
};
//+------------------------------------------------------------------+
//| Constants |
//+------------------------------------------------------------------+
const int CSActiveSets::m_maxbasisage=5;
const double CSActiveSets::m_maxbasisdecay=0.01;
const double CSActiveSets::m_minnormseparation=0.25;
//+------------------------------------------------------------------+
//| This subroutine is used to initialize active set. By default, |
//| empty N-variable model with no constraints is generated. |
//| Previously allocated buffer variables are reused as much as |
//| possible. |
//| Two use cases for this object are described below. |
//| CASE 1 - STEEPEST DESCENT : |
//| SASInit() |
//| repeat: |
//| SASReactivateConstraints() |
//| SASDescentDirection() |
//| SASExploreDirection() |
//| SASMoveTo() |
//| until convergence |
//| CASE 1 - PRECONDITIONED STEEPEST DESCENT: |
//| SASInit() |
//| repeat: |
//| SASReactivateConstraintsPrec() |
//| SASDescentDirectionPrec() |
//| SASExploreDirection() |
//| SASMoveTo() |
//| until convergence |
//+------------------------------------------------------------------+
void CSActiveSets::SASInit(int n,CSActiveSet &s)
{
s.m_n=n;
s.m_algostate=0;
//--- Constraints
s.m_constraintschanged=true;
s.m_nec=0;
s.m_nic=0;
s.m_bndl=vector<double>::Full(n,AL_NEGINF);
s.m_bndu=vector<double>::Full(n,AL_POSINF);
ArrayResize(s.m_HasBndL,n);
ArrayResize(s.m_HasBndU,n);
ArrayInitialize(s.m_HasBndL,false);
ArrayInitialize(s.m_HasBndU,false);
//--- current point, scale
s.m_hasxc=false;
s.m_xc=vector<double>::Zeros(n);
s.m_s=vector<double>::Ones(n);
s.m_h=s.m_s;
//--- Other
s.m_unitdiagonal=s.m_s;
}
//+------------------------------------------------------------------+
//| This function sets scaling coefficients for SAS object. |
//| ALGLIB optimizers use scaling matrices to test stopping |
//| conditions (step size and gradient are scaled before comparison |
//| with tolerances). Scale of the I-Th variable is a translation |
//| invariant measure of: |
//| a) "how large" the variable is |
//| b) how large the step should be to make significant changes in |
//| the function |
//| During orthogonalization phase, scale is used to calculate drop |
//| tolerances (whether vector is significantly non-zero or not). |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| S - array[N], non-zero scaling coefficients S[i] may |
//| be negative, sign doesn't matter. |
//+------------------------------------------------------------------+
void CSActiveSets::SASSetScale(CSActiveSet &State,CRowDouble &s)
{
//--- check
if(!CAp::Assert(State.m_algostate==0,__FUNCTION__+": you may change scale only in modification mode"))
return;
if(!CAp::Assert(s.Size()>=State.m_n,__FUNCTION__+": Length(S)<N"))
return;
for(int i=0; i<State.m_n; i++)
{
if(!CAp::Assert(MathIsValidNumber(s[i]),__FUNCTION__+": S contains infinite or NAN elements"))
return;
if(!CAp::Assert(s[i]!=0.0,__FUNCTION__+": S contains zero elements"))
return;
}
State.m_s=s.Abs()+0;
State.m_s.Resize(State.m_n);
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: diagonal of approximate |
//| Hessian is used. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| D - diagonal of the approximate Hessian, array[0..m_n-1],|
//| (if larger, only leading N elements are used). |
//| NOTE 1: D[i] should be positive. Exception will be thrown |
//| otherwise. |
//| NOTE 2: you should pass diagonal of approximate Hessian - NOT |
//| ITS INVERSE. |
//+------------------------------------------------------------------+
void CSActiveSets::SASSetPrecDiag(CSActiveSet &State,CRowDouble &d)
{
//--- check
if(!CAp::Assert(State.m_algostate==0,__FUNCTION__+": you may change preconditioner only in modification mode"))
return;
if(!CAp::Assert(d.Size()>=State.m_n,__FUNCTION__+": D is too short"))
return;
for(int i=0; i<State.m_n; i++)
{
if(!CAp::Assert(MathIsValidNumber(d[i]),__FUNCTION__+": D contains infinite or NAN elements"))
return;
if(!CAp::Assert(d[i]>0.0,__FUNCTION__+": D contains non-positive elements"))
return;
}
State.m_h=d;
State.m_h.Resize(State.m_n);
}
//+------------------------------------------------------------------+
//| This function sets / changes boundary constraints. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| BndL - lower bounds, array[N]. If some(all) variables are |
//| unbounded, you may specify very small number or |
//| -INF. |
//| BndU - upper bounds, array[N]. If some(all) variables are |
//| unbounded, you may specify very large number or |
//| +INF. |
//| NOTE 1: it is possible to specify BndL.Set(i, BndU[i]. In this case|
//| I-th variable will be "frozen" at X[i]=BndL[i]=BndU[i]. |
//+------------------------------------------------------------------+
void CSActiveSets::SASSetBC(CSActiveSet &State,CRowDouble &bndl,
CRowDouble &bndu)
{
//--- check
if(!CAp::Assert(State.m_algostate==0,__FUNCTION__+": you may change constraints only in modification mode"))
return;
int n=State.m_n;
if(!CAp::Assert(bndl.Size()>=n,__FUNCTION__+": Length(BndL)<N"))
return;
if(!CAp::Assert(bndu.Size()>=n,__FUNCTION__+": Length(BndU)<N"))
return;
for(int i=0; i<n; i++)
{
if(!CAp::Assert(MathIsValidNumber(bndl[i]) || AL_NEGINF==bndl[i],__FUNCTION__+": BndL contains NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(bndu[i]) || AL_POSINF==bndu[i],__FUNCTION__+": BndL contains NAN or -INF"))
return;
State.m_bndl.Set(i,bndl[i]);
State.m_HasBndL[i]=MathIsValidNumber(bndl[i]);
State.m_bndu.Set(i,bndu[i]);
State.m_HasBndU[i]=MathIsValidNumber(bndu[i]);
}
State.m_constraintschanged=true;
}
//+------------------------------------------------------------------+
//| This function sets linear constraints for SAS object. |
//| Linear constraints are inactive by default(after initial |
//| creation). |
//| INPUT PARAMETERS: |
//| State - SAS structure |
//| C - linear constraints, array[K, N + 1]. Each row of C |
//| represents one constraint, either equality or |
//| inequality (see below): |
//| * first N elements correspond to coefficients, |
//| * last element corresponds to the right part. |
//| All elements of C(including right part) must be |
//| finite. |
//| CT - type of constraints, array[K]: |
//| * if CT[i] > 0, then I-Th constraint is |
//| C[i, *] * x >= C[i, n + 1] |
//| * if CT.Set(i, 0, then I-Th constraint is |
//| C[i, *] * x = C[i, n + 1] |
//| * if CT[i] < 0, then I-Th constraint is |
//| C[i, *] * x <= C[i, n + 1] |
//| K - number of equality / inequality constraints, K >= 0|
//| NOTE 1: linear(non - bound) constraints are satisfied only |
//| approximately: |
//| * there always exists some minor violation(about Epsilon |
//| in magnitude) due to rounding errors |
//| * numerical differentiation, if used, may lead to |
//| function evaluations outside of the feasible area, |
//| because algorithm does NOT change numerical |
//| differentiation formula according to linear |
//| constraints. If you want constraints to be satisfied |
//| exactly, try to reformulate your problem in such manner|
//| that all constraints will become boundary ones (this |
//| kind of constraints is always satisfied exactly, both |
//| in the final solution and in all intermediate points). |
//+------------------------------------------------------------------+
void CSActiveSets::SASSetLC(CSActiveSet &State,CMatrixDouble &c,
CRowInt &ct,
int k)
{
//--- check
if(!CAp::Assert(State.m_algostate==0,__FUNCTION__+": you may change constraints only in modification mode"))
return;
int n=State.m_n;
//--- First, check for errors in the inputs
if(!CAp::Assert(k>=0,__FUNCTION__+": K<0"))
return;
if(!CAp::Assert(c.Cols()>=n+1 || k==0,__FUNCTION__+": Cols(C)<N+1"))
return;
if(!CAp::Assert(c.Rows()>=k,__FUNCTION__+": Rows(C)<K"))
return;
if(!CAp::Assert(ct.Size()>=k,__FUNCTION__+": Length(CT)<K"))
return;
if(!CAp::Assert(CApServ::IsFiniteMatrix(c,k,n+1),__FUNCTION__+": C contains infinite or NaN values!"))
return;
//--- Handle zero K
if(k==0)
{
State.m_nec=0;
State.m_nic=0;
State.m_constraintschanged=true;
return;
}
//--- Equality constraints are stored first, in the upper
//--- NEC rows of State.CLEIC matrix. Inequality constraints
//--- are stored in the next NIC rows.
//--- NOTE: we convert inequality constraints to the form
//--- A*x<=b before copying them.
State.m_cleic.Resize(k,n+1);
State.m_nec=0;
State.m_nic=0;
for(int i=0; i<k; i++)
{
if(ct[i]==0)
{
State.m_cleic.Row(State.m_nec,c[i]+0);
State.m_nec++;
}
}
for(int i=0; i<k; i++)
{
if(ct[i]!=0)
{
if(ct[i]>0)
State.m_cleic.Row(State.m_nec+State.m_nic,c[i]*(-1.0));
else
State.m_cleic.Row(State.m_nec+State.m_nic,c[i]+0);
State.m_nic++;
}
}
//--- Mark State as changed
State.m_constraintschanged=true;
}
//+------------------------------------------------------------------+
//| Another variation of SASSetLC(), which accepts linear constraints|
//| using another representation. |
//| Linear constraints are inactive by default(after initial |
//| creation). |
//| INPUT PARAMETERS: |
//| State - SAS structure |
//| CLEIC - linear constraints, array[NEC + NIC, N + 1]. Each |
//| row of C represents one constraint: |
//| * first N elements correspond to coefficients, |
//| * last element corresponds to the right part. First|
//| NEC rows store equality constraints, next NIC - |
//| are inequality ones. All elements of C(including |
//| right part) must be finite. |
//| NEC - number of equality constraints, NEC >= 0 |
//| NIC - number of inequality constraints, NIC >= 0 |
//| NOTE 1: linear(non - bound) constraints are satisfied only |
//| approximately: |
//| * there always exists some minor violation(about Epsilon |
//| in magnitude) due to rounding errors |
//| * numerical differentiation, if used, may lead to |
//| function evaluations outside of the feasible area, |
//| because algorithm does NOT change numerical |
//| differentiation formula according to linear constraints|
//| If you want constraints to be satisfied exactly, try |
//| to reformulate your problem in such manner that all |
//| constraints will become boundary ones (this kind of |
//| constraints is always satisfied exactly, both in the |
//| final solution and in all intermediate points). |
//+------------------------------------------------------------------+
void CSActiveSets::SASSetLCX(CSActiveSet &State,CMatrixDouble &cleic,
int nec,int nic)
{
//--- check
if(!CAp::Assert(State.m_algostate==0,__FUNCTION__+": you may change constraints only in modification mode"))
return;
int n=State.m_n;
//--- First, check for errors in the inputs
if(!CAp::Assert(nec>=0,__FUNCTION__+": NEC<0"))
return;
if(!CAp::Assert(nic>=0,__FUNCTION__+": NIC<0"))
return;
if(!CAp::Assert(cleic.Cols()>=n+1 || nec+nic==0,__FUNCTION__+": Cols(CLEIC)<N+1"))
return;
if(!CAp::Assert(cleic.Rows()>=nec+nic,__FUNCTION__+": Rows(CLEIC)<NEC+NIC"))
return;
if(!CAp::Assert(CApServ::IsFiniteMatrix(cleic,nec+nic,n+1),__FUNCTION__+": CLEIC contains infinite or NaN values!"))
return;
//--- Store constraints
State.m_cleic=cleic;
State.m_cleic.Resize(nec+nic,n+1);
State.m_nec=nec;
State.m_nic=nic;
//--- Mark State as changed
State.m_constraintschanged=true;
}
//+------------------------------------------------------------------+
//| This subroutine turns on optimization mode: |
//| 1. feasibility in X is enforced(in case X=S.XC and constraints |
//| have not changed, algorithm just uses X without any |
//| modifications at all) |
//| 2. constraints are marked as "candidate" or "inactive" |
//| INPUT PARAMETERS: |
//| S - active set object |
//| X - initial point(candidate), array[N]. It is expected |
//| that X contains only finite values(we do not check |
//| it). |
//| OUTPUT PARAMETERS: |
//| S - State is changed |
//| X - initial point can be changed to enforce feasibility|
//| RESULT: |
//| True in case feasible point was found(mode was changed |
//| to "optimization") |
//| False in case no feasible point was found(mode was not changed)|
//+------------------------------------------------------------------+
bool CSActiveSets::SASStartOptimization(CSActiveSet &State,
CRowDouble &x)
{
//--- create variables
bool result=false;
int n=0;
int nec=0;
int nic=0;
int i=0;
int j=0;
double v=0;
double v0=0;
double v1=0;
double vv=0;
double vc=0;
double vx=0;
int i_=0;
//--- check
if(!CAp::Assert(State.m_algostate==0,__FUNCTION__+": already in optimization mode"))
return(false);
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
//--- Enforce feasibility and calculate set of "candidate"/"active" constraints.
//--- Always active equality constraints are marked as "active", all other constraints
//--- are marked as "candidate".
State.m_cstatus.Resize(n+nec+nic);
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i] && State.m_HasBndU[i])
if(State.m_bndl[i]>State.m_bndu[i])
return(result);
}
State.m_xc=x;
if(State.m_nec+State.m_nic>0)
{
//--- General linear constraints are present.
//--- Try to use fast code for feasible initial point with modest
//--- memory requirements.
State.m_tmp0=x;
State.m_cstatus.Fill(-1,0,n);
State.m_feasinitpt=true;
for(i=0; i<n; i++)
{
if((State.m_HasBndL[i] && State.m_HasBndU[i]) && State.m_bndl[i]==State.m_bndu[i])
{
State.m_tmp0.Set(i,State.m_bndl[i]);
State.m_cstatus.Set(i,1);
continue;
}
if(State.m_HasBndL[i] && State.m_tmp0[i]<=State.m_bndl[i])
{
State.m_cstatus.Set(i,0);
State.m_tmp0.Set(i,State.m_bndl[i]);
}
if(State.m_HasBndU[i] && State.m_tmp0[i]>=State.m_bndu[i])
{
State.m_cstatus.Set(i,0);
State.m_tmp0.Set(i,State.m_bndu[i]);
}
}
for(i=0; i<State.m_nec+State.m_nic; i++)
{
v=-State.m_cleic.Get(i,n);
v0=0;
v1=0;
for(j=0; j<n; j++)
{
vx=State.m_tmp0[j]/State.m_s[j];
vc=State.m_cleic.Get(i,j)*State.m_s[j];
v+= vx*vc;
v0+=CMath::Sqr(vx);
v1+=CMath::Sqr(vc);
}
vv=MathSqrt(v0)*MathSqrt(v1)*1000*CMath::m_machineepsilon;
if(i<State.m_nec)
{
State.m_cstatus.Set(n+i,1);
State.m_feasinitpt=State.m_feasinitpt && MathAbs(v)<vv;
}
else
{
State.m_feasinitpt=State.m_feasinitpt && v<vv;
if(v<(-vv))
State.m_cstatus.Set(n+i,-1);
else
State.m_cstatus.Set(n+i,0);
}
}
if(State.m_feasinitpt)
{
State.m_xc=State.m_tmp0;
}
//--- Fast code failed? Use general code with ~(N+NIC)^2 memory requirements
if(!State.m_feasinitpt)
{
State.m_tmp0.Resize(n);
State.m_tmpfeas.Resize(n+State.m_nic);
State.m_tmpm0.Resize(State.m_nec+State.m_nic,n+State.m_nic+1);
for(i=0; i<State.m_nec+State.m_nic; i++)
{
for(i_=0; i_<n; i_++)
State.m_tmpm0.Set(i,i_,State.m_cleic.Get(i,i_));
for(j=n; j<n+State.m_nic; j++)
State.m_tmpm0.Set(i,j,0);
if(i>=State.m_nec)
State.m_tmpm0.Set(i,n+i-State.m_nec,1.0);
State.m_tmpm0.Set(i,n+State.m_nic,State.m_cleic.Get(i,n));
}
for(i_=0; i_<n; i_++)
State.m_tmpfeas.Set(i_,State.m_xc[i_]);
for(i=0; i<State.m_nic; i++)
{
v=CAblasF::RDotVR(n,State.m_xc,State.m_cleic,i+State.m_nec);
State.m_tmpfeas.Set(i+n,MathMax(State.m_cleic.Get(i+State.m_nec,n)-v,0.0));
}
if(!COptServ::FindFeasiblePoint(State.m_tmpfeas,State.m_bndl,State.m_HasBndL,State.m_bndu,State.m_HasBndU,n,State.m_nic,State.m_tmpm0,State.m_nec+State.m_nic,1.0E-6,i,j))
return(result);
for(i_=0; i_<n; i_++)
State.m_xc.Set(i_,State.m_tmpfeas[i_]);
for(i=0; i<n; i++)
{
if((State.m_HasBndL[i] && State.m_HasBndU[i]) && State.m_bndl[i]==State.m_bndu[i])
{
State.m_cstatus.Set(i,1);
continue;
}
if((State.m_HasBndL[i] && State.m_xc[i]==State.m_bndl[i]) || (State.m_HasBndU[i] && State.m_xc[i]==State.m_bndu[i]))
{
State.m_cstatus.Set(i,0);
continue;
}
State.m_cstatus.Set(i,-1);
}
for(i=0; i<State.m_nec; i++)
State.m_cstatus.Set(n+i,1);
for(i=0; i<State.m_nic; i++)
{
if(State.m_tmpfeas[n+i]==0.0)
State.m_cstatus.Set(n+State.m_nec+i,0);
else
State.m_cstatus.Set(n+State.m_nec+i,-1);
}
}
}
else
{
//--- Only box constraints are present, quick code can be used
for(i=0; i<n; i++)
{
State.m_cstatus.Set(i,-1);
if(State.m_HasBndL[i] && State.m_HasBndU[i] && State.m_bndl[i]==State.m_bndu[i])
{
State.m_cstatus.Set(i,1);
State.m_xc.Set(i,State.m_bndl[i]);
continue;
}
if(State.m_HasBndL[i] && State.m_xc[i]<=State.m_bndl[i])
{
State.m_xc.Set(i,State.m_bndl[i]);
State.m_cstatus.Set(i,0);
continue;
}
if(State.m_HasBndU[i] && State.m_xc[i]>=State.m_bndu[i])
{
State.m_xc.Set(i,State.m_bndu[i]);
State.m_cstatus.Set(i,0);
continue;
}
}
State.m_feasinitpt=true;
}
//--- Change State, allocate temporaries
result=true;
State.m_algostate=1;
State.m_basisisready=false;
State.m_hasxc=true;
return(result);
}
//+------------------------------------------------------------------+
//| This function explores search direction and calculates bound for |
//| step as well as information for activation of constraints. |
//| INPUT PARAMETERS: |
//| State - SAS structure which stores current point and all |
//| other active set related information |
//| D - descent direction to explore |
//| OUTPUT PARAMETERS: |
//| StpMax - upper limit on step length imposed by yet inactive |
//| constraints. Can be zero in case some constraints |
//| can be activated by zero step. Equal to some large |
//| value in case step is unlimited. |
//| CIdx - -1 for unlimited step, in [0, N + NEC + NIC) in |
//| case of limited step. |
//| VVal - value which is assigned to X[CIdx] during |
//| activation. For CIdx<0 or CIdx >= N some dummy |
//| value is assigned to this parameter. |
//+------------------------------------------------------------------+
void CSActiveSets::SASExploreDirection(CSActiveSet &State,CRowDouble &d,
double &stpmax,int &cidx,
double &vval)
{
//--- create variables
int n=0;
int nec=0;
int nic=0;
int i=0;
double prevmax=0;
double vc=0;
double vd=0;
int i_=0;
stpmax=0;
cidx=0;
vval=0;
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": is not in optimization mode"))
return;
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
cidx=-1;
vval=0;
stpmax=1.0E50;
for(i=0; i<n; i++)
{
if(State.m_cstatus[i]<=0)
{
//--- check
if(!CAp::Assert(!State.m_HasBndL[i] || State.m_xc[i]>=State.m_bndl[i],__FUNCTION__+": internal error - infeasible X"))
return;
if(!CAp::Assert(!State.m_HasBndU[i] || State.m_xc[i]<=State.m_bndu[i],__FUNCTION__+": internal error - infeasible X"))
return;
if(State.m_HasBndL[i] && d[i]<0.0)
{
prevmax=stpmax;
stpmax=CApServ::SafeMinPosRV(State.m_xc[i]-State.m_bndl[i],-d[i],stpmax);
if(stpmax<prevmax)
{
cidx=i;
vval=State.m_bndl[i];
}
}
if(State.m_HasBndU[i] && d[i]>0.0)
{
prevmax=stpmax;
stpmax=CApServ::SafeMinPosRV(State.m_bndu[i]-State.m_xc[i],d[i],stpmax);
if(stpmax<prevmax)
{
cidx=i;
vval=State.m_bndu[i];
}
}
}
}
for(i=nec; i<nec+nic; i++)
{
if(State.m_cstatus[n+i]<=0)
{
vc=0.0;
for(i_=0; i_<n; i_++)
{
vc+=State.m_cleic.Get(i,i_)*State.m_xc[i_];
}
vc=vc-State.m_cleic.Get(i,n);
vd=0.0;
for(i_=0; i_<n; i_++)
vd+=State.m_cleic.Get(i,i_)*d[i_];
if(vd<=0.0)
continue;
if(vc<0.0)
{
//--- XC is strictly feasible with respect to I-th constraint,
//--- we can perform non-zero step because there is non-zero distance
//--- between XC and bound.
prevmax=stpmax;
stpmax=CApServ::SafeMinPosRV(-vc,vd,stpmax);
if(stpmax<prevmax)
cidx=n+i;
}
else
{
//--- XC is at the boundary (or slightly beyond it), and step vector
//--- points beyond the boundary.
//--- The only thing we can do is to perform zero step and activate
//--- I-th constraint.
stpmax=0;
cidx=n+i;
}
}
}
}
//+------------------------------------------------------------------+
//| This subroutine moves current point to XN, which can be: |
//| a) point in the direction previously explored with |
//| SASExploreDirection() function (in this case NeedAct/CIdx/ |
//| CVal are used) |
//| b) point in arbitrary direction, not necessarily previously |
//| checked with SASExploreDirection() function. |
//| Step may activate one constraint. It is assumed than XN is |
//| approximately feasible(small error as large as several ulps |
//| is possible). Strict feasibility with respect to bound |
//| constraints is enforced during activation, feasibility with |
//| respect to general linear constraints is not enforced. |
//| This function activates boundary constraints, such that both is |
//| True: |
//| 1) XC[I] is not at the boundary |
//| 2) XN[I] is at the boundary or beyond it |
//| INPUT PARAMETERS: |
//| S - active set object |
//| XN - new point. |
//| NeedAct - True in case one constraint needs activation |
//| CIdx - index of constraint, in [0, N + NEC + NIC). Ignored|
//| if NeedAct is false. This value is calculated by |
//| SASExploreDirection(). |
//| CVal - for CIdx in [0, N) this field stores value which is|
//| assigned to XC[CIdx] during activation. CVal is |
//| ignored in other cases. This value is calculated by|
//| SASExploreDirection(). |
//| OUTPUT PARAMETERS: |
//| S - current point and list of active constraints are |
//| changed. |
//| RESULT: |
//| > 0, in case at least one inactive non - candidate constraint |
//| was activated |
//| =0, in case only "candidate" constraints were activated |
//| < 0, in case no constraints were activated by the step |
//| NOTE: in general case State.XC<>XN because activation of |
//| constraints may slightly change current point(to enforce |
//| feasibility). |
//+------------------------------------------------------------------+
int CSActiveSets::SASMoveTo(CSActiveSet &State,CRowDouble &xn,
bool needact,int cidx,double cval)
{
//--- create variables
int result=0;
int n=0;
int nec=0;
int nic=0;
int i=0;
bool wasactivation;
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": is not in optimization mode"))
return(result);
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
//--- Save previous State, update current point
State.m_mtx.Resize(n);
State.m_mtas.Resize(n+nec+nic);
for(i=0; i<n; i++)
{
State.m_mtx.Set(i,State.m_xc[i]);
State.m_xc.Set(i,xn[i]);
}
for(i=0; i<n+nec+nic ; i++)
State.m_mtas.Set(i,State.m_cstatus[i]);
//--- Activate constraints
ArrayResize(State.m_mtnew,n+nec+nic);
ArrayInitialize(State.m_mtnew,false);
wasactivation=false;
if(needact)
{
//--- Activation
//--- check
if(!CAp::Assert(cidx>=0 && cidx<n+nec+nic,__FUNCTION__+": incorrect CIdx"))
return(result);
if(cidx<n)
{
//--- CIdx in [0,N-1] means that bound constraint was activated.
//--- We activate it explicitly to avoid situation when roundoff-error
//--- prevents us from moving EXACTLY to x=CVal.
State.m_xc.Set(cidx,cval);
}
State.m_cstatus.Set(cidx,1);
State.m_mtnew[cidx]=true;
wasactivation=true;
}
for(i=0; i<n; i++)
{
//--- Post-check (some constraints may be activated because of numerical errors)
if((State.m_HasBndL[i] && State.m_xc[i]<=State.m_bndl[i]) && State.m_xc[i]!=State.m_mtx[i])
{
State.m_xc.Set(i,State.m_bndl[i]);
State.m_cstatus.Set(i,1);
State.m_mtnew[i]=true;
wasactivation=true;
}
if((State.m_HasBndU[i] && State.m_xc[i]>=State.m_bndu[i]) && State.m_xc[i]!=State.m_mtx[i])
{
State.m_xc.Set(i,State.m_bndu[i]);
State.m_cstatus.Set(i,1);
State.m_mtnew[i]=true;
wasactivation=true;
}
}
//--- Determine return status:
//--- * -1 in case no constraints were activated
//---* 0 in case only "candidate" constraints were activated
//---*+1 in case at least one "non-candidate" constraint was activated
if(wasactivation)
{
//--- Step activated one/several constraints, but sometimes it is spurious
//--- activation - RecalculateConstraints() tells us that constraint is
//--- inactive (negative Largrange multiplier), but step activates it
//--- because of numerical noise.
//--- This block of code checks whether step activated truly new constraints
//--- (ones which were not in the active set at the solution):
//--- * for non-boundary constraint it is enough to check that previous value
//--- of CStatus[i] is negative (=far from boundary), and new one is
//--- positive (=we are at the boundary, constraint is activated).
//--- * for boundary constraints previous criterion won't work. Each variable
//--- has two constraints, and simply checking their status is not enough -
//--- we have to correctly identify cases when we leave one boundary
//--- (PrevActiveSet[i]=0) and move to another boundary (CStatus[i]>0).
//--- Such cases can be identified if we compare previous X with new X.
//--- In case only "candidate" constraints were activated, result variable
//--- is set to 0. In case at least one new constraint was activated, result
//--- is set to 1.
result=0;
for(i=0; i<n; i++)
{
if(State.m_cstatus[i]>0 && State.m_xc[i]!=State.m_mtx[i])
result=1;
}
for(i=n; i<n+State.m_nec+State.m_nic; i++)
{
if(State.m_mtas[i]<0 && State.m_cstatus[i]>0)
result=1;
}
}
else
{
//--- No activation, return -1
result=-1;
}
//--- Update basis
SASAppendToBasis(State,State.m_mtnew);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This subroutine performs immediate activation of one constraint: |
//| *"immediate" means that we do not have to move to activate it |
//| * in case boundary constraint is activated, we enforce current |
//| point to be exactly at the boundary |
//| INPUT PARAMETERS: |
//| S - active set object |
//| CIdx - index of constraint, in [0, N + NEC + NIC). This |
//| value is calculated by SASExploreDirection(). |
//| CVal - for CIdx in [0, N) this field stores value which is|
//| assigned to XC[CIdx] during activation. CVal is |
//| ignored in other cases. This value is calculated |
//| by SASExploreDirection(). |
//+------------------------------------------------------------------+
void CSActiveSets::SASImmediateActivation(CSActiveSet &State,int cidx,
double cval)
{
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": is not in optimization mode"))
return;
if(cidx<State.m_n)
State.m_xc.Set(cidx,cval);
State.m_cstatus.Set(cidx,1);
CApServ::BVectorSetLengthAtLeast(State.m_mtnew,State.m_n+State.m_nec+State.m_nic);
ArrayInitialize(State.m_mtnew,false);
State.m_mtnew[cidx]=true;
SASAppendToBasis(State,State.m_mtnew);
}
//+------------------------------------------------------------------+
//| This subroutine calculates descent direction subject to current |
//| active set. |
//| INPUT PARAMETERS: |
//| S - active set object |
//| G - array[N], gradient |
//| D - possibly prealocated buffer; automatically resized |
//| if needed. |
//| OUTPUT PARAMETERS: |
//| D - descent direction projected onto current active set|
//| Components of D which correspond to active boundary|
//| constraints are forced to be exactly zero. In case |
//| D is non-zero, it is normalized to have unit norm. |
//| NOTE: in case active set has N active constraints (or more), |
//| descent direction is forced to be exactly zero. |
//+------------------------------------------------------------------+
void CSActiveSets::SASConstrainedDescent(CSActiveSet &State,
CRowDouble &g,
CRowDouble &d)
{
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": is not in optimization mode"))
return;
//--- function call
SASRebuildBasis(State);
ConstrainedDescent(State,g,State.m_unitdiagonal,State.m_idensebatch,true,d);
}
//+------------------------------------------------------------------+
//| This subroutine calculates preconditioned descent direction |
//| subject to current active set. |
//| INPUT PARAMETERS: |
//| S - active set object |
//| G - array[N], gradient |
//| D - possibly prealocated buffer; automatically resized |
//| if needed. |
//| OUTPUT PARAMETERS: |
//| D - descent direction projected onto current active set|
//| Components of D which correspond to active boundary|
//| constraints are forced to be exactly zero. In case |
//| D is non-zero, it is normalized to have unit norm. |
//| NOTE: in case active set has N active constraints (or more), |
//| descent direction is forced to be exactly zero. |
//+------------------------------------------------------------------+
void CSActiveSets::SASConstrainedDescentPrec(CSActiveSet &State,
CRowDouble &g,
CRowDouble &d)
{
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": is not in optimization mode"))
return;
//--- function call
SASRebuildBasis(State);
ConstrainedDescent(State,g,State.m_h,State.m_pdensebatch,true,d);
}
//+------------------------------------------------------------------+
//| This subroutine calculates projection of direction vector to |
//| current active set. |
//| INPUT PARAMETERS: |
//| S - active set object |
//| D - array[N], direction |
//| OUTPUT PARAMETERS: |
//| D - direction projected onto current active set. |
//| Components of D which correspond to active boundary|
//| constraints are forced to be exactly zero. |
//| NOTE: in case active set has N active constraints (or more), |
//| descent direction is forced to be exactly zero. |
//+------------------------------------------------------------------+
void CSActiveSets::SASConstrainedDirection(CSActiveSet &State,
CRowDouble &d)
{
int i=0;
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": is not in optimization mode"))
return;
//--- function call
SASRebuildBasis(State);
ConstrainedDescent(State,d,State.m_unitdiagonal,State.m_idensebatch,false,State.m_cdtmp);
//--- copy
d=State.m_cdtmp;
d*=(-1.0);
d.Resize(State.m_n);
}
//+------------------------------------------------------------------+
//| This subroutine calculates product of direction vector and |
//| preconditioner multiplied subject to current active set. |
//| INPUT PARAMETERS: |
//| S - active set object |
//| D - array[N], direction |
//| OUTPUT PARAMETERS: |
//| D - preconditioned direction projected onto current |
//| active set. Components of D which correspond to |
//| active boundary constraints are forced to be |
//| exactly zero. |
//| NOTE: in case active set has N active constraints (or more), |
//| descent direction is forced to be exactly zero. |
//+------------------------------------------------------------------+
void CSActiveSets::SASConstrainedDirectionPrec(CSActiveSet &State,
CRowDouble &d)
{
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": is not in optimization mode"))
return;
//--- function call
SASRebuildBasis(State);
ConstrainedDescent(State,d,State.m_h,State.m_pdensebatch,false,State.m_cdtmp);
//--- copy
d=State.m_cdtmp;
d*=(-1.0);
d.Resize(State.m_n);
}
//+------------------------------------------------------------------+
//| This subroutine performs correction of some (possibly infeasible)|
//| point with respect to a) current active set, b) all boundary |
//| constraints, both active and inactive: |
//| 0) we calculate L1 penalty term for violation of active linear |
//| constraints (one which is returned by SASActiveLCPenalty1() |
//| function). |
//| 1) first, it performs projection(orthogonal with respect |
//| to scale matrix S) of X into current active set: X -> X1. |
//| 2) next, we perform projection with respect to ALL boundary |
//| constraints which are violated at X1: X1 -> X2. |
//| 3) X is replaced by X2. |
//| The idea is that this function can preserve and enforce |
//| feasibility during optimization, and additional penalty parameter|
//| can be used to prevent algo from leaving feasible set because of |
//| rounding errors. |
//| INPUT PARAMETERS: |
//| S - active set object |
//| X - array[N], candidate point |
//| OUTPUT PARAMETERS: |
//| X - "improved" candidate point: |
//| a) feasible with respect to all boundary constraints |
//| b) feasibility with respect to active set is retained |
//| at good level. |
//| Penalty - penalty term, which can be added to function value |
//| if user wants to penalize violation of constraints |
//| (recommended). |
//| NOTE: this function is not intended to find exact projection |
//| (i.e. best approximation) of X into feasible set. It just |
//| improves situation a bit. |
//| Regular use of this function will help you to retain feasibility |
//| - if you already have something to start with and constrain your |
//| steps is such way that the only source of infeasibility are |
//| roundoff errors. |
//+------------------------------------------------------------------+
void CSActiveSets::SASCorrection(CSActiveSet &State,CRowDouble &x,
double &penalty)
{
//--- create variables
int i=0;
int j=0;
int n=0;
double v=0;
int i_=0;
penalty=0;
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": is not in optimization mode"))
return;
//--- function call
SASRebuildBasis(State);
n=State.m_n;
//--- Calculate penalty term.
penalty=SASActiveLCPenalty1(State,x);
//--- Perform projection 1.
//--- This projecton is given by:
//--- x_proj = x - S*S*As'*(As*x-b)
//--- where x is original x before projection, S is a scale matrix,
//--- As is a matrix of equality constraints (active set) which were
//--- orthogonalized with respect to inner product given by S (i.e. we
//--- have As*S*S'*As'=I), b is a right part of the orthogonalized
//--- constraints.
//--- NOTE: you can verify that x_proj is strictly feasible w.m_r.m_t.
//--- active set by multiplying it by As - you will get
//--- As*x_proj = As*x - As*x + b = b.
//--- This formula for projection can be obtained by solving
//--- following minimization problem.
//--- min ||inv(S)*(x_proj-x)||^2 s.m_t. As*x_proj=b
//--- NOTE: we apply sparse batch by examining CStatus[]; it is guaranteed
//--- to contain sparse batch, but avoids roundoff errors associated
//--- with the fact that some box constraints were moved to sparse
//--- storage
State.m_corrtmp=x;
State.m_corrtmp.Resize(n);
for(i=0; i<State.m_densebatchsize; i++)
{
v=-State.m_sdensebatch.Get(i,n);
for(j=0; j<n; j++)
v+= State.m_sdensebatch.Get(i,j)*State.m_corrtmp[j];
for(j=0; j<n; j++)
State.m_corrtmp.Add(j,- v*State.m_sdensebatch.Get(i,j)*CMath::Sqr(State.m_s[j]));
}
for(i=0; i<n; i++)
{
if(State.m_cstatus[i]>0)
State.m_corrtmp.Set(i,State.m_xc[i]);
}
//--- Perform projection 2
for(i=0; i<n; i++)
{
x.Set(i,State.m_corrtmp[i]);
if(State.m_HasBndL[i] && x[i]<State.m_bndl[i])
x.Set(i,State.m_bndl[i]);
if(State.m_HasBndU[i] && x[i]>State.m_bndu[i])
x.Set(i,State.m_bndu[i]);
}
}
//+------------------------------------------------------------------+
//| This subroutine returns L1 penalty for violation of active |
//| general linear constraints(violation of boundary or inactive |
//| linear constraints is not added to penalty). |
//| Penalty term is equal to: |
//| Penalty = SUM(Abs((C_i * x - R_i) / Alpha_i)) |
//| Here: |
//| * summation is performed for I = 0...NEC + NIC-1, |
//| CStatus[N+I] > 0 (only for rows of CLEIC which are in active |
//| set) |
//| * C_i is I-Th row of CLEIC |
//| * R_i is corresponding right part |
//| * S is a scale matrix |
//| * Alpha_i = || S * C_i || - is a scaling coefficient which |
//| "normalizes" I-Th summation term according to its scale. |
//| INPUT PARAMETERS: |
//| S - active set object |
//| X - array[N], candidate point |
//+------------------------------------------------------------------+
double CSActiveSets::SASActiveLCPenalty1(CSActiveSet &State,
CRowDouble &x)
{
//--- create variables
double result=0;
int i=0;
int j=0;
int n=0;
int nec=0;
int nic=0;
double v=0;
double alpha=0;
double p=0;
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": is not in optimization mode"))
return(result);
//--- function call
SASRebuildBasis(State);
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
//--- Calculate penalty term.
result=0;
for(i=0; i<nec+nic; i++)
{
if(State.m_cstatus[n+i]>0)
{
alpha=0;
p=-State.m_cleic.Get(i,n);
for(j=0; j<n; j++)
{
v=State.m_cleic.Get(i,j);
p=p+v*x[j];
alpha+=CMath::Sqr(v*State.m_s[j]);
}
alpha=MathSqrt(alpha);
if(alpha!=0.0)
result+=MathAbs(p/alpha);
}
}
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This subroutine calculates scaled norm of vector after projection|
//| onto subspace of active constraints. Most often this function is |
//| used to test stopping conditions. |
//| INPUT PARAMETERS: |
//| S - active set object |
//| D - vector whose norm is calculated |
//| RESULT: |
//| Vector norm (after projection and scaling) |
//| NOTE: projection is performed first, scaling is performed after |
//| projection |
//| NOTE: if we have N active constraints, zero value(exact zero) |
//| is returned |
//+------------------------------------------------------------------+
double CSActiveSets::SASScaledConstrainedNorm(CSActiveSet &State,
CRowDouble &d)
{
//--- create variables
double result=0;
int i=0;
int n=0;
double v=0;
int i_=0;
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": is not in optimization mode"))
return(result);
n=State.m_n;
//--- Prepare basis (if needed)
SASRebuildBasis(State);
//--- Calculate descent direction
if(State.m_sparsebatchsize+State.m_densebatchsize>=n)
{
//--- Quick exit if number of active constraints is N or larger
result=0.0;
return(result);
}
State.m_scntmp=d;
State.m_scntmp.Resize(n);
for(i=0; i<=State.m_densebatchsize-1; i++)
{
v=0.0;
for(i_=0; i_<n; i_++)
v+=State.m_idensebatch.Get(i,i_)*State.m_scntmp[i_];
for(i_=0; i_<n; i_++)
State.m_scntmp.Add(i_,- v*State.m_idensebatch.Get(i,i_));
}
for(i=0; i<n; i++)
{
if(State.m_cstatus[i]>0)
State.m_scntmp.Set(i,0);
}
v=0.0;
for(i=0; i<n; i++)
v+= CMath::Sqr(State.m_s[i]*State.m_scntmp[i]);
result=MathSqrt(v);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This subroutine turns off optimization mode. |
//| INPUT PARAMETERS: |
//| S - active set object |
//| OUTPUT PARAMETERS: |
//| S - State is changed |
//| NOTE: this function can be called many times for optimizer which |
//| was already stopped. |
//+------------------------------------------------------------------+
void CSActiveSets::SASStopOptimization(CSActiveSet &State)
{
State.m_algostate=0;
}
//+------------------------------------------------------------------+
//| This function recalculates constraints - activates and |
//| deactivates them according to gradient value at current point. |
//| Algorithm assumes that we want to make steepest descent step from|
//| current point; constraints are activated and deactivated in such |
//| way that we won't violate any constraint by steepest descent step|
//| After call to this function active set is ready to try steepest |
//| descent step (SASDescentDirection-SASExploreDirection-SASMoveTo).|
//| Only already "active" and "candidate" elements of ActiveSet are |
//| examined; constraints which are not active are not examined. |
//| INPUT PARAMETERS: |
//| State - active set object |
//| GC - array[N], gradient at XC |
//| OUTPUT PARAMETERS: |
//| State - active set object, with new set of constraint |
//+------------------------------------------------------------------+
void CSActiveSets::SASReactivateConstraints(CSActiveSet &State,
CRowDouble &gc)
{
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": must be in optimization mode"))
return;
//--- function call
ReactivateConstraints(State,gc,State.m_unitdiagonal);
}
//+------------------------------------------------------------------+
//| This function recalculates constraints - activates and |
//| deactivates them according to gradient value at current point. |
//| Algorithm assumes that we want to make Quasi - Newton step from |
//| current point with diagonal Quasi - Newton matrix H. Constraints |
//| are activated and deactivated in such way that we won't violate |
//| any constraint by step. |
//| After call to this function active set is ready to try |
//| preconditioned steepest descent step (SASDescentDirection - |
//| SASExploreDirection - SASMoveTo). |
//| Only already "active" and "candidate" elements of ActiveSet are |
//| examined; constraints which are not active are not examined. |
//| INPUT PARAMETERS: |
//| State - active set object |
//| GC - array[N], gradient at XC |
//| OUTPUT PARAMETERS: |
//| State - active set object, with new set of constraint |
//+------------------------------------------------------------------+
void CSActiveSets::SASReactivateConstraintsPrec(CSActiveSet &State,
CRowDouble &gc)
{
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": must be in optimization mode"))
return;
//--- function call
ReactivateConstraints(State,gc,State.m_h);
}
//+------------------------------------------------------------------+
//| This function builds three orthonormal basises for current active|
//| set: |
//| * P - orthogonal one, which is orthogonalized with inner |
//| product (x, y) = x'*P*y, where P=inv(H) is current |
//| preconditioner |
//| * S - orthogonal one, which is orthogonalized with inner |
//| product (x, y) = x'*S'*S * y, where S is diagonal |
//| scaling matrix |
//| * I - orthogonal one, which is orthogonalized with standard |
//| dot product |
//| NOTE: all sets of orthogonal vectors are guaranteed to have |
//| same size. P - orthogonal basis is built first, |
//| I / S - orthogonal basises are forced to have same number |
//| of vectors as P - orthogonal one(padded by zero vectors |
//| if needed). |
//| NOTE: this function tracks changes in active set; first call |
//| will result in reorthogonalization |
//| INPUT PARAMETERS: |
//| State - active set object |
//| H - diagonal preconditioner, H[i] > 0 |
//| OUTPUT PARAMETERS: |
//| State - active set object with new basis |
//+------------------------------------------------------------------+
void CSActiveSets::SASRebuildBasis(CSActiveSet &State)
{
//--- create variables
int n=0;
int nec=0;
int nic=0;
int i=0;
int j=0;
bool hasactivelin=false;
int candidatescnt=0;
double v=0;
double vv=0;
double vmax=0;
int kmax=0;
int i_=0;
if(State.m_basisisready)
return;
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
State.m_tmp0.Resize(n);
State.m_tmpprodp.Resize(n);
State.m_tmpprods.Resize(n);
State.m_tmpcp.Resize(n+1);
State.m_tmpcs.Resize(n+1);
State.m_tmpci.Resize(n+1);
State.m_tmpbasis.Resize(nec+nic,n+1);
State.m_pdensebatch.Resize(nec+nic,n+1);
State.m_idensebatch.Resize(nec+nic,n+1);
State.m_sdensebatch.Resize(nec+nic,n+1);
State.m_sparsebatch.Resize(n);
State.m_sparsebatchsize=0;
State.m_densebatchsize=0;
State.m_basisage=0;
State.m_basisisready=true;
//--- Determine number of active boundary and non-boundary
//--- constraints, move them to TmpBasis. Quick exit if no
//--- non-boundary constraints were detected.
hasactivelin=false;
for(i=0; i<nec+nic; i++)
{
if(State.m_cstatus[n+i]>0)
hasactivelin=true;
}
for(j=0; j<n; j++)
{
if(State.m_cstatus[j]>0)
{
State.m_sparsebatch.Set(State.m_sparsebatchsize,j);
State.m_sparsebatchsize++;
}
}
if(!hasactivelin)
return;
//--- Prepare precomputed values
State.m_tmpreciph=State.m_h.Pow(-1.0)+0;
State.m_tmpreciph.Resize(n);
//--- Prepare initial candidate set:
//--- * select active constraints
//--- * normalize (inner product is given by preconditioner)
//--- * orthogonalize with respect to active box constraints
//--- * copy normalized/orthogonalized candidates to PBasis/SBasis/IBasis
candidatescnt=0;
for(i=0; i<nec+nic; i++)
{
if(State.m_cstatus[n+i]>0)
{
State.m_tmpbasis.Row(candidatescnt,State.m_cleic[i]+0);
candidatescnt++;
}
}
for(i=0; i<candidatescnt; i++)
{
v=0.0;
for(j=0; j<n; j++)
v+=CMath::Sqr(State.m_tmpbasis.Get(i,j))*State.m_tmpreciph[j];
if(v>0.0)
{
v=1/MathSqrt(v);
State.m_tmpbasis.Row(i,State.m_tmpbasis[i]*v);
}
}
for(j=0; j<n; j++)
{
if(State.m_cstatus[j]>0)
{
for(i=0; i<candidatescnt; i++)
{
State.m_tmpbasis.Add(i,n,-State.m_tmpbasis.Get(i,j)*State.m_xc[j]);
State.m_tmpbasis.Set(i,j,0.0);
}
}
}
for(i=0; i<candidatescnt; i++)
State.m_pdensebatch.Row(i,State.m_tmpbasis[i]+0);
State.m_sdensebatch=State.m_pdensebatch;
State.m_idensebatch=State.m_pdensebatch;
//--- Perform orthogonalization of general linear constraints with respect
//--- to each other (constraints in P/S/IBasis are already normalized w.m_r.m_t.
//--- box constraints). During this process we select strictly active constraints
//--- from the candidate set, and drop ones which were detected as redundant
//--- during orthogonalization.
//--- Orthogonalization is performed with the help of Gram-Schmidt process.
//--- Due to accumulation of round-off errors it is beneficial to perform
//--- pivoting, i.e. to select candidate vector with largest norm at each
//--- step.
//--- First (basic) version of the algorithm is:
//--- 0. split all constraints into two sets: basis ones (initially empty)
//--- and candidate ones (all constraints)
//--- 1. fill PBasis with H-normalized candidate constraints, fill
//--- corresponding entries of S/IBasis with corresponding
//--- (non-normalized) constraints
//--- 2. select row of PBasis with largest norm, move it (and its S/IBasis
//--- counterparts) to the beginning of the candidate set, H-normalize
//--- this row (rows of S/IBasis are normalized using corresponding norms).
//--- Stop if largest row is nearly (or exactly) zero.
//--- 3. orthogonalize remaining rows of P/S/IBasis with respect to
//--- one chosen at step (2). It can be done efficiently using
//--- combination of DGEMV/DGER BLAS calls.
//--- 4. increase basis size by one, decrease candidate set size by one,
//--- goto (2)
//--- However, naive implementation of the algorithm above spends significant
//--- amount of time in step (2) - selection of row with largest H-norm. Step
//--- (3) can be efficiently implemented with optimized BLAS, but we have no
//--- optimized BLAS kernels for step(2). And because step (3) changes row norms,
//--- step (2) have to be re-calculated every time, which is quite slow.
//--- We can save significant amount of calculations by noticing that:
//--- * step (3) DECREASES row norms, but never increases it
//--- * we can maintain upper bounds for row H-norms is a separate array,
//--- use them for initial evaluation of best candidates, and update them
//--- after we find some promising row (all bounds are invalidated after
//--- step 3, but their old values still carry some information)
//--- * it is beneficial re-evaluate bounds only for rows which are
//--- significantly (at least few percents) larger than best one found so far
//--- * because rows are initially normalized, initial values for upper bounds
//--- can be set to 1.0
if(!CAp::Assert(State.m_densebatchsize==0,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(m_minnormseparation>0.0,__FUNCTION__+": integrity check failed"))
return;
State.m_tmpnormestimates=vector<double>::Ones(candidatescnt);
while(State.m_sparsebatchsize+State.m_densebatchsize<n)
{
//--- No candidates? We are done!
if(candidatescnt==0)
break;
//--- Find largest vector
vmax=0;
kmax=-1;
for(i=State.m_densebatchsize; i<State.m_densebatchsize+candidatescnt; i++)
{
//--- Use upper bound for row norm for initial evaluation.
//--- Skip rows whose upper bound is less than best candidate
//--- found so far.
//--- NOTE: in fact, we may skip rows whose upper bound is
//--- marginally higher than that of best candidate.
//--- No need to perform costly re-evaluation in order
//--- to get just few percents of improvement.
if(State.m_tmpnormestimates[i]<(vmax*(1+m_minnormseparation)))
continue;
//--- OK, upper bound is large enough... lets perform full
//--- re-evaluation and update of the estimate.
v=0.0;
for(j=0; j<n; j++)
{
vv=State.m_pdensebatch.Get(i,j);
v+= vv*vv*State.m_tmpreciph[j];
}
v=MathSqrt(v);
State.m_tmpnormestimates.Set(i,v);
//--- Now compare with best candidate so far
if(v>vmax)
{
vmax=v;
kmax=i;
}
}
if(vmax<(1.0E4*CMath::m_machineepsilon) || kmax<0)
{
//--- All candidates are either zero or too small (after orthogonalization)
candidatescnt=0;
break;
}
//--- Candidate is selected for inclusion into basis set.
//--- Move candidate row to the beginning of candidate array (which is
//--- right past the end of the approved basis). Normalize (for P-basis
//--- we perform preconditioner-based normalization, for S-basis - scale
//--- based, for I-basis - identity based).
State.m_pdensebatch.SwapRows(State.m_densebatchsize,kmax);
State.m_sdensebatch.SwapRows(State.m_densebatchsize,kmax);
State.m_idensebatch.SwapRows(State.m_densebatchsize,kmax);
State.m_tmpnormestimates.Swap(State.m_densebatchsize,kmax);
v=1/vmax;
State.m_pdensebatch.Row(State.m_densebatchsize,State.m_pdensebatch[State.m_densebatchsize]*v);
v=0;
for(j=0; j<n; j++)
{
vv=State.m_sdensebatch.Get(State.m_densebatchsize,j)*State.m_s[j];
v+= vv*vv;
}
//--- check
if(!CAp::Assert(v>0.0,__FUNCTION__+": integrity check failed,SNorm=0"))
return;
v=1/MathSqrt(v);
State.m_sdensebatch.Row(State.m_densebatchsize,State.m_sdensebatch[State.m_densebatchsize]*v);
v=0;
for(j=0; j<n; j++)
{
vv=State.m_idensebatch.Get(State.m_densebatchsize,j);
v+= vv*vv;
}
//--- check
if(!CAp::Assert(v>0.0,__FUNCTION__+": integrity check failed,INorm=0"))
return;
v=1/MathSqrt(v);
State.m_idensebatch.Row(State.m_densebatchsize,State.m_idensebatch[State.m_densebatchsize]*v);
//--- Reorthogonalize other candidates with respect to candidate #0:
//--- * calculate projections en masse with GEMV()
//--- * subtract projections with GER()
State.m_tmp0.Resize(candidatescnt-1);
for(j=0; j<n; j++)
{
State.m_tmpprodp.Set(j,State.m_pdensebatch.Get(State.m_densebatchsize,j)/State.m_h[j]);
State.m_tmpprods.Set(j,State.m_sdensebatch.Get(State.m_densebatchsize,j)*CMath::Sqr(State.m_s[j]));
}
State.m_tmpcp=State.m_pdensebatch[State.m_densebatchsize]+0;
State.m_tmpcs=State.m_sdensebatch[State.m_densebatchsize]+0;
State.m_tmpci=State.m_idensebatch[State.m_densebatchsize]+0;
CAblas::RMatrixGemVect(candidatescnt-1,n,1.0,State.m_pdensebatch,State.m_densebatchsize+1,0,0,State.m_tmpprodp,0,0.0,State.m_tmp0,0);
CAblas::RMatrixGer(candidatescnt-1,n+1,State.m_pdensebatch,State.m_densebatchsize+1,0,-1.0,State.m_tmp0,0,State.m_tmpcp,0);
CAblas::RMatrixGemVect(candidatescnt-1,n,1.0,State.m_sdensebatch,State.m_densebatchsize+1,0,0,State.m_tmpprods,0,0.0,State.m_tmp0,0);
CAblas::RMatrixGer(candidatescnt-1,n+1,State.m_sdensebatch,State.m_densebatchsize+1,0,-1.0,State.m_tmp0,0,State.m_tmpcs,0);
CAblas::RMatrixGemVect(candidatescnt-1,n,1.0,State.m_idensebatch,State.m_densebatchsize+1,0,0,State.m_tmpci,0,0.0,State.m_tmp0,0);
CAblas::RMatrixGer(candidatescnt-1,n+1,State.m_idensebatch,State.m_densebatchsize+1,0,-1.0,State.m_tmp0,0,State.m_tmpci,0);
//--- Increase basis, decrease candidates count
State.m_densebatchsize++;
candidatescnt--;
}
}
//+------------------------------------------------------------------+
//| This function appends new constraints(if possible; sometimes it |
//| isn't!) to three orthonormal basises for current active set: |
//| * P - orthogonal one, which is orthogonalized with inner |
//| product (x, y) = x'*P*y, where P=inv(H) is current |
//| preconditioner |
//| * S - orthogonal one, which is orthogonalized with inner |
//| product (x, y) = x'*S'*S*y, where S is diagonal scaling |
//| matrix |
//| * I - orthogonal one, which is orthogonalized with standard |
//| dot product |
//| NOTE: all sets of orthogonal vectors are guaranteed to have same |
//| size. P - orthogonal basis is built first, I/S - orthogonal|
//| basises are forced to have same number of vectors as |
//| P - orthogonal one(padded by zero vectors if needed). |
//| NOTE: this function may fail to update basis without full |
//| recalculation; in such case it will set BasisIsReady to |
//| False and silently return; if it succeeds, it will increase|
//| BasisSize. |
//| INPUT PARAMETERS: |
//| State - active set object |
//| NewEntries - array[N + NEC + NIC], indexes of constraints |
//| being added are marked as True; it is |
//| responsibility of the caller to specify only |
//| those constraints which were previously |
//| inactive; when some constraint is already in the|
//| active set, algorithm behavior is undefined. |
//| OUTPUT PARAMETERS: |
//| State - active set object with new basis |
//+------------------------------------------------------------------+
void CSActiveSets::SASAppendToBasis(CSActiveSet &State,
bool &newentries[])
{
//--- create variables
int n=0;
int nec=0;
int nic=0;
int i=0;
int j=0;
int t=0;
int nact=0;
double v=0;
double vp=0;
double vs=0;
double vi=0;
double initnormp=0;
double projnormp=0;
double projnorms=0;
double projnormi=0;
int i_=0;
//--- check
if(!State.m_basisisready)
return;
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
//--- Count number of constraints to activate;
//--- perform integrity check.
nact=0;
for(i=0; i<n; i++)
if(newentries[i])
nact++;
for(i=n; i<n+nec; i++)
{
//--- check
if(!CAp::Assert(!newentries[i],__FUNCTION__+": integrity check failed (appendtobasis.0)"))
return;
}
for(i=n+nec; i<n+nec+nic; i++)
{
if(newentries[i])
nact++;
}
if(nact+State.m_basisage>m_maxbasisage)
{
State.m_basisisready=false;
return;
}
//--- Resize basis matrices if needed
State.m_pdensebatch.Resize(State.m_densebatchsize+nact,n+1);
State.m_sdensebatch.Resize(State.m_densebatchsize+nact,n+1);
State.m_idensebatch.Resize(State.m_densebatchsize+nact,n+1);
//--- Try adding recommended entries to basis.
//--- If reorthogonalization removes too much of candidate constraint,
//--- we will invalidate basis and try to rebuild it from scratch.
State.m_tmp0.Resize(n+1);
State.m_tmpcp.Resize(n+1);
State.m_tmpcs.Resize(n+1);
State.m_tmpci.Resize(n+1);
State.m_tmpprodp.Resize(n);
State.m_tmpprods.Resize(n);
for(t=0; t<n+nec+nic; t++)
{
if(newentries[t])
{
//--- Basis is full? Quick skip!
if(State.m_sparsebatchsize+State.m_densebatchsize>=n)
{
//--- check
if(!CAp::Assert(State.m_sparsebatchsize+State.m_densebatchsize==n,__FUNCTION__+": integrity check failed (SASAppendToBasis)"))
return;
break;
}
//--- Copy constraint to temporary storage.
if(t<n)
{
//--- Copy box constraint
State.m_tmp0=vector<double>::Zeros(n+1);
State.m_tmp0.Set(t,1.0);
State.m_tmp0.Set(n,State.m_xc[t]);
}
else
{
//--- Copy general linear constraint
State.m_tmp0=State.m_cleic[t-n]+0;
}
//--- Calculate initial norm (preconditioner is used for norm calculation).
initnormp=0.0;
for(j=0; j<n; j++)
{
v=State.m_tmp0[j];
initnormp=initnormp+v*v/State.m_h[j];
}
initnormp=MathSqrt(initnormp);
if(initnormp==0.0)
{
//--- Well, it is not expected. Let's just rebuild basis
//--- from scratch and forget about this strange situation...
State.m_basisisready=false;
return;
}
//--- Orthogonalize Tmp0 w.m_r.m_t. sparse batch (box constraints stored in sparse storage).
//--- Copy to TmpCP/TmpCS/TmpCI (P for preconditioner-based inner product
//--- used for orthogonalization, S for scale-based orthogonalization,
//--- I for "traditional" inner product used for Gram-Schmidt orthogonalization).
for(i=0; i<State.m_sparsebatchsize; i++)
{
j=State.m_sparsebatch[i];
State.m_tmp0.Add(n,- State.m_tmp0[j]*State.m_xc[j]);
State.m_tmp0.Set(j,0.0);
}
State.m_tmpcp=State.m_tmp0;
State.m_tmpcs=State.m_tmp0;
State.m_tmpci=State.m_tmp0;
//--- Orthogonalize TmpCP/S/I with respect to active linear constraints from dense batch.
//--- Corresponding norm (preconditioner, scale, identity) is used in each case.
State.m_tmpprodp=State.m_h.Pow(-1.0)+0;
State.m_tmpprods=State.m_s.Pow(2)+0;
for(i=0; i<State.m_densebatchsize; i++)
{
vp=0;
vs=0;
vi=0;
for(j=0; j<n; j++)
{
vp+=State.m_pdensebatch.Get(i,j)*State.m_tmpcp[j]*State.m_tmpprodp[j];
vs+=State.m_sdensebatch.Get(i,j)*State.m_tmpcs[j]*State.m_tmpprods[j];
vi+=State.m_idensebatch.Get(i,j)*State.m_tmpci[j];
}
State.m_tmpcp-=State.m_pdensebatch[i]*vp;
State.m_tmpcs-=State.m_sdensebatch[i]*vs;
State.m_tmpci-=State.m_idensebatch[i]*vi;
}
projnormp=0.0;
projnorms=0.0;
projnormi=0.0;
for(j=0; j<n; j++)
{
projnormp+=CMath::Sqr(State.m_tmpcp[j])/State.m_h[j];
projnorms+=CMath::Sqr(State.m_tmpcs[j])*CMath::Sqr(State.m_s[j]);
projnormi+=CMath::Sqr(State.m_tmpci[j]);
}
projnormp=MathSqrt(projnormp);
projnorms=MathSqrt(projnorms);
projnormi=MathSqrt(projnormi);
if(projnormp<=(m_maxbasisdecay*initnormp))
{
State.m_basisisready=false;
return;
//--- Nearly zero row, skip
}
//--- check
if(!CAp::Assert(projnormp>0.0,__FUNCTION__+": integrity check failed,ProjNormP=0"))
return;
if(!CAp::Assert(projnorms>0.0,__FUNCTION__+": integrity check failed,ProjNormS=0"))
return;
if(!CAp::Assert(projnormi>0.0,__FUNCTION__+": integrity check failed,ProjNormI=0"))
return;
v=1/projnormp;
State.m_pdensebatch.Row(State.m_densebatchsize,State.m_tmpcp*v+0);
v=1/projnorms;
State.m_sdensebatch.Row(State.m_densebatchsize,State.m_tmpcs*v+0);
v=1/projnormi;
State.m_idensebatch.Row(State.m_densebatchsize,State.m_tmpci*v+0);
//--- Increase set size
State.m_densebatchsize++;
State.m_basisage++;
}
}
}
//+------------------------------------------------------------------+
//| This subroutine calculates preconditioned descent direction |
//| subject to current active set. |
//| INPUT PARAMETERS: |
//| State - active set object |
//| G - array[N], gradient |
//| H - array[N], Hessian matrix |
//| HA - active constraints orthogonalized in such way that |
//| HA*inv(H)*HA'= I. |
//| Normalize- whether we need normalized descent or not |
//| D - possibly preallocated buffer; automatically resized|
//| OUTPUT PARAMETERS: |
//| D - descent direction projected onto current active set|
//| Components of D which correspond to active boundary|
//| constraints are forced to be exactly zero. In case |
//| D is non-zero and Normalize is True, it is |
//| normalized to have unit norm. |
//| NOTE: if we have N active constraints, D is explicitly set to |
//| zero. |
//+------------------------------------------------------------------+
void CSActiveSets::ConstrainedDescent(CSActiveSet &State,
CRowDouble &g,
CRowDouble &h,
CMatrixDouble &ha,
bool normalize,
CRowDouble &d)
{
//--- create variables
int i=0;
int n=0;
double v=0;
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": internal error in ConstrainedDescent() - not in optimization mode"))
return;
if(!CAp::Assert(State.m_basisisready,__FUNCTION__+": internal error in ConstrainedDescent() - no basis"))
return;
n=State.m_n;
//--- Calculate preconditioned constrained descent direction:
//--- d := -inv(H)*( g - HA'*(HA*inv(H)*g) )
//--- Formula above always gives direction which is orthogonal to rows of HA.
//--- You can verify it by multiplication of both sides by HA[i] (I-th row),
//--- taking into account that HA*inv(H)*HA'= I (by definition of HA - it is
//--- orthogonal basis with inner product given by inv(H)).
d=g;
d.Resize(n);
for(i=0; i<State.m_densebatchsize; i++)
{
vector<double> row=ha[i];
row.Resize(n);
v=d.Dot(row/h.ToVector());
d-=row*v;
}
for(i=0; i<n; i++)
{
if(State.m_cstatus[i]>0)
d.Set(i,0);
}
d/=h.ToVector()*(-1);
v=MathSqrt(CAblasF::RDotV2(n,d));
if(State.m_sparsebatchsize+State.m_densebatchsize>=n)
{
v=0;
d=vector<double>::Zeros(n);
}
if(normalize && v>0.0)
d/=v;
}
//+------------------------------------------------------------------+
//| This function recalculates constraints - activates and |
//| deactivates them according to gradient value at current point. |
//| Algorithm assumes that we want to make Quasi - Newton step |
//| from current point with diagonal Quasi - Newton matrix H. |
//| Constraints are activated and deactivated in such way that we |
//| won't violate any constraint by step. |
//| Only already "active" and "candidate" elements of ActiveSet are |
//| examined; constraints which are not active are not examined. |
//| INPUT PARAMETERS: |
//| State - active set object |
//| GC - array[N], gradient at XC |
//| H - array[N], Hessian matrix |
//| OUTPUT PARAMETERS: |
//| State - active set object, with new set of constraint |
//+------------------------------------------------------------------+
void CSActiveSets::ReactivateConstraints(CSActiveSet &State,
CRowDouble &gc,
CRowDouble &h)
{
//--- return result
int n=0;
int nec=0;
int nic=0;
int i=0;
int j=0;
int idx0=0;
int idx1=0;
double v=0;
int nactivebnd=0;
int nactivelin=0;
int nactiveconstraints=0;
double rowscale=0;
int i_=0;
//--- check
if(!CAp::Assert(State.m_algostate==1,__FUNCTION__+": must be in optimization mode"))
return;
//--- Prepare
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
State.m_basisisready=false;
//--- Handle important special case - no linear constraints,
//--- only boundary constraints are present
if(nec+nic==0)
{
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i] && State.m_HasBndU[i] && State.m_bndl[i]==State.m_bndu[i])
{
State.m_cstatus.Set(i,1);
continue;
}
if(State.m_HasBndL[i] && State.m_xc[i]==State.m_bndl[i] && gc[i]>=0.0)
{
State.m_cstatus.Set(i,1);
continue;
}
if(State.m_HasBndU[i] && State.m_xc[i]==State.m_bndu[i] && gc[i]<=0.0)
{
State.m_cstatus.Set(i,1);
continue;
}
State.m_cstatus.Set(i,-1);
}
return;
}
//--- General case.
//--- Calculate descent direction
State.m_rctmpg=gc*(-1.0)+0;
//--- Allocate temporaries.
State.m_rctmpg.Resize(n);
State.m_rctmprightpart.Resize(n);
State.m_rctmps.Resize(n);
State.m_rctmpdense0.Resize(n,nec+nic);
State.m_rctmpdense1.Resize(n,nec+nic);
State.m_rctmpconstraintidx.Resize(n+nec+nic);
CApServ::BVectorSetLengthAtLeast(State.m_rctmpisequality,n+nec+nic);
//--- Determine candidates to the active set.
//--- After this block constraints become either "inactive" (CStatus[i]<0)
//--- or "candidates" (CStatus[i]=0). Previously active constraints always
//--- become "candidates".
State.m_cstatus.Fill(-1,0,n);
for(i=n; i<=n+nec+nic-1; i++)
{
if(State.m_cstatus[i]>0)
State.m_cstatus.Set(i,0);
else
State.m_cstatus.Set(i,-1);
}
nactiveconstraints=0;
nactivebnd=0;
nactivelin=0;
for(i=0; i<n; i++)
{
//--- Activate boundary constraints:
//--- * copy constraint index to RCTmpConstraintIdx
//---*set corresponding element of CStatus[] to "candidate"
//--- * fill RCTmpS by either +1 (lower bound) or -1 (upper bound)
//--- * set RCTmpIsEquality to False (BndL<BndU) or True (BndL=BndU)
//--- * increase counters
if(State.m_HasBndL[i] && State.m_HasBndU[i] && State.m_bndl[i]==State.m_bndu[i])
{
//--- Equality constraint is activated
State.m_rctmpconstraintidx.Set(nactiveconstraints,i);
State.m_cstatus.Set(i,0);
State.m_rctmps.Set(i,1.0);
State.m_rctmpisequality[nactiveconstraints]=true;
nactiveconstraints++;
nactivebnd++;
continue;
}
if(State.m_HasBndL[i] && State.m_xc[i]==State.m_bndl[i])
{
//--- Lower bound is activated
State.m_rctmpconstraintidx.Set(nactiveconstraints,i);
State.m_cstatus.Set(i,0);
State.m_rctmps.Set(i,-1.0);
State.m_rctmpisequality[nactiveconstraints]=false;
nactiveconstraints++;
nactivebnd++;
continue;
}
if(State.m_HasBndU[i] && State.m_xc[i]==State.m_bndu[i])
{
//--- Upper bound is activated
State.m_cstatus.Set(i,0);
State.m_rctmpconstraintidx.Set(nactiveconstraints,i);
State.m_rctmps.Set(i,1.0);
State.m_rctmpisequality[nactiveconstraints]=false;
nactiveconstraints++;
nactivebnd++;
continue;
}
}
for(i=0; i<nec+nic; i++)
{
if(i>=nec && State.m_cstatus[n+i]<0)
{
//--- Inequality constraints are skipped if both (a) constraint was
//--- not active, and (b) we are too far away from the boundary.
rowscale=0.0;
v=-State.m_cleic.Get(i,n);
for(j=0; j<n; j++)
{
v+= State.m_cleic.Get(i,j)*State.m_xc[j];
rowscale=MathMax(rowscale,MathAbs(State.m_cleic.Get(i,j)*State.m_s[j]));
}
if(v<=(-1.0E5*CMath::m_machineepsilon*rowscale))
{
//--- NOTE: it is important to check for non-strict inequality
//--- because we have to correctly handle zero constraint
//--- 0*x<=0
continue;
}
}
for(i_=0; i_<n; i_++)
State.m_rctmpdense0.Set(i_,nactivelin,State.m_cleic.Get(i,i_));
State.m_cstatus.Set(n+i,0);
State.m_rctmpconstraintidx.Set(nactiveconstraints,n+i);
State.m_rctmpisequality[nactiveconstraints]=i<nec;
nactiveconstraints++;
nactivelin++;
}
//--- Skip if no "candidate" constraints was found
if(nactiveconstraints==0)
{
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i] && State.m_HasBndU[i] && State.m_bndl[i]==State.m_bndu[i])
{
State.m_cstatus.Set(i,1);
continue;
}
if(State.m_HasBndL[i] && State.m_xc[i]==State.m_bndl[i] && gc[i]>=0.0)
{
State.m_cstatus.Set(i,1);
continue;
}
if(State.m_HasBndU[i] && State.m_xc[i]==State.m_bndu[i] && gc[i]<=0.0)
{
State.m_cstatus.Set(i,1);
continue;
}
}
return;
}
//--- General case.
//--- APPROACH TO CONSTRAINTS ACTIVATION/DEACTIVATION
//--- We have NActiveConstraints "candidates": NActiveBnd boundary candidates,
//--- NActiveLin linear candidates. Indexes of boundary constraints are stored
//--- in RCTmpConstraintIdx[0:NActiveBnd-1], indexes of linear ones are stored
//--- in RCTmpConstraintIdx[NActiveBnd:NActiveBnd+NActiveLin-1]. Some of the
//--- constraints are equality ones, some are inequality - as specified by
//--- RCTmpIsEquality[i].
//--- Now we have to determine active subset of "candidates" set. In order to
//--- do so we solve following constrained minimization problem:
//--- ( )^2
//--- min ( SUM(lambda[i]*A[i]) + G )
//--- ( )
//--- Here:
//--- * G is a gradient (column vector)
//--- * A[i] is a column vector, linear (left) part of I-th constraint.
//--- I=0..NActiveConstraints-1, first NActiveBnd elements of A are just
//--- subset of identity matrix (boundary constraints), next NActiveLin
//--- elements are subset of rows of the matrix of general linear constraints.
//--- * lambda[i] is a Lagrange multiplier corresponding to I-th constraint
//--- NOTE: for preconditioned setting A is replaced by A*H^(-0.5), G is
//--- replaced by G*H^(-0.5). We apply this scaling at the last stage,
//--- before passing data to NNLS m_solver.
//--- Minimization is performed subject to non-negativity constraints on
//--- lambda[i] corresponding to inequality constraints. Inequality constraints
//--- which correspond to non-zero lambda are activated, equality constraints
//--- are always considered active.
//--- Informally speaking, we "decompose" descent direction -G and represent
//--- it as sum of constraint vectors and "residual" part (which is equal to
//--- the actual descent direction subject to constraints).
//--- SOLUTION OF THE NNLS PROBLEM
//--- We solve this optimization problem with Non-Negative Least Squares m_solver,
//--- which can efficiently solve least squares problems of the form
//--- ( [ I | AU ] )^2
//--- min ( [ | ]*x-b ) s.m_t. non-negativity constraints on some x[i]
//--- ( [ 0 | AL ] )
//--- In order to use this m_solver we have to rearrange rows of A[] and G in
//--- such way that first NActiveBnd columns of A store identity matrix (before
//--- sorting non-zero elements are randomly distributed in the first NActiveBnd
//--- columns of A, during sorting we move them to first NActiveBnd rows).
//--- Then we create instance of NNLS m_solver (we reuse instance left from the
//--- previous run of the optimization problem) and solve NNLS problem.
idx0=0;
idx1=nactivebnd;
for(i=0; i<n; i++)
{
if(State.m_cstatus[i]>=0)
{
v=1/MathSqrt(h[i]);
for(j=0; j<=nactivelin-1; j++)
State.m_rctmpdense1.Set(idx0,j,State.m_rctmpdense0.Get(i,j)/State.m_rctmps[i]*v);
State.m_rctmprightpart.Set(idx0,State.m_rctmpg[i]/State.m_rctmps[i]*v);
idx0++;
}
else
{
v=1/MathSqrt(h[i]);
for(j=0; j<nactivelin; j++)
State.m_rctmpdense1.Set(idx1,j,State.m_rctmpdense0.Get(i,j)*v);
State.m_rctmprightpart.Set(idx1,State.m_rctmpg[i]*v);
idx1++;
}
}
CSNNLS::SNNLSInit(n,MathMin(nec+nic,n),n,State.m_solver);
CSNNLS::SNNLSSetProblem(State.m_solver,State.m_rctmpdense1,State.m_rctmprightpart,nactivebnd,nactiveconstraints-nactivebnd,n);
for(i=0; i<=nactiveconstraints-1; i++)
{
if(State.m_rctmpisequality[i])
CSNNLS::SNNLSDropNNC(State.m_solver,i);
}
CSNNLS::SNNLSSolve(State.m_solver,State.m_rctmplambdas);
//--- After solution of the problem we activate equality constraints (always active)
//--- and inequality constraints with non-zero Lagrange multipliers. Then we reorthogonalize
//--- active constraints.
State.m_cstatus.Fill(-1,0,n+nec+nic);
for(i=0; i<nactiveconstraints; i++)
{
if(State.m_rctmpisequality[i] || State.m_rctmplambdas[i]>0.0)
State.m_cstatus.Set(State.m_rctmpconstraintidx[i],1);
else
State.m_cstatus.Set(State.m_rctmpconstraintidx[i],0);
}
SASRebuildBasis(State);
}
//+------------------------------------------------------------------+
//| This object stores nonlinear optimizer State. |
//| You should use functions provided by MinBLEIC subpackage to work |
//| with this object |
//+------------------------------------------------------------------+
class CMinBLEICState
{
public:
//--- variables
int m_bufsize;
int m_cidx;
int m_maxits;
int m_mcstage;
int m_nec;
int m_nfev;
int m_nic;
int m_nmain;
int m_nonmonotoniccnt;
int m_nslack;
int m_prectype;
int m_repdebugfeasgpaits;
int m_repdebugfeasqpits;
int m_repinneriterationscount;
int m_repnfev;
int m_repouteriterationscount;
int m_repterminationtype;
int m_repvaridx;
int m_smoothnessguardlevel;
double m_activationstep;
double m_curstpmax;
double m_cval;
double m_diffstep;
double m_epsf;
double m_epsg;
double m_epsx;
double m_f;
double m_fbase;
double m_fc;
double m_fm1;
double m_fm2;
double m_fn;
double m_fp1;
double m_fp2;
double m_fp;
double m_gm1;
double m_gp1;
double m_lastgoodstep;
double m_lastscaledgoodstep;
double m_maxscaledgrad;
double m_repdebugdx;
double m_repdebugeqerr;
double m_repdebugff;
double m_repdebugfs;
double m_stp;
double m_stpmax;
double m_teststep;
double m_trimthreshold;
double m_xm1;
double m_xp1;
bool m_boundedstep;
bool m_drep;
bool m_lsstart;
bool m_needf;
bool m_needfg;
bool m_steepestdescentstep;
bool m_userterminationneeded;
bool m_xrep;
bool m_xupdated;
//--- objects
RCommState m_rstate;
CSmoothnessMonitor m_smonitor;
CSNNLSSolver m_solver;
CSActiveSet m_sas;
CLinMinState m_lstate;
//--- arrays
bool m_HasBndL[];
bool m_HasBndU[];
CRowDouble m_bndl;
CRowDouble m_bndu;
CRowDouble m_bufrho;
CRowDouble m_buftheta;
CRowDouble m_cgc;
CRowDouble m_cgn;
CRowDouble m_d;
CRowDouble m_diagh;
CRowDouble m_g;
CRowDouble m_invs;
CRowDouble m_lastscaleused;
CRowDouble m_s;
CRowDouble m_tmp0;
CRowDouble m_tmpprec;
CRowDouble m_ugc;
CRowDouble m_ugn;
CRowDouble m_work;
CRowDouble m_x;
CRowDouble m_xn;
CRowDouble m_xp;
CRowDouble m_xstart;
//--- matrix
CMatrixDouble m_bufsk;
CMatrixDouble m_bufyk;
CMatrixDouble m_cleic;
//--- constructor, destructor
CMinBLEICState(void);
~CMinBLEICState(void) {}
//--- copy
void Copy(const CMinBLEICState &obj);
//--- overloading
void operator=(const CMinBLEICState &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinBLEICState::CMinBLEICState(void)
{
m_bufsize=0;
m_cidx=0;
m_maxits=0;
m_mcstage=0;
m_nec=0;
m_nfev=0;
m_nic=0;
m_nmain=0;
m_nonmonotoniccnt=0;
m_nslack=0;
m_prectype=0;
m_repdebugfeasgpaits=0;
m_repdebugfeasqpits=0;
m_repinneriterationscount=0;
m_repnfev=0;
m_repouteriterationscount=0;
m_repterminationtype=0;
m_repvaridx=0;
m_smoothnessguardlevel=0;
m_activationstep=0;
m_curstpmax=0;
m_cval=0;
m_diffstep=0;
m_epsf=0;
m_epsg=0;
m_epsx=0;
m_f=0;
m_fbase=0;
m_fc=0;
m_fm1=0;
m_fm2=0;
m_fn=0;
m_fp1=0;
m_fp2=0;
m_fp=0;
m_gm1=0;
m_gp1=0;
m_lastgoodstep=0;
m_lastscaledgoodstep=0;
m_maxscaledgrad=0;
m_repdebugdx=0;
m_repdebugeqerr=0;
m_repdebugff=0;
m_repdebugfs=0;
m_stp=0;
m_stpmax=0;
m_teststep=0;
m_trimthreshold=0;
m_xm1=0;
m_xp1=0;
m_boundedstep=false;
m_drep=false;
m_lsstart=false;
m_needf=false;
m_needfg=false;
m_steepestdescentstep=false;
m_userterminationneeded=false;
m_xrep=false;
m_xupdated=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinBLEICState::Copy(const CMinBLEICState &obj)
{
m_bufsize=obj.m_bufsize;
m_cidx=obj.m_cidx;
m_maxits=obj.m_maxits;
m_mcstage=obj.m_mcstage;
m_nec=obj.m_nec;
m_nfev=obj.m_nfev;
m_nic=obj.m_nic;
m_nmain=obj.m_nmain;
m_nonmonotoniccnt=obj.m_nonmonotoniccnt;
m_nslack=obj.m_nslack;
m_prectype=obj.m_prectype;
m_repdebugfeasgpaits=obj.m_repdebugfeasgpaits;
m_repdebugfeasqpits=obj.m_repdebugfeasqpits;
m_repinneriterationscount=obj.m_repinneriterationscount;
m_repnfev=obj.m_repnfev;
m_repouteriterationscount=obj.m_repouteriterationscount;
m_repterminationtype=obj.m_repterminationtype;
m_repvaridx=obj.m_repvaridx;
m_smoothnessguardlevel=obj.m_smoothnessguardlevel;
m_activationstep=obj.m_activationstep;
m_curstpmax=obj.m_curstpmax;
m_cval=obj.m_cval;
m_diffstep=obj.m_diffstep;
m_epsf=obj.m_epsf;
m_epsg=obj.m_epsg;
m_epsx=obj.m_epsx;
m_f=obj.m_f;
m_fbase=obj.m_fbase;
m_fc=obj.m_fc;
m_fm1=obj.m_fm1;
m_fm2=obj.m_fm2;
m_fn=obj.m_fn;
m_fp1=obj.m_fp1;
m_fp2=obj.m_fp2;
m_fp=obj.m_fp;
m_gm1=obj.m_gm1;
m_gp1=obj.m_gp1;
m_lastgoodstep=obj.m_lastgoodstep;
m_lastscaledgoodstep=obj.m_lastscaledgoodstep;
m_maxscaledgrad=obj.m_maxscaledgrad;
m_repdebugdx=obj.m_repdebugdx;
m_repdebugeqerr=obj.m_repdebugeqerr;
m_repdebugff=obj.m_repdebugff;
m_repdebugfs=obj.m_repdebugfs;
m_stp=obj.m_stp;
m_stpmax=obj.m_stpmax;
m_teststep=obj.m_teststep;
m_trimthreshold=obj.m_trimthreshold;
m_xm1=obj.m_xm1;
m_xp1=obj.m_xp1;
m_boundedstep=obj.m_boundedstep;
m_drep=obj.m_drep;
ArrayCopy(m_HasBndL,obj.m_HasBndL);
ArrayCopy(m_HasBndU,obj.m_HasBndU);
m_lsstart=obj.m_lsstart;
m_needf=obj.m_needf;
m_needfg=obj.m_needfg;
m_steepestdescentstep=obj.m_steepestdescentstep;
m_userterminationneeded=obj.m_userterminationneeded;
m_xrep=obj.m_xrep;
m_xupdated=obj.m_xupdated;
m_rstate=obj.m_rstate;
m_smonitor=obj.m_smonitor;
m_solver=obj.m_solver;
m_sas=obj.m_sas;
m_bndl=obj.m_bndl;
m_bndu=obj.m_bndu;
m_bufrho=obj.m_bufrho;
m_buftheta=obj.m_buftheta;
m_cgc=obj.m_cgc;
m_cgn=obj.m_cgn;
m_d=obj.m_d;
m_diagh=obj.m_diagh;
m_g=obj.m_g;
m_invs=obj.m_invs;
m_lastscaleused=obj.m_lastscaleused;
m_s=obj.m_s;
m_tmp0=obj.m_tmp0;
m_tmpprec=obj.m_tmpprec;
m_ugc=obj.m_ugc;
m_ugn=obj.m_ugn;
m_work=obj.m_work;
m_x=obj.m_x;
m_xn=obj.m_xn;
m_xp=obj.m_xp;
m_xstart=obj.m_xstart;
m_bufsk=obj.m_bufsk;
m_bufyk=obj.m_bufyk;
m_cleic=obj.m_cleic;
m_lstate=obj.m_lstate;
}
//+------------------------------------------------------------------+
//| This object stores nonlinear optimizer State. |
//| You should use functions provided by MinBLEIC subpackage to work |
//| with this object |
//+------------------------------------------------------------------+
class CMinBLEICStateShell
{
private:
CMinBLEICState m_innerobj;
public:
//--- constructors, destructor
CMinBLEICStateShell(void) {}
CMinBLEICStateShell(CMinBLEICState &obj) { m_innerobj.Copy(obj); }
~CMinBLEICStateShell(void) {}
//--- methods
bool GetNeedF(void);
void SetNeedF(const bool b);
bool GetNeedFG(void);
void SetNeedFG(const bool b);
bool GetXUpdated(void);
void SetXUpdated(const bool b);
double GetF(void);
void SetF(const double d);
CMinBLEICState *GetInnerObj(void);
};
//+------------------------------------------------------------------+
//| Returns the value of the variable needf |
//+------------------------------------------------------------------+
bool CMinBLEICStateShell::GetNeedF(void)
{
return(m_innerobj.m_needf);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable needf |
//+------------------------------------------------------------------+
void CMinBLEICStateShell::SetNeedF(const bool b)
{
m_innerobj.m_needf=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable needfg |
//+------------------------------------------------------------------+
bool CMinBLEICStateShell::GetNeedFG(void)
{
return(m_innerobj.m_needfg);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable needfg |
//+------------------------------------------------------------------+
void CMinBLEICStateShell::SetNeedFG(const bool b)
{
m_innerobj.m_needfg=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable xupdated |
//+------------------------------------------------------------------+
bool CMinBLEICStateShell::GetXUpdated(void)
{
return(m_innerobj.m_xupdated);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable xupdated |
//+------------------------------------------------------------------+
void CMinBLEICStateShell::SetXUpdated(const bool b)
{
m_innerobj.m_xupdated=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable f |
//+------------------------------------------------------------------+
double CMinBLEICStateShell::GetF(void)
{
return(m_innerobj.m_f);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable f |
//+------------------------------------------------------------------+
void CMinBLEICStateShell::SetF(const double d)
{
m_innerobj.m_f=d;
}
//+------------------------------------------------------------------+
//| Return object of class |
//+------------------------------------------------------------------+
CMinBLEICState *CMinBLEICStateShell::GetInnerObj(void)
{
return(GetPointer(m_innerobj));
}
//+------------------------------------------------------------------+
//| This structure stores optimization report: |
//| * InnerIterationsCount number of inner iterations |
//| * OuterIterationsCount number of outer iterations |
//| * NFEV number of gradient evaluations |
//| * TerminationType termination type (see below) |
//| TERMINATION CODES |
//| TerminationType field contains completion code,which can be: |
//| -10 unsupported combination of algorithm Settings: |
//| 1) StpMax is set to non-zero value, |
//| AND 2) non-default preconditioner is used. |
//| You can't use both features at the same moment, |
//| so you have to choose one of them (and to turn |
//| off another one). |
//| -3 inconsistent constraints. Feasible point is |
//| either nonexistent or too hard to find. Try to |
//| restart optimizer with better initial |
//| approximation |
//| 4 conditions on constraints are fulfilled |
//| with error less than or equal to EpsC |
//| 5 MaxIts steps was taken |
//| 7 stopping conditions are too stringent, |
//| further improvement is impossible, |
//| X contains best point found so far. |
//| ADDITIONAL FIELDS |
//| There are additional fields which can be used for debugging: |
//| * DebugEqErr error in the equality constraints |
//| (2-norm) |
//| * DebugFS f,calculated at projection of initial|
//| point to the feasible set |
//| * DebugFF f,calculated at the final point |
//| * DebugDX |X_start-X_final| |
//+------------------------------------------------------------------+
class CMinBLEICReport
{
public:
//--- variables
int m_debugfeasgpaits;
int m_debugfeasqpits;
int m_inneriterationscount;
int m_iterationscount;
int m_nfev;
int m_outeriterationscount;
int m_terminationtype;
int m_varidx;
double m_debugdx;
double m_debugeqerr;
double m_debugff;
double m_debugfs;
//--- constructor, destructor
CMinBLEICReport(void) { ZeroMemory(this); }
~CMinBLEICReport(void) {}
//--- copy
void Copy(const CMinBLEICReport &obj);
//--- overloading
void operator=(const CMinBLEICReport &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinBLEICReport::Copy(const CMinBLEICReport &obj)
{
//--- copy variables
m_debugfeasgpaits=obj.m_debugfeasgpaits;
m_debugfeasqpits=obj.m_debugfeasqpits;
m_inneriterationscount=obj.m_inneriterationscount;
m_iterationscount=obj.m_iterationscount;
m_nfev=obj.m_nfev;
m_outeriterationscount=obj.m_outeriterationscount;
m_terminationtype=obj.m_terminationtype;
m_varidx=obj.m_varidx;
m_debugdx=obj.m_debugdx;
m_debugeqerr=obj.m_debugeqerr;
m_debugff=obj.m_debugff;
m_debugfs=obj.m_debugfs;
}
//+------------------------------------------------------------------+
//| This structure stores optimization report: |
//| * InnerIterationsCount number of inner iterations |
//| * OuterIterationsCount number of outer iterations |
//| * NFEV number of gradient evaluations |
//| * TerminationType termination type (see below) |
//| TERMINATION CODES |
//| TerminationType field contains completion code,which can be: |
//| -10 unsupported combination of algorithm Settings: |
//| 1) StpMax is set to non-zero value, |
//| AND 2) non-default preconditioner is used. |
//| You can't use both features at the same moment, |
//| so you have to choose one of them (and to turn |
//| off another one). |
//| -3 inconsistent constraints. Feasible point is |
//| either nonexistent or too hard to find. Try to |
//| restart optimizer with better initial |
//| approximation |
//| 4 conditions on constraints are fulfilled |
//| with error less than or equal to EpsC |
//| 5 MaxIts steps was taken |
//| 7 stopping conditions are too stringent, |
//| further improvement is impossible, |
//| X contains best point found so far. |
//| ADDITIONAL FIELDS |
//| There are additional fields which can be used for debugging: |
//| * DebugEqErr error in the equality constraints |
//| (2-norm) |
//| * DebugFS f,calculated at projection of initial|
//| point to the feasible set |
//| * DebugFF f,calculated at the final point |
//| * DebugDX |X_start-X_final| |
//+------------------------------------------------------------------+
class CMinBLEICReportShell
{
private:
CMinBLEICReport m_innerobj;
public:
//--- constructors, destructor
CMinBLEICReportShell(void) {}
CMinBLEICReportShell(CMinBLEICReport &obj) { m_innerobj.Copy(obj); }
~CMinBLEICReportShell(void) {}
//--- methods
int GetInnerIterationsCount(void);
void SetInnerIterationsCount(const int i);
int GetOuterIterationsCount(void);
void SetOuterIterationsCount(const int i);
int GetNFev(void);
void SetNFev(const int i);
int GetTerminationType(void);
void SetTerminationType(const int i);
double GetDebugEqErr(void);
void SetDebugEqErr(const double d);
double GetDebugFS(void);
void SetDebugFS(const double d);
double GetDebugFF(void);
void SetDebugFF(const double d);
double GetDebugDX(void);
void SetDebugDX(const double d);
CMinBLEICReport *GetInnerObj(void);
};
//+------------------------------------------------------------------+
//| Returns the value of the variable inneriterationscount |
//+------------------------------------------------------------------+
int CMinBLEICReportShell::GetInnerIterationsCount(void)
{
return(m_innerobj.m_inneriterationscount);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable inneriterationscount |
//+------------------------------------------------------------------+
void CMinBLEICReportShell::SetInnerIterationsCount(const int i)
{
m_innerobj.m_inneriterationscount=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable outeriterationscount |
//+------------------------------------------------------------------+
int CMinBLEICReportShell::GetOuterIterationsCount(void)
{
return(m_innerobj.m_outeriterationscount);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable outeriterationscount |
//+------------------------------------------------------------------+
void CMinBLEICReportShell::SetOuterIterationsCount(const int i)
{
m_innerobj.m_outeriterationscount=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable nfev |
//+------------------------------------------------------------------+
int CMinBLEICReportShell::GetNFev(void)
{
return(m_innerobj.m_nfev);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable nfev |
//+------------------------------------------------------------------+
void CMinBLEICReportShell::SetNFev(const int i)
{
m_innerobj.m_nfev=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable terminationtype |
//+------------------------------------------------------------------+
int CMinBLEICReportShell::GetTerminationType(void)
{
return(m_innerobj.m_terminationtype);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable terminationtype |
//+------------------------------------------------------------------+
void CMinBLEICReportShell::SetTerminationType(const int i)
{
m_innerobj.m_terminationtype=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable debugeqerr |
//+------------------------------------------------------------------+
double CMinBLEICReportShell::GetDebugEqErr(void)
{
return(m_innerobj.m_debugeqerr);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable debugeqerr |
//+------------------------------------------------------------------+
void CMinBLEICReportShell::SetDebugEqErr(const double d)
{
m_innerobj.m_debugeqerr=d;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable debugfs |
//+------------------------------------------------------------------+
double CMinBLEICReportShell::GetDebugFS(void)
{
return(m_innerobj.m_debugfs);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable debugfs |
//+------------------------------------------------------------------+
void CMinBLEICReportShell::SetDebugFS(const double d)
{
m_innerobj.m_debugfs=d;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable debugff |
//+------------------------------------------------------------------+
double CMinBLEICReportShell::GetDebugFF(void)
{
return(m_innerobj.m_debugff);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable debugff |
//+------------------------------------------------------------------+
void CMinBLEICReportShell::SetDebugFF(const double d)
{
m_innerobj.m_debugff=d;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable debugdx |
//+------------------------------------------------------------------+
double CMinBLEICReportShell::GetDebugDX(void)
{
return(m_innerobj.m_debugdx);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable debugdx |
//+------------------------------------------------------------------+
void CMinBLEICReportShell::SetDebugDX(const double d)
{
m_innerobj.m_debugdx=d;
}
//+------------------------------------------------------------------+
//| Return object of class |
//+------------------------------------------------------------------+
CMinBLEICReport *CMinBLEICReportShell::GetInnerObj(void)
{
return(GetPointer(m_innerobj));
}
//+------------------------------------------------------------------+
//| Bound constrained optimization with additional linear equality |
//| and inequality constraints |
//+------------------------------------------------------------------+
class CMinBLEIC
{
public:
//--- class constants
static const double m_maxnonmonotoniclen;
static const double m_gtol;
static const double m_initialdecay;
static const double m_mindecay;
static const double m_decaycorrection;
static const double m_penaltyfactor;
//--- public methods
static void MinBLEICCreate(const int n,double &x[],CMinBLEICState &State);
static void MinBLEICCreate(const int n,CRowDouble &x,CMinBLEICState &State);
static void MinBLEICCreateF(const int n,double &x[],const double diffstep,CMinBLEICState &State);
static void MinBLEICCreateF(const int n,CRowDouble &x,const double diffstep,CMinBLEICState &State);
static void MinBLEICSetBC(CMinBLEICState &State,double &bndl[],double &bndu[]);
static void MinBLEICSetBC(CMinBLEICState &State,CRowDouble &bndl,CRowDouble &bndu);
static void MinBLEICSetLC(CMinBLEICState &State,CMatrixDouble &c,int &ct[],const int k);
static void MinBLEICSetLC(CMinBLEICState &State,CMatrixDouble &c,CRowInt &ct,const int k);
static void MinBLEICSetInnerCond(CMinBLEICState &State,const double epsg,const double epsf,const double epsx);
static void MinBLEICSetOuterCond(CMinBLEICState &State,const double epsx,const double epsi);
static void MinBLEICSetCond(CMinBLEICState &State,double epsg,double epsf,double epsx,int m_maxits);
static void MinBLEICSetScale(CMinBLEICState &State,double &s[]);
static void MinBLEICSetScale(CMinBLEICState &State,CRowDouble &s);
static void MinBLEICSetPrecDefault(CMinBLEICState &State);
static void MinBLEICSetPrecDiag(CMinBLEICState &State,double &d[]);
static void MinBLEICSetPrecDiag(CMinBLEICState &State,CRowDouble &d);
static void MinBLEICSetPrecScale(CMinBLEICState &State);
static void MinBLEICSetMaxIts(CMinBLEICState &State,const int m_maxits);
static void MinBLEICSetXRep(CMinBLEICState &State,const bool needxrep);
static void MinBLEICSetDRep(CMinBLEICState &State,bool needdrep);
static void MinBLEICSetStpMax(CMinBLEICState &State,const double stpmax);
static void MinBLEICOptGuardGradient(CMinBLEICState &State,double &teststep);
static void MinBLEICOptGuardSmoothness(CMinBLEICState &State,int level);
static void MinBLEICOptGuardResults(CMinBLEICState &State,COptGuardReport &rep);
static void MinBLEICOptGuardNonC1Test0Results(CMinBLEICState &State,COptGuardNonC1Test0Report &strrep,COptGuardNonC1Test0Report &lngrep);
static void MinBLEICOptGuardNonC1Test1Results(CMinBLEICState &State,COptGuardNonC1Test1Report &strrep,COptGuardNonC1Test1Report &lngrep);
static void MinBLEICResults(CMinBLEICState &State,double &x[],CMinBLEICReport &rep);
static void MinBLEICResults(CMinBLEICState &State,CRowDouble &x,CMinBLEICReport &rep);
static void MinBLEICResultsBuf(CMinBLEICState &State,double &x[],CMinBLEICReport &rep);
static void MinBLEICResultsBuf(CMinBLEICState &State,CRowDouble &x,CMinBLEICReport &rep);
static void MinBLEICRestartFrom(CMinBLEICState &State,double &x[]);
static void MinBLEICRestartFrom(CMinBLEICState &State,CRowDouble &x);
static void MinBLEICRequestTermination(CMinBLEICState &State);
static void MinBLEICEmergencyTermination(CMinBLEICState &State);
static bool MinBLEICIteration(CMinBLEICState &State);
private:
static void ClearRequestFields(CMinBLEICState &State);
static void MinBLEICInitInternal(const int n,CRowDouble &x,const double diffstep,CMinBLEICState &State);
static void UpdateEstimateOfGoodStep(double &estimate,double newstep);
};
//+------------------------------------------------------------------+
//| Initialize constants |
//+------------------------------------------------------------------+
const double CMinBLEIC::m_maxnonmonotoniclen=1.0E-7;
const double CMinBLEIC::m_gtol=0.4;
const double CMinBLEIC::m_initialdecay=0.5;
const double CMinBLEIC::m_mindecay=0.1;
const double CMinBLEIC::m_decaycorrection=0.8;
const double CMinBLEIC::m_penaltyfactor=100;
//+------------------------------------------------------------------+
//| BOUND CONSTRAINED OPTIMIZATION |
//| WITH ADDITIONAL LINEAR EQUALITY AND INEQUALITY CONSTRAINTS|
//| DESCRIPTION: |
//| The subroutine minimizes function F(x) of N arguments subject to |
//| any combination of: |
//| * bound constraints |
//| * linear inequality constraints |
//| * linear equality constraints |
//| REQUIREMENTS: |
//| * user must provide function value and gradient |
//| * starting point X0 must be feasible or |
//| not too far away from the feasible set |
//| * grad(f) must be Lipschitz continuous on a level set: |
//| L = { x : f(x)<=f(x0) } |
//| * function must be defined everywhere on the feasible set F |
//| USAGE: |
//| Constrained optimization if far more complex than the |
//| unconstrained one. Here we give very brief outline of the BLEIC |
//| optimizer. We strongly recommend you to read examples in the |
//| ALGLIB Reference Manual and to read ALGLIB User Guide on |
//| optimization, which is available at |
//| http://www.alglib.net/optimization/ |
//| 1. User initializes algorithm State with MinBLEICCreate() call |
//| 2. USer adds boundary and/or linear constraints by calling |
//| MinBLEICSetBC() and MinBLEICSetLC() functions. |
//| 3. User sets stopping conditions for underlying unconstrained |
//| m_solver with MinBLEICSetInnerCond() call. |
//| This function controls accuracy of underlying optimization |
//| algorithm. |
//| 4. User sets stopping conditions for outer iteration by calling |
//| MinBLEICSetOuterCond() function. |
//| This function controls handling of boundary and inequality |
//| constraints. |
//| 5. Additionally, user may set limit on number of internal |
//| iterations by MinBLEICSetMaxIts() call. |
//| This function allows to prevent algorithm from looping |
//| forever. |
//| 6. User calls MinBLEICOptimize() function which takes algorithm |
//| State and pointer (delegate, etc.) to callback function |
//| which calculates F/G. |
//| 7. User calls MinBLEICResults() to get solution |
//| 8. Optionally user may call MinBLEICRestartFrom() to solve |
//| another problem with same N but another starting point. |
//| MinBLEICRestartFrom() allows to reuse already initialized |
//| structure. |
//| INPUT PARAMETERS: |
//| N - problem dimension, N>0: |
//| * if given, only leading N elements of X are |
//| used |
//| * if not given, automatically determined from |
//| size ofX |
//| X - starting point, array[N]: |
//| * it is better to set X to a feasible point |
//| * but X can be infeasible, in which case |
//| algorithm will try to find feasible point |
//| first, using X as initial approximation. |
//| OUTPUT PARAMETERS: |
//| State - structure stores algorithm State |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICCreate(const int n,double &x[],CMinBLEICState &State)
{
CRowDouble X=x;
MinBLEICCreate(n,X,State);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICCreate(const int n,CRowDouble &x,CMinBLEICState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- function call
MinBLEICInitInternal(n,x,0.0,State);
}
//+------------------------------------------------------------------+
//| The subroutine is finite difference variant of MinBLEICCreate(). |
//| It uses finite differences in order to differentiate target |
//| function. |
//| Description below contains information which is specific to this |
//| function only. We recommend to read comments on MinBLEICCreate() |
//| in order to get more information about creation of BLEIC |
//| optimizer. |
//| INPUT PARAMETERS: |
//| N - problem dimension, N>0: |
//| * if given, only leading N elements of X are used|
//| * if not given, automatically determined from |
//| size of X |
//| X - starting point, array[0..m_n-1]. |
//| DiffStep- differentiation step, >0 |
//| OUTPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NOTES: |
//| 1. algorithm uses 4-point central formula for differentiation. |
//| 2. differentiation step along I-th axis is equal to DiffStep*S[I]|
//| where S[] is scaling vector which can be set by |
//| MinBLEICSetScale() call. |
//| 3. we recommend you to use moderate values of differentiation |
//| step. Too large step will result in too large truncation |
//| errors, while too small step will result in too large |
//| numerical errors. 1.0E-6 can be good value to start with. |
//| 4. Numerical differentiation is very inefficient - one gradient |
//| calculation needs 4*N function evaluations. This function will|
//| work for any N - either small (1...10), moderate (10...100) or|
//| large (100...). However, performance penalty will be too |
//| severe for any N's except for small ones. |
//| We should also say that code which relies on numerical |
//| differentiation is less robust and precise. CG needs exact |
//| gradient values. Imprecise gradient may slow down convergence,|
//| especially on highly nonlinear problems. |
//| Thus we recommend to use this function for fast prototyping on|
//| small - dimensional problems only, and to implement analytical|
//| gradient as soon as possible. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICCreateF(const int n,double &x[],
const double diffstep,
CMinBLEICState &State)
{
CRowDouble X=x;
MinBLEICCreateF(n,X,diffstep,State);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICCreateF(const int n,CRowDouble &x,
const double diffstep,
CMinBLEICState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
//--- check
if(!CAp::Assert(x.Size()>=n,__FUNCTION__+": Length(X)<N"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(diffstep),__FUNCTION__+": DiffStep is infinite or NaN!"))
return;
//--- check
if(!CAp::Assert(diffstep>0.0,__FUNCTION__+": DiffStep is non-positive!"))
return;
//--- function call
MinBLEICInitInternal(n,x,diffstep,State);
}
//+------------------------------------------------------------------+
//| This function sets boundary constraints for BLEIC optimizer. |
//| Boundary constraints are inactive by default (after initial |
//| creation). They are preserved after algorithm restart with |
//| MinBLEICRestartFrom(). |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| BndL - lower bounds, array[N]. |
//| If some (all) variables are unbounded, you may |
//| specify very small number or -INF. |
//| BndU - upper bounds, array[N]. |
//| If some (all) variables are unbounded, you may |
//| specify very large number or +INF. |
//| NOTE 1: it is possible to specify BndL[i]=BndU[i]. In this case |
//| I-th variable will be "frozen" at X[i]=BndL[i]=BndU[i]. |
//| NOTE 2: this m_solver has following useful properties: |
//| * bound constraints are always satisfied exactly |
//| * function is evaluated only INSIDE area specified by bound |
//| constraints, even when numerical differentiation is used |
//| (algorithm adjusts nodes according to boundary constraints) |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetBC(CMinBLEICState &State,double &bndl[],
double &bndu[])
{
CRowDouble BndL=bndl;
CRowDouble BndU=bndu;
MinBLEICSetBC(State,BndL,BndU);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetBC(CMinBLEICState &State,CRowDouble &bndl,
CRowDouble &bndu)
{
int n=State.m_nmain;
//--- check
if(!CAp::Assert(CAp::Len(bndl)>=n,__FUNCTION__+": Length(BndL)<N"))
return;
//--- check
if(!CAp::Assert(CAp::Len(bndu)>=n,__FUNCTION__+": Length(BndU)<N"))
return;
for(int i=0; i<n; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(bndl[i]) || AL_NEGINF==bndl[i],__FUNCTION__+": BndL contains NAN or +INF"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(bndu[i]) || AL_POSINF==bndu[i],__FUNCTION__+": BndU contains NAN or -INF"))
return;
//--- change values
State.m_bndl.Set(i,bndl[i]);
State.m_bndu.Set(i,bndu[i]);
State.m_HasBndL[i]=MathIsValidNumber(bndl[i]);
State.m_HasBndU[i]=MathIsValidNumber(bndu[i]);
}
CSActiveSets::SASSetBC(State.m_sas,bndl,bndu);
}
//+------------------------------------------------------------------+
//| This function sets linear constraints for BLEIC optimizer. |
//| Linear constraints are inactive by default (after initial |
//| creation). They are preserved after algorithm restart with |
//| MinBLEICRestartFrom(). |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with |
//| MinBLEICCreate call. |
//| C - linear constraints, array[K,N+1]. |
//| Each row of C represents one constraint, either |
//| equality or inequality (see below): |
//| * first N elements correspond to coefficients, |
//| * last element corresponds to the right part. |
//| All elements of C (including right part) must be |
//| finite. |
//| CT - type of constraints, array[K]: |
//| * if CT[i]>0, then I-th constraint is |
//| C[i,*]*x >= C[i,n+1] |
//| * if CT[i]=0, then I-th constraint is |
//| C[i,*]*x = C[i,n+1] |
//| * if CT[i]<0, then I-th constraint is |
//| C[i,*]*x <= C[i,n+1] |
//| K - number of equality/inequality constraints, K>=0: |
//| * if given, only leading K elements of C/CT are |
//| used |
//| * if not given, automatically determined from |
//| sizes of C/CT |
//| NOTE 1: linear (non-bound) constraints are satisfied only |
//| approximately: |
//| * there always exists some minor violation (about Epsilon in |
//| magnitude) due to rounding errors |
//| * numerical differentiation, if used, may lead to function |
//| evaluations outside of the feasible area, because algorithm |
//| does NOT change numerical differentiation formula according to |
//| linear constraints. |
//| If you want constraints to be satisfied exactly, try to |
//| reformulate your problem in such manner that all constraints will|
//| become boundary ones (this kind of constraints is always |
//| satisfied exactly, both in the final solution and in all |
//| intermediate points). |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetLC(CMinBLEICState &State,CMatrixDouble &c,
int &ct[],const int k)
{
CRowInt CT=ct;
MinBLEICSetLC(State,c,CT,k);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetLC(CMinBLEICState &State,CMatrixDouble &c,
CRowInt &ct,const int k)
{
//--- create variables
int nmain=State.m_nmain;
int i=0;
//--- First,check for errors in the inputs
if(!CAp::Assert(k>=0,__FUNCTION__+": K<0"))
return;
//--- check
if(!CAp::Assert((int)CAp::Cols(c)>=nmain+1 || k==0,__FUNCTION__+": Cols(C)<N+1"))
return;
//--- check
if(!CAp::Assert((int)CAp::Rows(c)>=k,__FUNCTION__+": Rows(C)<K"))
return;
//--- check
if(!CAp::Assert(CAp::Len(ct)>=k,__FUNCTION__+": Length(CT)<K"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteMatrix(c,k,nmain+1),__FUNCTION__+": C contains infinite or NaN values!"))
return;
//-- Handle zero K
if(k==0)
{
State.m_nec=0;
State.m_nic=0;
CSActiveSets::SASSetLC(State.m_sas,c,ct,0);
return;
}
//-- Equality constraints are stored first, in the upper
//-- NEC rows of State.CLEIC matrix. Inequality constraints
//-- are stored in the next NIC rows.
//-- NOTE: we convert inequality constraints to the form
//-- A*x<=b before copying them.
State.m_cleic.Resize(k,nmain+1);
State.m_nec=0;
State.m_nic=0;
for(i=0; i<k; i++)
{
if(ct[i]==0)
{
State.m_cleic.Row(State.m_nec,c[i]+0);
State.m_nec++;
}
}
for(i=0; i<k; i++)
{
if(ct[i]!=0)
{
if(ct[i]>0)
State.m_cleic.Row(State.m_nec+State.m_nic,c[i]*(-1.0));
else
State.m_cleic.Row(State.m_nec+State.m_nic,c[i]+0);
State.m_nic++;
}
}
//-- Normalize rows of State.CLEIC: each row must have unit norm.
//-- Norm is calculated using first N elements (i.e. right part is
//-- not counted when we calculate norm).
for(i=0; i<k; i++)
{
double v=CAblasF::RDotRR(nmain,State.m_cleic,i,State.m_cleic,i);
if(v==0.0)
continue;
v=1/MathSqrt(v);
State.m_cleic.Row(i,State.m_cleic[i]*v);
}
CSActiveSets::SASSetLC(State.m_sas,c,ct,k);
}
//+------------------------------------------------------------------+
//| This function sets stopping conditions for the underlying |
//| nonlinear CG optimizer. It controls overall accuracy of solution.|
//| These conditions should be strict enough in order for algorithm |
//| to converge. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| EpsG - >=0 |
//| The subroutine finishes its work if the condition|
//| |v|<EpsG is satisfied, where: |
//| * |.| means Euclidian norm |
//| * v - scaled gradient vector, v[i]=g[i]*s[i] |
//| * g - gradient |
//| * s - scaling coefficients set by |
//| MinBLEICSetScale() |
//| EpsF - >=0 |
//| The subroutine finishes its work if on k+1-th |
//| iteration the condition |F(k+1)-F(k)| <= |
//| <= EpsF*max{|F(k)|,|F(k+1)|,1} is satisfied. |
//| EpsX - >=0 |
//| The subroutine finishes its work if on k+1-th |
//| iteration the condition |v|<=EpsX is fulfilled, |
//| where: |
//| * |.| means Euclidian norm |
//| * v - scaled step vector, v[i]=dx[i]/s[i] |
//| * dx - ste pvector, dx=X(k+1)-X(k) |
//| * s - scaling coefficients set by |
//| MinBLEICSetScale() |
//| Passing EpsG=0, EpsF=0 and EpsX=0 (simultaneously) will lead to |
//| automatic stopping criterion selection. |
//| These conditions are used to terminate inner iterations. However,|
//| you need to tune termination conditions for outer iterations too.|
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetInnerCond(CMinBLEICState &State,const double epsg,
const double epsf,const double epsx)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(epsg),__FUNCTION__+": EpsG is not finite number"))
return;
//--- check
if(!CAp::Assert(epsg>=0.0,__FUNCTION__+": negative EpsG"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(epsf),__FUNCTION__+": EpsF is not finite number"))
return;
//--- check
if(!CAp::Assert(epsf>=0.0,__FUNCTION__+": negative EpsF"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(epsx),__FUNCTION__+": EpsX is not finite number"))
return;
//--- check
if(!CAp::Assert(epsx>=0.0,__FUNCTION__+": negative EpsX"))
return;
//--- change values
State.m_epsg=epsg;
State.m_epsf=epsf;
State.m_epsx=epsx;
}
//+------------------------------------------------------------------+
//| This function sets stopping conditions for outer iteration of |
//| BLEIC algo. |
//| These conditions control accuracy of constraint handling and |
//| amount of infeasibility allowed in the solution. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| EpsX - >0, stopping condition on outer iteration step |
//| length |
//| EpsI - >0, stopping condition on infeasibility |
//| Both EpsX and EpsI must be non-zero. |
//| MEANING OF EpsX |
//| EpsX is a stopping condition for outer iterations. Algorithm will|
//| stop when solution of the current modified subproblem will be |
//| within EpsX (using 2-norm) of the previous solution. |
//| MEANING OF EpsI |
//| EpsI controls feasibility properties - algorithm won't stop until|
//| all inequality constraints will be satisfied with error (distance|
//| from current point to the feasible area) at most EpsI. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetOuterCond(CMinBLEICState &State,const double epsx,
const double epsi)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(epsx),__FUNCTION__+": EpsX is not finite number"))
return;
//--- check
if(!CAp::Assert(epsx>=0.0,__FUNCTION__+": non-positive EpsX"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(epsi),__FUNCTION__+": EpsI is not finite number"))
return;
//--- check
if(!CAp::Assert((double)(epsi)>0.0,__FUNCTION__+": non-positive EpsI"))
return;
//--- change values
State.m_epsx=epsx;
}
//+------------------------------------------------------------------+
//| This function sets stopping conditions for the optimizer. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| EpsG - >=0, The subroutine finishes its work if the |
//| condition |v|<EpsG is satisfied, where: |
//| * |.| means Euclidian norm |
//| * v - scaled gradient vector, v[i]=g[i]*s[i] |
//| * g - gradient |
//| * s - scaling coefficients set by |
//| MinBLEICSetScale() |
//| EpsF - >=0, The subroutine finishes its work if on k+1-th |
//| iteration the condition |
//| |F(k+1)-F(k)|<=EpsF*max{|F(k)|,|F(k+1)|,1} |
//| is satisfied. |
//| EpsX - >=0, The subroutine finishes its work if on k+1-th |
//| iteration the condition |v|<=EpsX is fulfilled|
//| where: |
//| * |.| means Euclidian norm |
//| * v - scaled step vector, v[i]=dx[i]/s[i] |
//| * dx - step vector, dx=X(k+1)-X(k) |
//| * s - scaling coefficients set by |
//| MinBLEICSetScale() |
//| MaxIts - maximum number of iterations. If MaxIts=0, the |
//| number of iterations is unlimited. |
//| Passing EpsG=0, EpsF=0 and EpsX=0 and MaxIts=0 (simultaneously) |
//| will lead to automatic stopping criterion selection. |
//| NOTE: when SetCond() called with non-zero MaxIts, BLEIC solver |
//| may perform slightly more than MaxIts iterations. I.e., |
//| MaxIts sets non-strict limit on iterations count. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetCond(CMinBLEICState &State,
double epsg,
double epsf,
double epsx,
int m_maxits)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(epsg),__FUNCTION__+": EpsG is not finite number"))
return;
if(!CAp::Assert(epsg>=0.0,__FUNCTION__+": negative EpsG"))
return;
if(!CAp::Assert(MathIsValidNumber(epsf),__FUNCTION__+": EpsF is not finite number"))
return;
if(!CAp::Assert(epsf>=0.0,__FUNCTION__+": negative EpsF"))
return;
if(!CAp::Assert(MathIsValidNumber(epsx),__FUNCTION__+": EpsX is not finite number"))
return;
if(!CAp::Assert(epsx>=0.0,__FUNCTION__+": negative EpsX"))
return;
if(!CAp::Assert(m_maxits>=0,__FUNCTION__+": negative MaxIts!"))
return;
if(epsg==0.0 && epsf==0.0 && epsx==0.0 && m_maxits==0)
epsx=1.0E-6;
State.m_epsg=epsg;
State.m_epsf=epsf;
State.m_epsx=epsx;
State.m_maxits=m_maxits;
}
//+------------------------------------------------------------------+
//| This function sets scaling coefficients for BLEIC optimizer. |
//| ALGLIB optimizers use scaling matrices to test stopping |
//| conditions (step size and gradient are scaled before comparison |
//| with tolerances). Scale of the I-th variable is a translation |
//| invariant measure of: |
//| a) "how large" the variable is |
//| b) how large the step should be to make significant changes in |
//| the function |
//| Scaling is also used by finite difference variant of the |
//| optimizer - step along I-th axis is equal to DiffStep*S[I]. |
//| In most optimizers (and in the BLEIC too) scaling is NOT a form |
//| of preconditioning. It just affects stopping conditions. You |
//| should set preconditioner by separate call to one of the |
//| MinBLEICSetPrec...() functions. |
//| There is a special preconditioning mode, however, which uses |
//| scaling coefficients to form diagonal preconditioning matrix. |
//| You can turn this mode on, if you want. But you should understand|
//| that scaling is not the same thing as preconditioning - these are|
//| two different, although related forms of tuning m_solver. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| S - array[N], non-zero scaling coefficients |
//| S[i] may be negative, sign doesn't matter. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetScale(CMinBLEICState &State,double &s[])
{
CRowDouble Scale=s;
MinBLEICSetScale(State,Scale);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetScale(CMinBLEICState &State,CRowDouble &s)
{
//--- check
if(!CAp::Assert(s.Size()>=State.m_nmain,__FUNCTION__+": Length(S)<N"))
return;
for(int i=0; i<State.m_nmain; i++)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(s[i]),__FUNCTION__+": S contains infinite or NAN elements"))
return;
if(!CAp::Assert(s[i]!=0.0,__FUNCTION__+": S contains zero elements"))
return;
State.m_s.Set(i,MathAbs(s[i]));
}
CSActiveSets::SASSetScale(State.m_sas,s);
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: preconditioning is turned |
//| off. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetPrecDefault(CMinBLEICState &State)
{
State.m_prectype=0;
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: diagonal of approximate |
//| Hessian is used. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| D - diagonal of the approximate Hessian, |
//| array[0..m_n-1], (if larger, only leading N |
//| elements are used). |
//| NOTE 1: D[i] should be positive. Exception will be thrown |
//| otherwise. |
//| NOTE 2: you should pass diagonal of approximate Hessian - NOT |
//| ITS INVERSE. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetPrecDiag(CMinBLEICState &State,double &d[])
{
CRowDouble D=d;
CMinBLEIC::MinBLEICSetPrecDiag(State,D);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetPrecDiag(CMinBLEICState &State,CRowDouble &d)
{
//--- check
if(!CAp::Assert(CAp::Len(d)>=State.m_nmain,__FUNCTION__+": D is too short"))
return;
for(int i=0; i<=State.m_nmain-1; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(d[i]),__FUNCTION__+": D contains infinite or NAN elements"))
return;
//--- check
if(!CAp::Assert(d[i]>0.0,__FUNCTION__+": D contains non-positive elements"))
return;
}
//--- function call
CApServ::RVectorSetLengthAtLeast(State.m_diagh,State.m_nmain);
State.m_prectype=2;
//--- copy
State.m_diagh=d;
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: scale-based diagonal |
//| preconditioning. |
//| This preconditioning mode can be useful when you don't have |
//| approximate diagonal of Hessian, but you know that your variables|
//| are badly scaled (for example, one variable is in [1,10], and |
//| another in [1000,100000]), and most part of the ill-conditioning |
//| comes from different scales of vars. |
//| In this case simple scale-based preconditioner, with H.Set(i, |
//| = 1/(s[i]^2), can greatly improve convergence. |
//| IMPRTANT: you should set scale of your variables with |
//| MinBLEICSetScale() call (before or after MinBLEICSetPrecScale() |
//| call). Without knowledge of the scale of your variables |
//| scale-based preconditioner will be just unit matrix. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetPrecScale(CMinBLEICState &State)
{
State.m_prectype=3;
}
//+------------------------------------------------------------------+
//| This function allows to stop algorithm after specified number of |
//| inner iterations. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| MaxIts - maximum number of inner iterations. |
//| If MaxIts=0, the number of iterations is |
//| unlimited. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetMaxIts(CMinBLEICState &State,
const int m_maxits)
{
//--- check
if(!CAp::Assert(m_maxits>=0,__FUNCTION__+": negative MaxIts!"))
return;
//--- change value
State.m_maxits=m_maxits;
}
//+------------------------------------------------------------------+
//| This function turns on/off reporting. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NeedXRep- whether iteration reports are needed or not |
//| If NeedXRep is True, algorithm will call rep() callback function |
//| if it is provided to MinBLEICOptimize(). |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetXRep(CMinBLEICState &State,
const bool needxrep)
{
State.m_xrep=needxrep;
}
//+------------------------------------------------------------------+
//| This function turns on/off line search reports. |
//| These reports are described in more details in developer-only |
//| comments on MinBLEICState object. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NeedDRep - whether line search reports are needed or not |
//| This function is intended for private use only. Turning it on |
//| artificially may cause program failure. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetDRep(CMinBLEICState &State,
bool needdrep)
{
State.m_drep=needdrep;
}
//+------------------------------------------------------------------+
//| This function sets maximum step length |
//| IMPORTANT: this feature is hard to combine with preconditioning. |
//| You can't set upper limit on step length, when you solve |
//| optimization problem with linear (non-boundary) constraints AND |
//| preconditioner turned on. |
//| When non-boundary constraints are present, you have to either a) |
//| use preconditioner, or b) use upper limit on step length. YOU |
//| CAN'T USE BOTH! In this case algorithm will terminate with |
//| appropriate error code. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| StpMax - maximum step length, >=0. Set StpMax to 0.0, if |
//| you don't want to limit step length. |
//| Use this subroutine when you optimize target function which |
//| contains exp() or other fast growing functions, and optimization |
//| algorithm makes too large steps which lead to overflow. This |
//| function allows us to reject steps that are too large (and |
//| therefore expose us to the possible overflow) without actually |
//| calculating function value at the x+stp*d. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICSetStpMax(CMinBLEICState &State,
const double stpmax)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(stpmax),__FUNCTION__+": StpMax is not finite!"))
return;
//--- check
if(!CAp::Assert(stpmax>=0.0,__FUNCTION__+": StpMax<0!"))
return;
//--- change value
State.m_stpmax=stpmax;
}
//+------------------------------------------------------------------+
//| This function activates/deactivates verification of the user- |
//| supplied analytic gradient. |
//| Upon activation of this option OptGuard integrity checker |
//| performs numerical differentiation of your target function at the|
//| initial point (note: future versions may also perform check at |
//| the final point) and compares numerical gradient with analytic |
//| one provided by you. |
//| If difference is too large, an error flag is set and optimization|
//| session continues. After optimization session is over, you can |
//| retrieve the report which stores both gradients and specific |
//| components highlighted as suspicious by the OptGuard. |
//| The primary OptGuard report can be retrieved with |
//| MinBLEICOptGuardResults(). |
//| IMPORTANT: gradient check is a high-overhead option which will |
//| cost you about 3*N additional function evaluations. |
//| In many cases it may cost as much as the rest of the |
//| optimization session. |
//| YOU SHOULD NOT USE IT IN THE PRODUCTION CODE UNLESS YOU WANT TO |
//| CHECK DERIVATIVES PROVIDED BY SOME THIRD PARTY. |
//| NOTE: unlike previous incarnation of the gradient checking code, |
//| OptGuard does NOT interrupt optimization even if it |
//| discovers bad gradient. |
//| INPUT PARAMETERS: |
//| State - structure used to store algorithm State |
//| TestStep - verification step used for numerical |
//| differentiation: |
//| * TestStep=0 turns verification off |
//| * TestStep>0 activates verification |
//| You should carefully choose TestStep. Value |
//| which is too large (so large that function |
//| behavior is non-cubic at this scale) will lead |
//| to false alarms. Too short step will result in |
//| rounding errors dominating numerical derivative.|
//| You may use different step for different parameters by means of |
//| setting scale with MinBLEICSetScale(). |
//| === EXPLANATION ================================================ |
//| In order to verify gradient algorithm performs following steps: |
//| * two trial steps are made to X[i]-TestStep*S[i] and |
//| X[i]+TestStep*S[i], where X[i] is i-th component of the |
//| initial point and S[i] is a scale of i-th parameter |
//| * F(X) is evaluated at these trial points |
//| * we perform one more evaluation in the middle point of the |
//| interval |
//| * we build cubic model using function values and derivatives at|
//| trial points and we compare its prediction with actual value |
//| in the middle point |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICOptGuardGradient(CMinBLEICState &State,
double &teststep)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(teststep),__FUNCTION__+": TestStep contains NaN or INF"))
return;
if(!CAp::Assert(teststep>=0.0,__FUNCTION__+": invalid argument TestStep(TestStep<0)"))
return;
State.m_teststep=teststep;
}
//+------------------------------------------------------------------+
//| This function activates/deactivates nonsmoothness monitoring |
//| option of the OptGuard integrity checker. Smoothness monitor |
//| silently observes solution process and tries to detect ill-posed |
//| problems, i.e. ones with: |
//| a) discontinuous target function (non-C0) |
//| b) nonsmooth target function (non-C1) |
//| Smoothness monitoring does NOT interrupt optimization even if it |
//| suspects that your problem is nonsmooth. It just sets |
//| corresponding flags in the OptGuard report which can be retrieved|
//| after optimization is over. |
//| Smoothness monitoring is a moderate overhead option which often |
//| adds less than 1% to the optimizer running time. Thus, you can |
//| use it even for large scale problems. |
//| NOTE: OptGuard does NOT guarantee that it will always detect |
//| C0/C1 continuity violations. |
//| First, minor errors are hard to catch - say, a 0.0001 difference |
//| in the model values at two sides of the gap may be due |
//| to discontinuity of the model - or simply because the model has |
//| changed. |
//| Second, C1-violations are especially difficult to detect in a |
//| noninvasive way. The optimizer usually performs very short steps |
//| near the nonsmoothness, and differentiation usually introduces a |
//| lot of numerical noise. It is hard to tell whether some tiny |
//| discontinuity in the slope is due to real nonsmoothness or just |
//| due to numerical noise alone. |
//| Our top priority was to avoid false positives, so in some rare |
//| cases minor errors may went unnoticed (however, in most cases |
//| they can be spotted with restart from different initial point). |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| Level - monitoring level: |
//| * 0 - monitoring is disabled |
//| * 1 - noninvasive low-overhead monitoring; function|
//| values and/or gradients are recorded, but |
//| OptGuard does not try to perform additional |
//| evaluations in order to get more information |
//| about suspicious locations. |
//| === EXPLANATION ================================================ |
//| One major source of headache during optimization is the |
//| possibility of the coding errors in the target function / |
//| constraints (or their gradients). Such errors most often manifest|
//| themselves as discontinuity or nonsmoothness of the target / |
//| constraints. |
//| Another frequent situation is when you try to optimize something |
//| involving lots of min() and max() operations, i.e. nonsmooth |
//| target. Although not a coding error, it is nonsmoothness anyway -|
//| and smooth optimizers usually stop right after encountering |
//| nonsmoothness, well before reaching solution. |
//| OptGuard integrity checker helps you to catch such situations: it|
//| monitors function values/gradients being passed to the optimizer |
//| and tries to errors. Upon discovering suspicious pair of points |
//| it raises appropriate flag (and allows you to continue |
//| optimization). When optimization is done, you can study OptGuard |
//| result. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICOptGuardSmoothness(CMinBLEICState &State,
int level)
{
//--- check
if(!CAp::Assert(level==0 || level==1,__FUNCTION__+": unexpected value of level parameter"))
return;
State.m_smoothnessguardlevel=level;
}
//+------------------------------------------------------------------+
//| Results of OptGuard integrity check, should be called after |
//| optimization session is over. |
//| === PRIMARY REPORT ============================================= |
//| OptGuard performs several checks which are intended to catch |
//| common errors in the implementation of nonlinear function / |
//| gradient: |
//| * incorrect analytic gradient |
//| * discontinuous (non-C0) target functions (constraints) |
//| * nonsmooth (non-C1) target functions (constraints) |
//| Each of these checks is activated with appropriate function: |
//| * MinBLEICOptGuardGradient() for gradient verification |
//| * MinBLEICOptGuardSmoothness() for C0/C1 checks |
//| Following flags are set when these errors are suspected: |
//| * rep.badgradsuspected, and additionally: |
//| * rep.badgradvidx for specific variable (gradient element) |
//| suspected |
//| * rep.badgradxbase, a point where gradient is tested |
//| * rep.badgraduser, user-provided gradient (stored as 2D |
//| matrix with single row in order to make report structure |
//| compatible with more complex optimizers like MinNLC or |
//| MinLM) |
//| * rep.badgradnum, reference gradient obtained via numerical |
//| differentiation (stored as 2D matrix with single row in |
//| order to make report structure compatible with more |
//| complex optimizers like MinNLC or MinLM) |
//| * rep.nonc0suspected |
//| * rep.nonc1suspected |
//| === ADDITIONAL REPORTS/LOGS ==================================== |
//| Several different tests are performed to catch C0/C1 errors, you |
//| can find out specific test signaled error by looking to: |
//| * rep.nonc0test0positive, for non-C0 test #0 |
//| * rep.nonc1test0positive, for non-C1 test #0 |
//| * rep.nonc1test1positive, for non-C1 test #1 |
//| Additional information (including line search logs) can be |
//| obtained by means of: |
//| * MinBLEICOptGuardNonC1Test0Results() |
//| * MinBLEICOptGuardNonC1Test1Results() |
//| which return detailed error reports, specific points where |
//| discontinuities were found, and so on. |
//| ================================================================ |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| Rep - generic OptGuard report; more detailed reports |
//| can be retrieved with other functions. |
//| NOTE: false negatives (nonsmooth problems are not identified as |
//| nonsmooth ones) are possible although unlikely. |
//| The reason is that you need to make several evaluations around |
//| nonsmoothness in order to accumulate enough information about |
//| function curvature. Say, if you start right from the nonsmooth |
//| point, optimizer simply won't get enough data to understand what |
//| is going wrong before it terminates due to abrupt changes in the |
//| derivative. It is also possible that "unlucky" step will move us |
//| to the termination too quickly. |
//| Our current approach is to have less than 0.1% false negatives in|
//| our test examples (measured with multiple restarts from random |
//| points), and to have exactly 0% false positives. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICOptGuardResults(CMinBLEICState &State,
COptGuardReport &rep)
{
COptServ::SmoothnessMonitorExportReport(State.m_smonitor,rep);
}
//+------------------------------------------------------------------+
//| Detailed results of the OptGuard integrity check for |
//| nonsmoothness test #0 |
//| Nonsmoothness (non-C1) test #0 studies function values (not |
//| gradient!) obtained during line searches and monitors behavior |
//| of the directional derivative estimate. |
//| This test is less powerful than test #1, but it does not depend |
//| on the gradient values and thus it is more robust against |
//| artifacts introduced by numerical differentiation. |
//| Two reports are returned: |
//| *a "strongest" one, corresponding to line search which had |
//| highest value of the nonsmoothness indicator |
//| *a "longest" one, corresponding to line search which had more |
//| function evaluations, and thus is more detailed |
//| In both cases following fields are returned: |
//| * positive - is TRUE when test flagged suspicious point; |
//| FALSE if test did not notice anything |
//| (in the latter cases fields below are empty). |
//| * x0[], d[] - arrays of length N which store initial point and |
//| direction for line search (d[] can be normalized,|
//| but does not have to) |
//| * stp[], f[]- arrays of length CNT which store step lengths and|
//| function values at these points; f[i] is |
//| evaluated in x0+stp[i]*d. |
//| * stpidxa, stpidxb - we suspect that function violates C1 |
//| continuity between steps #stpidxa and #stpidxb |
//| (usually we have stpidxb=stpidxa+3, with most |
//| likely position of the violation between |
//| stpidxa+1 and stpidxa+2. |
//| ================================================================ |
//| = SHORTLY SPEAKING: build a 2D plot of (stp,f) and look at it - |
//| = you will see where C1 continuity is violated.|
//| ================================================================ |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| StrRep - C1 test #0 "strong" report |
//| LngRep - C1 test #0 "long" report |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICOptGuardNonC1Test0Results(CMinBLEICState &State,
COptGuardNonC1Test0Report &strrep,
COptGuardNonC1Test0Report &lngrep)
{
COptGuardApi::SmoothnessMonitorExportC1Test0Report(State.m_smonitor.m_nonc1test0strrep,State.m_lastscaleused,strrep);
COptGuardApi::SmoothnessMonitorExportC1Test0Report(State.m_smonitor.m_nonc1test0lngrep,State.m_lastscaleused,lngrep);
}
//+------------------------------------------------------------------+
//| Detailed results of the OptGuard integrity check for |
//| nonsmoothness test #1 |
//| Nonsmoothness (non-C1) test #1 studies individual components of |
//| the gradient computed during line search. |
//| When precise analytic gradient is provided this test is more |
//| powerful than test #0 which works with function values and |
//| ignores user-provided gradient. However, test #0 becomes more |
//| powerful when numerical differentiation is employed (in such |
//| cases test #1 detects higher levels of numerical noise and |
//| becomes too conservative). |
//| This test also tells specific components of the gradient which |
//| violate C1 continuity, which makes it more informative than #0, |
//| which just tells that continuity is violated. |
//| Two reports are returned: |
//| *a "strongest" one, corresponding to line search which had |
//| highest value of the nonsmoothness indicator |
//| *a "longest" one, corresponding to line search which had more |
//| function evaluations, and thus is more detailed |
//| In both cases following fields are returned: |
//| * positive - is TRUE when test flagged suspicious point; |
//| FALSE if test did not notice anything |
//| (in the latter cases fields below are empty).|
//| * vidx - is an index of the variable in [0,N) with |
//| nonsmooth derivative |
//| * x0[], d[] - arrays of length N which store initial point and|
//| direction for line search (d[] can be normalized|
//| but does not have to) |
//| * stp[], g[]- arrays of length CNT which store step lengths |
//| and gradient values at these points; g[i] is |
//| evaluated in x0+stp[i]*d and contains vidx-th |
//| component of the gradient. |
//| * stpidxa, stpidxb - we suspect that function violates C1 |
//| continuity between steps #stpidxa and #stpidxb |
//| (usually we have stpidxb=stpidxa+3, with most |
//| likely position of the violation between |
//| stpidxa+1 and stpidxa+2. |
//| ================================================================ |
//| = SHORTLY SPEAKING: build a 2D plot of (stp,f) and look at it - |
//| = you will see where C1 continuity is violated.|
//| ================================================================ |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| StrRep - C1 test #1 "strong" report |
//| LngRep - C1 test #1 "long" report |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICOptGuardNonC1Test1Results(CMinBLEICState &State,
COptGuardNonC1Test1Report &strrep,
COptGuardNonC1Test1Report &lngrep)
{
COptGuardApi::SmoothnessMonitorExportC1Test1Report(State.m_smonitor.m_nonc1test1strrep,State.m_lastscaleused,strrep);
COptGuardApi::SmoothnessMonitorExportC1Test1Report(State.m_smonitor.m_nonc1test1lngrep,State.m_lastscaleused,lngrep);
}
//+------------------------------------------------------------------+
//| BLEIC results |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| X - array[0..m_n-1], solution |
//| Rep - optimization report. You should check Rep. |
//| TerminationType in order to distinguish |
//| successful termination from unsuccessful one. |
//| More information about fields of this structure |
//| can be found in the comments on MinBLEICReport |
//| datatype. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICResults(CMinBLEICState &State,double &x[],
CMinBLEICReport &rep)
{
//--- reset memory
ArrayResize(x,0);
//--- function call
MinBLEICResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICResults(CMinBLEICState &State,CRowDouble &x,
CMinBLEICReport &rep)
{
//--- reset memory
x.Resize(0);
//--- function call
MinBLEICResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| BLEIC results |
//| Buffered implementation of MinBLEICResults() which uses |
//| pre-allocated buffer to store X[]. If buffer size is too small, |
//| it resizes buffer. It is intended to be used in the inner cycles |
//| of performance critical algorithms where array reallocation |
//| penalty is too large to be ignored. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICResultsBuf(CMinBLEICState &State,double &x[],
CMinBLEICReport &rep)
{
CRowDouble X;
MinBLEICResultsBuf(State,X,rep);
X.ToArray(x);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICResultsBuf(CMinBLEICState &State,CRowDouble &x,
CMinBLEICReport &rep)
{
//--- change values
rep.m_iterationscount=State.m_repinneriterationscount;
rep.m_inneriterationscount=State.m_repinneriterationscount;
rep.m_outeriterationscount=State.m_repouteriterationscount;
rep.m_nfev=State.m_repnfev;
rep.m_varidx=State.m_repvaridx;
rep.m_terminationtype=State.m_repterminationtype;
//--- check
if(State.m_repterminationtype>0)
x=State.m_sas.m_xc;
else
x=vector<double>::Full(State.m_nmain,AL_NaN);
//--- change values
rep.m_debugeqerr=State.m_repdebugeqerr;
rep.m_debugfs=State.m_repdebugfs;
rep.m_debugff=State.m_repdebugff;
rep.m_debugdx=State.m_repdebugdx;
rep.m_debugfeasqpits=State.m_repdebugfeasqpits;
rep.m_debugfeasgpaits=State.m_repdebugfeasgpaits;
}
//+------------------------------------------------------------------+
//| This subroutine restarts algorithm from new point. |
//| All optimization parameters (including constraints) are left |
//| unchanged. |
//| This function allows to solve multiple optimization problems |
//| (which must have same number of dimensions) without object |
//| reallocation penalty. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with |
//| MinBLEICCreate call. |
//| X - new starting point. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICRestartFrom(CMinBLEICState &State,double &x[])
{
CRowDouble X=x;
MinBLEICRestartFrom(State,X);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICRestartFrom(CMinBLEICState &State,CRowDouble &x)
{
int n=State.m_nmain;
//--- First,check for errors in the inputs
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- Set XC
State.m_xstart=x;
State.m_xstart.Resize(n);
//--- prepare RComm facilities
State.m_rstate.ia.Resize(7);
ArrayResize(State.m_rstate.ba,1);
State.m_rstate.ra.Resize(6);
State.m_rstate.stage=-1;
//--- function call
ClearRequestFields(State);
CSActiveSets::SASStopOptimization(State.m_sas);
}
//+------------------------------------------------------------------+
//| This subroutine submits request for termination of running |
//| optimizer. It should be called from user-supplied callback when |
//| user decides that it is time to "smoothly" terminate optimization|
//| process. As result, optimizer stops at point which was "current |
//| accepted" when termination request was submitted and returns |
//| error code 8 (successful termination). |
//| INPUT PARAMETERS: |
//| State - optimizer structure |
//| NOTE: after request for termination optimizer may perform |
//| several additional calls to user-supplied callbacks. It |
//| does NOT guarantee to stop immediately - it just guarantees|
//| that these additional calls will be discarded later. |
//| NOTE: calling this function on optimizer which is NOT running |
//| will have no effect. |
//| NOTE: multiple calls to this function are possible. First call is|
//| counted, subsequent calls are silently ignored. |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICRequestTermination(CMinBLEICState &State)
{
State.m_userterminationneeded=true;
}
//+------------------------------------------------------------------+
//| This subroutine finalizes internal structures after emergency |
//| termination from State.LSStart report (see comments on |
//| MinBLEICState for more information). |
//| INPUT PARAMETERS: |
//| State - structure after exit from LSStart report |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICEmergencyTermination(CMinBLEICState &State)
{
CSActiveSets::SASStopOptimization(State.m_sas);
}
//+------------------------------------------------------------------+
//| Clears request fileds (to be sure that we don't forget to clear |
//| something) |
//+------------------------------------------------------------------+
void CMinBLEIC::ClearRequestFields(CMinBLEICState &State)
{
//--- change values
State.m_needf=false;
State.m_needfg=false;
State.m_xupdated=false;
State.m_lsstart=false;
}
//+------------------------------------------------------------------+
//| Internal initialization subroutine |
//+------------------------------------------------------------------+
void CMinBLEIC::MinBLEICInitInternal(const int n,CRowDouble &x,
const double diffstep,
CMinBLEICState &State)
{
//--- create matrix
CMatrixDouble c;
//--- create array
CRowInt ct;
//--- initialization
State.m_teststep=0;
State.m_smoothnessguardlevel=0;
COptServ::SmoothnessMonitorInit(State.m_smonitor,State.m_s,0,0,false);
State.m_nmain=n;
State.m_diffstep=diffstep;
CSActiveSets::SASInit(n,State.m_sas);
//--- allocation
ArrayResize(State.m_HasBndL,n);
ArrayResize(State.m_HasBndU,n);
State.m_bndl=vector<double>::Full(n,AL_NEGINF);
State.m_bndu=vector<double>::Full(n,AL_POSINF);
State.m_xstart.Resize(n);
State.m_cgc.Resize(n);
State.m_ugc.Resize(n);
State.m_xn.Resize(n);
State.m_cgn.Resize(n);
State.m_ugn.Resize(n);
State.m_xp.Resize(n);
State.m_d.Resize(n);
State.m_s=vector<double>::Ones(n);
State.m_invs=vector<double>::Ones(n);
State.m_lastscaleused=vector<double>::Ones(n);
State.m_x.Resize(n);
State.m_g.Resize(n);
State.m_work.Resize(n);
ArrayInitialize(State.m_HasBndL,false);
ArrayInitialize(State.m_HasBndU,false);
//--- function call
MinBLEICSetLC(State,c,ct,0);
//--- function call
MinBLEICSetCond(State,0.0,0.0,0.0,0);
//--- function call
MinBLEICSetXRep(State,false);
//--- function call
MinBLEICSetDRep(State,false);
//--- function call
MinBLEICSetStpMax(State,0.0);
//--- function call
MinBLEICSetPrecDefault(State);
//--- function call
MinBLEICRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| NOTES: |
//| 1. This function has two different implementations: one which |
//| uses exact (analytical) user-supplied gradient, and one which |
//| uses function value only and numerically differentiates |
//| function in order to obtain gradient. |
//| Depending on the specific function used to create optimizer |
//| object (either MinBLEICCreate() for analytical gradient or |
//| MinBLEICCreateF() for numerical differentiation) you should |
//| choose appropriate variant of MinBLEICOptimize() - one which |
//| accepts function AND gradient or one which accepts function |
//| ONLY. |
//| Be careful to choose variant of MinBLEICOptimize() which |
//| corresponds to your optimization scheme! Table below lists |
//| different combinations of callback (function/gradient) passed |
//| to MinBLEICOptimize() and specific function used to create |
//| optimizer. |
//| | USER PASSED TO MinBLEICOptimize() |
//| CREATED WITH | function only | function and gradient |
//| ------------------------------------------------------------ |
//| MinBLEICCreateF() | work FAIL |
//| MinBLEICCreate() | FAIL work |
//| Here "FAIL" denotes inappropriate combinations of optimizer |
//| creation function and MinBLEICOptimize() version. Attemps to |
//| use such combination (for example, to create optimizer with |
//| MinBLEICCreateF() and to pass gradient information to |
//| MinCGOptimize()) will lead to exception being thrown. Either |
//| you did not pass gradient when it WAS needed or you passed |
//| gradient when it was NOT needed. |
//+------------------------------------------------------------------+
bool CMinBLEIC::MinBLEICIteration(CMinBLEICState &State)
{
//--- create variables
int n=0;
int m=0;
int i=0;
int j=0;
double v=0;
double vv=0;
double v0=0;
bool b=false;
int mcinfo=0;
int actstatus=0;
int itidx=0;
double penalty=0;
double ginit=0;
double gdecay=0;
int i_=0;
int label=-1;
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(State.m_rstate.stage>=0)
{
n=State.m_rstate.ia[0];
m=State.m_rstate.ia[1];
i=State.m_rstate.ia[2];
j=State.m_rstate.ia[3];
mcinfo=State.m_rstate.ia[4];
actstatus=State.m_rstate.ia[5];
itidx=State.m_rstate.ia[6];
b=State.m_rstate.ba[0];
v=State.m_rstate.ra[0];
vv=State.m_rstate.ra[1];
v0=State.m_rstate.ra[2];
penalty=State.m_rstate.ra[3];
ginit=State.m_rstate.ra[4];
gdecay=State.m_rstate.ra[5];
}
else
{
n=359;
m=-58;
i=-919;
j=-909;
mcinfo=81;
actstatus=255;
itidx=74;
b=false;
v=809;
vv=205;
v0=-838;
penalty=939;
ginit=-526;
gdecay=763;
}
switch(State.m_rstate.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
case 3:
label=3;
break;
case 4:
label=4;
break;
case 5:
label=5;
break;
case 6:
label=6;
break;
case 7:
label=7;
break;
case 8:
label=8;
break;
case 9:
label=9;
break;
case 10:
label=10;
break;
case 11:
label=11;
break;
case 12:
label=12;
break;
case 13:
label=13;
break;
case 14:
label=14;
break;
case 15:
label=15;
break;
case 16:
label=16;
break;
case 17:
label=17;
break;
case 18:
label=18;
break;
case 19:
label=19;
break;
case 20:
label=20;
break;
case 21:
label=21;
break;
default:
//--- Routine body
//--- Algorithm parameters:
//--- * M number of L-BFGS corrections.
//--- This coefficient remains fixed during iterations.
//--- * GDecay desired decrease of constrained gradient during L-BFGS iterations.
//--- This coefficient is decreased after each L-BFGS round until
//--- it reaches minimum decay.
m=MathMin(5,State.m_nmain);
gdecay=m_initialdecay;
//--- Init
n=State.m_nmain;
State.m_steepestdescentstep=false;
State.m_userterminationneeded=false;
State.m_repterminationtype=0;
State.m_repinneriterationscount=0;
State.m_repouteriterationscount=0;
State.m_repnfev=0;
State.m_repvaridx=-1;
State.m_repdebugeqerr=0.0;
State.m_repdebugfs=AL_NaN;
State.m_repdebugff=AL_NaN;
State.m_repdebugdx=AL_NaN;
if(State.m_stpmax!=0.0 && State.m_prectype!=0)
{
State.m_repterminationtype=-10;
return(false);
}
CApServ::RMatrixSetLengthAtLeast(State.m_bufyk,m+1,n);
CApServ::RMatrixSetLengthAtLeast(State.m_bufsk,m+1,n);
CApServ::RVectorSetLengthAtLeast(State.m_bufrho,m);
CApServ::RVectorSetLengthAtLeast(State.m_buftheta,m);
CApServ::RVectorSetLengthAtLeast(State.m_tmp0,n);
COptServ::SmoothnessMonitorInit(State.m_smonitor,State.m_s,n,1,State.m_smoothnessguardlevel>0);
State.m_lastscaleused=State.m_s;
State.m_invs=vector<double>::Ones(n)/State.m_s.ToVector();
//--- Check analytic derivative
ClearRequestFields(State);
if(!(State.m_diffstep==0.0 && State.m_teststep>0.0))
{
label=22;
break;
}
label=24;
break;
}
//--- main loop
while(label>=0)
switch(label)
{
case 24:
if(!COptServ::SmoothnessMonitorCheckGradientATX0(State.m_smonitor,State.m_xstart,State.m_s,State.m_bndl,State.m_bndu,true,State.m_teststep))
{
label=25;
break;
}
State.m_x=State.m_smonitor.m_x;
State.m_needfg=true;
State.m_rstate.stage=0;
label=-1;
break;
case 0:
State.m_needfg=false;
State.m_smonitor.m_fi.Set(0,State.m_f);
State.m_smonitor.m_j.Row(0,State.m_g);
label=24;
break;
case 25:
case 22:
//--- Fill TmpPrec with current preconditioner
switch(State.m_prectype)
{
case 2:
State.m_tmpprec=State.m_diagh;
break;
case 3:
State.m_tmpprec=State.m_s.Pow(-2.0)+0;
break;
default:
State.m_tmpprec=vector<double>::Ones(n);
}
CSActiveSets::SASSetPrecDiag(State.m_sas,State.m_tmpprec);
//--- Start optimization
if(!CSActiveSets::SASStartOptimization(State.m_sas,State.m_xstart))
{
State.m_repterminationtype=-3;
return(false);
}
//--- Main cycle of BLEIC-PG algorithm
State.m_repterminationtype=0;
State.m_lastgoodstep=0;
State.m_lastscaledgoodstep=0;
State.m_maxscaledgrad=0;
State.m_nonmonotoniccnt=(int)MathRound(1.5*(n+State.m_nic))+5;
State.m_x=State.m_sas.m_xc;
ClearRequestFields(State);
if(State.m_diffstep!=0.0)
{
label=26;
break;
}
State.m_needfg=true;
State.m_rstate.stage=1;
label=-1;
break;
case 1:
State.m_needfg=false;
label=27;
break;
case 26:
State.m_needf=true;
State.m_rstate.stage=2;
label=-1;
break;
case 2:
State.m_needf=false;
case 27:
State.m_fc=State.m_f;
COptServ::TrimPrepare(State.m_f,State.m_trimthreshold);
State.m_repnfev++;
if(!State.m_xrep)
{
label=28;
break;
}
//--- Report current point
State.m_x=State.m_sas.m_xc;
State.m_f=State.m_fc;
State.m_xupdated=true;
State.m_rstate.stage=3;
label=-1;
break;
case 3:
State.m_xupdated=false;
case 28:
if(State.m_userterminationneeded)
{
//--- User requested termination
CSActiveSets::SASStopOptimization(State.m_sas);
State.m_repterminationtype=8;
return(false);
}
case 30:
//--- Preparations
//--- (a) calculate unconstrained gradient
//--- (b) determine initial active set
//--- (c) update MaxScaledGrad
//--- (d) check F/G for NAN/INF, abnormally terminate algorithm if needed
State.m_x=State.m_sas.m_xc;
ClearRequestFields(State);
if(State.m_diffstep!=0.0)
{
label=32;
break;
}
//--- Analytic gradient
State.m_needfg=true;
State.m_rstate.stage=4;
label=-1;
break;
case 4:
State.m_needfg=false;
label=33;
break;
case 32:
//--- Numerical differentiation
State.m_needf=true;
State.m_rstate.stage=5;
label=-1;
break;
case 5:
State.m_fbase=State.m_f;
i=0;
case 34:
if(i>n-1)
{
label=36;
break;
}
v=State.m_x[i];
b=false;
if(State.m_HasBndL[i])
b=b || (v-State.m_diffstep*State.m_s[i])<State.m_bndl[i];
if(State.m_HasBndU[i])
b=b || (v+State.m_diffstep*State.m_s[i])>State.m_bndu[i];
if(b)
{
label=37;
break;
}
State.m_x.Set(i,v-State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=6;
label=-1;
break;
case 6:
State.m_fm2=State.m_f;
State.m_x.Set(i,v-0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=7;
label=-1;
break;
case 7:
State.m_fm1=State.m_f;
State.m_x.Set(i,v+0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=8;
label=-1;
break;
case 8:
State.m_fp1=State.m_f;
State.m_x.Set(i,v+State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=9;
label=-1;
break;
case 9:
State.m_fp2=State.m_f;
State.m_g.Set(i,(8*(State.m_fp1-State.m_fm1)-(State.m_fp2-State.m_fm2))/(6*State.m_diffstep*State.m_s[i]));
label=38;
break;
case 37:
State.m_xm1=v-State.m_diffstep*State.m_s[i];
State.m_xp1=v+State.m_diffstep*State.m_s[i];
if(State.m_HasBndL[i] && State.m_xm1<State.m_bndl[i])
State.m_xm1=State.m_bndl[i];
if(State.m_HasBndU[i] && State.m_xp1>State.m_bndu[i])
State.m_xp1=State.m_bndu[i];
State.m_x.Set(i,State.m_xm1);
State.m_rstate.stage=10;
label=-1;
break;
case 10:
State.m_fm1=State.m_f;
State.m_x.Set(i,State.m_xp1);
State.m_rstate.stage=11;
label=-1;
break;
case 11:
State.m_fp1=State.m_f;
if(State.m_xm1!=State.m_xp1)
State.m_g.Set(i,(State.m_fp1-State.m_fm1)/(State.m_xp1-State.m_xm1));
else
State.m_g.Set(i,0);
case 38:
State.m_x.Set(i,v);
i++;
label=34;
break;
case 36:
State.m_f=State.m_fbase;
State.m_needf=false;
case 33:
State.m_fc=State.m_f;
State.m_ugc=State.m_g;
State.m_cgc=State.m_g;
CSActiveSets::SASReactivateConstraintsPrec(State.m_sas,State.m_ugc);
CSActiveSets::SASConstrainedDirection(State.m_sas,State.m_cgc);
ginit=MathPow(State.m_cgc*State.m_s+0,2).Sum();
ginit=MathSqrt(ginit);
State.m_maxscaledgrad=MathMax(State.m_maxscaledgrad,ginit);
if(!MathIsValidNumber(ginit) || !MathIsValidNumber(State.m_fc))
{
//--- Abnormal termination - infinities in function/gradient
CSActiveSets::SASStopOptimization(State.m_sas);
State.m_repterminationtype=-8;
return(false);
}
if(State.m_userterminationneeded)
{
//--- User requested termination
CSActiveSets::SASStopOptimization(State.m_sas);
State.m_repterminationtype=8;
return(false);
}
//--- LBFGS stage:
//--- * during LBFGS iterations we activate new constraints, but never
//--- deactivate already active ones.
//--- * we perform at most N iterations of LBFGS before re-evaluating
//--- active set and restarting LBFGS.
//--- * first iteration of LBFGS is a special - it is performed with
//--- minimum set of active constraints, algorithm termination can
//--- be performed only at this State.m_ We call this iteration
//--- "steepest descent step".
//--- About termination:
//--- * LBFGS iterations can be terminated because of two reasons:
//--- *"termination" - non-zero termination code in RepTerminationType,
//--- which means that optimization is done
//--- *"restart" - zero RepTerminationType, which means that we
//--- have to re-evaluate active set and resume LBFGS stage.
//---*one more option is "refresh" - to continue LBFGS iterations,
//--- but with all BFGS updates (Sk/Yk pairs) being dropped;
//--- it happens after changes in active set
State.m_bufsize=0;
State.m_steepestdescentstep=true;
itidx=-1;
case 39:
if(itidx>=n-1)
{
label=40;
break;
}
//--- Increment iterations counter
//--- NOTE: we have strong reasons to use such complex scheme
//--- instead of just for() loop - this counter may be
//--- decreased at some occasions to perform "restart"
//--- of an iteration.
itidx++;
//--- At the beginning of each iteration:
//--- * SAS.XC stores current point
//--- * FC stores current function value
//--- * UGC stores current unconstrained gradient
//--- * CGC stores current constrained gradient
//--- * D stores constrained step direction (calculated at this block)
//--- Check gradient-based stopping criteria
//--- This stopping condition is tested only for step which is the
//--- first step of LBFGS (subsequent steps may accumulate active
//--- constraints thus they should NOT be used for stopping - gradient
//--- may be small when constrained, but these constraints may be
//--- deactivated by the subsequent steps)
if(State.m_steepestdescentstep &&
CSActiveSets::SASScaledConstrainedNorm(State.m_sas,State.m_ugc)<=State.m_epsg)
{
//--- Gradient is small enough.
//--- Optimization is terminated
State.m_repterminationtype=4;
label=40;
break;
}
//--- 1. Calculate search direction D according to L-BFGS algorithm
//--- using constrained preconditioner to perform inner multiplication.
//--- 2. Evaluate scaled length of direction D; restart LBFGS if D is zero
//--- (it may be possible that we found minimum, but it is also possible
//--- that some constraints need deactivation)
//--- 3. If D is non-zero, try to use previous scaled step length as initial estimate for new step.
State.m_work=State.m_cgc;
for(i=State.m_bufsize-1; i>=0; i--)
{
v=CAblasF::RDotVR(n,State.m_work,State.m_bufsk,i);
State.m_buftheta.Set(i,v);
vv=v*State.m_bufrho[i];
for(i_=0; i_<n; i_++)
State.m_work.Add(i_,- State.m_bufyk.Get(i,i_)*vv);
}
CSActiveSets::SASConstrainedDirectionPrec(State.m_sas,State.m_work);
for(i=0; i<State.m_bufsize; i++)
{
v=CAblasF::RDotVR(n,State.m_work,State.m_bufyk,i);
vv=State.m_bufrho[i]*(-v+State.m_buftheta[i]);
for(i_=0; i_<n; i_++)
State.m_work.Add(i_,State.m_bufsk.Get(i,i_)*vv);
}
CSActiveSets::SASConstrainedDirection(State.m_sas,State.m_work);
State.m_d=State.m_work.ToVector()*(-1.0);
v=MathPow(State.m_d/State.m_s+0,2.0).Sum();
v=MathSqrt(v);
if(v==0.0)
{
//--- Search direction is zero.
//--- If we perform "steepest descent step", algorithm is terminated.
//--- Otherwise we just restart LBFGS.
if(State.m_steepestdescentstep)
State.m_repterminationtype=4;
label=40;
break;
}
//--- check
if(!CAp::Assert(v>0.0,__FUNCTION__+": internal error"))
return(false);
if(State.m_lastscaledgoodstep>0.0 && v>0.0)
State.m_stp=State.m_lastscaledgoodstep/v;
else
State.m_stp=1.0/v;
//--- Calculate bound on step length.
//--- Step direction is stored
CSActiveSets::SASExploreDirection(State.m_sas,State.m_d,State.m_curstpmax,State.m_cidx,State.m_cval);
State.m_activationstep=State.m_curstpmax;
if(State.m_cidx>=0 && State.m_activationstep==0.0)
{
//--- We are exactly at the boundary, immediate activation
//--- of constraint is required. LBFGS stage is continued
//--- with "refreshed" model.
//--- ! IMPORTANT: we do not clear SteepestDescent flag here,
//--- ! it is very important for correct stopping
//--- ! of algorithm.
//--- ! IMPORTANT: we decrease iteration counter in order to
//--- preserve computational budget for iterations.
CSActiveSets::SASImmediateActivation(State.m_sas,State.m_cidx,State.m_cval);
State.m_bufsize=0;
itidx=itidx-1;
label=39;
break;
}
if(State.m_stpmax>0.0)
{
v=CAblasF::RDotV2(n,State.m_d);
v=MathSqrt(v);
if(v>0.0)
State.m_curstpmax=MathMin(State.m_curstpmax,State.m_stpmax/v);
}
//--- Report beginning of line search (if requested by caller).
//--- See description of the MinBLEICState for more information
//--- about fields accessible to caller.
//--- Caller may do following:
//--- * change State.Stp and load better initial estimate of
//--- the step length.
//--- Caller may not terminate algorithm.
if(!State.m_drep)
{
label=41;
break;
}
ClearRequestFields(State);
State.m_lsstart=true;
State.m_boundedstep=State.m_cidx>=0;
State.m_x=State.m_sas.m_xc;
State.m_rstate.stage=12;
label=-1;
break;
case 12:
State.m_lsstart=false;
case 41:
//--- Minimize F(x+alpha*d)
State.m_xn=State.m_sas.m_xc;
State.m_cgn=State.m_cgc;
State.m_ugn=State.m_ugc;
State.m_fn=State.m_fc;
State.m_mcstage=0;
COptServ::SmoothnessMonitorStartLineSearch1u(State.m_smonitor,State.m_s,State.m_invs,State.m_xn,State.m_fn,State.m_ugn);
CLinMin::MCSrch(n,State.m_xn,State.m_fn,State.m_ugn,State.m_d,State.m_stp,State.m_curstpmax,m_gtol,mcinfo,State.m_nfev,State.m_work,State.m_lstate,State.m_mcstage);
case 43:
if(State.m_mcstage==0)
{
label=44;
break;
}
//--- Perform correction (constraints are enforced)
//--- Copy XN to X
CSActiveSets::SASCorrection(State.m_sas,State.m_xn,penalty);
State.m_x=State.m_xn;
//--- Gradient, either user-provided or numerical differentiation
ClearRequestFields(State);
if(State.m_diffstep!=0.0)
{
label=45;
break;
}
//--- Analytic gradient
State.m_needfg=true;
State.m_rstate.stage=13;
label=-1;
break;
case 13:
State.m_needfg=false;
State.m_repnfev++;
label=46;
break;
case 45:
//--- Numerical differentiation
State.m_needf=true;
State.m_rstate.stage=14;
label=-1;
break;
case 14:
State.m_fbase=State.m_f;
i=0;
case 47:
if(i>n-1)
{
label=49;
break;
}
v=State.m_x[i];
b=false;
if(State.m_HasBndL[i])
b=b || (v-State.m_diffstep*State.m_s[i])<State.m_bndl[i];
if(State.m_HasBndU[i])
b=b || (v+State.m_diffstep*State.m_s[i])>State.m_bndu[i];
if(b)
{
label=50;
break;
}
State.m_x.Set(i,v-State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=15;
label=-1;
break;
case 15:
State.m_fm2=State.m_f;
State.m_x.Set(i,v-0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=16;
label=-1;
break;
case 16:
State.m_fm1=State.m_f;
State.m_x.Set(i,v+0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=17;
label=-1;
break;
case 17:
State.m_fp1=State.m_f;
State.m_x.Set(i,v+State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=18;
label=-1;
break;
case 18:
State.m_fp2=State.m_f;
State.m_g.Set(i,(8*(State.m_fp1-State.m_fm1)-(State.m_fp2-State.m_fm2))/(6*State.m_diffstep*State.m_s[i]));
State.m_repnfev+=4;
label=51;
break;
case 50:
State.m_xm1=v-State.m_diffstep*State.m_s[i];
State.m_xp1=v+State.m_diffstep*State.m_s[i];
if(State.m_HasBndL[i] && State.m_xm1<State.m_bndl[i])
State.m_xm1=State.m_bndl[i];
if(State.m_HasBndU[i] && State.m_xp1>State.m_bndu[i])
State.m_xp1=State.m_bndu[i];
State.m_x.Set(i,State.m_xm1);
State.m_rstate.stage=19;
label=-1;
break;
case 19:
State.m_fm1=State.m_f;
State.m_x.Set(i,State.m_xp1);
State.m_rstate.stage=20;
label=-1;
break;
case 20:
State.m_fp1=State.m_f;
if(State.m_xm1!=State.m_xp1)
State.m_g.Set(i,(State.m_fp1-State.m_fm1)/(State.m_xp1-State.m_xm1));
else
State.m_g.Set(i,0);
State.m_repnfev+=2;
case 51:
State.m_x.Set(i,v);
i++;
label=47;
break;
case 49:
State.m_f=State.m_fbase;
State.m_needf=false;
case 46:
//--- Back to MCSRCH
//--- NOTE: penalty term from correction is added to FN in order
//--- to penalize increase in infeasibility.
COptServ::SmoothnessMonitorEnqueuePoint1u(State.m_smonitor,State.m_s,State.m_invs,State.m_d,State.m_stp,State.m_x,State.m_f,State.m_g);
State.m_fn=State.m_f+m_penaltyfactor*State.m_maxscaledgrad*penalty;
State.m_cgn=State.m_g;
State.m_ugn=State.m_g;
CSActiveSets::SASConstrainedDirection(State.m_sas,State.m_cgn);
COptServ::TrimFunction(State.m_fn,State.m_cgn,n,State.m_trimthreshold);
CLinMin::MCSrch(n,State.m_xn,State.m_fn,State.m_ugn,State.m_d,State.m_stp,State.m_curstpmax,m_gtol,mcinfo,State.m_nfev,State.m_work,State.m_lstate,State.m_mcstage);
label=43;
break;
case 44:
State.m_bufsk.Row(State.m_bufsize,State.m_xn-State.m_sas.m_xc+0);
State.m_bufyk.Row(State.m_bufsize,State.m_cgn-State.m_cgc+0);
COptServ::SmoothnessMonitorFinalizeLineSearch(State.m_smonitor);
//--- Check for presence of NAN/INF in function/gradient
v=State.m_fn;
for(i=0; i<n; i++)
v=0.1*v+State.m_ugn[i];
if(!MathIsValidNumber(v))
{
//--- Abnormal termination - infinities in function/gradient
State.m_repterminationtype=-8;
label=40;
break;
}
//--- Handle possible failure of the line search or request for termination
if(mcinfo!=1 && mcinfo!=5)
{
//--- We can not find step which decreases function value. We have
//--- two possibilities:
//--- (a) numerical properties of the function do not allow us to
//--- find good step.
//--- (b) we are close to activation of some constraint, and it is
//--- so close that step which activates it leads to change in
//--- target function which is smaller than numerical noise.
//--- Optimization algorithm must be able to handle case (b), because
//--- inability to handle it will cause failure when algorithm
//--- started very close to boundary of the feasible area.
//--- In order to correctly handle such cases we allow limited amount
//--- of small steps which increase function value.
v=MathPow(State.m_d/State.m_s*State.m_curstpmax,2.0).Sum();
v=MathSqrt(v);
b=false;
if(State.m_cidx>=0 && v<=m_maxnonmonotoniclen && State.m_nonmonotoniccnt>0)
{
//--- We try to enforce non-monotonic step:
//--- * Stp := CurStpMax
//--- * MCINFO := 5
//--- * XN := XC+CurStpMax*D
//--- * non-monotonic counter is decreased
//--- NOTE: UGN/CGN are not updated because step is so short that we assume that
//--- GN is approximately equal to GC.
//--- NOTE: prior to enforcing such step we check that it does not increase infeasibility
//--- of constraints beyond tolerable level
v=State.m_curstpmax;
State.m_tmp0=State.m_sas.m_xc.ToVector()+State.m_d*v;
State.m_stp=State.m_curstpmax;
mcinfo=5;
State.m_xn=State.m_tmp0;
State.m_nonmonotoniccnt--;
b=true;
}
if(!b)
{
//--- Numerical properties of the function do not allow
//--- us to solve problem. Here we have two possibilities:
//---*if it is "steepest descent" step, we can terminate
//--- algorithm because we are close to minimum
//---*if it is NOT "steepest descent" step, we should restart
//--- LBFGS iterations.
if(State.m_steepestdescentstep)
{
//--- Algorithm is terminated
State.m_repterminationtype=7;
label=40;
break;
}
else
{
//--- Re-evaluate active set and restart LBFGS
label=40;
break;
}
}
}
if(State.m_userterminationneeded)
{
label=40;
break;
}
//--- Current point is updated:
//--- * move XC/FC/GC to XP/FP/GP
//--- * change current point remembered by SAS structure
//--- * move XN/FN/GN to XC/FC/GC
//--- * report current point and update iterations counter
//--- * if MCINFO=1, push new pair SK/YK to LBFGS buffer
State.m_fp=State.m_fc;
State.m_xp=State.m_sas.m_xc;
State.m_fc=State.m_fn;
State.m_cgc=State.m_cgn;
State.m_ugc=State.m_ugn;
actstatus=CSActiveSets::SASMoveTo(State.m_sas,State.m_xn,State.m_cidx>=0 && State.m_stp>=State.m_activationstep,State.m_cidx,State.m_cval);
if(!State.m_xrep)
{
label=52;
break;
}
State.m_x=State.m_sas.m_xc;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=21;
label=-1;
break;
case 21:
State.m_xupdated=false;
case 52:
State.m_repinneriterationscount++;
if(mcinfo==1)
{
//--- Accept new LBFGS update given by Sk,Yk
if(State.m_bufsize==m)
{
//--- Buffer is full, shift contents by one row
for(i=0; i<State.m_bufsize; i++)
{
State.m_bufsk.Row(i,State.m_bufsk,i+1);
State.m_bufyk.Row(i,State.m_bufyk,i+1);
}
for(i=0; i<State.m_bufsize-1; i++)
{
State.m_bufrho.Set(i,State.m_bufrho[i+1]);
State.m_buftheta.Set(i,State.m_buftheta[i+1]);
}
}
else
{
//--- Buffer is not full, increase buffer size by 1
State.m_bufsize++;
}
v=CAblasF::RDotRR(n,State.m_bufyk,State.m_bufsize-1,State.m_bufsk,State.m_bufsize-1);
vv=CAblasF::RDotRR(n,State.m_bufyk,State.m_bufsize-1,State.m_bufyk,State.m_bufsize-1);
if(v==0.0 || vv==0.0)
{
//--- Strange internal error in LBFGS - either YK=0
//--- (which should not have been) or (SK,YK)=0 (again,
//--- unexpected). It should not take place because
//--- MCINFO=1, which signals "good" step. But just
//--- to be sure we have special branch of code which
//--- restarts LBFGS
label=40;
break;
}
State.m_bufrho.Set(State.m_bufsize-1,1/v);
//--- check
if(!CAp::Assert(State.m_bufsize<=m,__FUNCTION__+": internal error"))
return(false);
//--- Update length of the good step
vector<double> temp=State.m_sas.m_xc-State.m_xp;
v=MathPow(temp/State.m_s.ToVector(),2.0).Sum();
vv=temp.Dot(temp);
State.m_lastgoodstep=MathSqrt(vv);
UpdateEstimateOfGoodStep(State.m_lastscaledgoodstep,MathSqrt(v));
}
//--- Check stopping criteria
//--- Step size and function-based stopping criteria are tested only
//--- for step which satisfies Wolfe conditions and is the first step of
//--- LBFGS (subsequent steps may accumulate active constraints thus
//--- they should NOT be used for stopping; step size or function change
//--- may be small when constrained, but these constraints may be
//--- deactivated by the subsequent steps).
//--- MaxIts-based stopping condition is checked for all kinds of steps.
if(mcinfo==1 && State.m_steepestdescentstep)
{
//--- Step is small enough
v=MathPow((State.m_sas.m_xc-State.m_xp)/State.m_s.ToVector(),2.0).Sum();
v=MathSqrt(v);
if(v<=State.m_epsx)
{
State.m_repterminationtype=2;
label=40;
break;
}
//--- Function change is small enough
if(MathAbs(State.m_fp-State.m_fc)<=(State.m_epsf*MathMax(MathAbs(State.m_fc),MathMax(MathAbs(State.m_fp),1.0))))
{
State.m_repterminationtype=1;
label=40;
break;
}
}
if(State.m_maxits>0 && State.m_repinneriterationscount>=State.m_maxits)
{
State.m_repterminationtype=5;
label=40;
break;
}
//--- Clear "steepest descent" flag.
State.m_steepestdescentstep=false;
//--- Smooth reset (LBFGS memory model is refreshed) or hard restart:
//--- * LBFGS model is refreshed, if line search was performed with activation of constraints
//--- * algorithm is restarted if scaled gradient decreased below GDecay
if(actstatus>=0)
{
State.m_bufsize=0;
label=39;
break;
}
v=MathPow(State.m_cgc*State.m_s+0,2.0).Sum();
if(MathSqrt(v)<(gdecay*ginit))
{
label=40;
break;
}
label=39;
break;
case 40:
if(State.m_userterminationneeded)
{
//--- User requested termination
State.m_repterminationtype=8;
label=31;
break;
}
if(State.m_repterminationtype!=0)
{
//--- Algorithm terminated
label=31;
break;
}
//--- Decrease decay coefficient. Subsequent L-BFGS stages will
//--- have more stringent stopping criteria.
gdecay=MathMax(gdecay*m_decaycorrection,m_mindecay);
label=30;
break;
case 31:
CSActiveSets::SASStopOptimization(State.m_sas);
State.m_repouteriterationscount=1;
return(false);
}
//-- Saving State
State.m_rstate.ba[0]=b;
State.m_rstate.ia.Set(0,n);
State.m_rstate.ia.Set(1,m);
State.m_rstate.ia.Set(2,i);
State.m_rstate.ia.Set(3,j);
State.m_rstate.ia.Set(4,mcinfo);
State.m_rstate.ia.Set(5,actstatus);
State.m_rstate.ia.Set(6,itidx);
State.m_rstate.ra.Set(0,v);
State.m_rstate.ra.Set(1,vv);
State.m_rstate.ra.Set(2,v0);
State.m_rstate.ra.Set(3,penalty);
State.m_rstate.ra.Set(4,ginit);
State.m_rstate.ra.Set(5,gdecay);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| This subroutine updates estimate of the good step length given: |
//| 1) previous estimate |
//| 2) new length of the good step |
//| It makes sure that estimate does not change too rapidly - ratio |
//| of new and old estimates will be at least 0.01, at most 100.0 |
//| In case previous estimate of good step is zero (no estimate), |
//| new estimate is used unconditionally. |
//+------------------------------------------------------------------+
void CMinBLEIC::UpdateEstimateOfGoodStep(double &estimate,
double newstep)
{
//--- check
if(estimate==0.0)
{
estimate=newstep;
return;
}
if(newstep<(estimate*0.01))
{
estimate=estimate*0.01;
return;
}
if(newstep>(estimate*100))
{
estimate=estimate*100;
return;
}
estimate=newstep;
}
//+------------------------------------------------------------------+
//| Auxiliary class for CMinLBFGS |
//+------------------------------------------------------------------+
class CMinLBFGSState
{
public:
//--- variables
int m_k;
int m_m;
int m_maxits;
int m_mcstage;
int m_n;
int m_nfev;
int m_p;
int m_preck;
int m_prectype;
int m_q;
int m_repiterationscount;
int m_repnfev;
int m_repterminationtype;
int m_smoothnessguardlevel;
double m_diffstep;
double m_epsf;
double m_epsg;
double m_epsx;
double m_f;
double m_fbase;
double m_fm1;
double m_fm2;
double m_fold;
double m_fp1;
double m_fp2;
double m_gammak;
double m_stp;
double m_stpmax;
double m_teststep;
double m_trimthreshold;
bool m_needf;
bool m_needfg;
bool m_userterminationneeded;
bool m_xrep;
bool m_xupdated;
RCommState m_rstate;
CSmoothnessMonitor m_smonitor;
CRowDouble m_autobuf;
CRowDouble m_d;
CRowDouble m_diagh;
CRowDouble m_g;
CRowDouble m_invs;
CRowDouble m_lastscaleused;
CRowDouble m_precc;
CRowDouble m_precd;
CRowDouble m_rho;
CRowDouble m_s;
CRowDouble m_theta;
CRowDouble m_work;
CRowDouble m_x;
CRowDouble m_xbase;
CRowDouble m_xp;
CPrecBufLowRank m_lowrankbuf;
CPrecBufLBFGS m_precbuf;
CMatrixDouble m_denseh;
CMatrixDouble m_precw;
CMatrixDouble m_sk;
CMatrixDouble m_yk;
CLinMinState m_lstate;
//--- constructor, destructor
CMinLBFGSState(void);
~CMinLBFGSState(void) {}
//--- copy
void Copy(CMinLBFGSState &obj);
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinLBFGSState::CMinLBFGSState(void)
{
m_k=0;
m_m=0;
m_maxits=0;
m_mcstage=0;
m_n=0;
m_nfev=0;
m_p=0;
m_preck=0;
m_prectype=0;
m_q=0;
m_repiterationscount=0;
m_repnfev=0;
m_repterminationtype=0;
m_smoothnessguardlevel=0;
m_diffstep=0;
m_epsf=0;
m_epsg=0;
m_epsx=0;
m_f=0;
m_fbase=0;
m_fm1=0;
m_fm2=0;
m_fold=0;
m_fp1=0;
m_fp2=0;
m_gammak=0;
m_stp=0;
m_stpmax=0;
m_teststep=0;
m_trimthreshold=0;
m_needf=false;
m_needfg=false;
m_userterminationneeded=false;
m_xrep=false;
m_xupdated=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinLBFGSState::Copy(CMinLBFGSState &obj)
{
//--- copy variables
m_k=obj.m_k;
m_m=obj.m_m;
m_maxits=obj.m_maxits;
m_mcstage=obj.m_mcstage;
m_n=obj.m_n;
m_nfev=obj.m_nfev;
m_p=obj.m_p;
m_preck=obj.m_preck;
m_prectype=obj.m_prectype;
m_q=obj.m_q;
m_repiterationscount=obj.m_repiterationscount;
m_repnfev=obj.m_repnfev;
m_repterminationtype=obj.m_repterminationtype;
m_smoothnessguardlevel=obj.m_smoothnessguardlevel;
m_diffstep=obj.m_diffstep;
m_epsf=obj.m_epsf;
m_epsg=obj.m_epsg;
m_epsx=obj.m_epsx;
m_f=obj.m_f;
m_fbase=obj.m_fbase;
m_fm1=obj.m_fm1;
m_fm2=obj.m_fm2;
m_fold=obj.m_fold;
m_fp1=obj.m_fp1;
m_fp2=obj.m_fp2;
m_gammak=obj.m_gammak;
m_stp=obj.m_stp;
m_stpmax=obj.m_stpmax;
m_teststep=obj.m_teststep;
m_trimthreshold=obj.m_trimthreshold;
m_needf=obj.m_needf;
m_needfg=obj.m_needfg;
m_userterminationneeded=obj.m_userterminationneeded;
m_xrep=obj.m_xrep;
m_xupdated=obj.m_xupdated;
m_rstate=obj.m_rstate;
m_smonitor=obj.m_smonitor;
m_autobuf=obj.m_autobuf;
m_d=obj.m_d;
m_diagh=obj.m_diagh;
m_g=obj.m_g;
m_invs=obj.m_invs;
m_lastscaleused=obj.m_lastscaleused;
m_precc=obj.m_precc;
m_precd=obj.m_precd;
m_rho=obj.m_rho;
m_s=obj.m_s;
m_theta=obj.m_theta;
m_work=obj.m_work;
m_x=obj.m_x;
m_xbase=obj.m_xbase;
m_xp=obj.m_xp;
m_lowrankbuf=obj.m_lowrankbuf;
m_precbuf=obj.m_precbuf;
m_denseh=obj.m_denseh;
m_precw=obj.m_precw;
m_sk=obj.m_sk;
m_yk=obj.m_yk;
m_lstate=obj.m_lstate;
}
//+------------------------------------------------------------------+
//| This class is a shell for class CMinLBFGSState |
//+------------------------------------------------------------------+
class CMinLBFGSStateShell
{
private:
CMinLBFGSState m_innerobj;
public:
//--- constructors, destructor
CMinLBFGSStateShell(void) {}
CMinLBFGSStateShell(CMinLBFGSState &obj) { m_innerobj.Copy(obj); }
~CMinLBFGSStateShell(void) {}
//--- methods
bool GetNeedF(void);
void SetNeedF(const bool b);
bool GetNeedFG(void);
void SetNeedFG(const bool b);
bool GetXUpdated(void);
void SetXUpdated(const bool b);
double GetF(void);
void SetF(const double d);
CMinLBFGSState *GetInnerObj(void);
};
//+------------------------------------------------------------------+
//| Returns the value of the variable needf |
//+------------------------------------------------------------------+
bool CMinLBFGSStateShell::GetNeedF(void)
{
return(m_innerobj.m_needf);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable needf |
//+------------------------------------------------------------------+
void CMinLBFGSStateShell::SetNeedF(const bool b)
{
m_innerobj.m_needf=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable needfg |
//+------------------------------------------------------------------+
bool CMinLBFGSStateShell::GetNeedFG(void)
{
return(m_innerobj.m_needfg);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable needfg |
//+------------------------------------------------------------------+
void CMinLBFGSStateShell::SetNeedFG(const bool b)
{
m_innerobj.m_needfg=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable xupdated |
//+------------------------------------------------------------------+
bool CMinLBFGSStateShell::GetXUpdated(void)
{
return(m_innerobj.m_xupdated);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable xupdated |
//+------------------------------------------------------------------+
void CMinLBFGSStateShell::SetXUpdated(const bool b)
{
m_innerobj.m_xupdated=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable f |
//+------------------------------------------------------------------+
double CMinLBFGSStateShell::GetF(void)
{
return(m_innerobj.m_f);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable f |
//+------------------------------------------------------------------+
void CMinLBFGSStateShell::SetF(const double d)
{
m_innerobj.m_f=d;
}
//+------------------------------------------------------------------+
//| Return object of class |
//+------------------------------------------------------------------+
CMinLBFGSState *CMinLBFGSStateShell::GetInnerObj(void)
{
return(GetPointer(m_innerobj));
}
//+------------------------------------------------------------------+
//| Auxiliary class for CMinLFBFGS |
//+------------------------------------------------------------------+
class CMinLBFGSReport
{
public:
//--- variables
int m_iterationscount;
int m_nfev;
int m_terminationtype;
//--- constructor, destructor
CMinLBFGSReport(void) { ZeroMemory(this); }
~CMinLBFGSReport(void) {}
//--- copy
void Copy(CMinLBFGSReport &obj);
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinLBFGSReport::Copy(CMinLBFGSReport &obj)
{
//--- copy variables
m_iterationscount=obj.m_iterationscount;
m_nfev=obj.m_nfev;
m_terminationtype=obj.m_terminationtype;
}
//+------------------------------------------------------------------+
//| This class is a shell for class CMinLBFGSReport |
//+------------------------------------------------------------------+
class CMinLBFGSReportShell
{
private:
CMinLBFGSReport m_innerobj;
public:
//--- constructors, destructor
CMinLBFGSReportShell(void) {}
CMinLBFGSReportShell(CMinLBFGSReport &obj) { m_innerobj.Copy(obj); }
~CMinLBFGSReportShell(void) {}
//--- methods
int GetIterationsCount(void);
void SetIterationsCount(const int i);
int GetNFev(void);
void SetNFev(const int i);
int GetTerminationType(void);
void SetTerminationType(const int i);
CMinLBFGSReport *GetInnerObj(void);
};
//+------------------------------------------------------------------+
//| Returns the value of the variable iterationscount |
//+------------------------------------------------------------------+
int CMinLBFGSReportShell::GetIterationsCount(void)
{
return(m_innerobj.m_iterationscount);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable iterationscount |
//+------------------------------------------------------------------+
void CMinLBFGSReportShell::SetIterationsCount(const int i)
{
m_innerobj.m_iterationscount=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable nfev |
//+------------------------------------------------------------------+
int CMinLBFGSReportShell::GetNFev(void)
{
return(m_innerobj.m_nfev);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable nfev |
//+------------------------------------------------------------------+
void CMinLBFGSReportShell::SetNFev(const int i)
{
m_innerobj.m_nfev=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable terminationtype |
//+------------------------------------------------------------------+
int CMinLBFGSReportShell::GetTerminationType(void)
{
return(m_innerobj.m_terminationtype);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable terminationtype |
//+------------------------------------------------------------------+
void CMinLBFGSReportShell::SetTerminationType(const int i)
{
m_innerobj.m_terminationtype=i;
}
//+------------------------------------------------------------------+
//| Return object of class |
//+------------------------------------------------------------------+
CMinLBFGSReport *CMinLBFGSReportShell::GetInnerObj(void)
{
return(GetPointer(m_innerobj));
}
//+------------------------------------------------------------------+
//| Limited memory BFGS method for large scale optimization |
//+------------------------------------------------------------------+
class CMinLBFGS
{
public:
//--- constant
static const double m_gtol;
//--- public methods
static void MinLBFGSCreate(const int n,const int m,double &x[],CMinLBFGSState &State);
static void MinLBFGSCreate(const int n,const int m,CRowDouble &x,CMinLBFGSState &State);
static void MinLBFGSCreateF(const int n,const int m,double &x[],const double diffstep,CMinLBFGSState &State);
static void MinLBFGSSetCond(CMinLBFGSState &State,const double epsg,const double epsf,double epsx,const int m_maxits);
static void MinLBFGSSetXRep(CMinLBFGSState &State,const bool needxrep);
static void MinLBFGSSetStpMax(CMinLBFGSState &State,const double stpmax);
static void MinLBFGSSetScale(CMinLBFGSState &State,double &s[]);
static void MinLBFGSCreateX(const int n,const int m,double &x[],int flags,const double diffstep,CMinLBFGSState &State);
static void MinLBFGSCreateX(const int n,const int m,CRowDouble &x,int flags,const double diffstep,CMinLBFGSState &State);
static void MinLBFGSSetPrecDefault(CMinLBFGSState &State);
static void MinLBFGSSetPrecCholesky(CMinLBFGSState &State,CMatrixDouble &p,const bool IsUpper);
static void MinLBFGSSetPrecDiag(CMinLBFGSState &State,double &d[]);
static void MinLBFGSSetPrecScale(CMinLBFGSState &State);
static void MinLBFGSSetPrecRankKLBFGSFast(CMinLBFGSState &State,CRowDouble &d,CRowDouble &c,CMatrixDouble &w,int cnt);
static void MinLBFGSSetPrecLowRankExact(CMinLBFGSState &State,CRowDouble &d,CRowDouble &c,CMatrixDouble &w,int cnt);
static void MinLBFGSResults(CMinLBFGSState &State,double &x[],CMinLBFGSReport &rep);
static void MinLBFGSResults(CMinLBFGSState &State,CRowDouble &x,CMinLBFGSReport &rep);
static void MinLBFGSResultsBuf(CMinLBFGSState &State,double &x[],CMinLBFGSReport &rep);
static void MinLBFGSResultsBuf(CMinLBFGSState &State,CRowDouble &x,CMinLBFGSReport &rep);
static void MinLBFGSRestartFrom(CMinLBFGSState &State,double &x[]);
static void MinLBFGSRestartFrom(CMinLBFGSState &State,CRowDouble &x);
static void MinLBFGSRequestTermination(CMinLBFGSState &State);
static bool MinLBFGSIteration(CMinLBFGSState &State);
private:
static void ClearRequestFields(CMinLBFGSState &State);
};
//+------------------------------------------------------------------+
//| Initialize constants |
//+------------------------------------------------------------------+
const double CMinLBFGS::m_gtol=0.4;
//+------------------------------------------------------------------+
//| LIMITED MEMORY BFGS METHOD FOR LARGE SCALE OPTIMIZATION |
//| DESCRIPTION: |
//| The subroutine minimizes function F(x) of N arguments by using a |
//| quasi - Newton method (LBFGS scheme) which is optimized to use a |
//| minimum amount of memory. |
//| The subroutine generates the approximation of an inverse Hessian |
//| matrix by using information about the last M steps of the |
//| algorithm (instead of N). It lessens a required amount of memory |
//| from a value of order N^2 to a value of order 2*N*M. |
//| REQUIREMENTS: |
//| Algorithm will request following information during its |
//| operation: |
//| * function value F and its gradient G (simultaneously) at given |
//| point X |
//| USAGE: |
//| 1. User initializes algorithm State with MinLBFGSCreate() call |
//| 2. User tunes m_solver parameters with MinLBFGSSetCond() |
//| MinLBFGSSetStpMax() and other functions |
//| 3. User calls MinLBFGSOptimize() function which takes algorithm |
//| State and pointer (delegate, etc.) to callback function which |
//| calculates F/G. |
//| 4. User calls MinLBFGSResults() to get solution |
//| 5. Optionally user may call MinLBFGSRestartFrom() to solve |
//| another problem with same N/M but another starting point |
//| and/or another function. MinLBFGSRestartFrom() allows to reuse|
//| already initialized structure. |
//| INPUT PARAMETERS: |
//| N - problem dimension. N>0 |
//| M - number of corrections in the BFGS scheme of |
//| Hessian approximation update. Recommended value: |
//| 3<=M<=7. The smaller value causes worse |
//| convergence, the bigger will not cause a |
//| considerably better convergence, but will cause |
//| a fall in the performance. M<=N. |
//| X - initial solution approximation, array[0..m_n-1]. |
//| OUTPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NOTES: |
//| 1. you may tune stopping conditions with MinLBFGSSetCond() |
//| function |
//| 2. if target function contains exp() or other fast growing |
//| functions, and optimization algorithm makes too large steps |
//| which leads to overflow, use MinLBFGSSetStpMax() function to |
//| bound algorithm's steps. However, L-BFGS rarely needs such a |
//| tuning. |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSCreate(const int n,const int m,double &x[],
CMinLBFGSState &State)
{
CRowDouble X=x;
MinLBFGSCreate(n,m,X,State);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSCreate(const int n,const int m,CRowDouble &x,
CMinLBFGSState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1!"))
return;
//--- check
if(!CAp::Assert(m>=1,__FUNCTION__+": M<1"))
return;
//--- check
if(!CAp::Assert(m<=n,__FUNCTION__+": M>N"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- function call
MinLBFGSCreateX(n,m,x,0,0.0,State);
}
//+------------------------------------------------------------------+
//| The subroutine is finite difference variant of MinLBFGSCreate(). |
//| It uses finite differences in order to differentiate target |
//| function. |
//| Description below contains information which is specific to this |
//| function only. We recommend to read comments on MinLBFGSCreate() |
//| in order to get more information about creation of LBFGS |
//| optimizer. |
//| INPUT PARAMETERS: |
//| N - problem dimension, N>0: |
//| * if given, only leading N elements of X are used|
//| * if not given, automatically determined from |
//| size of X |
//| M - number of corrections in the BFGS scheme of |
//| Hessian approximation update. Recommended value: |
//| 3<=M<=7. The smaller value causes worse |
//| convergence, the bigger will not cause a |
//| considerably better convergence, but will cause a|
//| fall in the performance. M<=N. |
//| X - starting point, array[0..m_n-1]. |
//| DiffStep- differentiation step, >0 |
//| OUTPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NOTES: |
//| 1. algorithm uses 4-point central formula for differentiation. |
//| 2. differentiation step along I-th axis is equal to DiffStep*S[I]|
//| where S[] is scaling vector which can be set by |
//| MinLBFGSSetScale() call. |
//| 3. we recommend you to use moderate values of differentiation |
//| step. Too large step will result in too large truncation |
//| errors, while too small step will result in too large |
//| numerical errors. 1.0E-6 can be good value to start with. |
//| 4. Numerical differentiation is very inefficient - one gradient |
//| calculation needs 4*N function evaluations. This function will|
//| work for any N - either small (1...10), moderate (10...100) or|
//| large (100...). However, performance penalty will be too |
//| severe for any N's except for small ones. |
//| We should also say that code which relies on numerical |
//| differentiation is less robust and precise. LBFGS needs exact |
//| gradient values. Imprecise gradient may slow down convergence,|
//| especially on highly nonlinear problems. |
//| Thus we recommend to use this function for fast prototyping on|
//| small- dimensional problems only, and to implement analytical |
//| gradient as soon as possible. |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSCreateF(const int n,const int m,double &x[],
const double diffstep,CMinLBFGSState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N too small!"))
return;
//--- check
if(!CAp::Assert(m>=1,__FUNCTION__+": M<1"))
return;
//--- check
if(!CAp::Assert(m<=n,__FUNCTION__+": M>N"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(diffstep),__FUNCTION__+": DiffStep is infinite or NaN!"))
return;
//--- check
if(!CAp::Assert(diffstep>0.0,__FUNCTION__+": DiffStep is non-positive!"))
return;
//--- function call
MinLBFGSCreateX(n,m,x,0,diffstep,State);
}
//+------------------------------------------------------------------+
//| This function sets stopping conditions for L-BFGS optimization |
//| algorithm. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| EpsG - >=0 |
//| The subroutine finishes its work if the condition|
//| |v|<EpsG is satisfied, where: |
//| * |.| means Euclidian norm |
//| * v - scaled gradient vector, v[i]=g[i]*s[i] |
//| * g - gradient |
//| * s - scaling coefficients set by |
//| MinLBFGSSetScale() |
//| EpsF - >=0 |
//| The subroutine finishes its work if on k+1-th |
//| iteration the condition |F(k+1)-F(k)| <= |
//| <= EpsF*max{|F(k)|,|F(k+1)|,1} is satisfied. |
//| EpsX - >=0 |
//| The subroutine finishes its work if on k+1-th |
//| iteration the condition |v|<=EpsX is fulfilled, |
//| where: |
//| * |.| means Euclidian norm |
//| * v - scaled step vector, v[i]=dx[i]/s[i] |
//| * dx - ste pvector, dx=X(k+1)-X(k) |
//| * s - scaling coefficients set by |
//| MinLBFGSSetScale() |
//| MaxIts - maximum number of iterations. If MaxIts=0, the |
//| number of iterations is unlimited. |
//| Passing EpsG=0, EpsF=0, EpsX=0 and MaxIts=0 (simultaneously) will|
//| lead to automatic stopping criterion selection (small EpsX). |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSSetCond(CMinLBFGSState &State,const double epsg,
const double epsf,double epsx,
const int m_maxits)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(epsg),__FUNCTION__+": EpsG is not finite number!"))
return;
//--- check
if(!CAp::Assert(epsg>=0.0,__FUNCTION__+": negative EpsG!"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(epsf),__FUNCTION__+": EpsF is not finite number!"))
return;
//--- check
if(!CAp::Assert(epsf>=0.0,__FUNCTION__+": negative EpsF!"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(epsx),__FUNCTION__+": EpsX is not finite number!"))
return;
//--- check
if(!CAp::Assert(epsx>=0.0,__FUNCTION__+": negative EpsX!"))
return;
//--- check
if(!CAp::Assert(m_maxits>=0,__FUNCTION__+": negative MaxIts!"))
return;
//--- check
if(((epsg==0.0 && epsf==0.0) && epsx==0.0) && m_maxits==0)
epsx=1.0E-6;
//--- change values
State.m_epsg=epsg;
State.m_epsf=epsf;
State.m_epsx=epsx;
State.m_maxits=m_maxits;
}
//+------------------------------------------------------------------+
//| This function turns on/off reporting. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NeedXRep- whether iteration reports are needed or not |
//| If NeedXRep is True, algorithm will call rep() callback function |
//| if it is provided to MinLBFGSOptimize(). |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSSetXRep(CMinLBFGSState &State,const bool needxrep)
{
State.m_xrep=needxrep;
}
//+------------------------------------------------------------------+
//| This function sets maximum step length |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| StpMax - maximum step length, >=0. Set StpMax to 0.0 |
//| (default), if you don't want to limit step |
//| length. |
//| Use this subroutine when you optimize target function which |
//| contains exp() or other fast growing functions, and optimization |
//| algorithm makes too large steps which leads to overflow. This |
//| function allows us to reject steps that are too large (and |
//| therefore expose us to the possible overflow) without actually |
//| calculating function value at the x+stp*d. |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSSetStpMax(CMinLBFGSState &State,
const double stpmax)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(stpmax),__FUNCTION__+": StpMax is not finite!"))
return;
//--- check
if(!CAp::Assert(stpmax>=0.0,__FUNCTION__+": StpMax<0!"))
return;
//--- change value
State.m_stpmax=stpmax;
}
//+------------------------------------------------------------------+
//| This function sets scaling coefficients for LBFGS optimizer. |
//| ALGLIB optimizers use scaling matrices to test stopping |
//| conditions (step size and gradient are scaled before comparison |
//| with tolerances). Scale of the I-th variable is a translation |
//| invariant measure of: |
//| a) "how large" the variable is |
//| b) how large the step should be to make significant changes in |
//| the function |
//| Scaling is also used by finite difference variant of the |
//| optimizer - step along I-th axis is equal to DiffStep*S[I]. |
//| In most optimizers (and in the LBFGS too) scaling is NOT a form |
//| of preconditioning. It just affects stopping conditions. You |
//| should set preconditioner by separate call to one of the |
//| MinLBFGSSetPrec...() functions. |
//| There is special preconditioning mode, however, which uses |
//| scaling coefficients to form diagonal preconditioning matrix. |
//| You can turn this mode on, if you want. But you should |
//| understand that scaling is not the same thing as |
//| preconditioning - these are two different, although related |
//| forms of tuning m_solver. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| S - array[N], non-zero scaling coefficients |
//| S[i] may be negative, sign doesn't matter. |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSSetScale(CMinLBFGSState &State,double &s[])
{
//--- check
if(!CAp::Assert(CAp::Len(s)>=State.m_n,__FUNCTION__+": Length(S)<N"))
return;
for(int i=0; i<State.m_n; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(s[i]),__FUNCTION__+": S contains infinite or NAN elements"))
return;
//--- check
if(!CAp::Assert(s[i]!=0.0,__FUNCTION__+": S contains zero elements"))
return;
State.m_s.Set(i,MathAbs(s[i]));
}
}
//+------------------------------------------------------------------+
//| Extended subroutine for internal use only. |
//| Accepts additional parameters: |
//| Flags - additional Settings: |
//| * Flags = 0 means no additional Settings |
//| *Flags=1 "do not allocate memory". used when |
//| solving a many subsequent tasks with |
//| same N/M values. First call MUST be |
//| without this flag bit set, subsequent|
//| calls of MinLBFGS with same |
//| MinLBFGSState structure can set Flags|
//| to 1. |
//| DiffStep - numerical differentiation step |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSCreateX(const int n,const int m,double &x[],
int flags,const double diffstep,
CMinLBFGSState &State)
{
CRowDouble X=x;
MinLBFGSCreateX(n,m,X,flags,diffstep,State);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSCreateX(const int n,const int m,CRowDouble &x,
int flags,const double diffstep,
CMinLBFGSState &State)
{
//--- create variable
bool allocatemem;
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N too small!"))
return;
//--- check
if(!CAp::Assert(m>=1,__FUNCTION__+": M too small!"))
return;
//--- check
if(!CAp::Assert(m<=n,__FUNCTION__+": M too large!"))
return;
//--- Initialize
State.m_diffstep=diffstep;
State.m_n=n;
State.m_m=m;
allocatemem=flags%2==0;
flags=flags/2;
//--- check
if(allocatemem)
{
//--- allocation
State.m_rho.Resize(m);
State.m_theta.Resize(m);
State.m_yk.Resize(m,n);
State.m_sk.Resize(m,n);
State.m_d.Resize(n);
State.m_xp.Resize(n);
State.m_x.Resize(n);
State.m_xbase.Resize(n);
State.m_g.Resize(n);
State.m_work.Resize(n);
}
//--- function call
MinLBFGSSetCond(State,0,0,0,0);
//--- function call
MinLBFGSSetXRep(State,false);
//--- function call
MinLBFGSSetStpMax(State,0);
//--- function call
MinLBFGSRestartFrom(State,x);
//--- change values
State.m_s=vector<double>::Ones(n);
State.m_invs=vector<double>::Ones(n);
State.m_lastscaleused=vector<double>::Ones(n);
State.m_prectype=0;
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: default preconditioner |
//| (simple scaling, same for all elements of X) is used. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NOTE: you can change preconditioner "on the fly", during |
//| algorithm iterations. |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSSetPrecDefault(CMinLBFGSState &State)
{
State.m_prectype=0;
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: Cholesky factorization of |
//| approximate Hessian is used. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| P - triangular preconditioner, Cholesky factorization|
//| of the approximate Hessian. array[0..m_n-1,0..m_n-1],|
//| (if larger, only leading N elements are used). |
//| IsUpper - whether upper or lower triangle of P is given |
//| (other triangle is not referenced) |
//| After call to this function preconditioner is changed to P (P is |
//| copied into the internal buffer). |
//| NOTE: you can change preconditioner "on the fly", during |
//| algorithm iterations. |
//| NOTE 2: P should be nonsingular. Exception will be thrown |
//| otherwise. |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSSetPrecCholesky(CMinLBFGSState &State,
CMatrixDouble &p,
const bool IsUpper)
{
//--- check
if(!CAp::Assert(CApServ::IsFiniteRTrMatrix(p,State.m_n,IsUpper),__FUNCTION__+": P contains infinite or NAN values!"))
return;
//--- initialization
double mx=0;
for(int i=0; i<State.m_n; i++)
mx=MathMax(mx,MathAbs(p[i][i]));
//--- check
if(!CAp::Assert(mx>0.0,__FUNCTION__+": P is strictly singular!"))
return;
//--- check
if((int)CAp::Rows(State.m_denseh)<State.m_n || (int)CAp::Cols(State.m_denseh)<State.m_n)
State.m_denseh.Resize(State.m_n,State.m_n);
//--- initialization
State.m_prectype=1;
//--- check
if(IsUpper)
CAblas::RMatrixCopy(State.m_n,State.m_n,p,0,0,State.m_denseh,0,0);
else
CAblas::RMatrixTranspose(State.m_n,State.m_n,p,0,0,State.m_denseh,0,0);
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: diagonal of approximate |
//| Hessian is used. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| D - diagonal of the approximate Hessian, |
//| array[0..m_n-1], (if larger, only leading N |
//| elements are used). |
//| NOTE: you can change preconditioner "on the fly", during |
//| algorithm iterations. |
//| NOTE 2: D[i] should be positive. Exception will be thrown |
//| otherwise. |
//| NOTE 3: you should pass diagonal of approximate Hessian - NOT |
//| ITS INVERSE. |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSSetPrecDiag(CMinLBFGSState &State,double &d[])
{
//--- check
if(!CAp::Assert(CAp::Len(d)>=State.m_n,__FUNCTION__+": D is too short"))
return;
for(int i=0; i<State.m_n; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(d[i]),__FUNCTION__+": D contains infinite or NAN elements"))
return;
//--- check
if(!CAp::Assert(d[i]>0.0,__FUNCTION__+": D contains non-positive elements"))
return;
}
//--- change values
State.m_prectype=2;
State.m_diagh=d;
State.m_diagh.Resize(State.m_n);
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: scale-based diagonal |
//| preconditioning. |
//| This preconditioning mode can be useful when you don't have |
//| approximate diagonal of Hessian, but you know that your variables|
//| are badly scaled (for example, one variable is in [1,10], and |
//| another in [1000,100000]), and most part of the ill-conditioning |
//| comes from different scales of vars. |
//| In this case simple scale-based preconditioner, with H.Set(i, |
//| = 1/(s[i]^2), can greatly improve convergence. |
//| IMPRTANT: you should set scale of your variables with |
//| MinLBFGSSetScale() call (before or after MinLBFGSSetPrecScale() |
//| call). Without knowledge of the scale of your variables |
//| scale-based preconditioner will be just unit matrix. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSSetPrecScale(CMinLBFGSState &State)
{
State.m_prectype=3;
}
//+------------------------------------------------------------------+
//| This function sets low-rank preconditioner for Hessian matrix |
//| H=D+W'*C*W, |
//| where: |
//| * H is a Hessian matrix, which is approximated by D/W/C |
//| * D is a NxN diagonal positive definite matrix |
//| * W is a KxN low-rank correction |
//| * C is a KxK positive definite diagonal factor of low-rank |
//| correction |
//| This preconditioner is inexact but fast - it requires O(N*K) time|
//| to be applied. Preconditioner P is calculated by artificially |
//| constructing a set of BFGS updates which tries to reproduce |
//| behavior of H: |
//| * Sk = Wk (k-th row of W) |
//| * Yk = (D+Wk'*Ck*Wk)*Sk |
//| * Yk/Sk are reordered by ascending of C[k]*norm(Wk)^2 |
//| Here we assume that rows of Wk are orthogonal or nearly |
//| orthogonal, which allows us to have O(N*K+K^2) update instead |
//| of O(N*K^2) one. Reordering of updates is essential for having |
//| good performance on non-orthogonal problems (updates which do not|
//| add much of curvature are added first, and updates which add very|
//| large eigenvalues are added last and override effect of the first|
//| updates). |
//| In practice, this preconditioner is perfect when ortogonal |
//| correction is applied; on non-orthogonal problems sometimes it |
//| allows to achieve 5x speedup (when compared to non-preconditioned|
//| solver). |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSSetPrecRankKLBFGSFast(CMinLBFGSState &State,
CRowDouble &d,CRowDouble &c,CMatrixDouble &w,int cnt)
{
int n=State.m_n;
State.m_prectype=4;
State.m_preck=cnt;
//--- copy
State.m_precd=d;
State.m_precc=c;
State.m_precw=w;
State.m_precc.Resize(cnt);
State.m_precd.Resize(n);
State.m_precw.Resize(cnt,n);
}
//+------------------------------------------------------------------+
//| This function sets exact low-rank preconditioner for Hessian |
//| matrix H=D+W'*C*W, where: |
//| * H is a Hessian matrix, which is approximated by D/W/C |
//| * D is a NxN diagonal positive definite matrix |
//| * W is a KxN low-rank correction |
//| * C is a KxK semidefinite diagonal factor of low-rank |
//| correction |
//| This preconditioner is exact but slow - it requires O(N*K^2) time|
//| to be built and O(N*K) time to be applied. Woodbury matrix |
//| identity is used to build inverse matrix. |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSSetPrecLowRankExact(CMinLBFGSState &State,
CRowDouble &d,
CRowDouble &c,
CMatrixDouble &w,
int cnt)
{
State.m_prectype=5;
COptServ::PrepareLowRankPreconditioner(d,c,w,State.m_n,cnt,State.m_lowrankbuf);
}
//+------------------------------------------------------------------+
//| L-BFGS algorithm results |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| X - array[0..m_n-1], solution |
//| Rep - optimization report: |
//| * Rep.TerminationType completetion code: |
//| * -2 rounding errors prevent further |
//| improvement. X contains best point |
//| found. |
//| * -1 incorrect parameters were specified |
//| * 1 relative function improvement is no |
//| more than EpsF. |
//| * 2 relative step is no more than EpsX. |
//| * 4 gradient norm is no more than EpsG |
//| * 5 MaxIts steps was taken |
//| * 7 stopping conditions are too |
//| stringent, further improvement is |
//| impossible |
//| * Rep.IterationsCount contains iterations count |
//| * NFEV countains number of function calculations |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSResults(CMinLBFGSState &State,double &x[],
CMinLBFGSReport &rep)
{
//--- reset memory
ArrayResize(x,0);
//--- function call
MinLBFGSResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSResults(CMinLBFGSState &State,CRowDouble &x,
CMinLBFGSReport &rep)
{
//--- reset memory
x.Resize(0);
//--- function call
MinLBFGSResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| L-BFGS algorithm results |
//| Buffered implementation of MinLBFGSResults which uses |
//| pre-allocated buffer to store X[]. If buffer size is too small, |
//| it resizes buffer. It is intended to be used in the inner cycles |
//| of performance critical algorithms where array reallocation |
//| penalty is too large to be ignored. |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSResultsBuf(CMinLBFGSState &State,double &x[],
CMinLBFGSReport &rep)
{
CRowDouble X;
MinLBFGSResultsBuf(State,X,rep);
X.ToArray(x);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSResultsBuf(CMinLBFGSState &State,CRowDouble &x,
CMinLBFGSReport &rep)
{
//--- copy
x=State.m_x;
//--- change values
rep.m_iterationscount=State.m_repiterationscount;
rep.m_nfev=State.m_repnfev;
rep.m_terminationtype=State.m_repterminationtype;
}
//+------------------------------------------------------------------+
//| This subroutine restarts LBFGS algorithm from new point. All |
//| optimization parameters are left unchanged. |
//| This function allows to solve multiple optimization problems |
//| (which must have same number of dimensions) without object |
//| reallocation penalty. |
//| INPUT PARAMETERS: |
//| State - structure used to store algorithm State |
//| X - new starting point. |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSRestartFrom(CMinLBFGSState &State,double &x[])
{
CRowDouble X=x;
MinLBFGSRestartFrom(State,X);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSRestartFrom(CMinLBFGSState &State,CRowDouble &x)
{
//--- check
if(!CAp::Assert(CAp::Len(x)>=State.m_n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,State.m_n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- copy
State.m_xbase=x;
//--- allocation
State.m_rstate.ia.Resize(6);
State.m_rstate.ra.Resize(2);
State.m_rstate.stage=-1;
//--- function call
ClearRequestFields(State);
}
//+------------------------------------------------------------------+
//| This subroutine submits request for termination of running |
//| optimizer. It should be called from user-supplied callback when |
//| user decides that it is time to "smoothly" terminate optimization|
//| process. As result, optimizer stops at point which was "current |
//| accepted" when termination request was submitted and returns |
//| error code 8 (successful termination). |
//| INPUT PARAMETERS: |
//| State - optimizer structure |
//| NOTE: after request for termination optimizer may perform several|
//| additional calls to user-supplied callbacks. It does NOT |
//| guarantee to stop immediately - it just guarantees that |
//| these additional calls will be discarded later. |
//| NOTE: calling this function on optimizer which is NOT running |
//| will have no effect. |
//| NOTE: multiple calls to this function are possible. First call is|
//| counted, subsequent calls are silently ignored. |
//+------------------------------------------------------------------+
void CMinLBFGS::MinLBFGSRequestTermination(CMinLBFGSState &State)
{
State.m_userterminationneeded=true;
}
//+------------------------------------------------------------------+
//| Clears request fileds (to be sure that we don't forgot to clear |
//| something) |
//+------------------------------------------------------------------+
void CMinLBFGS::ClearRequestFields(CMinLBFGSState &State)
{
//--- change values
State.m_needf=false;
State.m_needfg=false;
State.m_xupdated=false;
}
//+------------------------------------------------------------------+
//| NOTES: |
//| 1. This function has two different implementations: one which |
//| uses exact (analytical) user-supplied gradient, and one which |
//| uses function value only and numerically differentiates |
//| function in order to obtain gradient. |
//| Depending on the specific function used to create optimizer |
//| object (either MinLBFGSCreate() for analytical gradient or |
//| MinLBFGSCreateF() for numerical differentiation) you should |
//| choose appropriate variant of MinLBFGSOptimize() - one which |
//| accepts function AND gradient or one which accepts function |
//| ONLY. |
//| Be careful to choose variant of MinLBFGSOptimize() which |
//| corresponds to your optimization scheme! Table below lists |
//| different combinations of callback (function/gradient) passed |
//| to MinLBFGSOptimize() and specific function used to create |
//| optimizer. |
//| | USER PASSED TO MinLBFGSOptimize() |
//| CREATED WITH | function only | function and gradient |
//| ------------------------------------------------------------ |
//| MinLBFGSCreateF() | work FAIL |
//| MinLBFGSCreate() | FAIL work |
//| Here "FAIL" denotes inappropriate combinations of optimizer |
//| creation function and MinLBFGSOptimize() version. Attemps to |
//| use such combination (for example, to create optimizer with |
//| MinLBFGSCreateF() and to pass gradient information to |
//| MinCGOptimize()) will lead to exception being thrown. Either |
//| you did not pass gradient when it WAS needed or you passed |
//| gradient when it was NOT needed. |
//+------------------------------------------------------------------+
bool CMinLBFGS::MinLBFGSIteration(CMinLBFGSState &State)
{
//--- create variables
bool result=false;
int n=0;
int m=0;
int i=0;
int j=0;
int ic=0;
int mcinfo=0;
double v=0;
double vv=0;
int i_=0;
int label=-1;
//--- Reverse communication preparations
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(State.m_rstate.stage>=0)
{
n=State.m_rstate.ia[0];
m=State.m_rstate.ia[1];
i=State.m_rstate.ia[2];
j=State.m_rstate.ia[3];
ic=State.m_rstate.ia[4];
mcinfo=State.m_rstate.ia[5];
v=State.m_rstate.ra[0];
vv=State.m_rstate.ra[1];
}
else
{
n=359;
m=-58;
i=-919;
j=-909;
ic=81;
mcinfo=255;
v=74;
vv=-788;
}
switch(State.m_rstate.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
case 3:
label=3;
break;
case 4:
label=4;
break;
case 5:
label=5;
break;
case 6:
label=6;
break;
case 7:
label=7;
break;
case 8:
label=8;
break;
case 9:
label=9;
break;
case 10:
label=10;
break;
case 11:
label=11;
break;
case 12:
label=12;
break;
case 13:
label=13;
break;
case 14:
label=14;
break;
default:
//--- Routine body
//--- Unload frequently used variables from State structure
//--- (just for typing convinience)
n=State.m_n;
m=State.m_m;
//--- Init
State.m_userterminationneeded=false;
State.m_repterminationtype=0;
State.m_repiterationscount=0;
State.m_repnfev=0;
COptServ::SmoothnessMonitorInit(State.m_smonitor,State.m_s,n,1,State.m_smoothnessguardlevel>0);
State.m_invs.Resize(n);
State.m_lastscaleused=State.m_s;
State.m_invs=State.m_s.Pow(-1)+0;
//--- Check, that transferred derivative value is right
State.m_stp=0;
ClearRequestFields(State);
if(!(State.m_diffstep==0.0 && State.m_teststep>0.0))
{
label=15;
break;
}
label=17;
break;
}
//--- main loop
while(label>=0)
{
switch(label)
{
case 17:
if(!COptServ::SmoothnessMonitorCheckGradientATX0(State.m_smonitor,State.m_xbase,State.m_s,State.m_s,State.m_s,false,State.m_teststep))
{
label=18;
break;
}
State.m_x=State.m_smonitor.m_x;
State.m_needfg=true;
State.m_rstate.stage=0;
label=-1;
break;
case 0:
State.m_needfg=false;
State.m_smonitor.m_fi.Set(0,State.m_f);
State.m_smonitor.m_j.Row(0,State.m_g);
label=17;
break;
case 18:
case 15:
//--- Calculate F/G at the initial point
State.m_x=State.m_xbase;
State.m_stp=0;
ClearRequestFields(State);
if(State.m_diffstep!=0.0)
{
label=19;
break;
}
State.m_needfg=true;
State.m_rstate.stage=1;
label=-1;
break;
case 1:
State.m_needfg=false;
label=20;
break;
case 19:
State.m_needf=true;
State.m_rstate.stage=2;
label=-1;
break;
case 2:
State.m_fbase=State.m_f;
i=0;
case 21:
if(i>n-1)
{
label=23;
break;
}
v=State.m_x[i];
State.m_x.Set(i,v-State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=3;
label=-1;
break;
case 3:
State.m_fm2=State.m_f;
State.m_x.Set(i,v-0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=4;
label=-1;
break;
case 4:
State.m_fm1=State.m_f;
State.m_x.Set(i,v+0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=5;
label=-1;
break;
case 5:
State.m_fp1=State.m_f;
State.m_x.Set(i,v+State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=6;
label=-1;
break;
case 6:
State.m_fp2=State.m_f;
State.m_x.Set(i,v);
State.m_g.Set(i,(8*(State.m_fp1-State.m_fm1)-(State.m_fp2-State.m_fm2))/(6*State.m_diffstep*State.m_s[i]));
i++;
label=21;
break;
case 23:
State.m_f=State.m_fbase;
State.m_needf=false;
case 20:
COptServ::TrimPrepare(State.m_f,State.m_trimthreshold);
if(!State.m_xrep)
{
label=24;
break;
}
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=7;
label=-1;
break;
case 7:
State.m_xupdated=false;
case 24:
if(State.m_userterminationneeded)
{
//--- User requested termination
State.m_repterminationtype=8;
result=false;
return(result);
}
State.m_repnfev=1;
State.m_fold=State.m_f;
v=MathPow(State.m_g*State.m_s+0,2.0).Sum();
if(MathSqrt(v)<=State.m_epsg)
{
State.m_repterminationtype=4;
result=false;
return(result);
}
//--- Choose initial step and direction.
//--- Apply preconditioner, if we have something other than default.
State.m_d=State.m_g.ToVector()*(-1.0);
switch(State.m_prectype)
{
case 0:
//--- Default preconditioner is used, but we can't use it before iterations will start
v=State.m_g.Dot(State.m_g);
v=MathSqrt(v);
if(State.m_stpmax==0.0)
State.m_stp=MathMin(1.0/v,1);
else
State.m_stp=MathMin(1.0/v,State.m_stpmax);
break;
case 1:
//--- Cholesky preconditioner is used
CFbls::FblsCholeskySolve(State.m_denseh,1.0,n,true,State.m_d,State.m_autobuf);
State.m_stp=1;
break;
case 2:
//--- diagonal approximation is used
State.m_d/=State.m_diagh;
State.m_stp=1;
break;
case 3:
//--- scale-based preconditioner is used
State.m_d*=State.m_s.Pow(2.0)+0;
State.m_stp=1;
break;
case 4:
//--- rank-k BFGS-based preconditioner is used
COptServ::InexactLBFGSPreconditioner(State.m_d,n,State.m_precd,State.m_precc,State.m_precw,State.m_preck,State.m_precbuf);
State.m_stp=1;
break;
case 5:
//--- exact low-rank preconditioner is used
COptServ::ApplyLowRankPreconditioner(State.m_d,State.m_lowrankbuf);
State.m_stp=1;
break;
}
//--- Main cycle
State.m_k=0;
case 26:
//--- Main cycle: prepare to 1-D line search
State.m_p=State.m_k%m;
State.m_q=MathMin(State.m_k,m-1);
//--- Store X[k], G[k]
State.m_xp=State.m_x;
State.m_sk.Row(State.m_p,State.m_x*(-1)+0);
State.m_yk.Row(State.m_p,State.m_g*(-1)+0);
//--- Minimize F(x+alpha*d)
//--- Calculate S[k], Y[k]
State.m_mcstage=0;
if(State.m_k!=0)
State.m_stp=1.0;
CLinMin::LinMinNormalized(State.m_d,State.m_stp,n);
COptServ::SmoothnessMonitorStartLineSearch1u(State.m_smonitor,State.m_s,State.m_invs,State.m_x,State.m_f,State.m_g);
CLinMin::MCSrch(n,State.m_x,State.m_f,State.m_g,State.m_d,State.m_stp,State.m_stpmax,m_gtol,mcinfo,State.m_nfev,State.m_work,State.m_lstate,State.m_mcstage);
case 28:
if(State.m_mcstage==0)
{
label=29;
break;
}
ClearRequestFields(State);
if(State.m_diffstep!=0.0)
{
label=30;
break;
}
State.m_needfg=true;
State.m_rstate.stage=8;
label=-1;
break;
case 8:
State.m_needfg=false;
label=31;
break;
case 30:
State.m_needf=true;
State.m_rstate.stage=9;
label=-1;
break;
case 9:
State.m_fbase=State.m_f;
i=0;
case 32:
if(i>n-1)
{
label=34;
break;
}
v=State.m_x[i];
State.m_x.Set(i,v-State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=10;
label=-1;
break;
case 10:
State.m_fm2=State.m_f;
State.m_x.Set(i,v-0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=11;
label=-1;
break;
case 11:
State.m_fm1=State.m_f;
State.m_x.Set(i,v+0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=12;
label=-1;
break;
case 12:
State.m_fp1=State.m_f;
State.m_x.Set(i,v+State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=13;
label=-1;
break;
case 13:
State.m_fp2=State.m_f;
State.m_x.Set(i,v);
State.m_g.Set(i,(8*(State.m_fp1-State.m_fm1)-(State.m_fp2-State.m_fm2))/(6*State.m_diffstep*State.m_s[i]));
i++;
label=32;
break;
case 34:
State.m_f=State.m_fbase;
State.m_needf=false;
case 31:
COptServ::SmoothnessMonitorEnqueuePoint1u(State.m_smonitor,State.m_s,State.m_invs,State.m_d,State.m_stp,State.m_x,State.m_f,State.m_g);
COptServ::TrimFunction(State.m_f,State.m_g,n,State.m_trimthreshold);
CLinMin::MCSrch(n,State.m_x,State.m_f,State.m_g,State.m_d,State.m_stp,State.m_stpmax,m_gtol,mcinfo,State.m_nfev,State.m_work,State.m_lstate,State.m_mcstage);
label=28;
break;
case 29:
COptServ::SmoothnessMonitorFinalizeLineSearch(State.m_smonitor);
if(State.m_userterminationneeded)
{
//--- User requested termination.
//--- Restore previous point and return.
State.m_x=State.m_xp;
State.m_repterminationtype=8;
result=false;
return(result);
}
if(!State.m_xrep)
{
label=35;
break;
}
//--- report
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=14;
label=-1;
break;
case 14:
State.m_xupdated=false;
case 35:
State.m_repnfev=State.m_repnfev+State.m_nfev;
State.m_repiterationscount=State.m_repiterationscount+1;
State.m_sk.Row(State.m_p,State.m_x.ToVector()+State.m_sk[State.m_p]);
State.m_yk.Row(State.m_p,State.m_g.ToVector()+State.m_yk[State.m_p]);
//--- Stopping conditions
v=MathPow(State.m_g*State.m_s+0,2.0).Sum();
if(!MathIsValidNumber(v) || !MathIsValidNumber(State.m_f))
{
//--- Abnormal termination - infinities in function/gradient
State.m_repterminationtype=-8;
result=false;
return(result);
}
if(State.m_repiterationscount>=State.m_maxits && State.m_maxits>0)
{
//--- Too many iterations
State.m_repterminationtype=5;
result=false;
return(result);
}
if(MathSqrt(v)<=State.m_epsg)
{
//--- Gradient is small enough
State.m_repterminationtype=4;
result=false;
return(result);
}
if((State.m_fold-State.m_f)<=(State.m_epsf*MathMax(MathAbs(State.m_fold),MathMax(MathAbs(State.m_f),1.0))))
{
//--- F(k+1)-F(k) is small enough
State.m_repterminationtype=1;
result=false;
return(result);
}
v=MathPow(State.m_sk[State.m_p]/State.m_s.ToVector(),2.0).Sum();
if(MathSqrt(v)<=State.m_epsx)
{
//--- X(k+1)-X(k) is small enough
State.m_repterminationtype=2;
result=false;
return(result);
}
//--- If Wolfe conditions are satisfied, we can update
//--- limited memory model.
//--- However, if conditions are not satisfied (NFEV limit is met,
//--- function is too wild, ...), we'll skip L-BFGS update
if(mcinfo!=1)
{
//--- Skip update.
//--- In such cases we'll initialize search direction by
//--- antigradient vector, because it leads to more
//--- transparent code with less number of special cases
State.m_fold=State.m_f;
State.m_d=State.m_g.ToVector()*(-1.0);
}
else
{
//--- Calculate Rho[k], GammaK
v=0.0;
for(i_=0; i_<n; i_++)
v+=State.m_yk.Get(State.m_p,i_)*State.m_sk.Get(State.m_p,i_);
vv=0.0;
for(i_=0; i_<n; i_++)
vv+=State.m_yk.Get(State.m_p,i_)*State.m_yk.Get(State.m_p,i_);
if(v==0.0 || vv==0.0)
{
//--- Rounding errors make further iterations impossible.
State.m_repterminationtype=-2;
result=false;
return(result);
}
State.m_rho.Set(State.m_p,1/v);
State.m_gammak=v/vv;
//--- Calculate d(k+1) = -H(k+1)*g(k+1)
//--- for I:=K downto K-Q do
//--- V = s(i)^T * work(iteration:I)
//--- theta(i) = V
//--- work(iteration:I+1) = work(iteration:I) - V*Rho(i)*y(i)
//--- work(last iteration) = H0*work(last iteration) - preconditioner
//--- for I:=K-Q to K do
//--- V = y(i)^T*work(iteration:I)
//--- work(iteration:I+1) = work(iteration:I) +(-V+theta(i))*Rho(i)*s(i)
//--- NOW WORK CONTAINS d(k+1)
State.m_work=State.m_g;
for(i=State.m_k; i>=State.m_k-State.m_q; i--)
{
ic=i%m;
v=State.m_work.DotR(State.m_sk,ic);
State.m_theta.Set(ic,v);
vv=v*State.m_rho[ic];
State.m_work-=State.m_yk[ic]*vv;
}
switch(State.m_prectype)
{
case 0:
//--- Simple preconditioner is used
v=State.m_gammak;
State.m_work*=v;
break;
case 1:
//--- Cholesky preconditioner is used
CFbls::FblsCholeskySolve(State.m_denseh,1,n,true,State.m_work,State.m_autobuf);
break;
case 2:
//--- diagonal approximation is used
State.m_work/=State.m_diagh;
break;
case 3:
//--- scale-based preconditioner is used
State.m_work*=State.m_s.Pow(2.0)+0;
break;
case 4:
//--- Rank-K BFGS-based preconditioner is used
COptServ::InexactLBFGSPreconditioner(State.m_work,n,State.m_precd,State.m_precc,State.m_precw,State.m_preck,State.m_precbuf);
break;
case 5:
//--- Exact low-rank preconditioner is used
COptServ::ApplyLowRankPreconditioner(State.m_work,State.m_lowrankbuf);
break;
}
for(i=State.m_k-State.m_q; i<=State.m_k; i++)
{
ic=i%m;
v=State.m_work.DotR(State.m_yk,ic);
vv=State.m_rho[ic]*(-v+State.m_theta[ic]);
State.m_work+=State.m_sk[ic]*vv;
}
State.m_d=State.m_work.ToVector()*(-1.0);
//--- Next step
State.m_fold=State.m_f;
State.m_k++;
}
label=26;
break;
case 27:
result=false;
return(result);
}
}
//--- Saving State
result=true;
State.m_rstate.ia.Set(0,n);
State.m_rstate.ia.Set(1,m);
State.m_rstate.ia.Set(2,i);
State.m_rstate.ia.Set(3,j);
State.m_rstate.ia.Set(4,ic);
State.m_rstate.ia.Set(5,mcinfo);
State.m_rstate.ra.Set(0,v);
State.m_rstate.ra.Set(1,vv);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Variables of IPM method(primal and dual, slacks) |
//+------------------------------------------------------------------+
struct CVIPMVars
{
int m_m;
int m_n;
CRowDouble m_g;
CRowDouble m_p;
CRowDouble m_q;
CRowDouble m_s;
CRowDouble m_t;
CRowDouble m_v;
CRowDouble m_w;
CRowDouble m_x;
CRowDouble m_y;
CRowDouble m_z;
//--- constructor / destructor
CVIPMVars(void) { m_m=0; m_n=0; }
~CVIPMVars(void) {}
//---
void Copy(const CVIPMVars &obj);
//--- overloading
void operator=(const CVIPMVars &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CVIPMVars::Copy(const CVIPMVars &obj)
{
m_m=obj.m_m;
m_n=obj.m_n;
m_g=obj.m_g;
m_p=obj.m_p;
m_q=obj.m_q;
m_s=obj.m_s;
m_t=obj.m_t;
m_v=obj.m_v;
m_w=obj.m_w;
m_x=obj.m_x;
m_y=obj.m_y;
m_z=obj.m_z;
}
//+------------------------------------------------------------------+
//| Reduced(M + N)*(M + N) KKT system stored in sparse format |
//+------------------------------------------------------------------+
struct CVIPMReducedSparseSystem
{
int m_ntotal;
bool m_isdiagonal[];
CSparseMatrix m_rawsystem;
CSpCholAnalysis m_analysis;
CRowInt m_coldegrees;
CRowInt m_priorities;
CRowInt m_rowdegrees;
CRowDouble m_effectivediag;
//--- constructor / destructor
CVIPMReducedSparseSystem(void) { m_ntotal=0; }
~CVIPMReducedSparseSystem(void) {}
void Copy(const CVIPMReducedSparseSystem &obj);
//--- overloading
void operator=(const CVIPMReducedSparseSystem &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CVIPMReducedSparseSystem::Copy(const CVIPMReducedSparseSystem &obj)
{
m_ntotal=obj.m_ntotal;
ArrayCopy(m_isdiagonal,obj.m_isdiagonal);
m_rawsystem=obj.m_rawsystem;
m_analysis=obj.m_analysis;
m_coldegrees=obj.m_coldegrees;
m_priorities=obj.m_priorities;
m_rowdegrees=obj.m_rowdegrees;
m_effectivediag=obj.m_effectivediag;
}
//+------------------------------------------------------------------+
//| Right - hand side for KKT system: |
//| * Rho corresponds to Ax - w = b |
//| * Nu corresponds to x - g = l |
//| * Tau corresponds to x + t = u |
//| * Alpha corresponds to w + p = r |
//| * Sigma corresponds to A'y+z-s-Hx=c |
//| * Beta corresponds to y + q - v = 0 |
//| * GammaZ, GammaS, GammaQ, GammaW correspond to complementarity |
//| conditions |
//+------------------------------------------------------------------+
struct CVIPMRightHandSide
{
CRowDouble m_alpha;
CRowDouble m_beta;
CRowDouble m_gammaq;
CRowDouble m_gammas;
CRowDouble m_gammaw;
CRowDouble m_gammaz;
CRowDouble m_nu;
CRowDouble m_rho;
CRowDouble m_sigma;
CRowDouble m_tau;
//--- constructor / destructor
CVIPMRightHandSide(void) {}
~CVIPMRightHandSide(void) {}
//---
void Copy(const CVIPMRightHandSide &obj);
//--- overloading
void operator=(const CVIPMRightHandSide &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CVIPMRightHandSide::Copy(const CVIPMRightHandSide &obj)
{
m_alpha=obj.m_alpha;
m_beta=obj.m_beta;
m_gammaq=obj.m_gammaq;
m_gammas=obj.m_gammas;
m_gammaw=obj.m_gammaw;
m_gammaz=obj.m_gammaz;
m_nu=obj.m_nu;
m_rho=obj.m_rho;
m_sigma=obj.m_sigma;
m_tau=obj.m_tau;
}
//+------------------------------------------------------------------+
//| VIPM State |
//+------------------------------------------------------------------+
struct CVIPMState
{
int m_cntgz;
int m_cntpq;
int m_cntts;
int m_cntwv;
int m_factorizationtype;
int m_hkind;
int m_mdense;
int m_msparse;
int m_n;
int m_nmain;
int m_repiterationscount;
int m_repncholesky;
double m_epsd;
double m_epsgap;
double m_epsp;
double m_targetscale;
bool m_aflips[];
bool m_dodetailedtrace;
bool m_dotrace;
bool m_factorizationpoweredup;
bool m_factorizationpresent;
bool m_HasBndL[];
bool m_HasBndU[];
bool m_hasgz[];
bool m_haspq[];
bool m_hasr[];
bool m_hasts[];
bool m_haswv[];
bool m_isdiagonalh;
bool m_isfrozen[];
bool m_islinear;
bool m_slacksforequalityconstraints;
CSparseMatrix m_combinedaslack;
CSparseMatrix m_sparseafull;
CSparseMatrix m_sparseamain;
CSparseMatrix m_sparseh;
CSparseMatrix m_tmpsparse0;
CVIPMVars m_best;
CVIPMVars m_current;
CVIPMVars m_deltaaff;
CVIPMVars m_deltacorr;
CVIPMVars m_trial;
CVIPMVars m_zerovars;
CVIPMRightHandSide m_rhs;
CVIPMReducedSparseSystem m_reducedsparsesystem;
CRowInt m_tmpi;
CRowDouble m_ascales;
CRowDouble m_b;
CRowDouble m_bndl;
CRowDouble m_bndu;
CRowDouble m_c;
CRowDouble m_deltaxy;
CRowDouble m_diagddr;
CRowDouble m_diagde;
CRowDouble m_diagder;
CRowDouble m_diagdq;
CRowDouble m_diagdqi;
CRowDouble m_diagdqiri;
CRowDouble m_diagds;
CRowDouble m_diagdsi;
CRowDouble m_diagdsiri;
CRowDouble m_diagdw;
CRowDouble m_diagdwi;
CRowDouble m_diagdwir;
CRowDouble m_diagdz;
CRowDouble m_diagdzi;
CRowDouble m_diagdziri;
CRowDouble m_diagr;
CRowDouble m_dummyr;
CRowDouble m_factinvregdzrz;
CRowDouble m_factregdhrh;
CRowDouble m_factregewave;
CRowDouble m_facttmpdiag;
CRowDouble m_invscl;
CRowDouble m_tmp0;
CRowDouble m_tmp1;
CRowDouble m_r;
CRowDouble m_rawbndl;
CRowDouble m_rawbndu;
CRowDouble m_rhsalphacap;
CRowDouble m_rhsbetacap;
CRowDouble m_rhsnucap;
CRowDouble m_rhstaucap;
CRowDouble m_scl;
CRowDouble m_tmp2;
CRowDouble m_tmpaty;
CRowDouble m_tmpax;
CRowDouble m_tmphx;
CRowDouble m_tmplaggrad;
CRowDouble m_tmpy;
CRowDouble m_xorigin;
CMatrixDouble m_denseafull;
CMatrixDouble m_denseamain;
CMatrixDouble m_denseh;
CMatrixDouble m_factdensehaug;
CMatrixDouble m_tmpr2;
//--- constructor / destructor
CVIPMState(void);
~CVIPMState(void) {}
//---
void Copy(const CVIPMState &obj);
//--- overloading
void operator=(const CVIPMState &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CVIPMState::CVIPMState(void)
{
m_cntgz=0;
m_cntpq=0;
m_cntts=0;
m_cntwv=0;
m_factorizationtype=0;
m_hkind=0;
m_mdense=0;
m_msparse=0;
m_n=0;
m_nmain=0;
m_repiterationscount=0;
m_repncholesky=0;
m_epsd=0;
m_epsgap=0;
m_epsp=0;
m_targetscale=0;
m_dodetailedtrace=false;
m_dotrace=false;
m_factorizationpoweredup=false;
m_factorizationpresent=false;
m_isdiagonalh=false;
m_islinear=false;
m_slacksforequalityconstraints=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CVIPMState::Copy(const CVIPMState &obj)
{
m_cntgz=obj.m_cntgz;
m_cntpq=obj.m_cntpq;
m_cntts=obj.m_cntts;
m_cntwv=obj.m_cntwv;
m_factorizationtype=obj.m_factorizationtype;
m_hkind=obj.m_hkind;
m_mdense=obj.m_mdense;
m_msparse=obj.m_msparse;
m_n=obj.m_n;
m_nmain=obj.m_nmain;
m_repiterationscount=obj.m_repiterationscount;
m_repncholesky=obj.m_repncholesky;
m_epsd=obj.m_epsd;
m_epsgap=obj.m_epsgap;
m_epsp=obj.m_epsp;
m_targetscale=obj.m_targetscale;
ArrayCopy(m_aflips,obj.m_aflips);
m_dodetailedtrace=obj.m_dodetailedtrace;
m_dotrace=obj.m_dotrace;
m_factorizationpoweredup=obj.m_factorizationpoweredup;
m_factorizationpresent=obj.m_factorizationpresent;
ArrayCopy(m_HasBndL,obj.m_HasBndL);
ArrayCopy(m_HasBndU,obj.m_HasBndU);
ArrayCopy(m_hasgz,obj.m_hasgz);
ArrayCopy(m_haspq,obj.m_haspq);
ArrayCopy(m_hasr,obj.m_hasr);
ArrayCopy(m_hasts,obj.m_hasts);
ArrayCopy(m_haswv,obj.m_haswv);
m_isdiagonalh=obj.m_isdiagonalh;
ArrayCopy(m_isfrozen,obj.m_isfrozen);
m_islinear=obj.m_islinear;
m_slacksforequalityconstraints=obj.m_slacksforequalityconstraints;
m_combinedaslack=obj.m_combinedaslack;
m_sparseafull=obj.m_sparseafull;
m_sparseamain=obj.m_sparseamain;
m_sparseh=obj.m_sparseh;
m_tmpsparse0=obj.m_tmpsparse0;
m_best=obj.m_best;
m_current=obj.m_current;
m_deltaaff=obj.m_deltaaff;
m_deltacorr=obj.m_deltacorr;
m_trial=obj.m_trial;
m_zerovars=obj.m_zerovars;
m_rhs=obj.m_rhs;
m_reducedsparsesystem=obj.m_reducedsparsesystem;
m_tmpi=obj.m_tmpi;
m_ascales=obj.m_ascales;
m_b=obj.m_b;
m_bndl=obj.m_bndl;
m_bndu=obj.m_bndu;
m_c=obj.m_c;
m_deltaxy=obj.m_deltaxy;
m_diagddr=obj.m_diagddr;
m_diagde=obj.m_diagde;
m_diagder=obj.m_diagder;
m_diagdq=obj.m_diagdq;
m_diagdqi=obj.m_diagdqi;
m_diagdqiri=obj.m_diagdqiri;
m_diagds=obj.m_diagds;
m_diagdsi=obj.m_diagdsi;
m_diagdsiri=obj.m_diagdsiri;
m_diagdw=obj.m_diagdw;
m_diagdwi=obj.m_diagdwi;
m_diagdwir=obj.m_diagdwir;
m_diagdz=obj.m_diagdz;
m_diagdzi=obj.m_diagdzi;
m_diagdziri=obj.m_diagdziri;
m_diagr=obj.m_diagr;
m_dummyr=obj.m_dummyr;
m_factinvregdzrz=obj.m_factinvregdzrz;
m_factregdhrh=obj.m_factregdhrh;
m_factregewave=obj.m_factregewave;
m_facttmpdiag=obj.m_facttmpdiag;
m_invscl=obj.m_invscl;
m_tmp0=obj.m_tmp0;
m_tmp1=obj.m_tmp1;
m_r=obj.m_r;
m_rawbndl=obj.m_rawbndl;
m_rawbndu=obj.m_rawbndu;
m_rhsalphacap=obj.m_rhsalphacap;
m_rhsbetacap=obj.m_rhsbetacap;
m_rhsnucap=obj.m_rhsnucap;
m_rhstaucap=obj.m_rhstaucap;
m_scl=obj.m_scl;
m_tmp2=obj.m_tmp2;
m_tmpaty=obj.m_tmpaty;
m_tmpax=obj.m_tmpax;
m_tmphx=obj.m_tmphx;
m_tmplaggrad=obj.m_tmplaggrad;
m_tmpy=obj.m_tmpy;
m_xorigin=obj.m_xorigin;
m_denseafull=obj.m_denseafull;
m_denseamain=obj.m_denseamain;
m_denseh=obj.m_denseh;
m_factdensehaug=obj.m_factdensehaug;
m_tmpr2=obj.m_tmpr2;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CVIPMSolver
{
public:
//--- constants
static const double m_muquasidense;
static const int m_maxipmits;
static const double m_initslackval;
static const double m_steplengthdecay;
static const double m_stagnationdelta;
static const double m_primalinfeasible1;
static const double m_dualinfeasible1;
static const double m_bigy;
static const double m_ygrowth;
static const int m_itersfortoostringentcond;
static const int m_minitersbeforedroppingbounds;
static const int m_minitersbeforeinfeasible;
static const int m_minitersbeforestagnation;
static const int m_minitersbeforeeworststagnation;
static const int m_primalstagnationlen;
static const int m_dualstagnationlen;
static const double m_bigconstrxtol;
static const double m_bigconstrmag;
static const double m_minitersbeforesafeguards;
static const double m_badsteplength;
static void VIPMInitDense(CVIPMState &State,CRowDouble &s,CRowDouble &xorigin,int n);
static void VIPMInitDenseWithSlacks(CVIPMState &State,CRowDouble &s,CRowDouble &xorigin,int nmain,int n);
static void VIPMInitSparse(CVIPMState &State,CRowDouble &s,CRowDouble &xorigin,int n);
static void VIPMSetQuadraticLinear(CVIPMState &State,CMatrixDouble &denseh,CSparseMatrix &sparseh,int hkind,bool IsUpper,CRowDouble &c);
static void VIPMSetConstraints(CVIPMState &State,CRowDouble &bndl,CRowDouble &bndu,CSparseMatrix &sparsea,int msparse,CMatrixDouble &densea,int mdense,CRowDouble &cl,CRowDouble &cu);
static void VIPMSetCond(CVIPMState &State,double epsp,double epsd,double epsgap);
static void VIPMOptimize(CVIPMState &State,bool dropbigbounds,CRowDouble &xs,CRowDouble &lagbc,CRowDouble &laglc,int &terminationtype);
private:
static void VARSInitByZero(CVIPMVars &vstate,int n,int m);
static void VARSInitFrom(CVIPMVars &vstate,CVIPMVars &vsrc);
static void VarsAddStep(CVIPMVars &vstate,CVIPMVars &vdir,double stpp,double stpd);
static double VarsComputeComplementarityGap(CVIPMVars &vstate);
static double VARSComputeMu(CVIPMState &State,CVIPMVars &vstate);
static void ReducedSystemInit(CVIPMReducedSparseSystem &s,CVIPMState &solver);
static bool ReducedSystemFactorizeWithAddEnd(CVIPMReducedSparseSystem &s,CRowDouble &d,double modeps,double badchol,double &sumsq,double &errsq);
static void ReducedSystemSolve(CVIPMReducedSparseSystem &s,CRowDouble &b);
static void VIPMInit(CVIPMState &State,CRowDouble &s,CRowDouble &xorigin,int n,int nmain,int ftype);
static double VIPMTarget(CVIPMState &State,CRowDouble &x);
static void MultiplyGEAX(CVIPMState &State,double alpha,CRowDouble &x,int offsx,double beta,CRowDouble &y,int offsax);
static void MultiplyGEATX(CVIPMState &State,double alpha,CRowDouble &x,int offsx,double beta,CRowDouble &y,int offsy);
static void MultiplyHX(CVIPMState &State,CRowDouble &x,CRowDouble &hx);
static void VIPMMultiply(CVIPMState &State,CRowDouble &x,CRowDouble &y,CRowDouble &hx,CRowDouble &ax,CRowDouble &aty);
static void VIPMPowerUp(CVIPMState &State,double regfree);
static bool VIPMFactorize(CVIPMState &State,double alpha0,CRowDouble &d,double beta0,CRowDouble &e,double alpha11,double beta11,double modeps,double dampeps);
static void SolveReducedKKTSystem(CVIPMState &State,CRowDouble &deltaxy);
static bool VIPMPrecomputeNewtonFactorization(CVIPMState &State,CVIPMVars &v0,double regeps,double modeps,double dampeps,double dampfree);
static void SolveKKTSystem(CVIPMState &State,CVIPMRightHandSide &rhs,CVIPMVars &sol);
static bool VIPMComputeStepDirection(CVIPMState &State,CVIPMVars &v0,double muestimate,CVIPMVars &vdestimate,CVIPMVars &vdresult,double reg,bool isdampepslarge);
static void VIPMComputeStepLength(CVIPMState &State,CVIPMVars &v0,CVIPMVars &vs,double stepdecay,double &alphap,double &alphad);
static void VIPMPerformStep(CVIPMState &State,double alphap,double alphad);
static void ComputeErrors(CVIPMState &State,double &errp2,double &errd2,double &errpinf,double &errdinf,double &egap);
static void RunIntegrityChecks(CVIPMState &State,CVIPMVars &v0,CVIPMVars &vd,double alphap,double alphad);
static void TraceProgress(CVIPMState &State,double mu,double muaff,double sigma,double alphap,double alphad);
static void RHSCompute(CVIPMState &State,CVIPMVars &v0,double muestimate,CVIPMVars &direstimate,CVIPMRightHandSide &rhs,double reg);
static void RHSSubtract(CVIPMState &State,CVIPMRightHandSide &rhs,CVIPMVars &v0,CVIPMVars &vdcandidate,double reg);
static double RHSPrimal2(CVIPMRightHandSide &rhs);
static double RHSDual2(CVIPMRightHandSide &rhs);
static double RHSPrimalInf(CVIPMRightHandSide &rhs);
static double RHSDualInf(CVIPMRightHandSide &rhs);
static double RHSCompl2(CVIPMRightHandSide &rhs);
static double MinNZ(CRowDouble &x,int n);
static double MinProdNZ(CRowDouble &x,CRowDouble &y,int n);
static double MaxProdNZ(CRowDouble &x,CRowDouble &y,int n);
};
//+------------------------------------------------------------------+
//| Constants |
//+------------------------------------------------------------------+
const double CVIPMSolver::m_muquasidense=2.0;
const int CVIPMSolver::m_maxipmits=200;
const double CVIPMSolver::m_initslackval=100.0;
const double CVIPMSolver::m_steplengthdecay=0.95;
const double CVIPMSolver::m_stagnationdelta=0.99999;
const double CVIPMSolver::m_primalinfeasible1=1.0E-3;
const double CVIPMSolver::m_dualinfeasible1=1.0E-3;
const double CVIPMSolver::m_bigy=1.0E8;
const double CVIPMSolver::m_ygrowth=1.0E6;
const int CVIPMSolver::m_itersfortoostringentcond=25;
const int CVIPMSolver::m_minitersbeforedroppingbounds=3;
const int CVIPMSolver::m_minitersbeforeinfeasible=3;
const int CVIPMSolver::m_minitersbeforestagnation=5;
const int CVIPMSolver::m_minitersbeforeeworststagnation=50;
const int CVIPMSolver::m_primalstagnationlen=5;
const int CVIPMSolver::m_dualstagnationlen=7;
const double CVIPMSolver::m_bigconstrxtol=1.0E-5;
const double CVIPMSolver::m_bigconstrmag=1.0E3;
const double CVIPMSolver::m_minitersbeforesafeguards=5;
const double CVIPMSolver::m_badsteplength=1.0E-3;
//+------------------------------------------------------------------+
//| Initializes QP - IPM State and prepares it to receive quadratic/ |
//| linear terms and constraints. |
//| The solver is configured to work internally with dense NxN |
//| factorization, no matter what exactly is passed - dense or sparse|
//| matrices. |
//| INPUT PARAMETERS: |
//| State - solver State to be configured; previously allocated|
//| memory is reused as much as possible |
//| S - scale vector, array[N]: |
//| * I-th element contains scale of I-th variable, |
//| * S[I] > 0 |
//| XOrigin - origin term, array[N]. Can be zero. The solver |
//| solves problem of the form |
//| > |
//| > min(0.5*(x-x_origin)'*A*(x-x_origin)+b'*(x-x_origin)) |
//| > |
//| The terms A and b(as well as constraints) will be specified later|
//| with separate calls. |
//+------------------------------------------------------------------+
void CVIPMSolver::VIPMInitDense(CVIPMState &State,
CRowDouble &s,
CRowDouble &xorigin,
int n)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(s,n),__FUNCTION__+": S contains infinite or NaN elements"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(xorigin,n),__FUNCTION__+": XOrigin contains infinite or NaN elements"))
return;
VIPMInit(State,s,xorigin,n,n,0);
}
//+------------------------------------------------------------------+
//| Initializes QP-IPM State and prepares it to receive quadratic/ |
//| linear terms and constraints. |
//| The solver is configured to work internally with dense NxN |
//| problem divided into two distinct parts-"main" and slack one: |
//| * dense quadratic term is a NMain*NMain matrix(NMain <= N),|
//| quadratic coefficients are zero for variables outside of |
//| [0, NMain) range) |
//| * linear term is general vector of length N |
//| * linear constraints have special structure for variable with |
//| indexes in [NMain, N) range: at most one element per column |
//| can be nonzero. |
//| This mode is intended for problems arising during SL1QP nonlinear|
//| programming. |
//| INPUT PARAMETERS: |
//| State - m_solver State to be configured; previously |
//| allocated memory is reused as much as possible |
//| S - scale vector, array[N]: |
//| * I-th element contains scale of I-th variable, |
//| * S[I] > 0 |
//| XOrigin - origin term, array[N]. Can be zero. The solver |
//| solves problem of the form |
//| > |
//| > min(0.5 * (x - x_origin)'*A*(x-x_origin)+b' * (x - x_origin))|
//| > |
//| The terms A and b(as well as constraints) will be specified later|
//| with separate calls. |
//| NMain - number of "main" variables, 1 <= NMain <= N |
//| N - total number of variables including slack ones |
//+------------------------------------------------------------------+
void CVIPMSolver::VIPMInitDenseWithSlacks(CVIPMState &State,
CRowDouble &s,
CRowDouble &xorigin,
int nmain,
int n)
{
//--- check
if(!CAp::Assert(nmain>=1,__FUNCTION__+": NMain<1"))
return;
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
if(!CAp::Assert(nmain<=n,__FUNCTION__+": NMain>N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(s,n),__FUNCTION__+": S contains infinite or NaN elements"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(xorigin,n),__FUNCTION__+": XOrigin contains infinite or NaN elements"))
return;
VIPMInit(State,s,xorigin,n,nmain,0);
}
//+------------------------------------------------------------------+
//| Initializes QP-IPM State and prepares it to receive quadratic/ |
//| linear terms and constraints. |
//| The solver is configured to work internally with sparse |
//| (N + M)x(N + M) factorization no matter what exactly is passed - |
//| dense or sparse matrices. Dense quadratic term will be sparsified|
//| prior to storage. |
//| INPUT PARAMETERS: |
//| State - solver State to be configured; previously allocated|
//| memory is reused as much as possible |
//| S - scale vector, array[N]: |
//| * I-th element contains scale of I-th variable, |
//| * S[I] > 0 |
//| XOrigin - origin term, array[N]. Can be zero. The solver |
//| solves problem of the form |
//| > |
//| > min(0.5*(x - x_origin)'*A*(x-x_origin)+b'*(x - x_origin)) |
//| > |
//| The terms A and b(as well as constraints) will be specified later|
//| with separate calls. |
//| N - total number of variables, N >= 1 |
//| This optimization mode assumes that no slack variables is present|
//+------------------------------------------------------------------+
void CVIPMSolver::VIPMInitSparse(CVIPMState &State,
CRowDouble &s,
CRowDouble &xorigin,
int n)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(s,n),__FUNCTION__+": S contains infinite or NaN elements"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(xorigin,n),__FUNCTION__+": XOrigin contains infinite or NaN elements"))
return;
VIPMInit(State,s,xorigin,n,n,1);
}
//+------------------------------------------------------------------+
//| Sets linear / quadratic terms for QP - IPM m_solver |
//| If you initialized m_solver with VIMPInitDenseWithSlacks(), NMain|
//| below is a number of non-slack variables. In other cases, |
//| NMain = N. |
//| INPUT PARAMETERS: |
//| State - instance initialized with one of the initialization|
//| functions |
//| DenseH - if HKind = 0: array[NMain, NMain], dense quadratic |
//| term (either upper or lower triangle) |
//| SparseH - if HKind = 1: array[NMain, NMain], sparse quadratic|
//| term (either upper or lower triangle) |
//| HKind - 0 or 1, quadratic term format |
//| IsUpper - whether dense / sparse H contains lower or upper |
//| triangle of the quadratic term |
//| C - array[N], linear term |
//+------------------------------------------------------------------+
void CVIPMSolver::VIPMSetQuadraticLinear(CVIPMState &State,
CMatrixDouble &denseh,
CSparseMatrix &sparseh,
int hkind,
bool IsUpper,
CRowDouble &c)
{
//--- create variables
int nmain=State.m_nmain;
int n=State.m_n;
int i=0;
int j=0;
int k=0;
int j0=0;
int j1=0;
double v=0;
double vv=0;
int nnz=0;
int offs=0;
//--- check
if(!CAp::Assert(hkind==0 || hkind==1,__FUNCTION__+": incorrect HKind"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(c,n),__FUNCTION__+": C contains infinite or NaN elements"))
return;
if(!CAp::Assert(State.m_factorizationtype==0 || State.m_factorizationtype==1,__FUNCTION__+": unexpected factorization type"))
return;
//--- Set problem Info, reset factorization flag
State.m_islinear=false;
State.m_factorizationpresent=false;
State.m_factorizationpoweredup=false;
//--- Linear term
State.m_c=c;
State.m_c.Resize(n);
//--- Quadratic term and normalization
//--- NOTE: we perform integrity check for inifinities/NANs by
//--- computing sum of all matrix elements and checking its
//--- value for being finite. It is a bit faster than checking
//--- each element individually.
State.m_hkind=-1;
State.m_targetscale=1.0;
switch(State.m_factorizationtype)
{
case 0:
//--- Quadratic term is stored in dense format: either copy dense
//--- term of densify sparse one
State.m_hkind=0;
switch(hkind)
{
case 0:
//--- Copy dense quadratic term
if(IsUpper)
State.m_denseh=denseh.Transpose()+0;
else
State.m_denseh=denseh;
State.m_denseh.Resize(nmain,nmain);
break;
case 1:
//--- Extract sparse quadratic term
if(!CAp::Assert(sparseh.m_MatrixType==1,__FUNCTION__+": unexpected sparse matrix format"))
return;
if(!CAp::Assert(sparseh.m_M==nmain,__FUNCTION__+": unexpected sparse matrix size"))
return;
if(!CAp::Assert(sparseh.m_N==nmain,__FUNCTION__+": unexpected sparse matrix size"))
return;
State.m_denseh.Resize(nmain,nmain);
for(i=0; i<nmain; i++)
for(j=0; j<=i; j++)
State.m_denseh.Set(i,j,0);
for(i=0; i<nmain; i++)
{
//--- diagonal element
if(sparseh.m_DIdx[i]!=sparseh.m_UIdx[i])
State.m_denseh.Set(i,i,sparseh.m_Vals[sparseh.m_DIdx[i]]);
//--- Off-diagonal elements
if(IsUpper)
{
//--- superdiagonal elements are moved to subdiagonal part
j0=sparseh.m_UIdx[i];
j1=sparseh.m_RIdx[i+1];
for(j=j0; j<j1; j++)
State.m_denseh.Set(sparseh.m_Idx[j],i,sparseh.m_Vals[j]);
}
else
{
//--- subdiagonal elements are moved to subdiagonal part
j0=sparseh.m_RIdx[i];
j1=sparseh.m_DIdx[i];
for(j=j0; j<j1; j++)
State.m_denseh.Set(i,sparseh.m_Idx[j],sparseh.m_Vals[j]);
}
}
break;
}
vv=0;
for(i=0; i<nmain; i++)
{
for(j=0; j<=i; j++)
vv+=State.m_denseh.Get(i,j);
}
//--- check
if(!CAp::Assert(MathIsValidNumber(vv),__FUNCTION__+": DenseH contains infinite or NaN values!"))
return;
CLPQPServ::ScaleDenseQPInplace(State.m_denseh,false,nmain,State.m_c,n,State.m_scl);
State.m_targetscale=CLPQPServ::NormalizeDenseQPInplace(State.m_denseh,false,nmain,State.m_c,n);
State.m_isdiagonalh=false;
break;
case 1:
//--- check
if(!CAp::Assert(nmain==n,__FUNCTION__+": critical integrity check failed,NMain!=N"))
return;
//--- Quadratic term is stored in sparse format: either sparsify dense
//--- term or copy the sparse one
State.m_hkind=1;
State.m_sparseh.m_MatrixType=1;
State.m_sparseh.m_M=n;
State.m_sparseh.m_N=n;
switch(hkind)
{
case 0:
//--- Sparsify dense term
nnz=0;
for(i=0; i<n; i++)
{
nnz++;
if(IsUpper)
{
j0=i+1;
j1=n-1;
}
else
{
j0=0;
j1=i-1;
}
for(j=j0; j<=j1; j++)
if(denseh.Get(i,j)!=0)
nnz++;
}
State.m_sparseh.m_RIdx.Resize(n+1);
State.m_sparseh.m_Idx.Resize(nnz);
State.m_sparseh.m_Vals.Resize(nnz);
State.m_sparseh.m_RIdx.Set(0,0);
offs=0;
vv=0;
for(i=0; i<n; i++)
{
//--- Off-diagonal elements are copied only when nonzero
if(!IsUpper)
{
for(j=0; j<=i-1; j++)
{
if(denseh.Get(i,j)!=0)
{
v=denseh.Get(i,j);
State.m_sparseh.m_Idx.Set(offs,j);
State.m_sparseh.m_Vals.Set(offs,v);
vv+=v;
offs++;
}
}
}
//--- Diagonal element is always copied
v=denseh.Get(i,i);
State.m_sparseh.m_Idx.Set(offs,i);
State.m_sparseh.m_Vals.Set(offs,v);
vv+=v;
offs++;
//--- Off-diagonal elements are copied only when nonzero
if(IsUpper)
{
for(j=i+1; j<n; j++)
{
v=denseh.Get(i,j);
if(v!=0)
{
State.m_sparseh.m_Idx.Set(offs,j);
State.m_sparseh.m_Vals.Set(offs,v);
vv+=v;
offs++;
}
}
}
//--- Finalize row
State.m_sparseh.m_RIdx.Set(i+1,offs);
}
//--- check
if(!CAp::Assert(MathIsValidNumber(vv),__FUNCTION__+": DenseH contains infinite or NaN values!"))
return;
if(!CAp::Assert(offs==nnz,__FUNCTION__+": integrity check failed"))
return;
CSparse::SparseCreateCRSInplace(State.m_sparseh);
break;
case 1:
//--- Copy sparse quadratic term, but make sure that we have diagonal elements
//--- present (we add diagonal if it is not present)
if(!CAp::Assert(sparseh.m_MatrixType==1,__FUNCTION__+": unexpected sparse matrix format"))
return;
if(!CAp::Assert(sparseh.m_M==n,__FUNCTION__+": unexpected sparse matrix size"))
return;
if(!CAp::Assert(sparseh.m_N==n,__FUNCTION__+": unexpected sparse matrix size"))
return;
State.m_sparseh.m_RIdx.Resize(n+1);
State.m_sparseh.m_Idx.Resize(sparseh.m_RIdx[n]+n);
State.m_sparseh.m_Vals.Resize(sparseh.m_RIdx[n]+n);
State.m_sparseh.m_RIdx.Set(0,0);
offs=0;
vv=0;
for(i=0; i<n; i++)
{
//--- Copy subdiagonal elements (if needed)
if(!IsUpper)
{
j0=sparseh.m_RIdx[i];
j1=sparseh.m_DIdx[i]-1;
for(k=j0; k<=j1; k++)
{
v=sparseh.m_Vals[k];
State.m_sparseh.m_Idx.Set(offs,sparseh.m_Idx[k]);
State.m_sparseh.m_Vals.Set(offs,v);
vv+=v;
offs++;
}
}
//--- Diagonal element is always copied
v=0;
if(sparseh.m_UIdx[i]!=sparseh.m_DIdx[i])
v=sparseh.m_Vals[sparseh.m_DIdx[i]];
State.m_sparseh.m_Idx.Set(offs,i);
State.m_sparseh.m_Vals.Set(offs,v);
vv+=v;
offs++;
//--- Copy superdiagonal elements (if needed)
if(IsUpper)
{
j0=sparseh.m_UIdx[i];
j1=sparseh.m_RIdx[i+1]-1;
for(k=j0; k<=j1; k++)
{
v=sparseh.m_Vals[k];
State.m_sparseh.m_Idx.Set(offs,sparseh.m_Idx[k]);
State.m_sparseh.m_Vals.Set(offs,v);
vv+=v;
offs++;
}
}
//--- Finalize row
State.m_sparseh.m_RIdx.Set(i+1,offs);
}
//--- check
if(!CAp::Assert(MathIsValidNumber(vv),__FUNCTION__+": SparseH contains infinite or NaN values!"))
return;
if(!CAp::Assert(offs<=State.m_sparseh.m_Vals.Size() && offs<=State.m_sparseh.m_Idx.Size(),__FUNCTION__+": integrity check failed"))
return;
CSparse::SparseCreateCRSInplace(State.m_sparseh);
if(IsUpper)
{
CSparse::SparseCopyTransposeCRSBuf(State.m_sparseh,State.m_tmpsparse0);
CSparse::SparseCopyBuf(State.m_tmpsparse0,State.m_sparseh);
}
break;
}
CLPQPServ::ScaleSparseQPInplace(State.m_scl,n,State.m_sparseh,State.m_c);
State.m_targetscale=CLPQPServ::NormalizeSparseQPInplace(State.m_sparseh,false,State.m_c,n);
State.m_isdiagonalh=State.m_sparseh.m_RIdx[n]==n;
break;
}
CAp::Assert(State.m_hkind>=0,__FUNCTION__+": integrity check failed");
}
//+------------------------------------------------------------------+
//| Sets constraints for QP - IPM m_solver |
//| INPUT PARAMETERS: |
//| State - instance initialized with one of the initialization|
//| functions |
//| BndL, BndU - lower and upper bound. BndL[] can be - INF, |
//| BndU[] can be + INF. |
//| SparseA - sparse constraint matrix, CRS format |
//| MSparse - number of sparse constraints |
//| DenseA - array[MDense, N], dense part of the constraints |
//| MDense - number of dense constraints |
//| CL, CU - lower and upper bounds for constraints, first |
//| MSparse are bounds for sparse part, following MDense ones are |
//| bounds for dense part, MSparse + MDense in total. |
//| - INF <= CL[I] <= CU[I] <= +INF. |
//| This function throws exception if constraints have inconsistent |
//| bounds, i.e. either BndL[I] > BndU[I] or CL[I] > CU[I]. In all |
//| other cases it succeeds. |
//+------------------------------------------------------------------+
void CVIPMSolver::VIPMSetConstraints(CVIPMState &State,
CRowDouble &bndl,
CRowDouble &bndu,
CSparseMatrix &sparsea,
int msparse,
CMatrixDouble &densea,
int mdense,
CRowDouble &cl,
CRowDouble &cu)
{
//--- create variables
int n=State.m_n;
int nmain=State.m_nmain;
int nslack=n-nmain;
int m=0;
int i=0;
int j=0;
int j0=0;
int j1=0;
int k=0;
int offsmain=0;
int offscombined=0;
double vs=0;
double v=0;
//--- check
if(!CAp::Assert(mdense>=0,__FUNCTION__+": MDense<0"))
return;
if(!CAp::Assert(msparse>=0,__FUNCTION__+": MSparse<0"))
return;
if(!CAp::Assert(CApServ::IsFiniteMatrix(densea,mdense,n),__FUNCTION__+": DenseA contains infinite or NaN values!"))
return;
if(!CAp::Assert(msparse==0 || sparsea.m_MatrixType==1,__FUNCTION__+": non-CRS constraint matrix!"))
return;
if(!CAp::Assert(msparse==0 || (sparsea.m_M==msparse && sparsea.m_N==n),__FUNCTION__+": constraint matrix has incorrect size"))
return;
if(!CAp::Assert(cl.Size()>=mdense+msparse,__FUNCTION__+": CL is too short!"))
return;
if(!CAp::Assert(cu.Size()>=mdense+msparse,__FUNCTION__+": CU is too short!"))
return;
//--- Reset factorization flag
State.m_factorizationpresent=false;
State.m_factorizationpoweredup=false;
//--- Box constraints
State.m_bndl.Resize(n);
State.m_bndu.Resize(n);
State.m_rawbndl.Resize(n);
State.m_rawbndu.Resize(n);
ArrayResize(State.m_HasBndL,n);
ArrayResize(State.m_HasBndU,n);
for(i=0; i<n; i++)
{
State.m_HasBndL[i]=MathIsValidNumber(bndl[i]);
State.m_HasBndU[i]=MathIsValidNumber(bndu[i]);
//--- check
if(!CAp::Assert(!(State.m_HasBndL[i] && State.m_HasBndU[i] && bndl[i]>bndu[i]),__FUNCTION__+": inconsistent range for box constraints"))
return;
State.m_bndl.Set(i,bndl[i]);
State.m_bndu.Set(i,bndu[i]);
State.m_rawbndl.Set(i,bndl[i]);
State.m_rawbndu.Set(i,bndu[i]);
}
CLPQPServ::ScaleShiftBCInplace(State.m_scl,State.m_xorigin,State.m_bndl,State.m_bndu,n);
//--- Linear constraints (full matrices)
m=mdense+msparse;
State.m_b.Resize(m);
State.m_r.Resize(m);
State.m_ascales.Resize(m);
ArrayResize(State.m_aflips,m);
ArrayResize(State.m_hasr,m);
if(msparse>0)
CSparse::SparseCopyToCRSBuf(sparsea,State.m_sparseafull);
if(mdense>0)
State.m_denseafull=densea;
State.m_denseafull.Resize(mdense,n);
for(i=0; i<m; i++)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(cl[i]) || AL_NEGINF==cl[i],__FUNCTION__+": CL is not finite number or -INF"))
return;
if(!CAp::Assert(MathIsValidNumber(cu[i]) || AL_POSINF==cu[i],__FUNCTION__+": CU is not finite number or +INF"))
return;
//--- Store range
if(MathIsValidNumber(cl[i]) || MathIsValidNumber(cu[i]))
{
//--- Non-degenerate constraint, at least one of bounds is present
if(MathIsValidNumber(cl[i]))
{
if(!CAp::Assert(!MathIsValidNumber(cu[i]) || cu[i]>=cl[i],__FUNCTION__+": inconsistent range (right-hand side) for linear constraint"))
return;
if(MathIsValidNumber(cu[i]))
{
//--- We have both CL and CU, i.e. CL <= A*x <= CU.
//--- It can be either equality constraint (no slacks) or range constraint
//--- (two pairs of slacks variables).
//--- Try to arrange things in such a way that |CU|>=|CL| (it can be done
//--- by multiplication by -1 and boundaries swap).
//--- Having |CU|>=|CL| will allow us to drop huge irrelevant bound CU,
//--- if we find it irrelevant during computations. Due to limitations
//--- of our slack variable substitution, it can be done only for CU.
if(MathAbs(cu[i])>=MathAbs(cl[i]))
{
State.m_b.Set(i,cl[i]);
State.m_r.Set(i,cu[i]-cl[i]);
State.m_hasr[i]=true;
State.m_aflips[i]=false;
vs=1;
}
else
{
State.m_b.Set(i,-cu[i]);
State.m_r.Set(i,cu[i]-cl[i]);
State.m_hasr[i]=true;
State.m_aflips[i]=true;
vs=-1;
}
}
else
{
//--- Only lower bound: CL <= A*x.
//--- One pair of slack variables added.
State.m_b.Set(i,cl[i]);
State.m_r.Set(i,AL_POSINF);
State.m_hasr[i]=false;
State.m_aflips[i]=false;
vs=1;
}
}
else
{
//--- Only upper bound: A*x <= CU
//--- One pair of slack variables added.
State.m_b.Set(i,-cu[i]);
State.m_r.Set(i,AL_POSINF);
State.m_hasr[i]=false;
State.m_aflips[i]=true;
vs=-1;
}
}
else
{
//--- Degenerate constraint -inf <= Ax <= +inf.
//--- Generate dummy formulation.
State.m_b.Set(i,-1);
State.m_r.Set(i,2);
State.m_hasr[i]=true;
State.m_aflips[i]=false;
vs=0;
}
//--- Store matrix row and its scaling coefficient
if(i<msparse)
{
j0=State.m_sparseafull.m_RIdx[i];
j1=State.m_sparseafull.m_RIdx[i+1];
for(j=j0; j<j1; j++)
State.m_sparseafull.m_Vals.Mul(j,vs);
}
else
State.m_denseafull.Row(i-msparse,State.m_denseafull[i-msparse]*vs);
State.m_ascales.Set(i,vs);
}
CLPQPServ::ScaleShiftMixedBRLCInplace(State.m_scl,State.m_xorigin,n,State.m_sparseafull,msparse,State.m_denseafull,mdense,State.m_b,State.m_r);
CLPQPServ::NormalizeMixedBRLCInplace(State.m_sparseafull,msparse,State.m_denseafull,mdense,State.m_b,State.m_r,n,true,State.m_tmp0,true);
State.m_ascales*=State.m_tmp0;
State.m_mdense=mdense;
State.m_msparse=msparse;
//--- Separate main and slack parts of the constraint matrices
State.m_tmpi.Resize(nslack);
State.m_tmpi.Fill(0);
State.m_combinedaslack.m_M=mdense+msparse;
State.m_combinedaslack.m_N=nslack;
State.m_combinedaslack.m_RIdx.Resize(mdense+msparse+1);
State.m_combinedaslack.m_Idx.Resize(nslack);
State.m_combinedaslack.m_Vals.Resize(nslack);
State.m_combinedaslack.m_RIdx.Set(0,0);
State.m_sparseamain.m_M=msparse;
State.m_sparseamain.m_N=nmain;
if(msparse>0)
{
State.m_sparseamain.m_RIdx.Resize(msparse+1);
State.m_sparseamain.m_Idx.Resize(sparsea.m_RIdx[msparse]);
State.m_sparseamain.m_Vals.Resize(sparsea.m_RIdx[msparse]);
State.m_sparseamain.m_RIdx.Set(0,0);
for(i=0; i<msparse; i++)
{
offsmain=State.m_sparseamain.m_RIdx[i];
offscombined=State.m_combinedaslack.m_RIdx[i];
j0=State.m_sparseafull.m_RIdx[i];
j1=State.m_sparseafull.m_RIdx[i+1];
for(j=j0; j<j1; j++)
{
v=State.m_sparseafull.m_Vals[j];
k=State.m_sparseafull.m_Idx[j];
if(k<nmain)
{
State.m_sparseamain.m_Idx.Set(offsmain,k);
State.m_sparseamain.m_Vals.Set(offsmain,v);
offsmain++;
}
else
{
//--- check
if(!CAp::Assert(State.m_tmpi[k-nmain]==0,__FUNCTION__+": slack column contains more than one nonzero element"))
return;
State.m_combinedaslack.m_Idx.Set(offscombined,k-nmain);
State.m_combinedaslack.m_Vals.Set(offscombined,v);
State.m_tmpi.Add(k-nmain,1);
offscombined++;
}
}
State.m_sparseamain.m_RIdx.Set(i+1,offsmain);
State.m_combinedaslack.m_RIdx.Set(i+1,offscombined);
}
}
CSparse::SparseCreateCRSInplace(State.m_sparseamain);
if(mdense>0)
{
State.m_denseamain=State.m_denseafull;
State.m_denseamain.Resize(mdense,nmain);
for(i=0; i<mdense; i++)
{
offscombined=State.m_combinedaslack.m_RIdx[msparse+i];
for(k=nmain; k<n; k++)
{
if(State.m_denseafull.Get(i,k)!=0)
{
//--- check
if(!CAp::Assert(State.m_tmpi[k-nmain]==0,__FUNCTION__+": slack column contains more than one nonzero element"))
return;
State.m_combinedaslack.m_Idx.Set(offscombined,k-nmain);
State.m_combinedaslack.m_Vals.Set(offscombined,State.m_denseafull.Get(i,k));
State.m_tmpi.Add(k-nmain,1);
offscombined++;
}
}
State.m_combinedaslack.m_RIdx.Set(msparse+i+1,offscombined);
}
}
CSparse::SparseCreateCRSInplace(State.m_combinedaslack);
}
//+------------------------------------------------------------------+
//| Sets stopping criteria for QP - IPM m_solver. |
//| You can set all epsilon-values to one small value, about 1.0E-6. |
//| INPUT PARAMETERS: |
//| State - instance initialized with one of the |
//| initialization functions |
//| EpsP - maximum primal error allowed in the solution, |
//| EpsP >= 0. Zero will be automatically replaced |
//| by recommended default value, which is equal |
//| to 10 * Sqrt(Epsilon) in the current version |
//| EpsD - maximum dual error allowed in the solution, |
//| EpsP >= 0. Zero will be automatically replaced |
//| by recommended default value, which is equal |
//| to 10 * Sqrt(Epsilon) in the current version |
//| EpsGap - maximum duality gap allowed in the solution, |
//| EpsP >= 0. Zero will be automatically replaced |
//| by recommended default value, which is equal |
//| to 10 * Sqrt(Epsilon) in the current version |
//+------------------------------------------------------------------+
void CVIPMSolver::VIPMSetCond(CVIPMState &State,
double epsp,
double epsd,
double epsgap)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(epsp) && epsp>=0.0,__FUNCTION__+": EpsP is infinite or negative"))
return;
if(!CAp::Assert(MathIsValidNumber(epsd) && epsd>=0.0,__FUNCTION__+": EpsD is infinite or negative"))
return;
if(!CAp::Assert(MathIsValidNumber(epsgap) && epsgap>=0.0,__FUNCTION__+": EpsP is infinite or negative"))
return;
double sml=MathSqrt(CMath::m_machineepsilon);
State.m_epsp=CApServ::Coalesce(epsp,sml);
State.m_epsd=CApServ::Coalesce(epsd,sml);
State.m_epsgap=CApServ::Coalesce(epsgap,sml);
}
//+------------------------------------------------------------------+
//| Solve QP problem. |
//| INPUT PARAMETERS: |
//| State - solver instance |
//| DropBigBounds - If True, algorithm may drop box and linear |
//| constraints with huge bound values that destabilize|
//| algorithm. |
//| OUTPUT PARAMETERS: |
//| XS - array[N], solution |
//| LagBC - array[N], Lagrange multipliers for box constraints |
//| LagLC - array[M], Lagrange multipliers for linear |
//| constraints |
//| TerminationType - completion code, positive values for success,|
//| negative for failures(XS constrains best point |
//| found so far): |
//| * -2 the task is either unbounded or infeasible; |
//| the IPM solver has difficulty distinguishing |
//| between these two. |
//| * +1 stopping criteria are met |
//| * +7 stopping criteria are too stringent |
//| RESULT: |
//| This function ALWAYS returns something meaningful in XS, LagBC,|
//| LagLC - either solution or the best point so far, even for |
//| negative TerminationType. |
//+------------------------------------------------------------------+
void CVIPMSolver::VIPMOptimize(CVIPMState &State,
bool dropbigbounds,
CRowDouble &xs,
CRowDouble &lagbc,
CRowDouble &laglc,
int &terminationtype)
{
//--- create variables
int n=State.m_n;
int m=State.m_mdense+State.m_msparse;
int i=0;
int iteridx=0;
double mu=0;
double muaff=0;
double sigma=0;
double alphaaffp=0;
double alphaaffd=0;
double alphap=0;
double alphad=0;
int primalstagnationcnt=0;
int dualstagnationcnt=0;
double regeps=0;
double dampeps=0;
double safedampeps=0;
double modeps=0;
double maxdampeps=0;
double regfree=0;
double dampfree=0;
int droppedbounds=0;
double primalxscale=0;
double errp2=0;
double errd2=0;
double errpinf=0;
double errdinf=0;
double preverrp2=0;
double preverrd2=0;
double errgap=0;
double eprimal=0;
double edual=0;
double egap=0;
double mumin=0;
double mustop=0;
double y0nrm=0;
double bady=0;
double mxprimal=0;
double mxdeltaprimal=0;
int bestiteridx=0;
double besterr=0;
double bestegap=0;
double besteprimal=0;
double bestedual=0;
bool loadbest=false;
terminationtype=0;
State.m_dotrace=CAp::IsTraceEnabled("IPM");
State.m_dodetailedtrace=State.m_dotrace && CAp::IsTraceEnabled("IPM.DETAILED");
//--- Prepare outputs
xs=vector<double>::Zeros(n);
lagbc=vector<double>::Zeros(n);
laglc=vector<double>::Zeros(m);
//--- Some integrity checks:
//--- * we need PrimalStagnationLen<DualStagnationLen in order to be able to correctly
//--- detect infeasible instances (stagnated dual error is present in both infeasible
//--- and unbounded instances, so we should check for primal stagnation a few iters
//--- before checking for dual stagnation)
if(!CAp::Assert(m_primalstagnationlen<m_dualstagnationlen,__FUNCTION__+": critical integrity failure - incorrect configuration parameters"))
return;
//--- Trace output (if needed)
if(State.m_dotrace)
{
CAp::Trace("\n\n");
CAp::Trace("////////////////////////////////////////////////////////////////////////////////////////////////////\n");
CAp::Trace("//--- IPM SOLVER STARTED //\n");
CAp::Trace("////////////////////////////////////////////////////////////////////////////////////////////////////\n");
}
//--- Prepare regularization coefficients:
//--- * RegEps - one that is applied to initial (5N+5M)x(5N+5M) KKT system. This one has to be
//--- small because it perturbs solution returned by the algorithm. Essential in order to
//--- avoid stalling at extremely large points.
//--- * ModEps - small modification applied to LDLT decomposition in order to preserve sign
//--- of diagonal elements
//--- * DampEps - damping coefficient for damped Newton step. Comes along with SafeDampEps
//--- (threshold value when some safeguards are turned off in order to preserve convergence
//--- speed) and MaxDampEps - threshold value when we consider problem overregularized and stop.
//--- * DampFree - additional damping coefficient for free variables
regfree=MathPow(CMath::m_machineepsilon,0.75);
dampfree=0;
regeps=100*CMath::m_machineepsilon;
modeps=(100+MathSqrt(n))*CMath::m_machineepsilon;
dampeps=(100+MathSqrt(n))*CMath::m_machineepsilon;
safedampeps=MathSqrt(CMath::m_machineepsilon);
maxdampeps=MathSqrt(MathSqrt(CMath::m_machineepsilon));
//--- Set up initial State
State.m_repiterationscount=0;
State.m_repncholesky=0;
mustop=(100+MathSqrt(n))*CMath::m_machineepsilon;
mumin=0.01*mustop;
VIPMPowerUp(State,regfree);
VARSInitFrom(State.m_best,State.m_current);
VARSInitByZero(State.m_zerovars,n,m);
VARSInitByZero(State.m_deltaaff,n,m);
VARSInitByZero(State.m_deltacorr,n,m);
bestiteridx=-1;
besterr=CMath::m_maxrealnumber;
bestegap=CMath::m_maxrealnumber;
besteprimal=CMath::m_maxrealnumber;
bestedual=CMath::m_maxrealnumber;
TraceProgress(State,0.0,0.0,0.0,0.0,0.0);
y0nrm=0;
y0nrm=MathMax(y0nrm,CAblasF::RMaxAbsV(m,State.m_current.m_y));
y0nrm=MathMax(y0nrm,CAblasF::RMaxAbsV(m,State.m_current.m_v));
y0nrm=MathMax(y0nrm,CAblasF::RMaxAbsV(m,State.m_current.m_q));
y0nrm=MathMax(y0nrm,CAblasF::RMaxAbsV(n,State.m_current.m_z));
y0nrm=MathMax(y0nrm,CAblasF::RMaxAbsV(n,State.m_current.m_s));
//--- Start iteration
loadbest=true;
primalstagnationcnt=0;
dualstagnationcnt=0;
terminationtype=7;
errp2=CMath::m_maxrealnumber;
errd2=CMath::m_maxrealnumber;
for(iteridx=0; iteridx<m_maxipmits; iteridx++)
{
//--- Trace beginning
if(State.m_dotrace)
CAp::Trace(StringFormat("=== PREDICTOR-CORRECTOR STEP %2d ====================================================================\n",iteridx));
//--- Check regularization status, terminate if overregularized
if(dampeps>=maxdampeps)
{
if(State.m_dotrace)
{
CAp::Trace("> tried to increase regularization parameter,but it is too large\n");
CAp::Trace("> it is likely that stopping conditions are too stringent,stopping at the best point found so far\n");
}
terminationtype=7;
break;
}
//--- Precompute factorization
//--- NOTE: we use "m_solver" regularization coefficient at this moment
if(!VIPMPrecomputeNewtonFactorization(State,State.m_current,regeps,modeps,dampeps,dampfree))
{
//--- KKT factorization failed.
//--- Increase regularization parameter and skip this iteration.
dampeps*=10;
if(State.m_dotrace)
{
CAp::Trace("> LDLT factorization failed due to rounding errors\n");
CAp::Trace(StringFormat("> increasing damping coefficient to %.2E,skipping iteration\n",dampeps));
}
continue;
}
//--- Compute Mu
mu=VARSComputeMu(State,State.m_current);
//--- Compute affine scaling step for Mehrotra's predictor-corrector algorithm
if(!VIPMComputeStepDirection(State,State.m_current,0.0,State.m_zerovars,State.m_deltaaff,regeps,dampeps>=safedampeps))
{
//--- Affine scaling step failed due to numerical errors.
//--- Increase regularization parameter and skip this iteration.
dampeps*=10;
if(State.m_dotrace)
{
CAp::Trace("> affine scaling step failed to decrease residual due to rounding errors\n");
CAp::Trace(StringFormat("> increasing damping coefficient to %.2E,skipping iteration\n",dampeps));
}
continue;
}
VIPMComputeStepLength(State,State.m_current,State.m_deltaaff,m_steplengthdecay,alphaaffp,alphaaffd);
//--- Compute MuAff and centering parameter
VARSInitFrom(State.m_trial,State.m_current);
VarsAddStep(State.m_trial,State.m_deltaaff,alphaaffp,alphaaffd);
muaff=VARSComputeMu(State,State.m_trial);
sigma=MathMin(MathPow((muaff+mumin)/(mu+mumin),3),1.0);
//--- check
if(!CAp::Assert(MathIsValidNumber(sigma) && sigma<=1.0,__FUNCTION__+": critical integrity check failed for Sigma (infinite or greater than 1)"))
return;
//--- Compute corrector step
if(!VIPMComputeStepDirection(State,State.m_current,sigma*mu+mumin,State.m_deltaaff,State.m_deltacorr,regeps,dampeps>=safedampeps))
{
//--- Affine scaling step failed due to numerical errors.
//--- Increase regularization parameter and skip this iteration.
dampeps*=10;
if(State.m_dotrace)
{
CAp::Trace("> corrector step failed to decrease residual due to rounding errors\n");
CAp::Trace(StringFormat("> increasing damping coefficient to %.2E,skipping iteration\n",dampeps));
}
continue;
}
VIPMComputeStepLength(State,State.m_current,State.m_deltacorr,m_steplengthdecay,alphap,alphad);
if((double)(iteridx)>=m_minitersbeforesafeguards && (alphap<=m_badsteplength || alphad<=m_badsteplength))
{
//--- Affine scaling step failed due to numerical errors.
//--- Increase regularization parameter and skip this iteration.
dampeps*=10;
if(State.m_dotrace)
{
CAp::Trace("> step length is too short,suspecting rounding errors\n");
CAp::Trace(StringFormat("> increasing damping coefficient to %.2E,skipping iteration\n",dampeps));
}
continue;
}
//--- Perform a step
RunIntegrityChecks(State,State.m_current,State.m_deltacorr,alphap,alphad);
VIPMPerformStep(State,alphap,alphad);
TraceProgress(State,mu,muaff,sigma,alphap,alphad);
//--- Check for excessive bounds (one that are so large that they are both irrelevant
//--- and destabilizing due to their magnitude)
if(dropbigbounds && iteridx>=m_minitersbeforedroppingbounds)
{
//--- check
if(!CAp::Assert((10.0*m_bigconstrmag)<=(1.0/m_bigconstrxtol),__FUNCTION__+": integrity check failed (incorrect BigConstr Settings)"))
return;
droppedbounds=0;
//--- Determine variable and step scales.
//--- Both quantities are bounded from below by 1.0
mxprimal=1.0;
mxprimal=MathMax(mxprimal,CAblasF::RMaxAbsV(n,State.m_current.m_x));
mxprimal=MathMax(mxprimal,CAblasF::RMaxAbsV(n,State.m_current.m_g));
mxprimal=MathMax(mxprimal,CAblasF::RMaxAbsV(n,State.m_current.m_t));
mxprimal=MathMax(mxprimal,CAblasF::RMaxAbsV(m,State.m_current.m_w));
mxprimal=MathMax(mxprimal,CAblasF::RMaxAbsV(m,State.m_current.m_p));
mxdeltaprimal=1.0;
mxdeltaprimal=MathMax(mxdeltaprimal,alphap*CAblasF::RMaxAbsV(n,State.m_deltacorr.m_x));
mxdeltaprimal=MathMax(mxdeltaprimal,alphap*CAblasF::RMaxAbsV(n,State.m_deltacorr.m_g));
mxdeltaprimal=MathMax(mxdeltaprimal,alphap*CAblasF::RMaxAbsV(n,State.m_deltacorr.m_t));
mxdeltaprimal=MathMax(mxdeltaprimal,alphap*CAblasF::RMaxAbsV(m,State.m_deltacorr.m_w));
mxdeltaprimal=MathMax(mxdeltaprimal,alphap*CAblasF::RMaxAbsV(m,State.m_deltacorr.m_p));
//--- If changes in primal variables are small enough, try dropping too large bounds
if(mxdeltaprimal<(mxprimal*m_bigconstrxtol))
{
//--- Drop irrelevant box constraints
primalxscale=1.0;
primalxscale=MathMax(primalxscale,CAblasF::RMaxAbsV(n,State.m_current.m_x));
for(i=0; i<n; i++)
{
if((State.m_HasBndL[i] && State.m_hasgz[i]) && MathAbs(State.m_bndl[i])>(m_bigconstrmag*primalxscale))
{
State.m_hasgz[i]=false;
State.m_current.m_g.Set(i,0);
State.m_current.m_z.Set(i,0);
State.m_cntgz--;
droppedbounds++;
}
if((State.m_HasBndU[i] && State.m_hasts[i]) && MathAbs(State.m_bndu[i])>(m_bigconstrmag*primalxscale))
{
State.m_hasts[i]=false;
State.m_current.m_t.Set(i,0);
State.m_current.m_s.Set(i,0);
State.m_cntts--;
droppedbounds++;
}
}
//--- Drop irrelevant linear constraints. Due to specifics of the m_solver
//--- we can drop only right part part of b<=Ax<=b+r.
//--- We can't drop b<=A from b<=A<=b+r because it impossible with our choice of
//--- slack variables. Usually we do not need to do so because we reorder constraints
//--- during initialization in such a way that |b+r|>|b| and because typical
//--- applications do not have excessively large lower AND upper bound (user may
//--- specify large value for 'absent' bound, but usually he does not mark both bounds as absent).
MultiplyGEAX(State,1.0,State.m_current.m_x,0,0.0,State.m_tmpax,0);
primalxscale=1.0;
primalxscale=MathMax(primalxscale,CAblasF::RMaxAbsV(n,State.m_current.m_x));
primalxscale=MathMax(primalxscale,CAblasF::RMaxAbsV(m,State.m_tmpax));
for(i=0; i<m; i++)
{
if(State.m_hasr[i] && State.m_haspq[i] && MathAbs(State.m_b[i]+State.m_r[i])>(m_bigconstrmag*primalxscale) && MathAbs(State.m_b[i])<(m_bigconstrmag*primalxscale))
{
//--- check
if(!CAp::Assert(State.m_haswv[i] && State.m_haspq[i],__FUNCTION__+": unexpected integrity check failure (4y64)"))
return;
State.m_haspq[i]=false;
State.m_current.m_p.Set(i,0);
State.m_current.m_q.Set(i,0);
State.m_cntpq--;
droppedbounds++;
}
}
//--- Trace output
if(droppedbounds>0)
if(State.m_dotrace)
CAp::Trace(StringFormat("[NOTICE] detected %d irrelevant constraints with huge bounds,X converged to values well below them,dropping...\n",droppedbounds));
}
}
//--- Check stopping criteria
//--- * primal and dual stagnation are checked only when following criteria are met:
//--- 1) Mu is smaller than 1 (we already converged close enough)
//--- 2) we performed more than MinItersBeforeStagnation iterations
preverrp2=errp2;
preverrd2=errd2;
ComputeErrors(State,errp2,errd2,errpinf,errdinf,errgap);
mu=VARSComputeMu(State,State.m_current);
egap=errgap;
eprimal=errpinf;
edual=errdinf;
if(MathMax(egap,MathMax(eprimal,edual))<besterr)
{
//--- Save best point found so far
VARSInitFrom(State.m_best,State.m_current);
bestiteridx=iteridx;
besterr=MathMax(egap,MathMax(eprimal,edual));
bestegap=egap;
besteprimal=eprimal;
bestedual=edual;
}
if(bestiteridx>0 && iteridx>bestiteridx+m_minitersbeforeeworststagnation)
{
if(State.m_dotrace)
CAp::Trace(StringFormat("> worst of primal/dual/gap errors stagnated for %d its,stopping at the best point found so far\n",m_minitersbeforeeworststagnation));
break;
}
if(egap<=State.m_epsgap && errp2>=(m_stagnationdelta*preverrp2) && errpinf>=m_primalinfeasible1 && iteridx>=m_minitersbeforestagnation)
{
primalstagnationcnt++;
if(primalstagnationcnt>=m_primalstagnationlen)
{
if(State.m_dotrace)
CAp::Trace(StringFormat("> primal error stagnated for %d its,stopping at the best point found so far\n",m_primalstagnationlen));
break;
}
}
else
primalstagnationcnt=0;
if(egap<=State.m_epsgap && errd2>=(m_stagnationdelta*preverrd2) && errdinf>=m_dualinfeasible1 && iteridx>=m_minitersbeforestagnation)
{
dualstagnationcnt++;
if(dualstagnationcnt>=m_dualstagnationlen)
{
if(State.m_dotrace)
CAp::Trace(StringFormat("> dual error stagnated for %d its,stopping at the best point found so far\n",m_dualstagnationlen));
break;
}
}
else
dualstagnationcnt=0;
if(mu<=mustop && iteridx>=m_itersfortoostringentcond)
{
if(State.m_dotrace)
CAp::Trace("> stopping conditions are too stringent,stopping at the best point found so far\n");
terminationtype=7;
break;
}
if(egap<=State.m_epsgap && eprimal<=State.m_epsp && edual<=State.m_epsd)
{
if(State.m_dotrace)
CAp::Trace("> stopping criteria are met\n");
terminationtype=1;
loadbest=false;
break;
}
bady=m_bigy;
bady=MathMax(bady,m_ygrowth*y0nrm);
bady=MathMax(bady,m_ygrowth*CAblasF::RMaxAbsV(n,State.m_current.m_x));
bady=MathMax(bady,m_ygrowth*CAblasF::RMaxAbsV(n,State.m_current.m_g));
bady=MathMax(bady,m_ygrowth*CAblasF::RMaxAbsV(n,State.m_current.m_t));
bady=MathMax(bady,m_ygrowth*CAblasF::RMaxAbsV(m,State.m_current.m_w));
bady=MathMax(bady,m_ygrowth*CAblasF::RMaxAbsV(m,State.m_current.m_p));
if(CAblasF::RMaxAbsV(m,State.m_current.m_y)>=bady && iteridx>=m_minitersbeforeinfeasible)
{
if(State.m_dotrace)
CAp::Trace(StringFormat("> |Y| increased beyond %.1E,stopping at the best point found so far\n",bady));
break;
}
}
//--- Load best point, perform some checks
if(loadbest)
{
//--- Load best point
//--- NOTE: TouchReal() is used to avoid spurious compiler warnings about 'set but unused'
if(State.m_dotrace)
CAp::Trace(StringFormat("> the best point so far is one from iteration %d\n",bestiteridx));
VARSInitFrom(State.m_current,State.m_best);
//--- If no error flags were set yet, check solution quality
bady=m_bigy;
bady=MathMax(bady,m_ygrowth*y0nrm);
bady=MathMax(bady,m_ygrowth*CAblasF::RMaxAbsV(n,State.m_current.m_x));
bady=MathMax(bady,m_ygrowth*CAblasF::RMaxAbsV(n,State.m_current.m_g));
bady=MathMax(bady,m_ygrowth*CAblasF::RMaxAbsV(n,State.m_current.m_t));
bady=MathMax(bady,m_ygrowth*CAblasF::RMaxAbsV(m,State.m_current.m_w));
bady=MathMax(bady,m_ygrowth*CAblasF::RMaxAbsV(m,State.m_current.m_p));
if(terminationtype>0 && CAblasF::RMaxAbsV(m,State.m_current.m_y)>=bady)
{
terminationtype=-2;
if(State.m_dotrace)
CAp::Trace(StringFormat("> |Y| increased beyond %.1E,declaring infeasibility/unboundedness\n",bady));
}
if(terminationtype>0 && besteprimal>=m_primalinfeasible1)
{
terminationtype=-2;
if(State.m_dotrace)
CAp::Trace("> primal error at the best point is too high,declaring infeasibility/unboundedness\n");
}
if(terminationtype>0 && bestedual>=m_dualinfeasible1)
{
terminationtype=-2;
if(State.m_dotrace)
CAp::Trace("> dual error at the best point is too high,declaring infeasibility/unboundedness\n");
}
}
//--- Output
MultiplyHX(State,State.m_current.m_x,State.m_tmp0);
CAblasF::RAddV(n,1.0,State.m_c,State.m_tmp0);
MultiplyGEATX(State,-1.0,State.m_current.m_y,0,1.0,State.m_tmp0,0);
for(i=0; i<n; i++)
{
if(State.m_isfrozen[i])
{
//--- I-th variable is frozen, use its frozen value.
//--- By the definition, I-th Lagrangian multiplier is an I-th component of Lagrangian gradient
xs.Set(i,State.m_current.m_x[i]);
lagbc.Set(i,-State.m_tmp0[i]);
}
else
{
xs.Set(i,State.m_current.m_x[i]);
lagbc.Set(i,0.0);
if(State.m_hasgz[i])
lagbc.Add(i,- State.m_current.m_z[i]);
if(State.m_hasts[i])
lagbc.Add(i,State.m_current.m_s[i]);
}
}
laglc=State.m_current.m_y*(-1.0)+0;
//--- Unscale point and Lagrange multipliers
CLPQPServ::UnscaleUnshiftPointBC(State.m_scl,State.m_xorigin,State.m_rawbndl,State.m_rawbndu,State.m_bndl,State.m_bndu,State.m_HasBndL,State.m_HasBndU,xs,n);
lagbc*=State.m_scl.Pow(-1.0)*State.m_targetscale;
for(i=0; i<m; i++)
laglc.Mul(i,State.m_targetscale/CApServ::Coalesce(State.m_ascales[i],1.0));
}
//+------------------------------------------------------------------+
//| Allocates place for variables of IPM and fills by zeros. |
//+------------------------------------------------------------------+
void CVIPMSolver::VARSInitByZero(CVIPMVars &vstate,
int n,
int m)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
if(!CAp::Assert(m>=0,__FUNCTION__+": M<0"))
return;
vstate.m_n=n;
vstate.m_m=m;
vstate.m_x=vector<double>::Zeros(n);
vstate.m_g=vector<double>::Zeros(n);
vstate.m_t=vector<double>::Zeros(n);
vstate.m_z=vector<double>::Zeros(n);
vstate.m_s=vector<double>::Zeros(n);
vstate.m_y=vector<double>::Zeros(m);
vstate.m_w=vector<double>::Zeros(m);
vstate.m_p=vector<double>::Zeros(m);
vstate.m_v=vector<double>::Zeros(m);
vstate.m_q=vector<double>::Zeros(m);
}
//+------------------------------------------------------------------+
//| Allocates place for variables of IPM and fills them by values of |
//| the source |
//+------------------------------------------------------------------+
void CVIPMSolver::VARSInitFrom(CVIPMVars &vstate,
CVIPMVars &vsrc)
{
//--- check
if(!CAp::Assert(vsrc.m_n>=1,__FUNCTION__+": N<1"))
return;
if(!CAp::Assert(vsrc.m_m>=0,__FUNCTION__+": M<0"))
return;
//--- copy
vstate=vsrc;
}
//+------------------------------------------------------------------+
//| Adds to variables direction vector times step length. Different |
//| lengths are used for primal and dual steps. |
//+------------------------------------------------------------------+
void CVIPMSolver::VarsAddStep(CVIPMVars &vstate,
CVIPMVars &vdir,
double stpp,
double stpd)
{
//--- create variables
int n=vstate.m_n;
int m=vstate.m_m;
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
if(!CAp::Assert(m>=0,__FUNCTION__+": M<0"))
return;
if(!CAp::Assert(n==vdir.m_n,__FUNCTION__+": sizes mismatch"))
return;
if(!CAp::Assert(m==vdir.m_m,__FUNCTION__+": sizes mismatch"))
return;
vstate.m_x+=vdir.m_x*stpp+0;
vstate.m_g+=vdir.m_g*stpp+0;
vstate.m_t+=vdir.m_t*stpp+0;
vstate.m_z+=vdir.m_z*stpd+0;
vstate.m_s+=vdir.m_s*stpd+0;
vstate.m_w+=vdir.m_w*stpp+0;
vstate.m_p+=vdir.m_p*stpp+0;
vstate.m_y+=vdir.m_y*stpd+0;
vstate.m_v+=vdir.m_v*stpd+0;
vstate.m_q+=vdir.m_q*stpd+0;
}
//+------------------------------------------------------------------+
//| Computes complementarity gap |
//+------------------------------------------------------------------+
double CVIPMSolver::VarsComputeComplementarityGap(CVIPMVars &vstate)
{
//--- create variables
double result=0;
int n=vstate.m_n;
int m=vstate.m_m;
result=CAblasF::RDotV(n,vstate.m_z,vstate.m_g)+CAblasF::RDotV(n,vstate.m_s,vstate.m_t);
result+=CAblasF::RDotV(m,vstate.m_v,vstate.m_w)+CAblasF::RDotV(m,vstate.m_p,vstate.m_q);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Computes empirical value of the barrier parameter Mu |
//+------------------------------------------------------------------+
double CVIPMSolver::VARSComputeMu(CVIPMState &State,CVIPMVars &vstate)
{
//--- create variable
double result=0;
result=CAblasF::RDotV(vstate.m_n,vstate.m_z,vstate.m_g)+CAblasF::RDotV(vstate.m_n,vstate.m_s,vstate.m_t);
result+=CAblasF::RDotV(vstate.m_m,vstate.m_v,vstate.m_w)+CAblasF::RDotV(vstate.m_m,vstate.m_p,vstate.m_q);
result/=CApServ::Coalesce(State.m_cntgz+State.m_cntts+State.m_cntwv+State.m_cntpq,1);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Initializes reduced sparse system. |
//| Works only for sparse IPM. |
//+------------------------------------------------------------------+
void CVIPMSolver::ReducedSystemInit(CVIPMReducedSparseSystem &s,
CVIPMState &solver)
{
//--- create variables
int ntotal=0;
int nnzmax=0;
int factldlt=0;
int permpriorityamd=0;
int offs=0;
int rowoffs=0;
int i=0;
int j=0;
int k=0;
int k0=0;
int k1=0;
int sumdeg=0;
int colthreshold=0;
int rowthreshold=0;
int eligiblecols=0;
int eligiblerows=0;
//--- check
if(!CAp::Assert(solver.m_factorizationtype==1,__FUNCTION__+": unexpected factorization type"))
return;
if(!CAp::Assert(solver.m_hkind==1,__FUNCTION__+": unexpected HKind"))
return;
ntotal=solver.m_n+solver.m_mdense+solver.m_msparse;
s.m_ntotal=ntotal;
s.m_effectivediag.Resize(ntotal);
//--- Determine maximum amount of memory required to store sparse matrices
nnzmax=solver.m_sparseh.m_RIdx[solver.m_n];
if(solver.m_msparse>0)
nnzmax+=solver.m_sparseafull.m_RIdx[solver.m_msparse];
if(solver.m_mdense>0)
nnzmax+=solver.m_n*solver.m_mdense;
nnzmax+=ntotal;
//--- Prepare strictly lower triangle of template KKT matrix (KKT system without D and E
//--- terms being added to diagonals)
s.m_rawsystem.m_M=ntotal;
s.m_rawsystem.m_N=ntotal;
s.m_rawsystem.m_Idx.Resize(nnzmax);
s.m_rawsystem.m_Vals.Resize(nnzmax);
s.m_rawsystem.m_RIdx.Resize(ntotal+1);
s.m_rawsystem.m_RIdx.Set(0,0);
//if(solver.m_dodetailedtrace)
//--- CSparse::SparseTrace(s.m_rawsystem);
offs=0;
rowoffs=0;
sumdeg=0;
CAblasF::ISetAllocV(solver.m_n,0,s.m_coldegrees);
CAblasF::ISetAllocV(solver.m_msparse+solver.m_mdense,0,s.m_rowdegrees);
CAblasF::BSetAllocV(solver.m_n,true,s.m_isdiagonal);
for(i=0; i<solver.m_n; i++)
{
//--- check
if(!CAp::Assert(solver.m_sparseh.m_DIdx[i]+1==solver.m_sparseh.m_UIdx[i],__FUNCTION__+": critical integrity check failed for diagonal of H"))
return;
if(!solver.m_isfrozen[i])
{
//--- Entire row is not frozen, but some of its entries can be.
//--- Output non-frozen offdiagonal entries.
k0=solver.m_sparseh.m_RIdx[i];
k1=solver.m_sparseh.m_DIdx[i];
for(k=k0; k<k1; k++)
{
j=solver.m_sparseh.m_Idx[k];
if(!solver.m_isfrozen[j])
{
s.m_rawsystem.m_Idx.Set(offs,j);
s.m_rawsystem.m_Vals.Set(offs,-solver.m_sparseh.m_Vals[k]);
s.m_isdiagonal[i]=false;
s.m_isdiagonal[j]=false;
offs++;
}
}
//--- Output diagonal entry (it is always not frozen)
s.m_rawsystem.m_Idx.Set(offs,i);
s.m_rawsystem.m_Vals.Set(offs,-solver.m_sparseh.m_Vals[solver.m_sparseh.m_DIdx[i]]);
offs++;
}
else
{
//--- Entire row is frozen, output just -1
s.m_rawsystem.m_Idx.Set(offs,i);
s.m_rawsystem.m_Vals.Set(offs,-1.0);
offs++;
}
rowoffs++;
s.m_rawsystem.m_RIdx.Set(rowoffs,offs);
//if(solver.m_dodetailedtrace)
//--- CSparse::SparseTrace(s.m_rawsystem);
}
for(i=0; i<solver.m_msparse; i++)
{
k0=solver.m_sparseafull.m_RIdx[i];
k1=solver.m_sparseafull.m_RIdx[i+1];
for(k=k0; k<k1; k++)
{
j=solver.m_sparseafull.m_Idx[k];
if(!solver.m_isfrozen[j])
{
s.m_rawsystem.m_Idx.Set(offs,j);
s.m_rawsystem.m_Vals.Set(offs,solver.m_sparseafull.m_Vals[k]);
s.m_rowdegrees.Add(i,1);
s.m_coldegrees.Add(j,1);
sumdeg++;
offs++;
}
}
s.m_rawsystem.m_Idx.Set(offs,rowoffs);
s.m_rawsystem.m_Vals.Set(offs,0.0);
offs++;
rowoffs++;
s.m_rawsystem.m_RIdx.Set(rowoffs,offs);
//if(solver.m_dodetailedtrace)
//--- CSparse::SparseTrace(s.m_rawsystem);
}
for(i=0; i<solver.m_mdense; i++)
{
for(k=0; k<solver.m_n; k++)
{
if(solver.m_denseafull.Get(i,k)!=0.0 && !solver.m_isfrozen[k])
{
s.m_rawsystem.m_Idx.Set(offs,k);
s.m_rawsystem.m_Vals.Set(offs,solver.m_denseafull.Get(i,k));
s.m_rowdegrees.Add(solver.m_msparse+i,1);
s.m_coldegrees.Add(k,1);
sumdeg++;
offs++;
}
}
s.m_rawsystem.m_Idx.Set(offs,rowoffs);
s.m_rawsystem.m_Vals.Set(offs,0.0);
offs++;
rowoffs++;
s.m_rawsystem.m_RIdx.Set(rowoffs,offs);
//if(solver.m_dodetailedtrace)
//--- CSparse::SparseTrace(s.m_rawsystem);
}
//--- check
if(!CAp::Assert(rowoffs==ntotal,__FUNCTION__+": critical integrity check failed"))
return;
if(!CAp::Assert(offs<=nnzmax,__FUNCTION__+": critical integrity check failed"))
return;
CSparse::SparseCreateCRSInplace(s.m_rawsystem);
//if(solver.m_dodetailedtrace)
//--- CSparse::SparseTrace(s.m_rawsystem);
//--- Prepare reordering
colthreshold=(int)MathRound(m_muquasidense*sumdeg/solver.m_n)+2;
rowthreshold=(int)MathRound(m_muquasidense*sumdeg/(solver.m_msparse+solver.m_mdense+1))+2;
eligiblecols=0;
eligiblerows=0;
CAblasF::ISetAllocV(ntotal,0,s.m_priorities);
for(i=0; i<solver.m_n; i++)
{
if(s.m_isdiagonal[i] && s.m_coldegrees[i]<=colthreshold)
eligiblecols++;
}
for(i=0; i<(solver.m_mdense+solver.m_msparse); i++)
{
if(s.m_rowdegrees[i]<=rowthreshold)
eligiblerows++;
}
if(solver.m_dotrace)
CAp::Trace("> initializing KKT system; no priority ordering being applied\n");
//--- Perform factorization analysis using sparsity pattern (but not numerical values)
factldlt=1;
permpriorityamd=3;
if(!CSpChol::SpSymmAnalyze(s.m_rawsystem,s.m_priorities,factldlt,permpriorityamd,s.m_analysis))
CAp::Assert(false,__FUNCTION__+": critical integrity check failed,symbolically degenerate KKT system encountered");
}
//+------------------------------------------------------------------+
//| Computes factorization of A + D, where A is internally stored |
//| KKT matrix and D is user-supplied diagonal term.The factorization|
//| is stored internally and should never be accessed directly. |
//| ModEps and BadChol are user supplied tolerances for modified |
//| Cholesky / LDLT. |
//| Returns True on success, False on LDLT failure. |
//| On success outputs diagonal reproduction error ErrSq, and sum of |
//| squared diagonal elements SumSq |
//+------------------------------------------------------------------+
bool CVIPMSolver::ReducedSystemFactorizeWithAddEnd(CVIPMReducedSparseSystem &s,
CRowDouble &d,
double modeps,
double badchol,
double &sumsq,
double &errsq)
{
//--- create variables
bool result=true;
int ntotal=s.m_ntotal;
sumsq=0;
errsq=0;
for(int i=0; i<ntotal; i++)
s.m_effectivediag.Set(i,s.m_rawsystem.m_Vals[s.m_rawsystem.m_DIdx[i]]+d[i]);
CSpChol::SpSymmReloadDiagonal(s.m_analysis,s.m_effectivediag);
CSpChol::SpSymmSetModificationStrategy(s.m_analysis,1,modeps,badchol,0.0,0.0);
if(CSpChol::SpSymmFactorize(s.m_analysis))
CSpChol::SpSymmDiagErr(s.m_analysis,sumsq,errsq);
else
{
sumsq=0;
errsq=0;
result=false;
}
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Solve reduced KKT system, replacing right part by its solution. |
//+------------------------------------------------------------------+
void CVIPMSolver::ReducedSystemSolve(CVIPMReducedSparseSystem &s,
CRowDouble &b)
{
CSpChol::SpSymmSolve(s.m_analysis,b);
}
//+------------------------------------------------------------------+
//| Initializes QP-IPM State and prepares it to receive quadratic / |
//| linear terms and constraints. |
//| The solver is configured to work internally with factorization |
//| FType |
//| INPUT PARAMETERS: |
//| State - solver State to be configured; previously |
//| allocated memory is reused as much as possible |
//| S - scale vector, array[N]: |
//| * I-th element contains scale of I-th variable, |
//| * S[I] > 0 |
//| XOrigin - origin term, array[N]. Can be zero. The solver |
//| solves problem of the form |
//| > |
//| > min(0.5 * (x-x_origin)'*A*(x-x_origin)+b' * (x-x_origin)) |
//| > |
//| The terms A and b (as well as constraints) will be specified |
//| later with separate calls. |
//| FType - factorization type: |
//| * 0 for dense NxN factorization (normal equations) |
//| * 1 for sparse(N + M) |
//| x(N + M) factorization |
//+------------------------------------------------------------------+
void CVIPMSolver::VIPMInit(CVIPMState &State,
CRowDouble &s,
CRowDouble &xorigin,
int n,
int nmain,
int ftype)
{
//--- create variables
int nslack=n-nmain;
int i=0;
int j=0;
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(s,n),__FUNCTION__+": S contains infinite or NaN elements"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(xorigin,n),__FUNCTION__+": XOrigin contains infinite or NaN elements"))
return;
if(!CAp::Assert(ftype==0 || ftype==1,__FUNCTION__+": unexpected FType"))
return;
if(!CAp::Assert(nmain>=1,__FUNCTION__+": NMain<1"))
return;
if(!CAp::Assert(nmain<=n,__FUNCTION__+": NMain>N"))
return;
//--- Problem metrics, Settings and type
State.m_n=n;
State.m_nmain=nmain;
State.m_islinear=true;
State.m_factorizationtype=ftype;
State.m_factorizationpresent=false;
State.m_factorizationpoweredup=false;
State.m_slacksforequalityconstraints=true;
VIPMSetCond(State,0.0,0.0,0.0);
//--- Reports
State.m_repiterationscount=0;
State.m_repncholesky=0;
//--- Trace
State.m_dotrace=false;
State.m_dodetailedtrace=false;
//--- Scale and origin
State.m_scl.Resize(n);
State.m_invscl.Resize(n);
State.m_xorigin.Resize(n);
for(i=0; i<n; i++)
{
//--- check
if(!CAp::Assert(s[i]>0.0,__FUNCTION__+": S[i] is non-positive"))
return;
State.m_scl.Set(i,s[i]);
State.m_invscl.Set(i,1/s[i]);
State.m_xorigin.Set(i,xorigin[i]);
}
State.m_targetscale=1.0;
//--- Linear and quadratic terms - default value
State.m_c=vector<double>::Zeros(n);
State.m_hkind=-1;
switch(ftype)
{
case 0:
//--- Dense quadratic term
State.m_denseh.Resize(nmain,nmain);
for(i=0; i<nmain; i++)
for(j=0; j<=i; j++)
State.m_denseh.Set(i,j,0);
State.m_hkind=0;
State.m_isdiagonalh=false;
break;
case 1:
//--- Sparse quadratic term
State.m_sparseh.m_MatrixType=1;
State.m_sparseh.m_M=n;
State.m_sparseh.m_N=n;
State.m_sparseh.m_NInitialized=n;
State.m_sparseh.m_Idx.Resize(n);
State.m_sparseh.m_Vals=vector<double>::Zeros(n);
State.m_sparseh.m_RIdx.Resize(n+1);
for(i=0; i<n; i++)
{
State.m_sparseh.m_Idx.Set(i,i);
State.m_sparseh.m_RIdx.Set(i,i);
}
State.m_sparseh.m_RIdx.Set(n,n);
CSparse::SparseCreateCRSInplace(State.m_sparseh);
State.m_hkind=1;
State.m_isdiagonalh=true;
break;
}
//--- check
if(!CAp::Assert(State.m_hkind>=0,__FUNCTION__+": integrity check failed"))
return;
//--- Box constraints - default values
State.m_bndl=vector<double>::Full(n,AL_NEGINF);
State.m_bndu=vector<double>::Full(n,AL_POSINF);
CApServ::BVectorSetLengthAtLeast(State.m_HasBndL,n);
CApServ::BVectorSetLengthAtLeast(State.m_HasBndU,n);
ArrayInitialize(State.m_HasBndL,false);
ArrayInitialize(State.m_HasBndU,false);
//--- Linear constraints - empty
State.m_mdense=0;
State.m_msparse=0;
State.m_combinedaslack.m_M=0;
State.m_combinedaslack.m_N=nslack;
State.m_sparseamain.m_M=0;
State.m_sparseamain.m_N=nmain;
CSparse::SparseCreateCRSInplace(State.m_sparseamain);
CSparse::SparseCreateCRSInplace(State.m_combinedaslack);
}
//+------------------------------------------------------------------+
//| Computes target function 0.5 * x'*H*x+c'*x |
//+------------------------------------------------------------------+
double CVIPMSolver::VIPMTarget(CVIPMState &State,
CRowDouble &x)
{
//--- create variables
double result=0;
int n=State.m_n;
int nmain=State.m_nmain;
int i=0;
int j=0;
int k=0;
int j0=0;
int j1=0;
double v=0;
//--- check
if(!CAp::Assert(State.m_hkind==0 || State.m_hkind==1,__FUNCTION__+": unexpected HKind"))
return(result);
switch(State.m_hkind)
{
//--- Dense
case 0:
for(i=0; i<nmain; i++)
{
for(j=0; j<=i-1; j++)
result+=x[i]*State.m_denseh.Get(i,j)*x[j];
result +=+0.5*CMath::Sqr(x[i])*State.m_denseh.Get(i,i);
}
for(i=0; i<n; i++)
result+=State.m_c[i]*x[i];
break;
//--- Sparse
case 1:
for(i=0; i<n; i++)
{
result+=State.m_c[i]*x[i];
j0=State.m_sparseh.m_RIdx[i];
j1=State.m_sparseh.m_DIdx[i]-1;
for(k=j0; k<=j1; k++)
{
v=State.m_sparseh.m_Vals[k];
j=State.m_sparseh.m_Idx[k];
result+=v*x[i]*x[j];
}
//--- check
if(!CAp::Assert(State.m_sparseh.m_UIdx[i]!=State.m_sparseh.m_DIdx[i],__FUNCTION__+": sparse diagonal not found"))
return(0);
v=State.m_sparseh.m_Vals[State.m_sparseh.m_DIdx[i]];
result+=0.5*v*CMath::Sqr(x[i]);
}
break;
}
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Computes Y := alpha * A * x + beta * Y |
//| where A is constraint matrix, |
//| X is user - specified source, |
//| Y is target. |
//| Beta can be zero(in this case original contents of Y is ignored).|
//| If Beta is nonzero, we expect that Y contains preallocated array.|
//+------------------------------------------------------------------+
void CVIPMSolver::MultiplyGEAX(CVIPMState &State,
double alpha,
CRowDouble &x,
int offsx,
double beta,
CRowDouble &y,
int offsax)
{
//--- create variables
int n=State.m_n;
int m=State.m_mdense+State.m_msparse;
int mdense=State.m_mdense;
int msparse=State.m_msparse;
if(beta==0.0)
y.Resize(offsax+m);
else
{
//--- check
if(!CAp::Assert(y.Size()>=offsax+m,__FUNCTION__+": Y is too short"))
return;
}
if(msparse>0)
CSparse::SparseGemV(State.m_sparseafull,alpha,0,x,offsx,beta,y,offsax);
if(mdense>0)
CAblas::RMatrixGemVect(mdense,n,alpha,State.m_denseafull,0,0,0,x,offsx,beta,y,offsax+msparse);
}
//+------------------------------------------------------------------+
//| Computes Y := alpha * A'*x + beta*Y |
//| where A is constraint matrix, |
//| X is user - specified source, |
//| Y is target. |
//| Beta can be zero, in this case we automatically reallocate target|
//| if it is too short (but do NOT reallocate it if its size is large|
//| enough). If Beta is nonzero, we expect that Y contains |
//| preallocated array. |
//+------------------------------------------------------------------+
void CVIPMSolver::MultiplyGEATX(CVIPMState &State,
double alpha,
CRowDouble &x,
int offsx,
double beta,
CRowDouble &y,
int offsy)
{
//--- create variables
int n=State.m_n;
int mdense=State.m_mdense;
int msparse=State.m_msparse;
if(beta==0.0)
{
y.Resize(offsy+n);
CAblasF::RSetVX(n,0.0,y,offsy);
}
else
{
//--- check
if(!CAp::Assert(y.Size()>=offsy+n,__FUNCTION__+": Y is too short"))
return;
CAblasF::RMulVX(n,beta,y,offsy);
}
if(msparse>0)
CSparse::SparseGemV(State.m_sparseafull,alpha,1,x,offsx,1.0,y,offsy);
if(mdense>0)
CAblas::RMatrixGemVect(n,mdense,alpha,State.m_denseafull,0,0,1,x,offsx+msparse,1.0,y,offsy);
}
//+------------------------------------------------------------------+
//| Computes H*x, does not support advanced functionality of GEAX / |
//| GEATX |
//+------------------------------------------------------------------+
void CVIPMSolver::MultiplyHX(CVIPMState &State,
CRowDouble &x,
CRowDouble &hx)
{
//--- create variables
int n=State.m_n;
int nmain=State.m_nmain;
int i=0;
hx.Resize(n);
//--- check
if(!CAp::Assert(State.m_hkind==0 || State.m_hkind==1,__FUNCTION__+": unexpected HKind"))
return;
switch(State.m_hkind)
{
case 0:
CAblas::RMatrixSymVect(nmain,1.0,State.m_denseh,0,0,false,x,0,0.0,hx,0);
for(i=nmain; i<n; i++)
hx.Set(i,0);
for(i=0; i<n; i++)
hx.Add(i,x[i]*State.m_diagr[i]);
break;
case 1:
//--- check
if(!CAp::Assert(State.m_sparseh.m_N==n && State.m_sparseh.m_M==n,__FUNCTION__+": sparse H has incorrect size"))
return;
if(State.m_isdiagonalh)
{
//--- H is known to be diagonal, much faster code can be used
CAblasF::RCopyV(n,State.m_diagr,hx);
CAblasF::RAddV(n,1.0,State.m_sparseh.m_Vals,hx);
CAblasF::RMergeMulV(n,x,hx);
}
else
{
//--- H is a general sparse matrix, use generic sparse matrix-vector multiply
CSparse::SparseSMV(State.m_sparseh,false,x,hx);
for(i=0; i<n; i++)
hx.Add(i,x[i]*State.m_diagr[i]);
}
break;
}
}
//+------------------------------------------------------------------+
//| Computes products H*x, A*x, A^T*y |
//+------------------------------------------------------------------+
void CVIPMSolver::VIPMMultiply(CVIPMState &State,
CRowDouble &x,
CRowDouble &y,
CRowDouble &hx,
CRowDouble &ax,
CRowDouble &aty)
{
MultiplyGEAX(State,1.0,x,0,0.0,ax,0);
MultiplyGEATX(State,1.0,y,0,0.0,aty,0);
MultiplyHX(State,x,hx);
}
//+------------------------------------------------------------------+
//| This function "powers up" factorization, i.e. prepares some |
//| important temporaries. It should be called once prior to the |
//| first call to VIPMInitialPoint() or VIPMFactorize(). |
//| Parameters: |
//| RegFree - regularization for free variables; good value |
//| sqrt(MachineEpsilon) |
//+------------------------------------------------------------------+
void CVIPMSolver::VIPMPowerUp(CVIPMState &State,
double regfree)
{
//--- create variables
int n=State.m_n;
int m=State.m_mdense+State.m_msparse;
int i=0;
double v=0;
double vrhs=0;
double priorcoeff=0;
double initprimslack=0;
double initdualslack=0;
double maxinitialnoncentrality=0;
double maxinitialimbalance=0;
double mu0=0;
double mumin=0;
bool success=false;
//--- check
if(!CAp::Assert(State.m_factorizationtype==0 || State.m_factorizationtype==1,__FUNCTION__+": unexpected factorization type"))
return;
//--- check
if(!CAp::Assert(!State.m_factorizationpoweredup,__FUNCTION__+": repeated call"))
return;
maxinitialnoncentrality=1.0E-6;
maxinitialimbalance=1.0E-6;
//--- Set up information about presence of slack variables.
//--- Decide which components of X should be frozen.
//--- Compute diagonal regularization matrix R.
CAblasF::BCopyAllocV(n,State.m_HasBndL,State.m_hasgz);
CAblasF::BCopyAllocV(n,State.m_HasBndU,State.m_hasts);
CAblasF::BSetAllocV(n,false,State.m_isfrozen);
CAblasF::RSetAllocV(n,0.0,State.m_diagr);
VARSInitByZero(State.m_current,n,m);
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i] && State.m_HasBndU[i] && State.m_bndl[i]==State.m_bndu[i])
{
State.m_isfrozen[i]=true;
State.m_hasgz[i]=false;
State.m_hasts[i]=false;
State.m_current.m_x.Set(i,State.m_bndl[i]);
}
if(!State.m_HasBndL[i] && !State.m_HasBndU[i])
State.m_diagr.Set(i,regfree);
}
CAblasF::BAllocV(m,State.m_haspq);
CAblasF::BAllocV(m,State.m_haswv);
for(i=0; i<m; i++)
{
State.m_haswv[i]=(State.m_slacksforequalityconstraints || !State.m_hasr[i] || State.m_r[i]>0.0);
State.m_haspq[i]=(State.m_hasr[i] && State.m_haswv[i]);
}
State.m_cntgz=0;
State.m_cntts=0;
State.m_cntwv=0;
State.m_cntpq=0;
for(i=0; i<n; i++)
{
if(State.m_hasgz[i])
State.m_cntgz++;
if(State.m_hasts[i])
State.m_cntts++;
}
for(i=0; i<m; i++)
{
if(State.m_haswv[i])
State.m_cntwv++;
if(State.m_haspq[i])
State.m_cntpq++;
}
//--- Special initialization for sparse version
if(State.m_factorizationtype==1)
ReducedSystemInit(State.m_reducedsparsesystem,State);
State.m_factorizationpoweredup=true;
//--- Set up initial values of primal and dual variables X and Y by solving
//--- modified KKT system which tries to enforce linear constraints (ignoring
//--- box constraints for a while) subject to minimization of additional prior
//--- term which moves solution towards some interior point.
//--- Here we expect that State.Current.X contains zeros in non-fixed variables
//--- and their fixed values for fixed ones.
priorcoeff=1.0;
success=VIPMFactorize(State,0.0,State.m_diagddr,0.0,State.m_diagder,priorcoeff,priorcoeff,CMath::m_machineepsilon,CMath::m_machineepsilon);
//--- check
if(!CAp::Assert(success,__FUNCTION__+": impossible failure of LDLT factorization"))
return;
MultiplyHX(State,State.m_current.m_x,State.m_tmp0);
MultiplyGEAX(State,1.0,State.m_current.m_x,0,0.0,State.m_tmp1,0);
CAblasF::RAllocV(n+m,State.m_deltaxy);
for(i=0; i<n; i++)
State.m_deltaxy.Set(i,State.m_c[i]+State.m_tmp0[i]);
for(i=0; i<m; i++)
{
//--- We need to specify target right-hand sides for constraints.
//--- Ether zero, b or b+r is used (depending on presence of r and
//--- magnitudes of b and b+r, and subject to current State of frozen
//--- variables).
vrhs=State.m_b[i]-State.m_tmp1[i];
if(State.m_hasr[i])
{
//--- Range constraint b<=Ax<=b+r
if(vrhs>=0.0)
{
//--- 0<=b<=b+r, select target at lower bound
v=vrhs;
}
else
{
//--- b<=0, b+r can have any sign.
//--- Select zero target if possible, if not - one with smallest absolute value.
v=MathMin(vrhs+State.m_r[i],0.0);
}
}
else
{
//--- Single-sided constraint Ax>=b.
//--- Select zero target if possible, if not - one with smallest absolute value.
v=MathMax(vrhs,0.0);
}
State.m_deltaxy.Set(n+i,v);
}
SolveReducedKKTSystem(State,State.m_deltaxy);
for(i=0; i<n; i++)
{
if(!State.m_isfrozen[i])
State.m_current.m_x.Set(i,State.m_deltaxy[i]);
}
for(i=0; i<m; i++)
State.m_current.m_y.Set(i,State.m_deltaxy[n+i]);
//--- Set up slacks according to our own heuristic
initprimslack=MathMax(m_initslackval,CAblasF::RMaxAbsV(n,State.m_current.m_x));
initdualslack=MathMax(m_initslackval,CAblasF::RMaxAbsV(m,State.m_current.m_y));
MultiplyGEAX(State,1.0,State.m_current.m_x,0,0.0,State.m_tmpax,0);
mu0=1.0;
for(i=0; i<n; i++)
{
if(State.m_hasgz[i])
{
State.m_current.m_g.Set(i,MathMax(MathAbs(State.m_current.m_x[i]-State.m_bndl[i]),initprimslack));
State.m_current.m_z.Set(i,MathMax(State.m_current.m_g[i]*maxinitialimbalance,initdualslack));
mu0=MathMax(mu0,State.m_current.m_g[i]*State.m_current.m_z[i]);
}
if(State.m_hasts[i])
{
State.m_current.m_t.Set(i,MathMax(MathAbs(State.m_current.m_x[i]-State.m_bndu[i]),initprimslack));
State.m_current.m_s.Set(i,MathMax(State.m_current.m_t[i]*maxinitialimbalance,initdualslack));
mu0=MathMax(mu0,State.m_current.m_t[i]*State.m_current.m_s[i]);
}
}
for(i=0; i<m; i++)
{
if(State.m_haswv[i])
{
State.m_current.m_w.Set(i,MathMax(MathAbs(State.m_tmpax[i]-State.m_b[i]),initprimslack));
State.m_current.m_v.Set(i,MathMax(State.m_current.m_w[i]*maxinitialimbalance,MathMax(MathAbs(State.m_current.m_y[i]),m_initslackval)));
mu0=MathMax(mu0,State.m_current.m_w[i]*State.m_current.m_v[i]);
}
if(State.m_haspq[i])
{
State.m_current.m_p.Set(i,MathMax(MathAbs(State.m_r[i]-State.m_current.m_w[i]),initprimslack));
State.m_current.m_q.Set(i,MathMax(State.m_current.m_p[i]*maxinitialimbalance,MathMax(MathAbs(State.m_current.m_y[i]),m_initslackval)));
mu0=MathMax(mu0,State.m_current.m_p[i]*State.m_current.m_q[i]);
}
}
//--- Additional shift to ensure that initial point is not too non-centered
mumin=mu0*maxinitialnoncentrality;
for(i=0; i<n; i++)
{
if(State.m_hasgz[i] && (State.m_current.m_g[i]*State.m_current.m_z[i])<mumin)
{
v=MathSqrt(mumin/(State.m_current.m_g[i]*State.m_current.m_z[i]));
State.m_current.m_g.Mul(i,v);
State.m_current.m_z.Mul(i,v);
}
if(State.m_hasts[i] && (double)(State.m_current.m_t[i]*State.m_current.m_s[i])<mumin)
{
v=MathSqrt(mumin/(State.m_current.m_t[i]*State.m_current.m_s[i]));
State.m_current.m_t.Mul(i,v);
State.m_current.m_s.Mul(i,v);
}
}
for(i=0; i<m; i++)
{
if(State.m_haswv[i] && (State.m_current.m_w[i]*State.m_current.m_v[i])<mumin)
{
v=MathSqrt(mumin/(State.m_current.m_w[i]*State.m_current.m_v[i]));
State.m_current.m_w.Mul(i,v);
State.m_current.m_v.Mul(i,v);
}
if(State.m_haspq[i] && (State.m_current.m_p[i]*State.m_current.m_q[i])<mumin)
{
v=MathSqrt(mumin/(State.m_current.m_p[i]*State.m_current.m_q[i]));
State.m_current.m_p.Mul(i,v);
State.m_current.m_q.Mul(i,v);
}
}
//--- Almost done
if(State.m_dotrace)
CAp::Trace("> initial point was generated\n");
}
//+------------------------------------------------------------------+
//| This function performs factorization of modified KKT system |
//| ( | ) |
//| (-(H + alpha0 * D + alpha1*I) | A^T ) |
//| ( | ) |
//| (---------------------------- | -------------------) |
//| ( | ) |
//| ( A | beta0 * E + beta1*I) |
//| ( | ) |
//| where: |
//| * H is an NxN quadratic term |
//| * A is an MxN matrix of linear constraint |
//| * alpha0, alpha1, beta0, beta1 are nonnegative scalars |
//| * D and E are diagonal matrices with nonnegative entries |
//| (which are ignored if alpha0 and beta0 are zero - arrays |
//| are not referenced at all) |
//| * I is an NxN or MxM identity matrix |
//| Additionally, regularizing term |
//| ( | ) |
//| ( -reg * I | ) |
//| ( | ) |
//| (---------- | ----------) |
//| ( | ) |
//| ( | +reg*I ) |
//| ( | ) |
//| is added to the entire KKT system prior to factorization in order|
//| to improve its numerical stability. |
//| Returns True on success, False on falure of factorization (it is |
//| recommended to increase regularization parameter and try one more|
//| time). |
//+------------------------------------------------------------------+
bool CVIPMSolver::VIPMFactorize(CVIPMState &State,
double alpha0,
CRowDouble &d,
double beta0,
CRowDouble &e,
double alpha11,
double beta11,
double modeps,
double dampeps)
{
//--- create variables
int n=State.m_n;
int nmain=State.m_nmain;
int nslack=n-nmain;
int m=State.m_mdense+State.m_msparse;
int mdense=State.m_mdense;
int msparse=State.m_msparse;
int i=0;
int j=0;
int k=0;
int k0=0;
int k1=0;
int ka=0;
int kb=0;
int ja=0;
int jb=0;
double va=0;
double vb=0;
double v=0;
double vv=0;
double badchol=1.0E50;
double sumsq=0;
double errsq=0;
//--- check
if(!CAp::Assert(MathIsValidNumber(alpha0) && alpha0>=0.0,__FUNCTION__+": Alpha0 is infinite or negative"))
return(false);
if(!CAp::Assert(MathIsValidNumber(alpha11) && alpha11>=0.0,__FUNCTION__+": Alpha1 is infinite or negative"))
return(false);
if(!CAp::Assert(MathIsValidNumber(beta0) && beta0>=0.0,__FUNCTION__+": Beta0 is infinite or negative"))
return(false);
if(!CAp::Assert(MathIsValidNumber(beta11) && beta11>=0.0,__FUNCTION__+": Beta1 is infinite or negative"))
return(false);
if(!CAp::Assert(State.m_factorizationtype==0 || State.m_factorizationtype==1,__FUNCTION__+": unexpected factorization type"))
return(false);
if(!CAp::Assert(State.m_factorizationpoweredup,__FUNCTION__+": critical integrity check failed (no powerup stage)"))
return(false);
State.m_factorizationpresent=false;
//--- Dense NxN normal equations approach
if(State.m_factorizationtype==0)
{
//--- A problem formulation with possible slacks.
//--- === A FORMULATION WITHOUT FROZEN VARIABLES ===
//--- We have to solve following system:
//--- [ -(H+Dh+Rh) Ah' ] [ Xh ] [ Bh ]
//--- [ -(Dz+Rz) Az' ] [ Xz ] = [ Bz ]
//--- [ Ah Az E ] [ Y ] [ By ]
//--- with Xh being NMain-dimensional vector, Xz being NSlack-dimensional vector, constraint
//--- matrix A being divided into non-slack and slack parts Ah and Az (and Ah, in turn, being
//--- divided into sparse and dense parts), Rh and Rz being diagonal regularization matrix,
//--- Y being M-dimensional vector.
//--- NOTE: due to definition of slack variables following holds: for any diagonal matrix W
//--- a product Az*W*Az' is a diagonal matrix.
//--- From the second line we get
//--- Xz = inv(Dz+Rz)*Az'*y - inv(Dz+Rz)*Bz
//--- = inv(Dz+Rz)*Az'*y - BzWave
//--- Using this value for Zx, third line gives us
//--- Y = inv(E+Az*inv(Dz+Rz)*Az')*(By+Az*BzWave-Ah*Xh)
//--- = inv(EWave)*(ByWave-Ah*Xh)
//--- with EWave = E+Az*inv(Dz+Rz)*Az' and ByWave = By+Az*BzWave
//--- Finally, first line gives us
//--- Xh = -inv(H+Dh+Rh+Ah'*inv(EWave)*Ah)*(Bh-Ah'*inv(EWave)*ByWave)
//--- = -inv(HWave)*BhWave
//--- with HWave = H+Dh+Rh+Ah'*inv(EWave)*Ah and BhWave = Bh-Ah'*inv(EWave)*ByWave
//--- In order to prepare factorization we need to compute:
//--- (a) diagonal matrices Dh, Rh, Dz and Rz (and precomputed inverse of Dz+Rz)
//--- (b) EWave
//--- (c) HWave
//--- === SPECIAL HANDLING OF FROZEN VARIABLES ===
//--- Frozen variables result in zero steps, i.e. zero components of Xh and Xz.
//--- It could be implemented by explicit modification of KKT system (zeroing out
//--- columns/rows of KKT matrix, rows of right part, putting 1's to diagonal).
//--- However, it is possible to do without actually modifying quadratic term and
//--- constraints:
//--- * freezing elements of Xz can be implemented by zeroing out corresponding
//--- columns of inv(Dz+Rz) because Az always appears in computations along with diagonal Dz+Rz.
//--- * freezing elements of Xh is a bit more complex - it needs:
//--- * zeroing out columns/rows of HWave and setting up unit diagonal prior to solving for Xh
//--- * explicitly zeroing out computed elements of Xh prior to computing Y and Xz
State.m_factregdhrh.Resize(nmain);
State.m_factinvregdzrz.Resize(nslack);
for(i=0; i<n; i++)
{
v=0;
if(alpha0>0)
v+=alpha0*d[i];
if(alpha11>0)
v+=alpha11;
v+=State.m_diagr[i]+dampeps;
//--- check
if(!CAp::Assert(v>0,__FUNCTION__+": integrity check failed,degenerate diagonal matrix"))
return(false);
if(i>=nmain)
{
if(!State.m_isfrozen[i])
State.m_factinvregdzrz.Set(i-nmain,1/v);
else
State.m_factinvregdzrz.Set(i-nmain,0.0);
}
else
State.m_factregdhrh.Set(i,v);
}
//--- Now we are ready to compute EWave
State.m_factregewave.Resize(m);
for(i=0; i<m; i++)
{
//--- Compute diagonal element of E
v=0;
if(beta0>0)
v+=beta0*e[i];
if(beta11>0)
v+=beta11;
v+=dampeps;
//--- check
if(!CAp::Assert(v>0,__FUNCTION__+": integrity check failed,degenerate diagonal matrix"))
return(false);
//--- Compute diagonal modification Az*inv(Dz)*Az'
k0=State.m_combinedaslack.m_RIdx[i];
k1=State.m_combinedaslack.m_RIdx[i+1];
for(k=k0; k<k1; k++)
{
vv=State.m_combinedaslack.m_Vals[k];
v+=vv*vv*State.m_factinvregdzrz[State.m_combinedaslack.m_Idx[k]];
}
//--- Save EWave
State.m_factregewave.Set(i,v);
}
//--- Now we are ready to compute HWave:
//--- * store H
//--- * add Dh
//--- * add Ah'*inv(EWave)*Ah
//--- check
if(!CAp::Assert(State.m_hkind==0,__FUNCTION__+": unexpected HKind"))
return(false);
State.m_factdensehaug=State.m_denseh;
State.m_factdensehaug.Resize(nmain,nmain);
for(i=0; i<nmain; i++)
State.m_factdensehaug.Add(i,i,State.m_factregdhrh[i]);
if(msparse>0)
{
//--- Handle sparse part of Ah in Ah'*inv(EWave)*Ah
for(i=0; i<msparse; i++)
{
v=1.0/State.m_factregewave[i];
k0=State.m_sparseamain.m_RIdx[i];
k1=State.m_sparseamain.m_RIdx[i+1];
for(ka=k0; ka<k1; ka++)
{
ja=State.m_sparseamain.m_Idx[ka];
va=State.m_sparseamain.m_Vals[ka];
for(kb=k0; kb<=ka; kb++)
{
jb=State.m_sparseamain.m_Idx[kb];
vb=State.m_sparseamain.m_Vals[kb];
State.m_factdensehaug.Add(ja,jb,v*va*vb);
}
}
}
}
if(mdense>0)
{
//--- Handle dense part of Ah in Ah'*inv(EWave)*Ah
State.m_tmpr2=State.m_denseamain;
State.m_tmpr2.Resize(mdense,nmain);
for(i=0; i<mdense; i++)
{
v=1.0/MathSqrt(State.m_factregewave[msparse+i]);
for(j=0; j<nmain; j++)
State.m_tmpr2.Mul(i,j,v);
}
CAblas::RMatrixSyrk(nmain,mdense,1.0,State.m_tmpr2,0,0,2,1.0,State.m_factdensehaug,0,0,false);
}
//--- Zero out rows/cols of HWave corresponding to frozen variables, set up unit diagonal
State.m_tmp0=vector<double>::Ones(nmain);
for(i=0; i<nmain; i++)
{
if(State.m_isfrozen[i])
{
State.m_tmp0.Set(i,0.0);
//--- Entire row is nullified except for diagonal element
CAblasF::RSetR(i+1,0.0,State.m_factdensehaug,i);
State.m_factdensehaug.Set(i,i,1.0);
}
else
{
//--- Only some components are nullified
CAblasF::RMergeMulVR(i+1,State.m_tmp0,State.m_factdensehaug,i);
}
}
//--- Compute Cholesky factorization of HWave
if(!CTrFac::SPDMatrixCholesky(State.m_factdensehaug,nmain,false))
return(false);
v=(State.m_factdensehaug.Diag(0)+0).Sum();
if(!MathIsValidNumber(v) || v>badchol)
return(false);
State.m_factorizationpresent=true;
}
//--- Sparse (M+N)x(M+N) factorization
if(State.m_factorizationtype==1)
{
//--- Generate reduced KKT matrix
State.m_facttmpdiag.Resize(n+m);
for(i=0; i<n; i++)
{
vv=0;
if(alpha0>0)
vv+=alpha0*d[i];
if(alpha11>0)
vv+=alpha11;
vv+=State.m_diagr[i]+dampeps;
State.m_facttmpdiag.Set(i,-vv);
//--- check
if(!CAp::Assert(vv>0,__FUNCTION__+": integrity check failed,degenerate diagonal matrix"))
return(false);
}
for(i=0; i<msparse+mdense; i++)
{
vv=0;
if(beta0>0)
vv+=beta0*e[i];
if(beta11>0)
vv+=beta11;
vv+=dampeps;
State.m_facttmpdiag.Set(n+i,vv);
//--- check
if(!CAp::Assert(vv>0,__FUNCTION__+": integrity check failed,degenerate diagonal matrix"))
return(false);
}
//--- Perform factorization
//--- Perform additional integrity check: LDLT should reproduce diagonal of initial KKT system with good precision
if(!ReducedSystemFactorizeWithAddEnd(State.m_reducedsparsesystem,State.m_facttmpdiag,modeps,badchol,sumsq,errsq))
{
return(false);
}
if(MathSqrt(errsq/(1+sumsq))>MathSqrt(CMath::m_machineepsilon))
{
if(State.m_dotrace)
CAp::Trace(StringFormat("LDLT-diag-err= %.3E (diagonal reproduction error)\n",MathSqrt(errsq / (1 + sumsq))));
return(false);
}
State.m_factorizationpresent=true;
//--- Trace
if(State.m_dotrace)
{
CSpChol::SpSymmExtract(State.m_reducedsparsesystem.m_analysis,State.m_tmpsparse0,State.m_tmp0,State.m_tmpi);
CAp::Trace("--- sparse KKT factorization report ----------------------------------------------------------------\n");
CAp::Trace("> diagonal terms D and E\n");
if(alpha0!=0.0)
{
v=MathAbs(d[0]);
vv=MathAbs(d[0]);
for(i=1; i<n; i++)
{
v=MathMin(v,MathAbs(d[i]));
vv=MathMax(vv,MathAbs(d[i]));
}
CAp::Trace(StringFormat("diagD = %.3E (min) ... %.3E (max)\n",v,vv));
}
if(m>0 && beta0!=0.0)
{
v=MathAbs(e[0]);
vv=MathAbs(e[0]);
for(i=1; i<m; i++)
{
v=MathMin(v,MathAbs(e[i]));
vv=MathMax(vv,MathAbs(e[i]));
}
CAp::Trace(StringFormat("diagE = %.3E (min) ... %.3E (max)\n",v,vv));
}
CAp::Trace("> LDLT factorization of entire KKT matrix\n");
v=MathAbs(State.m_tmp0[0]);
vv=MathAbs(State.m_tmp0[0]);
for(i=1; i<State.m_tmpsparse0.m_N; i++)
{
v=MathMax(v,MathAbs(State.m_tmp0[i]));
vv=MathMin(vv,MathAbs(State.m_tmp0[i]));
}
CAp::Trace(StringFormat("|D| = %.3E (min) ... %.3E (max)\n",vv,v));
v=0.0;
for(i=0; i<State.m_tmpsparse0.m_N; i++)
{
k0=State.m_tmpsparse0.m_RIdx[i];
k1=State.m_tmpsparse0.m_DIdx[i];
for(k=k0; k<=k1; k++)
v=MathMax(v,MathAbs(State.m_tmpsparse0.m_Vals[k]));
}
CAp::Trace(StringFormat("max(|L|) = %.3E\n",v));
CAp::Trace(StringFormat("diag-err = %.3E (diagonal reproduction error)\n",MathSqrt(errsq / (1 + sumsq))));
}
}
//--- Done, integrity control
//--- check
if(!CAp::Assert(State.m_factorizationpresent,__FUNCTION__+": integrity check failed"))
return(false);
State.m_repncholesky++;
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| A low - level function which solves KKT system whose regularized |
//| (!) factorization was prepared by VIPMFactorize(). No iterative |
//| refinement is performed. |
//| On input, right - hand - side is stored in DeltaXY; on output, |
//| solution replaces DeltaXY. |
//+------------------------------------------------------------------+
void CVIPMSolver::SolveReducedKKTSystem(CVIPMState &State,
CRowDouble &deltaxy)
{
//--- create variables
int n=State.m_n;
int nmain=State.m_nmain;
int nslack=n-nmain;
int m=State.m_mdense+State.m_msparse;
int mdense=State.m_mdense;
int msparse=State.m_msparse;
int i=0;
//--- check
if(!CAp::Assert(State.m_factorizationpresent,__FUNCTION__+": integrity check failed - factorization is not present"))
return;
if(!CAp::Assert(State.m_factorizationtype==0 || State.m_factorizationtype==1,__FUNCTION__+": unexpected factorization type"))
return;
//--- Dense solving
switch(State.m_factorizationtype)
{
case 0:
//--- Compute
//--- BzWave = inv(Dz+Rz)*Bz
//--- ByWave = By+Az*BzWave
//--- BhWave = Bh-Ah'*inv(EWave)*ByWave
for(i=0; i<nslack; i++)
deltaxy.Mul(nmain+i,State.m_factinvregdzrz[i]);
CSparse::SparseGemV(State.m_combinedaslack,1.0,0,deltaxy,nmain,1.0,deltaxy,n);
State.m_tmp1.Resize(m);
for(i=0; i<m; i++)
State.m_tmp1.Set(i,deltaxy[n+i]/State.m_factregewave[i]);
CSparse::SparseGemV(State.m_sparseamain,-1.0,1,State.m_tmp1,0,1.0,deltaxy,0);
CAblas::RMatrixGemVect(nmain,mdense,-1.0,State.m_denseamain,0,0,1,State.m_tmp1,msparse,1.0,deltaxy,0);
//--- Compute Xh = -inv(HWave)*BhWave.
//--- Zero out components corresponding to frozen variables.
for(i=0; i<nmain; i++)
deltaxy.Mul(i,-1);
CAblas::RMatrixTrsVect(nmain,State.m_factdensehaug,0,0,false,false,0,deltaxy,0);
CAblas::RMatrixTrsVect(nmain,State.m_factdensehaug,0,0,false,false,1,deltaxy,0);
for(i=0; i<n; i++)
{
if(State.m_isfrozen[i])
deltaxy.Set(i,0);
}
//--- Compute Y = inv(EWave)*(ByWave-Ah*Xh)
CSparse::SparseGemV(State.m_sparseamain,-1.0,0,deltaxy,0,1.0,deltaxy,n);
CAblas::RMatrixGemVect(mdense,nmain,-1.0,State.m_denseamain,0,0,0,deltaxy,0,1.0,deltaxy,n+msparse);
for(i=0; i<m; i++)
deltaxy.Mul(n+i,1.0/State.m_factregewave[i]);
//--- Compute Xz = -(BzWave - inv(Dz+Rz)*Az'*y)
State.m_tmp0.Resize(nslack);
for(i=0; i<nslack; i++)
State.m_tmp0.Set(i,0);
CSparse::SparseGemV(State.m_combinedaslack,1.0,1,deltaxy,n,1.0,State.m_tmp0,0);
for(i=0; i<nslack; i++)
deltaxy.Set(nmain+i,-(deltaxy[nmain+i]-State.m_factinvregdzrz[i]*State.m_tmp0[i]));
//--- Done
break;
//--- Sparse solving
case 1:
ReducedSystemSolve(State.m_reducedsparsesystem,deltaxy);
for(i=0; i<n; i++)
{
if(State.m_isfrozen[i])
deltaxy.Set(i,0);
}
break;
default:
CAp::Assert(false,__FUNCTION__+": integrity check failed - unexpected factorization");
break;
}
}
//+------------------------------------------------------------------+
//| Generates precomputed temporary vectors and KKT factorization at |
//| the beginning of the current iteration. |
//| This function uses representation of KKT system inspired by |
//| Vanderbei slack variable approach, but with additional |
//| regularization being applied all along computations. |
//| On successful factorization returns True; on failure returns |
//| False - it is recommended to increase regularization parameter |
//| and try one more time. |
//| --- DESCRIPTION ------------------------------------------------ |
//| Initial KKT system proposed by Vanderbei has following structure:|
//| (1) -DS*deltaT-I*deltaS = -mu/T+s+DELTAT*DELTAS/T = -GammaS |
//| (2) -DZ*deltaG-I*deltaZ = -mu/G+z+DELTAG*DELTAZ/G = -GammaZ |
//| (3) -DQ*deltaP-I*deltaQ = -mu/P+q+DELTAP*DELTAQ/P = -GammaQ |
//| (4) -DW*deltaV-I*deltaW = -mu/V+w+DELTAV*DELTAW/V = -GammaW |
//| (5) - I*deltaY-I*deltaQ+I*deltaV = y-q+v = Beta |
//| (6) -H*deltaX+A'*deltaY+I*deltaZ-I*deltaS=c-A'*y-z+s+H*x=Sigma |
//| (7) A * deltaX - I * deltaW = b - A * x + w = Rho |
//| (8) I * deltaX - I * deltaG = l - x + g = Nu |
//| (9) -I * deltaX - I * deltaT = -u + x + t = -Tau |
//| (10) -I * deltaW - I * deltaP = -r + w + p = -Alpha |
//| where: |
//| DS = diag(S / T) |
//| DZ = diag(Z / G) |
//| DQ = diag(Q / P) |
//| DW = diag(W / V) |
//| This linear system is actually symmetric indefinite one, that can|
//| be regularized by modifying equations (5), (6), (7), (8), (9), |
//| (10): |
//| (5) -I*deltaY-I*deltaQ+I*deltaV-REG*deltaW = y+q-v+REG*w = Beta|
//| (6) -(H+REG)*deltaX+A'*deltaY+I*deltaZ-I*deltaS = |
//| c-A'*y-z+s+(H+REG)*x = Sigma |
//| (7) A*deltaX-I*deltaW+REG*deltaY = b-A*x+w-REG*y = Rho |
//| (8) I*deltaX-I*deltaG+REG*deltaZ = l-x+g-REG*z = Nu |
//| (9) -I*deltaX-I*deltaT+REG*deltaS = -u+x+t-REG*s = -Tau |
//| (10)-I*deltaW-I*deltaP+REG*deltaQ = -r+w+p-REG*q = -Alpha |
//| NOTE: regularizing equations (5)-(10) seems to be beneficial |
//| because their coefficients are well-normalized, usually |
//| having unit scale. Contrary to that, equations (1)-(4) are |
//| wildly nonnormalized, and regularization ruins algorithm |
//| convergence. |
//| From(1), (2), (3) and (4) we obtain |
//| deltaT = (GammaS - I * deltaS) / DS |
//| deltaG = (GammaZ - I * deltaZ) / DZ |
//| deltaP = (GammaQ - I * deltaQ) / DQ |
//| deltaV = (GammaW - I * deltaW) / DW |
//| and substitute them to equations to obtain |
//| (5) -I*deltaY-I*deltaQ-(inv(DW)+REG)*deltaW = |
//| Beta - inv(DW) * GammaW = BetaCap |
//| (8) I*deltaX+(inv(DZ)+REG)*deltaZ = Nu+inv(DZ)*GammaZ = NuCap |
//| (9) I*deltaX+(inv(DS)+REG)*deltaS=-(Tau-inv(DS)*GammaS)=-TauCap|
//| (10) -I * deltaW + (inv(DQ) + REG) * deltaQ = |
//| -(Alpha - inv(DQ) * GammaQ) = -AlphaCap |
//| (6) A'*deltaY + I*deltaZ - I*deltaS - (H+REG)*deltaX = |
//| c-A' * y - z + s + (H + REG) * x = Sigma |
//| (7) REG*deltaY+A*deltaX-I*deltaW = b-A*x+w-REG*y = Rho |
//| then, we obtain(here IRI stands for Invert - Regularize - Invert)|
//| DQIRI = inv(inv(DQ) + REG) |
//| DZIRI = inv(inv(DZ) + REG) |
//| DSIRI = inv(inv(DS) + REG) |
//| deltaQ = (I * deltaW - AlphaCap) * DQIRI |
//| deltaZ = (NuCap - I * deltaX) * DZIRI |
//| deltaS = (I * deltaX - TauCap) * DSIRI |
//| DWIR = inv(DW) + REG |
//| and after substitution |
//| (5) -I*deltaY-(DQIRI+DWIR)*deltaW = BetaCap-DQIRI*AlphaCap |
//| (6) A'*deltaY - (H+REG+DSIRI+DZIRI)*deltaX = |
//| Sigma-DSIRI*TauCap-DZIRI*NuCap |
//| (7) REG * deltaY + A * deltaX - I * deltaW = Rho |
//| finally, we obtain |
//| DE = inv(DQIRI + DWIR) |
//| DER = DE + REG |
//| DDR = DSIRI + DZIRI + REG |
//| deltaW = -(BetaCap - DQIRI * AlphaCap + I * deltaY) * DE |
//| and after substitution |
//| (6) - (H + DDR) * deltaX + A'*deltaY = |
//| Sigma-DSIRI*TauCap-DZIRI*NuCap |
//| (7) A*deltaX + DER*deltaY = Rho-DE*(BetaCap-DQIRI*AlphaCap) |
//+------------------------------------------------------------------+
bool CVIPMSolver::VIPMPrecomputeNewtonFactorization(CVIPMState &State,
CVIPMVars &v0,
double regeps,
double modeps,
double dampeps,
double dampfree)
{
//--- create variables
bool result=false;
int n=State.m_n;
int m=State.m_mdense+State.m_msparse;
int i=0;
State.m_diagdz=vector<double>::Zeros(n);
State.m_diagdzi=vector<double>::Zeros(n);
State.m_diagdziri=vector<double>::Zeros(n);
State.m_diagds=vector<double>::Zeros(n);
State.m_diagdsi=vector<double>::Zeros(n);
State.m_diagdsiri=vector<double>::Zeros(n);
State.m_diagdw=vector<double>::Zeros(m);
State.m_diagdwi=vector<double>::Zeros(m);
State.m_diagdwir=vector<double>::Zeros(m);
State.m_diagdq=vector<double>::Zeros(m);
State.m_diagdqi=vector<double>::Zeros(m);
State.m_diagdqiri=vector<double>::Zeros(m);
State.m_diagddr.Resize(n);
State.m_diagde.Resize(m);
State.m_diagder.Resize(m);
//--- Handle temporary matrices arising due to box constraints
for(i=0; i<n; i++)
{
//--- Lower bound: G*inv(Z) and Z*inv(G)
if(State.m_hasgz[i])
{
//--- check
if(!CAp::Assert(v0.m_g[i]>0.0 && v0.m_z[i]>0.0,__FUNCTION__+": integrity failure - G[i]<=0 or Z[i]<=0"))
return(false);
State.m_diagdz.Set(i,v0.m_z[i]/v0.m_g[i]);
State.m_diagdzi.Set(i,1.0/State.m_diagdz[i]);
State.m_diagdziri.Set(i,1.0/(State.m_diagdzi[i]+regeps));
}
else
{
//--- check
if(!CAp::Assert(v0.m_g[i]==0.0 && v0.m_z[i]==0.0,__FUNCTION__+": integrity failure - G[i]<>0 or Z[i]<>0 for absent lower bound"))
return(false);
}
//--- Upper bound: T*inv(S) and S*inv(T)
if(State.m_hasts[i])
{
//--- check
if(!CAp::Assert(v0.m_t[i]>0.0 && v0.m_s[i]>0.0,__FUNCTION__+": integrity failure - T[i]<=0 or S[i]<=0"))
return(false);
State.m_diagds.Set(i,v0.m_s[i]/v0.m_t[i]);
State.m_diagdsi.Set(i,1/State.m_diagds[i]);
State.m_diagdsiri.Set(i,1/(State.m_diagdsi[i]+regeps));
}
else
{
//--- check
if(!CAp::Assert(v0.m_t[i]==0.0 && v0.m_s[i]==0.0,__FUNCTION__+": integrity failure - T[i]<>0 or S[i]<>0 for absent upper bound"))
return(false);
}
//--- Diagonal term D
State.m_diagddr.Set(i,State.m_diagdziri[i]+State.m_diagdsiri[i]+regeps);
if(!State.m_hasgz[i] && !State.m_hasts[i])
State.m_diagddr.Add(i,dampfree);
}
//--- Handle temporary matrices arising due to linear constraints: with lower bound B[]
//--- or with lower and upper bounds.
for(i=0; i<m; i++)
{
//--- Lower bound
if(State.m_haswv[i])
{
//--- check
if(!CAp::Assert(v0.m_v[i]>0.0 && v0.m_w[i]>0.0,__FUNCTION__+": integrity failure - V[i]<=0 or W[i]<=0"))
return(false);
State.m_diagdw.Set(i,v0.m_w[i]/v0.m_v[i]);
State.m_diagdwi.Set(i,1/State.m_diagdw[i]);
State.m_diagdwir.Set(i,State.m_diagdwi[i]+regeps);
}
else
{
//--- check
if(!CAp::Assert(v0.m_v[i]==0.0 && v0.m_w[i]==0.0,__FUNCTION__+": integrity failure - V[i]<>0 or W[i]<>0 for linear equality constraint"))
return(false);
}
//--- Upper bound
if(State.m_haspq[i])
{
//--- check
if(!CAp::Assert(v0.m_p[i]>0.0 && v0.m_q[i]>0.0,__FUNCTION__+": integrity failure - P[i]<=0 or Q[i]<=0"))
return(false);
State.m_diagdq.Set(i,v0.m_q[i]/v0.m_p[i]);
State.m_diagdqi.Set(i,1/State.m_diagdq[i]);
State.m_diagdqiri.Set(i,1/(State.m_diagdqi[i]+regeps));
}
else
{
//--- check
if(!CAp::Assert(v0.m_p[i]==0.0 && v0.m_q[i]==0.0,__FUNCTION__+": integrity failure - P[i]<>0 or Q[i]<>0 for absent linear constraint"))
return(false);
}
//--- Diagonal term E
if(State.m_haswv[i] || State.m_haspq[i])
State.m_diagde.Set(i,1/(State.m_diagdwir[i]+State.m_diagdqiri[i]));
else
State.m_diagde.Set(i,0.0);
State.m_diagder.Set(i,State.m_diagde[i]+regeps);
}
//--- Perform factorization
result=VIPMFactorize(State,1.0,State.m_diagddr,1.0,State.m_diagder,0.0,0.0,modeps,dampeps);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Solves KKT system stored in VIPMState with user - passed RHS. |
//| Sol must be preallocated VIPMVars object whose initial values are|
//| ignored. |
//+------------------------------------------------------------------+
void CVIPMSolver::SolveKKTSystem(CVIPMState &State,
CVIPMRightHandSide &rhs,
CVIPMVars &sol)
{
//--- create variables
int n=State.m_n;
int m=State.m_mdense+State.m_msparse;
int i=0;
//--- Compute elimination temporaries
//--- RhsAlphaCap = RhsAlpha - InvDQ*GammaQ
//--- RhsNuCap = RhsNu + InvDZ*GammaZ
//--- RhsTauCap = RhsTau - InvDS*GammaS
//--- RhsBetaCap = RhsBeta - InvDW*GammaW
State.m_rhsnucap.Resize(n);
State.m_rhstaucap.Resize(n);
State.m_rhsbetacap.Resize(m);
State.m_rhsalphacap.Resize(m);
CAblasF::RCopyNegMulAddV(m,State.m_diagdqi,rhs.m_gammaq,rhs.m_alpha,State.m_rhsalphacap);
CAblasF::RCopyMulAddV(n,State.m_diagdzi,rhs.m_gammaz,rhs.m_nu,State.m_rhsnucap);
CAblasF::RCopyNegMulAddV(n,State.m_diagdsi,rhs.m_gammas,rhs.m_tau,State.m_rhstaucap);
CAblasF::RCopyNegMulAddV(m,State.m_diagdwi,rhs.m_gammaw,rhs.m_beta,State.m_rhsbetacap);
//--- Solve reduced KKT system
State.m_deltaxy.Resize(n+m);
for(i=0; i<n; i++)
State.m_deltaxy.Set(i,rhs.m_sigma[i]-State.m_diagdziri[i]*State.m_rhsnucap[i]-State.m_diagdsiri[i]*State.m_rhstaucap[i]);
for(i=0; i<m; i++)
State.m_deltaxy.Set(n+i,rhs.m_rho[i]-State.m_diagde[i]*(State.m_rhsbetacap[i]-State.m_diagdqiri[i]*State.m_rhsalphacap[i]));
SolveReducedKKTSystem(State,State.m_deltaxy);
//--- Perform backsubstitution
for(i=0; i<n; i++)
{
sol.m_x.Set(i,State.m_deltaxy[i]);
sol.m_s.Set(i,State.m_diagdsiri[i]*(sol.m_x[i]-State.m_rhstaucap[i]));
sol.m_z.Set(i,State.m_diagdziri[i]*(State.m_rhsnucap[i]-sol.m_x[i]));
sol.m_g.Set(i,State.m_diagdzi[i]*(rhs.m_gammaz[i]-sol.m_z[i]));
sol.m_t.Set(i,State.m_diagdsi[i]*(rhs.m_gammas[i]-sol.m_s[i]));
}
for(i=0; i<m; i++)
{
sol.m_y.Set(i,State.m_deltaxy[n+i]);
sol.m_w.Set(i,-(State.m_diagde[i]*(State.m_rhsbetacap[i]-State.m_diagdqiri[i]*State.m_rhsalphacap[i]+sol.m_y[i])));
sol.m_q.Set(i,State.m_diagdqiri[i]*(sol.m_w[i]-State.m_rhsalphacap[i]));
sol.m_v.Set(i,State.m_diagdwi[i]*(rhs.m_gammaw[i]-sol.m_w[i]));
sol.m_p.Set(i,State.m_diagdqi[i]*(rhs.m_gammaq[i]-sol.m_q[i]));
}
}
//+------------------------------------------------------------------+
//| Compute VIPM step by solving KKT system. |
//| VDResult must be preallocated VIPMVars object whose initial |
//| values are ignored. |
//| Returns False on failure to compute step direction with |
//| reasonable accuracy (it is advised to terminate iterations |
//| immediately). |
//+------------------------------------------------------------------+
bool CVIPMSolver::VIPMComputeStepDirection(CVIPMState &State,
CVIPMVars &v0,
double muestimate,
CVIPMVars &vdestimate,
CVIPMVars &vdresult,
double reg,
bool isdampepslarge)
{
//--- create variables
int n=State.m_n;
int m=State.m_mdense+State.m_msparse;
double vrhsprim2=0;
double vrhsdual2=0;
double vrhscmpl2=0;
double vresprim2=0;
double vresdual2=0;
double vrescmpl2=0;
double vrhspriminf=0;
double vrhsdualinf=0;
double vrespriminf=0;
double vresdualinf=0;
double badres=1.01;
double verybadres=1.0E3;
double residualgrowth=0;
bool primaldestabilized=false;
bool dualdestabilized=false;
//--- Initial m_solver report
if(State.m_dotrace)
CAp::Trace("--- detailed KKT solver report ---------------------------------------------------------------------\n");
//--- Solve KKT system with right-hand sides coming from primal, dual
//--- and complementary slackness conditions. Analyze solution,
//--- terminate immediately if primal/dual residuals are way too high.
RHSCompute(State,v0,muestimate,vdestimate,State.m_rhs,reg);
vrhsprim2=RHSPrimal2(State.m_rhs);
vrhsdual2=RHSDual2(State.m_rhs);
vrhscmpl2=RHSCompl2(State.m_rhs);
vrhspriminf=RHSPrimalInf(State.m_rhs);
vrhsdualinf=RHSDualInf(State.m_rhs);
if(State.m_dotrace)
{
CAp::Trace("> primal/dual/complementarity right-hand-side\n");
CAp::Trace(StringFormat("rhs-prim = %.3E (2-norm)\n",MathSqrt(vrhsprim2)));
CAp::Trace(StringFormat("rhs-dual = %.3E (2-norm)\n",MathSqrt(vrhsdual2)));
CAp::Trace(StringFormat("rhs-cmpl = %.3E (2-norm)\n",MathSqrt(vrhscmpl2)));
}
SolveKKTSystem(State,State.m_rhs,vdresult);
RHSSubtract(State,State.m_rhs,v0,vdresult,reg);
vresprim2=RHSPrimal2(State.m_rhs);
vresdual2=RHSDual2(State.m_rhs);
vrescmpl2=RHSCompl2(State.m_rhs);
vrespriminf=RHSPrimalInf(State.m_rhs);
vresdualinf=RHSDualInf(State.m_rhs);
if(State.m_dotrace)
{
CAp::Trace("> primal/dual/complementarity residuals compared with RHS\n");
CAp::Trace(StringFormat("res/rhs prim = %.3E\n",MathSqrt(vresprim2 / CApServ::Coalesce(vrhsprim2,1))));
CAp::Trace(StringFormat("res/rhs dual = %.3E\n",MathSqrt(vresdual2 / CApServ::Coalesce(vrhsdual2,1))));
CAp::Trace(StringFormat("res/rhs cmpl = %.3E\n",MathSqrt(vrescmpl2 / CApServ::Coalesce(vrhscmpl2,1))));
CAp::Trace(StringFormat("res/rhs all = %.3E\n",MathSqrt((vresprim2 + vresdual2 + vrescmpl2) / CApServ::Coalesce(vrhsprim2 + vrhsdual2 + vrhscmpl2,1))));
}
primaldestabilized=(vrhspriminf<=State.m_epsp && vrespriminf>=MathMax(verybadres*vrhspriminf,State.m_epsp));
dualdestabilized=(vrhsdualinf<=State.m_epsd && vresdualinf>=MathMax(verybadres*vrhsdualinf,State.m_epsd));
residualgrowth=MathSqrt((vresprim2+vresdual2+vrescmpl2)/CApServ::Coalesce(vrhsprim2+vrhsdual2+vrhscmpl2,1));
if((primaldestabilized || dualdestabilized) && residualgrowth>(0.01*MathSqrt(CMath::m_machineepsilon)) && !isdampepslarge)
{
if(State.m_dotrace)
CAp::Trace("> primal/dual residual growth is too high,signaling presence of numerical errors\n");
return(false);
}
if(residualgrowth>badres)
{
if(State.m_dotrace)
CAp::Trace("> total residual is too high,signaling presence of numerical errors\n");
return(false);
}
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| This function estimates primal and dual step lengths (subject to |
//| step decay parameter, which should be in [0, 1] range). |
//| Current version returns same step lengths for primal and dual |
//| steps. |
//| INPUT PARAMETERS: |
//| State - solver State |
//| V0 - current point (we ignore one stored in |
//| State.Current) |
//| VS - step direction |
//| StepDecay- decay parameter, the step is multiplied by this |
//| coefficient. 1.0 corresponds to full step length |
//| being returned. Values in (0, 1] range. |
//| OUTPUT PARAMETERS: |
//| AlphaP - primal step(after applying decay coefficient) |
//| AlphaD - dual step(after applying decay coefficient) |
//+------------------------------------------------------------------+
void CVIPMSolver::VIPMComputeStepLength(CVIPMState &State,
CVIPMVars &v0,
CVIPMVars &vs,
double stepdecay,
double &alphap,
double &alphad)
{
//--- create variables
int n=State.m_n;
int m=State.m_mdense+State.m_msparse;
int i=0;
double alpha=0;
alphap=0;
alphad=0;
//--- check
if(!CAp::Assert(n==v0.m_n && m==v0.m_m,__FUNCTION__+": sizes mismatch"))
return;
alphap=1;
alphad=1;
for(i=0; i<n; i++)
{
//--- Primal
if(vs.m_g[i]<0.0)
alphap=CApServ::SafeMinPosRV(v0.m_g[i],-vs.m_g[i],alphap);
if(vs.m_t[i]<0.0)
alphap=CApServ::SafeMinPosRV(v0.m_t[i],-vs.m_t[i],alphap);
//--- Dual
if(vs.m_z[i]<0.0)
alphad=CApServ::SafeMinPosRV(v0.m_z[i],-vs.m_z[i],alphad);
if(vs.m_s[i]<0.0)
alphad=CApServ::SafeMinPosRV(v0.m_s[i],-vs.m_s[i],alphad);
}
for(i=0; i<m; i++)
{
//--- Primal
if(vs.m_w[i]<0.0)
alphap=CApServ::SafeMinPosRV(v0.m_w[i],-vs.m_w[i],alphap);
if(vs.m_p[i]<0.0)
alphap=CApServ::SafeMinPosRV(v0.m_p[i],-vs.m_p[i],alphap);
//--- Dual
if(vs.m_v[i]<0.0)
alphad=CApServ::SafeMinPosRV(v0.m_v[i],-vs.m_v[i],alphad);
if(vs.m_q[i]<0.0)
alphad=CApServ::SafeMinPosRV(v0.m_q[i],-vs.m_q[i],alphad);
}
//--- Because we may solve QP problem, step length has to be same for primal and dual variables
alpha=MathMin(alphap,alphad);
//--- Apply decay
alphap=stepdecay*alpha;
alphad=stepdecay*alpha;
}
//+------------------------------------------------------------------+
//| This function performs IPM step, updates iteration counts and |
//| performs following additional checks: |
//| * it monitors status of box/linear constraints and smoothly |
//| drops ones with too large bounds (a variable or linear sum is|
//| well below constraint bound for several iterations) |
//| INPUT PARAMETERS: |
//| State - m_solver State |
//| AlphaP - primal step to perform |
//| AlphaD - dual step to perform |
//| OUTPUT PARAMETERS: |
//+------------------------------------------------------------------+
void CVIPMSolver::VIPMPerformStep(CVIPMState &State,
double alphap,
double alphad)
{
//--- Perform step
VarsAddStep(State.m_current,State.m_deltacorr,alphap,alphad);
//--- Update iterations count
State.m_repiterationscount++;
}
//+------------------------------------------------------------------+
//| Compute primal / dual errors and complementarity gap |
//+------------------------------------------------------------------+
void CVIPMSolver::ComputeErrors(CVIPMState &State,
double &errp2,
double &errd2,
double &errpinf,
double &errdinf,
double &egap)
{
//--- create variables
int n=State.m_n;
int m=State.m_mdense+State.m_msparse;
int i=0;
int cntp2=0;
int cntd2=0;
double v=0;
errp2=0;
errd2=0;
errpinf=0;
errdinf=0;
egap=0;
//--- Compute primal and dual infeasibilities
VIPMMultiply(State,State.m_current.m_x,State.m_current.m_y,State.m_tmphx,State.m_tmpax,State.m_tmpaty);
cntp2=0;
errp2=0;
errpinf=0;
for(i=0; i<m; i++)
{
v=State.m_tmpax[i]-State.m_current.m_w[i]-State.m_b[i];
errp2=errp2+v*v;
errpinf=MathMax(errpinf,MathAbs(v));
cntp2++;
if(State.m_haspq[i])
{
v=State.m_current.m_w[i]+State.m_current.m_p[i]-State.m_r[i];
errp2=errp2+v*v;
errpinf=MathMax(errpinf,MathAbs(v));
cntp2++;
}
}
for(i=0; i<n; i++)
{
if(State.m_hasgz[i])
{
v=State.m_current.m_x[i]-State.m_current.m_g[i]-State.m_bndl[i];
errp2+=v*v;
errpinf=MathMax(errpinf,MathAbs(v));
cntp2++;
}
if(State.m_hasts[i])
{
v=State.m_current.m_x[i]+State.m_current.m_t[i]-State.m_bndu[i];
errp2+=v*v;
errpinf=MathMax(errpinf,MathAbs(v));
cntp2++;
}
}
errp2=MathSqrt(errp2/CApServ::Coalesce(cntp2,1));
cntd2=0;
errd2=0;
errdinf=0;
for(i=0; i<n; i++)
{
if(!State.m_isfrozen[i])
{
v=State.m_tmphx[i]+State.m_c[i]-State.m_tmpaty[i];
if(State.m_hasgz[i])
v-=State.m_current.m_z[i];
if(State.m_hasts[i])
v+=State.m_current.m_s[i];
errd2+=v*v;
errdinf=MathMax(errdinf,MathAbs(v));
cntd2++;
}
}
for(i=0; i<m; i++)
{
v=0;
if(State.m_haswv[i])
v=State.m_current.m_y[i]-State.m_current.m_v[i];
if(State.m_haspq[i])
v+=State.m_current.m_q[i];
errd2+=v*v;
errdinf=MathMax(errdinf,MathAbs(v));
if(State.m_haswv[i] || State.m_haspq[i])
cntd2++;
}
errd2=MathSqrt(errd2/CApServ::Coalesce(cntd2,1));
egap=VarsComputeComplementarityGap(State.m_current)/(1.0+MathAbs(VIPMTarget(State,State.m_current.m_x)));
}
//+------------------------------------------------------------------+
//| Performs integrity checks for current point and step |
//+------------------------------------------------------------------+
void CVIPMSolver::RunIntegrityChecks(CVIPMState &State,
CVIPMVars &v0,
CVIPMVars &vd,
double alphap,
double alphad)
{
//--- create variables
int n=State.m_n;
int m=State.m_mdense+State.m_msparse;
int i=0;
//--- check
if(!CAp::Assert(MathIsValidNumber(alphap) && alphap>=0.0,__FUNCTION__+": bad AlphaP"))
return;
if(!CAp::Assert(MathIsValidNumber(alphad) && alphad>=0.0,__FUNCTION__+": bad AlphaD"))
return;
for(i=0; i<n; i++)
{
if(State.m_hasgz[i])
{
//--- check
if(!CAp::Assert(!State.m_isfrozen[i],__FUNCTION__+": integrity failure - X[I] is frozen"))
return;
if(!CAp::Assert(v0.m_g[i]>0.0 && v0.m_z[i]>0.0,__FUNCTION__+": integrity failure - G[i]<=0 or Z[i]<=0"))
return;
}
else
{
//--- check
if(!CAp::Assert(v0.m_g[i]==0.0 && v0.m_z[i]==0.0,__FUNCTION__+": integrity failure - G[i]<>0 or Z[i]<>0 for absent lower bound"))
return;
if(!CAp::Assert(vd.m_g[i]==0.0 && vd.m_z[i]==0.0,__FUNCTION__+": integrity failure - G[i]<>0 or Z[i]<>0 for absent lower bound"))
return;
}
if(State.m_hasts[i])
{
if(!CAp::Assert(!State.m_isfrozen[i],__FUNCTION__+": integrity failure - X[I] is frozen"))
return;
if(!CAp::Assert(v0.m_t[i]>0.0 && v0.m_s[i]>0.0,__FUNCTION__+": integrity failure - T[i]<=0 or S[i]<=0"))
return;
}
else
{
if(!CAp::Assert(v0.m_t[i]==0.0 && v0.m_s[i]==0.0,__FUNCTION__+": integrity failure - T[i]<>0 or S[i]<>0 for absent upper bound"))
return;
if(!CAp::Assert(vd.m_t[i]==0.0 && vd.m_s[i]==0.0,__FUNCTION__+": integrity failure - T[i]<>0 or S[i]<>0 for absent upper bound"))
return;
}
}
for(i=0; i<m; i++)
{
if(!CAp::Assert(State.m_haswv[i] || !State.m_haspq[i],__FUNCTION__+": inconsistent HasWV/HasPQ"))
return;
if(State.m_haswv[i])
{
if(!CAp::Assert(v0.m_v[i]>0.0 && v0.m_w[i]>0.0,__FUNCTION__+": integrity failure - V[i]<=0 or W[i]<=0"))
return;
}
else
{
if(!CAp::Assert(v0.m_v[i]==0.0 && v0.m_w[i]==0.0,__FUNCTION__+": integrity failure - V[i]<>0 or W[i]<>0 for linear equality constraint"))
return;
if(!CAp::Assert(vd.m_v[i]==0.0 && vd.m_w[i]==0.0,__FUNCTION__+": integrity failure - V[i]<>0 or W[i]<>0 for linear equality constraint"))
return;
}
if(State.m_haspq[i])
{
if(!CAp::Assert(v0.m_p[i]>0.0 && v0.m_q[i]>0.0,__FUNCTION__+": integrity failure - P[i]<=0 or Q[i]<=0"))
return;
}
else
{
if(!CAp::Assert(v0.m_p[i]==0.0 && v0.m_q[i]==0.0,__FUNCTION__+": integrity failure - P[i]<>0 or Q[i]<>0 for absent range of linear constraint"))
return;
if(!CAp::Assert(vd.m_p[i]==0.0 && vd.m_q[i]==0.0,__FUNCTION__+": integrity failure - P[i]<>0 or Q[i]<>0 for absent range of linear constraint"))
return;
}
}
}
//+------------------------------------------------------------------+
//| Evaluate progress so far, outputs trace data, if requested to do |
//| so. |
//+------------------------------------------------------------------+
void CVIPMSolver::TraceProgress(CVIPMState &State,
double mu,
double muaff,
double sigma,
double alphap,
double alphad)
{
//--- create variables
int n=State.m_n;
int m=State.m_mdense+State.m_msparse;
int i=0;
double v=0;
double errp2=0;
double errd2=0;
double errpinf=0;
double errdinf=0;
double errgap=0;
if(!State.m_dotrace)
return;
//--- Print high-level information
ComputeErrors(State,errp2,errd2,errpinf,errdinf,errgap);
CAp::Trace("--- step report ------------------------------------------------------------------------------------\n");
CAp::Trace("> step information\n");
CAp::Trace(StringFormat("mu_init = %.3E (at the beginning)\n",mu));
CAp::Trace(StringFormat("mu_aff = %.3E (by affine scaling step)\n",muaff));
CAp::Trace(StringFormat("sigma = %.3E (centering parameter)\n",sigma));
CAp::Trace(StringFormat("alphaP = %.3E (primal step)\n",alphap));
CAp::Trace(StringFormat("alphaD = %.3E (dual step)\n",alphad));
CAp::Trace(StringFormat("mu_cur = %.3E (after the step)\n",VARSComputeMu(State,State.m_current)));
CAp::Trace("> errors\n");
CAp::Trace(StringFormat("errP = %.3E (primal infeasibility,inf-norm)\n",errpinf));
CAp::Trace(StringFormat("errD = %.3E (dual infeasibility, inf-norm)\n",errdinf));
CAp::Trace(StringFormat("errGap = %.3E (complementarity gap)\n",errgap));
CAp::Trace("> current point information (inf-norm)\n");
CAp::Trace(StringFormat("|X|=%8.1E,|G|=%8.1E,|T|=%8.1E,|W|=%8.1E,|P|=%8.1E\n",CAblasF::RMaxAbsV(n,State.m_current.m_x),CAblasF::RMaxAbsV(n,State.m_current.m_g),CAblasF::RMaxAbsV(n,State.m_current.m_t),CAblasF::RMaxAbsV(m,State.m_current.m_w),CAblasF::RMaxAbsV(m,State.m_current.m_p)));
CAp::Trace(StringFormat("|Y|=%8.1E,|Z|=%8.1E,|S|=%8.1E,|V|=%8.1E,|Q|=%8.1E\n",CAblasF::RMaxAbsV(m,State.m_current.m_y),CAblasF::RMaxAbsV(n,State.m_current.m_z),CAblasF::RMaxAbsV(n,State.m_current.m_s),CAblasF::RMaxAbsV(m,State.m_current.m_v),CAblasF::RMaxAbsV(m,State.m_current.m_q)));
//--- Print variable stats, if required
if(State.m_dotrace)
{
CAp::Trace("--- variable statistics ----------------------------------------------------------------------------\n");
CAp::Trace("> smallest values for nonnegative vars\n");
CAp::Trace(StringFormat("primal: minG=%8.1E minT=%8.1E minW=%8.1E minP=%8.1E\n",MinNZ(State.m_current.m_g,n),MinNZ(State.m_current.m_t,n),MinNZ(State.m_current.m_w,m),MinNZ(State.m_current.m_p,m)));
CAp::Trace(StringFormat("dual: minZ=%8.1E minS=%8.1E minV=%8.1E minQ=%8.1E\n",MinNZ(State.m_current.m_z,n),MinNZ(State.m_current.m_s,n),MinNZ(State.m_current.m_v,m),MinNZ(State.m_current.m_q,m)));
CAp::Trace("> min and max complementary slackness\n");
CAp::Trace(StringFormat("min: GZ=%8.1E TS=%8.1E WV=%8.1E PQ=%8.1E\n",MinProdNZ(State.m_current.m_g,State.m_current.m_z,n),MinProdNZ(State.m_current.m_t,State.m_current.m_s,n),MinProdNZ(State.m_current.m_w,State.m_current.m_v,m),MinProdNZ(State.m_current.m_p,State.m_current.m_q,m)));
CAp::Trace(StringFormat("max: GZ=%8.1E TS=%8.1E WV=%8.1E PQ=%8.1E\n",MaxProdNZ(State.m_current.m_g,State.m_current.m_z,n),MaxProdNZ(State.m_current.m_t,State.m_current.m_s,n),MaxProdNZ(State.m_current.m_w,State.m_current.m_v,m),MaxProdNZ(State.m_current.m_p,State.m_current.m_q,m)));
}
//--- Detailed output (all variables values, not suited for high-dimensional problems)
if(State.m_dodetailedtrace)
{
VIPMMultiply(State,State.m_current.m_x,State.m_current.m_y,State.m_tmphx,State.m_tmpax,State.m_tmpaty);
State.m_tmplaggrad=vector<double>::Zeros(n);
for(i=0; i<n; i++)
{
if(!State.m_isfrozen[i])
{
v=State.m_tmphx[i]+State.m_c[i]-State.m_tmpaty[i];
if(State.m_hasgz[i])
v-=State.m_current.m_z[i];
if(State.m_hasts[i])
v+=State.m_current.m_s[i];
State.m_tmplaggrad.Set(i,v);
}
}
CAp::Trace("--- printing raw data (prior to applying variable scales and Shifting by XOrigin) ------------------\n");
CAp::Trace("X (raw) = ");
CApServ::TraceVectorUnscaledUnshiftedAutopRec(State.m_current.m_x,n,State.m_scl,true,State.m_xorigin,true);
CAp::Trace("\n");
CAp::Trace("--- printing scaled data (after applying variable scales and Shifting by XOrigin) ------------------\n");
CAp::Trace("> reporting X,Lagrangian gradient\n");
CAp::Trace("Xnew = ");
CApServ::TraceVectorAutopRec(State.m_current.m_x,0,n);
CAp::Trace("\n");
CAp::Trace("Lag-grad = ");
CApServ::TraceVectorAutopRec(State.m_tmplaggrad,0,n);
CAp::Trace("\n");
CAp::Trace("--- printing new point -----------------------------------------------------------------------------\n");
CAp::Trace("> primal slacks and dual multipliers for box constraints\n");
CAp::Trace("G (L prim slck) = ");
CApServ::TraceVectorAutopRec(State.m_current.m_g,0,n);
CAp::Trace("\n");
CAp::Trace("Z (L dual mult) = ");
CApServ::TraceVectorAutopRec(State.m_current.m_z,0,n);
CAp::Trace("\n");
CAp::Trace("T (U prim slck) = ");
CApServ::TraceVectorAutopRec(State.m_current.m_t,0,n);
CAp::Trace("\n");
CAp::Trace("S (U dual mult) = ");
CApServ::TraceVectorAutopRec(State.m_current.m_s,0,n);
CAp::Trace("\n");
CAp::Trace("> primal slacks and dual multipliers for linear constraints,B/R stand for B<=Ax<=B+R\n");
CAp::Trace("Y (lag mult) = ");
CApServ::TraceVectorAutopRec(State.m_current.m_y,0,m);
CAp::Trace("\n");
CAp::Trace("W (B prim slck) = ");
CApServ::TraceVectorAutopRec(State.m_current.m_w,0,m);
CAp::Trace("\n");
CAp::Trace("V (B dual mult) = ");
CApServ::TraceVectorAutopRec(State.m_current.m_v,0,m);
CAp::Trace("\n");
CAp::Trace("P (R prim slck) = ");
CApServ::TraceVectorAutopRec(State.m_current.m_p,0,m);
CAp::Trace("\n");
CAp::Trace("Q (R dual mult) = ");
CApServ::TraceVectorAutopRec(State.m_current.m_q,0,m);
CAp::Trace("\n");
}
CAp::Trace("\n");
}
//+------------------------------------------------------------------+
//| Compute right - hand side for KKT system. |
//| INPUT PARAMETERS: |
//| State - IPM State |
//| V0 - current point(used to compute RHS) |
//| MuEstimate - estimate of Mu(can be zero) |
//| DirEstimate - estimate of delta's (can be zero) |
//| OUTPUT PARAMETERS: |
//| Rhs - RHS |
//+------------------------------------------------------------------+
void CVIPMSolver::RHSCompute(CVIPMState &State,
CVIPMVars &v0,
double muestimate,
CVIPMVars &direstimate,
CVIPMRightHandSide &rhs,
double reg)
{
//--- create variables
int n=State.m_n;
int m=State.m_mdense+State.m_msparse;
int i=0;
//--- Allocate
rhs.m_sigma.Resize(n);
rhs.m_nu.Resize(n);
rhs.m_tau.Resize(n);
rhs.m_gammaz.Resize(n);
rhs.m_gammas.Resize(n);
rhs.m_gammaw.Resize(m);
rhs.m_gammaq.Resize(m);
rhs.m_beta=vector<double>::Zeros(m);
rhs.m_rho=vector<double>::Zeros(m);
rhs.m_alpha=vector<double>::Zeros(m);
//--- Compute products H*x, A*x, A^T*y
//--- We compute these products in one location for the sake of simplicity.
VIPMMultiply(State,v0.m_x,v0.m_y,State.m_tmphx,State.m_tmpax,State.m_tmpaty);
//--- Compute right-hand side:
//--- Rho = b - A*x + w
//--- Nu = l - x + g
//--- Tau = u - x - t
//--- Alpha = r - w - p
//--- Sigma = c - A^T*y - z + s + (H+REG)*x
//--- Beta = y + q - v
for(i=0; i<m; i++)
{
rhs.m_rho.Set(i,State.m_b[i]-State.m_tmpax[i]-reg*v0.m_y[i]);
if(State.m_haswv[i])
{
//--- Inequality/range constraint
rhs.m_rho.Add(i,v0.m_w[i]);
}
else
{
//--- Equality constraint without slack variables, W[i]=0
if(!CAp::Assert(v0.m_w[i]==0.0,__FUNCTION__+": W[i]<>0 for linear equality constraint"))
return;
}
}
for(i=0; i<n; i++)
{
if(State.m_hasgz[i])
{
//--- Lower bound is present
rhs.m_nu.Set(i,State.m_bndl[i]-v0.m_x[i]+v0.m_g[i]-reg*v0.m_z[i]);
}
else
{
//--- Lower bound is absent, g.Set(i, 0
if(!CAp::Assert(v0.m_g[i]==0.0,__FUNCTION__+": G[i]<>0 for absent constraint"))
return;
rhs.m_nu.Set(i,0);
}
}
for(i=0; i<n; i++)
{
if(State.m_hasts[i])
{
//--- Upper bound is present
rhs.m_tau.Set(i,State.m_bndu[i]-v0.m_x[i]-v0.m_t[i]+reg*v0.m_s[i]);
}
else
{
//--- Upper bound is absent, t.Set(i, 0
if(!CAp::Assert(v0.m_t[i]==0.0,__FUNCTION__+": T[i]<>0 for absent constraint"))
return;
rhs.m_tau.Set(i,0);
}
}
for(i=0; i<m; i++)
{
if(State.m_haspq[i])
rhs.m_alpha.Set(i,State.m_r[i]-v0.m_w[i]-v0.m_p[i]+reg*v0.m_q[i]);
}
for(i=0; i<n; i++)
{
if(!State.m_isfrozen[i])
{
rhs.m_sigma.Set(i,State.m_c[i]-State.m_tmpaty[i]+State.m_tmphx[i]+reg*v0.m_x[i]);
if(State.m_hasgz[i])
rhs.m_sigma.Add(i,-v0.m_z[i]);
if(State.m_hasts[i])
rhs.m_sigma.Add(i,v0.m_s[i]);
}
else
rhs.m_sigma.Set(i,0);
}
for(i=0; i<m; i++)
{
if(State.m_haswv[i])
rhs.m_beta.Add(i,v0.m_y[i]-v0.m_v[i]+reg*v0.m_w[i]);
if(State.m_haspq[i])
rhs.m_beta.Add(i,v0.m_q[i]);
}
//--- Compute right-hand side:
//--- GammaZ = mu*inv(G)*e - z - inv(G)*DELTAG*deltaZ
//--- GammaW = mu*inv(V)*e - w - inv(V)*DELTAV*deltaW
//--- GammaS = mu*inv(T)*e - s - inv(T)*DELTAT*deltaS
//--- GammaQ = mu*inv(P)*e - q - inv(P)*DELTAP*deltaQ
for(i=0; i<n; i++)
{
if(State.m_hasgz[i])
{
if(!CAp::Assert(v0.m_g[i]>0.0,__FUNCTION__+": G[i]<=0"))
return;
rhs.m_gammaz.Set(i,muestimate/v0.m_g[i]-v0.m_z[i]-direstimate.m_g[i]*direstimate.m_z[i]/v0.m_g[i]);
}
else
{
if(!CAp::Assert(v0.m_g[i]==0.0,__FUNCTION__+": G[i]<>0 for absent constraint"))
return;
if(!CAp::Assert(v0.m_z[i]==0.0,__FUNCTION__+": Z[i]<>0 for absent constraint"))
return;
rhs.m_gammaz.Set(i,0);
}
}
for(i=0; i<m; i++)
{
if(State.m_haswv[i])
{
//--- Inequality/range constraint
if(!CAp::Assert(v0.m_v[i]>0.0,__FUNCTION__+": V[i]<=0"))
return;
rhs.m_gammaw.Set(i,muestimate/v0.m_v[i]-v0.m_w[i]-direstimate.m_v[i]*direstimate.m_w[i]/v0.m_v[i]);
}
else
{
//--- Equality constraint
if(!CAp::Assert(v0.m_v[i]==0.0,__FUNCTION__+": V[i]<>0 for equality constraint"))
return;
if(!CAp::Assert(v0.m_w[i]==0.0,__FUNCTION__+": W[i]<>0 for equality constraint"))
return;
rhs.m_gammaw.Set(i,0);
}
}
for(i=0; i<n; i++)
{
if(State.m_hasts[i])
{
//--- Upper bound is present
if(!CAp::Assert(v0.m_t[i]>0.0,__FUNCTION__+": T[i]<=0"))
return;
rhs.m_gammas.Set(i,muestimate/v0.m_t[i]-v0.m_s[i]-direstimate.m_t[i]*direstimate.m_s[i]/v0.m_t[i]);
}
else
{
//--- Upper bound is absent
if(!CAp::Assert(v0.m_t[i]==0.0,__FUNCTION__+": T[i]<>0 for absent constraint"))
return;
if(!CAp::Assert(v0.m_s[i]==0.0,__FUNCTION__+": S[i]<>0 for absent constraint"))
return;
rhs.m_gammas.Set(i,0);
}
}
for(i=0; i<m; i++)
{
if(State.m_haspq[i])
{
if(!CAp::Assert(v0.m_p[i]>0.0,__FUNCTION__+": P[i]<=0"))
return;
rhs.m_gammaq.Set(i,muestimate/v0.m_p[i]-v0.m_q[i]-direstimate.m_p[i]*direstimate.m_q[i]/v0.m_p[i]);
}
else
{
if(!CAp::Assert(v0.m_p[i]==0.0,__FUNCTION__+": P[i]<>0 for absent range"))
return;
if(!CAp::Assert(v0.m_q[i]==0.0,__FUNCTION__+": Q[i]<>0 for absent range"))
return;
rhs.m_gammaq.Set(i,0);
}
}
}
//+------------------------------------------------------------------+
//| Subtracts KKT*cand from already computed RHS. |
//| A pair of RHSCompute/RHSSubtract calls results in residual being |
//| loaded into the RHS structure. |
//| INPUT PARAMETERS: |
//| State - IPM State |
//| V0 - current point(used to compute RHS) |
//| MuEstimate - estimate of Mu(can be zero) |
//| DirEstimate - estimate of delta's (can be zero) |
//| ResidualFrom- whether we want to compute RHS or residual |
//| computed using VDCandidate |
//| VDCandidate - solution candidate |
//| OUTPUT PARAMETERS: |
//| Rhs - either RHS or residual RHS - KKT*Cand |
//+------------------------------------------------------------------+
void CVIPMSolver::RHSSubtract(CVIPMState &State,
CVIPMRightHandSide &rhs,
CVIPMVars &v0,
CVIPMVars &vdcandidate,
double reg)
{
//--- create variables
int n=State.m_n;
int m=State.m_mdense+State.m_msparse;
int i=0;
VIPMMultiply(State,vdcandidate.m_x,vdcandidate.m_y,State.m_tmphx,State.m_tmpax,State.m_tmpaty);
//--- Residual for Rho, Nu, Tau, Alpha, Sigma, Beta
for(i=0; i<m; i++)
{
if(State.m_haswv[i])
{
rhs.m_rho.Add(i,-(State.m_tmpax[i]-vdcandidate.m_w[i]+reg*vdcandidate.m_y[i]));
rhs.m_beta.Add(i,-(-vdcandidate.m_y[i]+vdcandidate.m_v[i]-reg*vdcandidate.m_w[i]));
rhs.m_gammaw.Add(i,-(v0.m_w[i]/v0.m_v[i]*vdcandidate.m_v[i]+vdcandidate.m_w[i]));
}
else
rhs.m_rho.Add(i,-(State.m_tmpax[i]+reg*vdcandidate.m_y[i]));
//---
if(State.m_haspq[i])
{
rhs.m_alpha.Add(i,-(vdcandidate.m_w[i]+vdcandidate.m_p[i]-reg*vdcandidate.m_q[i]));
rhs.m_beta.Add(i,vdcandidate.m_q[i]);
rhs.m_gammaq.Add(i,-(v0.m_q[i]/v0.m_p[i]*vdcandidate.m_p[i]+vdcandidate.m_q[i]));
}
}
for(i=0; i<n; i++)
{
//--- Residual for GammaZ, GammaW, GammaS, GammaQ
if(State.m_hasgz[i])
{
rhs.m_nu.Add(i,-(vdcandidate.m_x[i]-vdcandidate.m_g[i]+reg*vdcandidate.m_z[i]));
rhs.m_gammaz.Add(i,-(v0.m_z[i]/v0.m_g[i]*vdcandidate.m_g[i]+vdcandidate.m_z[i]));
}
if(State.m_hasts[i])
{
rhs.m_tau.Add(i,-(vdcandidate.m_x[i]+vdcandidate.m_t[i]-reg*vdcandidate.m_s[i]));
rhs.m_gammas.Add(i,-(v0.m_s[i]/v0.m_t[i]*vdcandidate.m_t[i]+vdcandidate.m_s[i]));
}
//---
if(!State.m_isfrozen[i])
{
rhs.m_sigma.Add(i,-(State.m_tmpaty[i]-State.m_tmphx[i]-reg*vdcandidate.m_x[i]));
if(State.m_hasgz[i])
rhs.m_sigma.Add(i,-vdcandidate.m_z[i]);
if(State.m_hasts[i])
rhs.m_sigma.Add(i,+ vdcandidate.m_s[i]);
}
}
}
//+------------------------------------------------------------------+
//| Computes sum of squared primal terms of RHS |
//| INPUT PARAMETERS: |
//| Rhs - RHS structure |
//| N, M - problem metrics |
//| RESULT: |
//| sum(sqr()) computed over primal terms(Rho, Nu, Tau, Alpha) |
//+------------------------------------------------------------------+
double CVIPMSolver::RHSPrimal2(CVIPMRightHandSide &rhs)
{
double result=rhs.m_rho.Dot(rhs.m_rho);
result+=rhs.m_nu.Dot(rhs.m_nu);
result+=rhs.m_tau.Dot(rhs.m_tau);
result+=rhs.m_alpha.Dot(rhs.m_alpha);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Computes sum of squared dual terms of RHS |
//| INPUT PARAMETERS: |
//| Rhs - RHS structure |
//| N, M - problem metrics |
//| RESULT: |
//| sum(sqr()) computed over dual terms(Sigma, Beta) |
//+------------------------------------------------------------------+
double CVIPMSolver::RHSDual2(CVIPMRightHandSide &rhs)
{
double result=rhs.m_sigma.Dot(rhs.m_sigma);
result+=rhs.m_beta.Dot(rhs.m_beta);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Computes inf - norm of primal terms of RHS |
//| INPUT PARAMETERS: |
//| Rhs - RHS structure |
//| N, M - problem metrics |
//| RESULT: |
//| max(abs()) computed over primal terms(Rho, Nu, Tau, Alpha) |
//+------------------------------------------------------------------+
double CVIPMSolver::RHSPrimalInf(CVIPMRightHandSide &rhs)
{
double result=MathMax((rhs.m_rho.Abs()+0).Max(),(rhs.m_nu.Abs()+0).Max());
result=MathMax(result,(rhs.m_tau.Abs()+0).Max());
result=MathMax(result,(rhs.m_alpha.Abs()+0).Max());
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Computes inf - norm of dual terms of RHS |
//| INPUT PARAMETERS: |
//| Rhs - RHS structure |
//| N, M - problem metrics |
//| RESULT: |
//| max(abs()) computed over dual terms(Sigma, Beta) |
//+------------------------------------------------------------------+
double CVIPMSolver::RHSDualInf(CVIPMRightHandSide &rhs)
{
double result=MathMax((rhs.m_sigma.Abs()+0).Max(),(rhs.m_beta.Abs()+0).Max());
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Computes maximum over complementarity slackness terms of RHS |
//| INPUT PARAMETERS: |
//| Rhs - RHS structure |
//| N, M - problem metrics |
//| RESULT: |
//| max(abs()) computed over complementarity terms |
//| (GammaZ, GammaS, GammaW, GammaQ) |
//+------------------------------------------------------------------+
double CVIPMSolver::RHSCompl2(CVIPMRightHandSide &rhs)
{
double result=rhs.m_gammaz.Dot(rhs.m_gammaz);
result+=rhs.m_gammas.Dot(rhs.m_gammas);
result+=rhs.m_gammaw.Dot(rhs.m_gammaw);
result+=rhs.m_gammaq.Dot(rhs.m_gammaq);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Computes minimum nonzero value of the vector. Returns 0 if all |
//| components are nonpositive. |
//| INPUT PARAMETERS: |
//| X - vector |
//| N - length |
//+------------------------------------------------------------------+
double CVIPMSolver::MinNZ(CRowDouble &x,
int n)
{
//--- create variables
double result=0;
bool nz=false;
for(int i=0; i<n; i++)
{
if(x[i]>0.0)
{
if(!nz)
{
result=x[i];
nz=true;
}
else
result=MathMin(result,x[i]);
}
}
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Computes minimum product of nonzero components. |
//| Returns 0 if all components are nonpositive. |
//| INPUT PARAMETERS: |
//| X - vector |
//| Y - vector |
//| N - length |
//+------------------------------------------------------------------+
double CVIPMSolver::MinProdNZ(CRowDouble &x,
CRowDouble &y,
int n)
{
//--- create variables
double result=0;
bool nz=false;
for(int i=0; i<n; i++)
{
if(x[i]>0.0 && y[i]>0.0)
{
if(!nz)
{
result=x[i]*y[i];
nz=true;
}
else
result=MathMin(result,x[i]*y[i]);
}
}
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Computes maximum product of nonzero components. |
//| Returns 0 if all components are nonpositive. |
//| INPUT PARAMETERS: |
//| X - vector |
//| Y - vector |
//| N - length |
//+------------------------------------------------------------------+
double CVIPMSolver::MaxProdNZ(CRowDouble &x,
CRowDouble &y,
int n)
{
//--- create variables
double result=0;
bool nz=false;
for(int i=0; i<n; i++)
{
if(x[i]>0.0 && y[i]>0.0)
{
if(!nz)
{
result=x[i]*y[i];
nz=true;
}
else
result=MathMax(result,x[i]*y[i]);
}
}
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This structure describes convex quadratic model of the form: |
//|f(x)=0.5*(Alpha*x'*A*x+Tau*x'*D*x)+0.5*Theta*(Q*x-r)'*(Q*x-r)+b'*x|
//| where: |
//| * Alpha>=0, Tau>=0, Theta>=0, Alpha+Tau>0. |
//| * A is NxN matrix, Q is NxK matrix (N>=1, K>=0), b is Nx1 |
//| vector, D is NxN diagonal matrix. |
//| *"main" quadratic term Alpha*A+Lambda*D is symmetric positive |
//| definite |
//| Structure may contain optional equality constraints of the form |
//| x[i]=x0[i], in this case functions provided by this unit |
//| calculate Newton step subject to these equality constraints. |
//+------------------------------------------------------------------+
struct CConvexQuadraticModel
{
int m_ecakind;
int m_k;
int m_n;
int m_nfree;
double m_alpha;
double m_ec;
double m_tau;
double m_theta;
double m_tk0;
double m_tq0;
bool m_activeset[];
bool m_isactivesetchanged;
bool m_islineartermchanged;
bool m_ismaintermchanged;
bool m_issecondarytermchanged;
CRowDouble m_b;
CRowDouble m_d;
CRowDouble m_eb;
CRowDouble m_ecadiag;
CRowDouble m_r;
CRowDouble m_tb;
CRowDouble m_tk1;
CRowDouble m_tmp0;
CRowDouble m_tmp1;
CRowDouble m_tmpg;
CRowDouble m_tq1;
CRowDouble m_tq2diag;
CRowDouble m_txc;
CRowDouble m_xc;
CMatrixDouble m_a;
CMatrixDouble m_ecadense;
CMatrixDouble m_eccm;
CMatrixDouble m_eq;
CMatrixDouble m_q;
CMatrixDouble m_tk2;
CMatrixDouble m_tmp2;
CMatrixDouble m_tq2dense;
//--- constructor / destructor
CConvexQuadraticModel(void);
~CConvexQuadraticModel(void) {}
//---
void Copy(const CConvexQuadraticModel &obj);
//--- overloading
void operator=(const CConvexQuadraticModel &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CConvexQuadraticModel::CConvexQuadraticModel(void)
{
m_ecakind=0;
m_k=0;
m_n=0;
m_nfree=0;
m_alpha=0;
m_ec=0;
m_tau=0;
m_theta=0;
m_tk0=0;
m_tq0=0;
m_isactivesetchanged=false;
m_islineartermchanged=false;
m_ismaintermchanged=false;
m_issecondarytermchanged=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CConvexQuadraticModel::Copy(const CConvexQuadraticModel &obj)
{
m_ecakind=obj.m_ecakind;
m_k=obj.m_k;
m_n=obj.m_n;
m_nfree=obj.m_nfree;
m_alpha=obj.m_alpha;
m_ec=obj.m_ec;
m_tau=obj.m_tau;
m_theta=obj.m_theta;
m_tk0=obj.m_tk0;
m_tq0=obj.m_tq0;
ArrayCopy(m_activeset,obj.m_activeset);
m_isactivesetchanged=obj.m_isactivesetchanged;
m_islineartermchanged=obj.m_islineartermchanged;
m_ismaintermchanged=obj.m_ismaintermchanged;
m_issecondarytermchanged=obj.m_issecondarytermchanged;
m_b=obj.m_b;
m_d=obj.m_d;
m_eb=obj.m_eb;
m_ecadiag=obj.m_ecadiag;
m_r=obj.m_r;
m_tb=obj.m_tb;
m_tk1=obj.m_tk1;
m_tmp0=obj.m_tmp0;
m_tmp1=obj.m_tmp1;
m_tmpg=obj.m_tmpg;
m_tq1=obj.m_tq1;
m_tq2diag=obj.m_tq2diag;
m_txc=obj.m_txc;
m_xc=obj.m_xc;
m_a=obj.m_a;
m_ecadense=obj.m_ecadense;
m_eccm=obj.m_eccm;
m_eq=obj.m_eq;
m_q=obj.m_q;
m_tk2=obj.m_tk2;
m_tmp2=obj.m_tmp2;
m_tq2dense=obj.m_tq2dense;
}
//+------------------------------------------------------------------+
//| Convex Quadratic Models |
//+------------------------------------------------------------------+
class CCQModels
{
public:
static const int m_newtonrefinementits;
static void CQMInit(int n,CConvexQuadraticModel &s);
static void CQMSetA(CConvexQuadraticModel &s,CMatrixDouble &a,bool IsUpper,double alpha);
static void CQMGetA(CConvexQuadraticModel &s,CMatrixDouble &a);
static void CQMRewriteDenseDiagonal(CConvexQuadraticModel &s,CRowDouble &z);
static void CQMSetD(CConvexQuadraticModel &s,CRowDouble &d,double tau);
static void CQMDropA(CConvexQuadraticModel &s);
static void CQMSetB(CConvexQuadraticModel &s,CRowDouble &b);
static void CQMSetQ(CConvexQuadraticModel &s,CMatrixDouble &q,CRowDouble &r,int k,double theta);
static void CQMSetActiveSet(CConvexQuadraticModel &s,CRowDouble &x,bool &activeset[]);
static double CQMEval(CConvexQuadraticModel &s,CRowDouble &x);
static void CQMEvalX(CConvexQuadraticModel &s,CRowDouble &x,double &r,double &noise);
static void CQMGradUnconstrained(CConvexQuadraticModel &s,CRowDouble &x,CRowDouble &g);
static double CQMXTADX2(CConvexQuadraticModel &s,CRowDouble &x,CRowDouble &tmp);
static void CQMADX(CConvexQuadraticModel &s,CRowDouble &x,CRowDouble &y);
static bool CQMConstrainedOptimum(CConvexQuadraticModel &s,CRowDouble &x);
static void CQMScaleVector(CConvexQuadraticModel &s,CRowDouble &x);
static void CQMGetDiagA(CConvexQuadraticModel &s,CRowDouble &x);
static double CQMDebugConstrainedEvalT(CConvexQuadraticModel &s,CRowDouble &x);
static double CQMDebugConstrainedEvalE(CConvexQuadraticModel &s,CRowDouble &x);
private:
static bool CQMRebuild(CConvexQuadraticModel &s);
static void CQMSolveEA(CConvexQuadraticModel &s,CRowDouble &x,CRowDouble &tmp);
};
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
const int CCQModels::m_newtonrefinementits=3;
//+------------------------------------------------------------------+
//| This subroutine is used to initialize CQM. By default, empty NxN |
//| model is generated, with Alpha=Lambda=Theta=0.0 and zero b. |
//| Previously allocated buffer variables are reused as much as |
//| possible. |
//+------------------------------------------------------------------+
void CCQModels::CQMInit(int n,CConvexQuadraticModel &s)
{
//--- initialization
s.m_n=n;
s.m_k=0;
s.m_nfree=n;
s.m_ecakind=-1;
s.m_alpha=0.0;
s.m_tau=0.0;
s.m_theta=0.0;
s.m_ismaintermchanged=true;
s.m_issecondarytermchanged=true;
s.m_islineartermchanged=true;
s.m_isactivesetchanged=true;
ArrayResize(s.m_activeset,n);
s.m_xc=vector<double>::Zeros(n);
s.m_eb=vector<double>::Zeros(n);
s.m_tq1=vector<double>::Zeros(n);
s.m_txc=vector<double>::Zeros(n);
s.m_tb=vector<double>::Zeros(n);
s.m_b=vector<double>::Zeros(n);
s.m_tk1=vector<double>::Zeros(n);
ArrayInitialize(s.m_activeset,false);
}
//+------------------------------------------------------------------+
//| This subroutine changes main quadratic term of the model. |
//| INPUT PARAMETERS: |
//| S - model |
//| A - NxN matrix, only upper or lower triangle is |
//| referenced |
//| IsUpper - True, when matrix is stored in upper triangle |
//| Alpha - multiplier; when Alpha=0, A is not referenced at |
//| all |
//+------------------------------------------------------------------+
void CCQModels::CQMSetA(CConvexQuadraticModel &s,
CMatrixDouble &a,
bool IsUpper,
double alpha)
{
//--- create variables
int i=0;
int j=0;
double v=0;
//--- check
if(!CAp::Assert(MathIsValidNumber(alpha) && alpha>=0.0,__FUNCTION__+": Alpha<0 or is not finite number"))
return;
if(!CAp::Assert(alpha==0.0 || CApServ::IsFiniteRTrMatrix(a,s.m_n,IsUpper),__FUNCTION__+": A is not finite NxN matrix"))
return;
s.m_alpha=alpha;
if(alpha>0.0)
{
s.m_a.Resize(s.m_n,s.m_n);
s.m_ecadense.Resize(s.m_n,s.m_n);
s.m_tq2dense.Resize(s.m_n,s.m_n);
for(i=0; i<s.m_n; i++)
{
for(j=i; j<s.m_n; j++)
{
if(IsUpper)
v=a.Get(i,j);
else
v=a.Get(j,i);
s.m_a.Set(i,j,v);
s.m_a.Set(j,i,v);
}
}
}
s.m_ismaintermchanged=true;
}
//+------------------------------------------------------------------+
//| This subroutine changes main quadratic term of the model. |
//| INPUT PARAMETERS: |
//| S - model |
//| A - possibly preallocated buffer |
//| OUTPUT PARAMETERS: |
//| A - NxN matrix, full matrix is returned. Zero matrix is|
//| returned if model is empty. |
//+------------------------------------------------------------------+
void CCQModels::CQMGetA(CConvexQuadraticModel &s,
CMatrixDouble &a)
{
//--- create variables
double v=0;
//--- copy
if(s.m_alpha>0.0)
{
v=s.m_alpha;
a=s.m_a*v+0;
}
else
{
a=matrix<double>::Zeros(s.m_n,s.m_n);
}
}
//+------------------------------------------------------------------+
//| This subroutine rewrites diagonal of the main quadratic term of |
//| the model (dense A) by vector Z/Alpha (current value of the Alpha|
//| coefficient is used). |
//| IMPORTANT: in case model has no dense quadratic term, this |
//| function allocates N*N dense matrix of zeros, and |
//| fills its diagonal by non-zero values. |
//| INPUT PARAMETERS: |
//| S - model |
//| Z - new diagonal, array[N] |
//+------------------------------------------------------------------+
void CCQModels::CQMRewriteDenseDiagonal(CConvexQuadraticModel &s,
CRowDouble &z)
{
if(s.m_alpha==0.0)
{
s.m_a=matrix<double>::Zeros(s.m_n,s.m_n);
s.m_ecadense.Resize(s.m_n,s.m_n);
s.m_tq2dense.Resize(s.m_n,s.m_n);
s.m_alpha=1.0;
}
vector<double> diag=z/s.m_alpha;
diag.Resize(s.m_n);
s.m_a.Diag(diag);
s.m_ismaintermchanged=true;
}
//+------------------------------------------------------------------+
//| This subroutine changes diagonal quadratic term of the model. |
//| INPUT PARAMETERS: |
//| S - model |
//| D - array[N], semidefinite diagonal matrix |
//| Tau - multiplier; when Tau=0, D is not referenced at all |
//+------------------------------------------------------------------+
void CCQModels::CQMSetD(CConvexQuadraticModel &s,CRowDouble &d,double tau)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(tau) && tau>=0.0,__FUNCTION__+": Tau<0 or is not finite number"))
return;
if(!CAp::Assert(tau==0.0 || CApServ::IsFiniteVector(d,s.m_n),__FUNCTION__+": D is not finite Nx1 vector"))
return;
s.m_tau=tau;
if(tau>0.0)
{
s.m_d=d;
s.m_d.Resize(s.m_n);
//--- check
if(!CAp::Assert(s.m_d.Min()>=0.0,__FUNCTION__+": D[i]<0"))
return;
s.m_ecadiag=vector<double>::Zeros(s.m_n);
s.m_tq2diag=vector<double>::Zeros(s.m_n);
}
s.m_ismaintermchanged=true;
}
//+------------------------------------------------------------------+
//| This subroutine drops main quadratic term A from the model. It is|
//| same as call to CQMSetA() with zero A, but gives better |
//| performance because algorithm knows that matrix is zero and can |
//| optimize subsequent calculations. |
//| INPUT PARAMETERS: |
//| S - model |
//+------------------------------------------------------------------+
void CCQModels::CQMDropA(CConvexQuadraticModel &s)
{
s.m_alpha=0.0;
s.m_ismaintermchanged=true;
}
//+------------------------------------------------------------------+
//| This subroutine changes linear term of the model |
//+------------------------------------------------------------------+
void CCQModels::CQMSetB(CConvexQuadraticModel &s,CRowDouble &b)
{
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(b,s.m_n),__FUNCTION__+": B is not finite vector"))
return;
s.m_b=b;
s.m_b.Resize(s.m_n);
s.m_islineartermchanged=true;
}
//+------------------------------------------------------------------+
//| This subroutine changes linear term of the model |
//+------------------------------------------------------------------+
void CCQModels::CQMSetQ(CConvexQuadraticModel &s,
CMatrixDouble &q,
CRowDouble &r,
int k,
double theta)
{
//--- check
if(!CAp::Assert(k>=0,__FUNCTION__+": K<0"))
return;
if(!CAp::Assert(k==0 || theta==0.0 || CApServ::IsFiniteMatrix(q,k,s.m_n),__FUNCTION__+": Q is not finite matrix"))
return;
if(!CAp::Assert(k==0 || theta==0.0 || CApServ::IsFiniteVector(r,k),__FUNCTION__+": R is not finite vector"))
return;
if(!CAp::Assert(MathIsValidNumber(theta) && theta>=0.0,__FUNCTION__+": Theta<0 or is not finite number"))
return;
//--- degenerate case: K=0 or Theta=0
if(k==0 || theta==0.0)
{
s.m_k=0;
s.m_theta=0;
s.m_issecondarytermchanged=true;
return;
}
//--- General case: both Theta>0 and K>0
s.m_k=k;
s.m_theta=theta;
s.m_q=q;
s.m_r=r;
s.m_q.Resize(k,s.m_n);
s.m_r.Resize(k);
CApServ::RMatrixSetLengthAtLeast(s.m_eq,k,s.m_n);
CApServ::RMatrixSetLengthAtLeast(s.m_eccm,k,k);
CApServ::RMatrixSetLengthAtLeast(s.m_tk2,k,s.m_n);
s.m_issecondarytermchanged=true;
}
//+------------------------------------------------------------------+
//| This subroutine changes active set |
//| INPUT PARAMETERS: |
//| S - model |
//| X - array[N], constraint values |
//| ActiveSet- array[N], active set. If ActiveSet[I] = True, |
//| then I-th variables is constrained to X[I]. |
//+------------------------------------------------------------------+
void CCQModels::CQMSetActiveSet(CConvexQuadraticModel &s,CRowDouble &x,
bool &activeset[])
{
//--- check
if(!CAp::Assert(x.Size()>=s.m_n,__FUNCTION__+": Length(X)<N"))
return;
if(!CAp::Assert(ArraySize(activeset)>=s.m_n,__FUNCTION__+": Length(ActiveSet)<N"))
return;
for(int i=0; i<s.m_n; i++)
{
s.m_isactivesetchanged=s.m_isactivesetchanged || (s.m_activeset[i] && !activeset[i]);
s.m_isactivesetchanged=s.m_isactivesetchanged || (activeset[i] && !s.m_activeset[i]);
s.m_activeset[i]=activeset[i];
if(activeset[i])
{
//--- check
if(!CAp::Assert(MathIsValidNumber(x[i]),__FUNCTION__+": X[] contains infinite constraints"))
return;
s.m_isactivesetchanged=s.m_isactivesetchanged || (s.m_xc[i]!=x[i]);
s.m_xc.Set(i,x[i]);
}
}
}
//+------------------------------------------------------------------+
//| This subroutine evaluates model at X. Active constraints are |
//| ignored. |
//+------------------------------------------------------------------+
double CCQModels::CQMEval(CConvexQuadraticModel &s,CRowDouble &X)
{
//--- create variables
double result=0;
int n=s.m_n;
double v=0;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(X,n),__FUNCTION__+": X is not finite vector"))
return(result);
//--- main quadratic term
CRowDouble x=X;
x.Resize(n);
if(s.m_alpha>0.0)
result=x.Dot(x.MatMul(s.m_a)*(s.m_alpha*0.5));
if(s.m_tau>0.0)
result+=s.m_tau*s.m_d.Dot(x*x/2.0);
//--- secondary quadratic term
if(s.m_theta>0.0)
{
for(int i=0; i<s.m_k; i++)
{
v=x.Dot(s.m_q[i]+0);
result+=0.5*s.m_theta*CMath::Sqr(v-s.m_r[i]);
}
}
//--- linear term
result+=x.Dot(s.m_b);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This subroutine evaluates model at X. Active constraints are |
//| ignored. |
//| It returns: |
//| R - model value |
//| Noise - estimate of the numerical noise in data |
//+------------------------------------------------------------------+
void CCQModels::CQMEvalX(CConvexQuadraticModel &s,
CRowDouble &x,
double &r,
double &noise)
{
//--- create variables
int n=s.m_n;
int i=0;
int j=0;
double v=0;
double v2=0;
double mxq=0;
double eps=2*CMath::m_machineepsilon;
//--- initialization
r=0;
noise=0;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X is not finite vector"))
return;
//--- Main quadratic term.
//--- Noise from the main quadratic term is equal to the
//--- maximum summand in the term.
if(s.m_alpha>0.0)
{
for(i=0; i<n; i++)
for(j=0; j<n; j++)
{
v=s.m_alpha*0.5*x[i]*s.m_a.Get(i,j)*x[j];
r+=v;
noise=MathMax(noise,eps*MathAbs(v));
}
}
if(s.m_tau>0.0)
{
for(i=0; i<n; i++)
{
v=0.5*CMath::Sqr(x[i])*s.m_tau*s.m_d[i];
r+=v;
noise=MathMax(noise,eps*MathAbs(v));
}
}
//--- secondary quadratic term
//--- Noise from the secondary quadratic term is estimated as follows:
//--- * noise in qi*x-r[i] is estimated as
//--- Eps*MXQ = Eps*max(|r[i]|, |q[i,j]*x[j]|)
//--- * noise in (qi*x-r[i])^2 is estimated as
//--- NOISE = (|qi*x-r[i]|+Eps*MXQ)^2-(|qi*x-r[i]|)^2
//--- = Eps*MXQ*(2*|qi*x-r[i]|+Eps*MXQ)
if(s.m_theta>0.0)
{
for(i=0; i<s.m_k; i++)
{
v=0.0;
mxq=MathAbs(s.m_r[i]);
for(j=0; j<n; j++)
{
v2=s.m_q.Get(i,j)*x[j];
v+=v2;
mxq=MathMax(mxq,MathAbs(v2));
}
r+=0.5*s.m_theta*CMath::Sqr(v-s.m_r[i]);
noise=MathMax(noise,eps*mxq*(2*MathAbs(v-s.m_r[i])+eps*mxq));
}
}
//--- linear term
for(i=0; i<s.m_n; i++)
{
r+=x[i]
*s.m_b[i];
noise=MathMax(noise,eps*MathAbs(x[i]*s.m_b[i]));
}
//--- Final update of the noise
noise=n*noise;
}
//+------------------------------------------------------------------+
//| This subroutine evaluates gradient of the model; active |
//| constraints are ignored. |
//| INPUT PARAMETERS: |
//| S - convex model |
//| X - point, array[N] |
//| G - possibly preallocated buffer; resized, if too small|
//+------------------------------------------------------------------+
void CCQModels::CQMGradUnconstrained(CConvexQuadraticModel &s,
CRowDouble &x,
CRowDouble &g)
{
//--- create variables
int n=s.m_n;
int i=0;
double v=0;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X is not finite vector"))
return;
g=vector<double>::Zeros(n);
//--- main quadratic term
if(s.m_alpha>0.0)
{
for(i=0; i<n; i++)
{
v=x.Dot(s.m_a[i]*s.m_alpha);
g.Add(i,v);
}
}
if(s.m_tau>0.0)
g+=x*s.m_d*s.m_tau;
//--- secondary quadratic term
if(s.m_theta>0.0)
{
for(i=0; i<s.m_k; i++)
{
v=x.Dot(s.m_q[i]+0);
v=s.m_theta*(v-s.m_r[i]);
g+=s.m_q[i]*v;
}
}
//--- linear term
g+=s.m_b;
}
//+------------------------------------------------------------------+
//| This subroutine evaluates x'*(0.5*alpha*A+tau*D)*x |
//| NOTE: Tmp[] must be preallocated array whose length is at least N|
//+------------------------------------------------------------------+
double CCQModels::CQMXTADX2(CConvexQuadraticModel &s,
CRowDouble &x,
CRowDouble &tmp)
{
//--- create variables
double result=0;
int n=s.m_n;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X is not finite vector"))
return(0.0);
if(!CAp::Assert(tmp.Size()>=n,__FUNCTION__+": Length(Tmp)<N"))
return(0.0);
//--- main quadratic term
if(s.m_alpha>0.0)
result+=s.m_alpha*0.5*CAblas::RMatrixSyvMVect(n,s.m_a,0,0,true,x,0,tmp);
if(s.m_tau>0.0)
{
result+=s.m_d.Dot(x*x/2.0)*s.m_tau;
}
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This subroutine evaluates(0.5 * alpha * A + tau*D)*x |
//| Y is automatically resized if needed |
//+------------------------------------------------------------------+
void CCQModels::CQMADX(CConvexQuadraticModel &s,
CRowDouble &x,
CRowDouble &y)
{
int n=s.m_n;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X is not finite vector"))
return;
y=vector<double>::Zeros(n);
//--- main quadratic term
if(s.m_alpha>0.0)
CAblas::RMatrixSymVect(n,s.m_alpha,s.m_a,0,0,true,x,0,1.0,y,0);
if(s.m_tau>0.0)
y+=x*s.m_d*s.m_tau;
}
//+------------------------------------------------------------------+
//| This subroutine finds optimum of the model. It returns False on |
//| failure (indefinite / semidefinite matrix). Optimum is found |
//| subject to active constraints. |
//| INPUT PARAMETERS: |
//| S - model |
//| X - possibly preallocated buffer; automatically |
//| resized, if too small enough. |
//+------------------------------------------------------------------+
bool CCQModels::CQMConstrainedOptimum(CConvexQuadraticModel &s,
CRowDouble &x)
{
//--- create variables
int n=0;
int nfree=0;
int k=0;
int i=0;
double v=0;
int cidx0=0;
int itidx=0;
int i_=0;
//--- Rebuild internal structures
if(!CQMRebuild(s))
return(false);
n=s.m_n;
k=s.m_k;
nfree=s.m_nfree;
//--- Calculate initial point for the iterative refinement:
//--- * free components are set to zero
//--- * constrained components are set to their constrained values
x=vector<double>::Zeros(n);
for(i=0; i<n; i++)
{
if(s.m_activeset[i])
x.Set(i,s.m_xc[i]);
}
//--- Iterative refinement.
//--- In an ideal world without numerical errors it would be enough
//--- to make just one Newton step from initial point:
//--- x_new = -H^(-1)*grad(x=0)
//--- However, roundoff errors can significantly deteriorate quality
//--- of the solution. So we have to recalculate gradient and to
//--- perform Newton steps several times.
//--- Below we perform fixed number of Newton iterations.
for(itidx=0; itidx<m_newtonrefinementits; itidx++)
{
//--- Calculate gradient at the current point.
//--- Move free components of the gradient in the beginning.
CQMGradUnconstrained(s,x,s.m_tmpg);
cidx0=0;
for(i=0; i<n; i++)
{
if(!s.m_activeset[i])
{
s.m_tmpg.Set(cidx0,s.m_tmpg[i]);
cidx0++;
}
}
//--- Free components of the extrema are calculated in the first NFree elements of TXC.
//--- First, we have to calculate original Newton step, without rank-K perturbations
for(i_=0; i_<nfree; i_++)
s.m_txc.Set(i_,-s.m_tmpg[i_]);
CQMSolveEA(s,s.m_txc,s.m_tmp0);
//--- Then, we account for rank-K correction.
//--- Woodbury matrix identity is used.
if(s.m_k>0 && s.m_theta>0.0)
{
s.m_tmp0=vector<double>::Zeros(MathMax(nfree,k));
s.m_tmp1=vector<double>::Zeros(MathMax(nfree,k));
for(i_=0; i_<nfree; i_++)
s.m_tmp1.Set(i_,-s.m_tmpg[i_]);
CQMSolveEA(s,s.m_tmp1,s.m_tmp0);
for(i=0; i<k; i++)
{
v=0.0;
for(i_=0; i_<nfree; i_++)
v+=s.m_eq.Get(i,i_)*s.m_tmp1[i_];
s.m_tmp0.Set(i,v);
}
CFbls::FblsCholeskySolve(s.m_eccm,1.0,k,true,s.m_tmp0,s.m_tmp1);
for(i=0; i<nfree; i++)
s.m_tmp1.Set(i,0.0);
for(i=0; i<k; i++)
{
v=s.m_tmp0[i];
for(i_=0; i_<nfree; i_++)
s.m_tmp1.Add(i_,v*s.m_eq.Get(i,i_));
}
CQMSolveEA(s,s.m_tmp1,s.m_tmp0);
for(i_=0; i_<nfree; i_++)
s.m_txc.Add(i_,-s.m_tmp1[i_]);
}
//--- Unpack components from TXC into X. We pass through all
//--- free components of X and add our step.
cidx0=0;
for(i=0; i<n; i++)
{
if(!s.m_activeset[i])
{
x.Add(i,s.m_txc[cidx0]);
cidx0++;
}
}
}
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| This function scales vector by multiplying it by inverse of the|
//| diagonal of the Hessian matrix. It should be used to accelerate |
//| steepest descent phase of the QP m_solver. |
//| Although it is called "scale-grad", it can be called for any |
//| vector, whether it is gradient, anti-gradient, or just some |
//| vector. |
//| This function does NOT takes into account current set of |
//| constraints, it just performs matrix-vector multiplication |
//| without taking into account constraints. |
//| INPUT PARAMETERS: |
//| S - model |
//| X - vector to scale |
//| OUTPUT PARAMETERS: |
//| X - scaled vector |
//| NOTE: when called for non-SPD matrices, it silently skips |
//| components of X which correspond to zero or negative |
//| diagonal elements. |
//| NOTE: this function uses diagonals of A and D; it ignores |
//| Q-rank-K term of the quadratic model. |
//+------------------------------------------------------------------+
void CCQModels::CQMScaleVector(CConvexQuadraticModel &s,CRowDouble &x)
{
//--- create variables
int n=s.m_n;
double v=0;
for(int i=0; i<n; i++)
{
v=0.0;
if(s.m_alpha>0.0)
v+=s.m_a.Get(i,i);
if(s.m_tau>0.0)
v+=s.m_d[i];
if(v>0.0)
x.Mul(i,1.0/v);
}
}
//+------------------------------------------------------------------+
//| This function returns diagonal of the A - term. |
//| INPUT PARAMETERS: |
//| S - model |
//| OUTPUT PARAMETERS: |
//| D - diagonal of the A( or zero) |
//+------------------------------------------------------------------+
void CCQModels::CQMGetDiagA(CConvexQuadraticModel &s,CRowDouble &x)
{
int n=s.m_n;
if(s.m_alpha>0)
x=s.m_a.Diag(0)+0;
else
x=vector<double>::Zeros(n);
}
//+------------------------------------------------------------------+
//| This subroutine calls CQMRebuild() and evaluates model at X |
//| subject to active constraints. |
//| It is intended for debug purposes only, because it evaluates |
//| model by means of temporaries, which were calculated by |
//| CQMRebuild(). The only purpose of this function is to check |
//| correctness of CQMRebuild() by comparing results of this function|
//| with ones obtained by CQMEval(), which is used as reference point|
//| The idea is that significant deviation in results of these two |
//| functions is evidence of some error in the CQMRebuild(). |
//| NOTE: suffix T denotes that temporaries marked by T-prefix are |
//| used. There is one more variant of this function, which |
//| uses "effective" model built by CQMRebuild(). |
//| NOTE2: in case CQMRebuild() fails(due to model non-convexity), |
//| this function returns NAN. |
//+------------------------------------------------------------------+
double CCQModels::CQMDebugConstrainedEvalT(CConvexQuadraticModel &s,
CRowDouble &x)
{
//--- create variables
double result=0.0;
int n=s.m_n;
int nfree=0;
int i=0;
int j=0;
double v=0;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X is not finite vector"))
return(result);
if(!CQMRebuild(s))
return(AL_NaN);
nfree=s.m_nfree;
//--- Reorder variables
j=0;
for(i=0; i<n; i++)
{
if(!s.m_activeset[i])
{
//--- check
if(!CAp::Assert(j<nfree,__FUNCTION__+": internal error"))
return(AL_NaN);
s.m_txc.Set(j,x[i]);
j++;
}
}
//--- TQ2, TQ1, TQ0
if(s.m_alpha>0.0)
{
//--- Dense TQ2
for(i=0; i<nfree; i++)
for(j=0; j<nfree; j++)
result+=0.5*s.m_txc[i]*s.m_tq2dense.Get(i,j)*s.m_txc[j];
}
else
{
//--- Diagonal TQ2
for(i=0; i<nfree; i++)
result+=0.5*s.m_tq2diag[i]*CMath::Sqr(s.m_txc[i]);
}
result+=s.m_tq1.Dot(s.m_txc);
result=result+s.m_tq0;
//--- TK2, TK1, TK0
if(s.m_k>0 && s.m_theta>0.0)
{
for(i=0; i<s.m_k; i++)
{
v=CAblasF::RDotVR(nfree,s.m_txc,s.m_tk2,i);
result=result+0.5*CMath::Sqr(v);
}
result+=s.m_tk1.Dot(s.m_txc);
result=result+s.m_tk0;
}
//--- TB (Bf and Bc parts)
result+=s.m_tb.Dot(s.m_txc);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This subroutine calls CQMRebuild() and evaluates model at X |
//| subject to active constraints. |
//| It is intended for debug purposes only, because it evaluates |
//| model by means of "effective" matrices built by CQMRebuild(). The|
//| only purpose of this function is to check correctness of |
//| CQMRebuild() by comparing results of this function with ones |
//| obtained by CQMEval(), which is used as reference point. The idea|
//| is that significant deviation in results of these two functions |
//| is evidence of some error in the CQMRebuild(). |
//| NOTE: suffix E denotes that effective matrices. There is one more|
//| variant of this function, which uses temporary matrices |
//| built by CQMRebuild(). |
//| NOTE2: in case CQMRebuild() fails(due to model non-convexity), |
//| this function returns NAN. |
//+------------------------------------------------------------------+
double CCQModels::CQMDebugConstrainedEvalE(CConvexQuadraticModel &s,
CRowDouble &x)
{
//--- create variables
double result=0;
int n=s.m_n;
int nfree=0;
int i=0;
int j=0;
double v=0;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X is not finite vector"))
return(AL_NaN);
if(!CQMRebuild(s))
return(AL_NaN);
nfree=s.m_nfree;
//--- Reorder variables
j=0;
for(i=0; i<n; i++)
{
if(!s.m_activeset[i])
{
//--- check
if(!CAp::Assert(j<nfree,__FUNCTION__+": internal error"))
return(AL_NaN);
s.m_txc.Set(j,x[i]);
j++;
}
}
//--- ECA
//--- check
if(!CAp::Assert(s.m_ecakind==0 || s.m_ecakind==1 || (s.m_ecakind==-1 && nfree==0),__FUNCTION__+": unexpected ECAKind"))
return(AL_NaN);
if(s.m_ecakind==0)
{
//--- Dense ECA
for(i=0; i<nfree; i++)
{
v=0.0;
for(j=i; j<nfree; j++)
v+= s.m_ecadense.Get(i,j)*s.m_txc[j];
result+=0.5*CMath::Sqr(v);
}
}
if(s.m_ecakind==1)
{
//--- Diagonal ECA
for(i=0; i<nfree; i++)
result+=0.5*CMath::Sqr(s.m_ecadiag[i]*s.m_txc[i]);
}
//--- EQ
for(i=0; i<s.m_k; i++)
{
v=0.0;
for(j=0; j<nfree; j++)
v+=s.m_eq.Get(i,j)*s.m_txc[j];
result+=0.5*CMath::Sqr(v);
}
//--- EB
for(i=0; i<nfree; i++)
result+=s.m_eb[i]*s.m_txc[i];
//--- EC
result+=s.m_ec;
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Internal function, rebuilds "effective" model subject to |
//| constraints. Returns False on failure (non-SPD main quadratic |
//| term) |
//+------------------------------------------------------------------+
bool CCQModels::CQMRebuild(CConvexQuadraticModel &s)
{
//--- create variables
bool result=true;
int n=s.m_n;
int k=s.m_k;
int nfree=0;
int i=0;
int j=0;
int ridx0=0;
int ridx1=0;
int cidx0=0;
int cidx1=0;
double v=0;
int i_=0;
//--- Non-SPD model, quick exit
if(s.m_alpha==0.0 && s.m_tau==0.0)
return(false);
//--- Determine number of free variables.
//--- Fill TXC - array whose last N-NFree elements store constraints.
if(s.m_isactivesetchanged)
{
s.m_nfree=0;
for(i=0; i<n; i++)
{
if(!s.m_activeset[i])
s.m_nfree++;
}
j=s.m_nfree;
for(i=0; i<n; i++)
if(s.m_activeset[i])
{
s.m_txc.Set(j,s.m_xc[i]);
j++;
}
}
nfree=s.m_nfree;
//--- Re-evaluate TQ2/TQ1/TQ0, if needed
if(s.m_isactivesetchanged || s.m_ismaintermchanged)
{
//--- Handle cases Alpha>0 and Alpha=0 separately:
//--- * in the first case we have dense matrix
//--- * in the second one we have diagonal matrix, which can be
//--- handled more efficiently
if(s.m_alpha>0.0)
{
//--- Alpha>0, dense QP
//--- Split variables into two groups - free (F) and constrained (C). Reorder
//--- variables in such way that free vars come first, constrained are last:
//--- x = [xf, xc].
//--- Main quadratic term x'*(alpha*A+tau*D)*x now splits into quadratic part,
//--- linear part and constant part:
//--- ( alpha*Aff+tau*Df alpha*Afc ) ( xf )
//--- 0.5*( xf' xc' )*( )*( ) =
//--- ( alpha*Acf alpha*Acc+tau*Dc ) ( xc )
//--- = 0.5*xf'*(alpha*Aff+tau*Df)*xf + (alpha*Afc*xc)'*xf + 0.5*xc'(alpha*Acc+tau*Dc)*xc
//--- We store these parts into temporary variables:
//--- * alpha*Aff+tau*Df, alpha*Afc, alpha*Acc+tau*Dc are stored into upper
//--- triangle of TQ2
//--- * alpha*Afc*xc is stored into TQ1
//--- * 0.5*xc'(alpha*Acc+tau*Dc)*xc is stored into TQ0
//--- Below comes first part of the work - generation of TQ2:
//--- * we pass through rows of A and copy I-th row into upper block (Aff/Afc) or
//--- lower one (Acf/Acc) of TQ2, depending on presence of X[i] in the active set.
//--- RIdx0 variable contains current position for insertion into upper block,
//--- RIdx1 contains current position for insertion into lower one.
//--- * within each row, we copy J-th element into left half (Aff/Acf) or right
//--- one (Afc/Acc), depending on presence of X[j] in the active set. CIdx0
//--- contains current position for insertion into left block, CIdx1 contains
//--- position for insertion into right one.
//--- * during copying, we multiply elements by alpha and add diagonal matrix D.
ridx0=0;
ridx1=s.m_nfree;
for(i=0; i<n; i++)
{
cidx0=0;
cidx1=s.m_nfree;
for(j=0; j<n; j++)
{
if(!s.m_activeset[i] && !s.m_activeset[j])
{
//--- Element belongs to Aff
v=s.m_alpha*s.m_a.Get(i,j);
if(i==j && s.m_tau>0.0)
v+=s.m_tau*s.m_d[i];
s.m_tq2dense.Set(ridx0,cidx0,v);
}
if(!s.m_activeset[i] && s.m_activeset[j])
{
//--- Element belongs to Afc
s.m_tq2dense.Set(ridx0,cidx1,s.m_alpha*s.m_a.Get(i,j));
}
if(s.m_activeset[i] && !s.m_activeset[j])
{
//--- Element belongs to Acf
s.m_tq2dense.Set(ridx1,cidx0,s.m_alpha*s.m_a.Get(i,j));
}
if(s.m_activeset[i] && s.m_activeset[j])
{
//--- Element belongs to Acc
v=s.m_alpha*s.m_a.Get(i,j);
if(i==j && s.m_tau>0.0)
v+=s.m_tau*s.m_d[i];
s.m_tq2dense.Set(ridx1,cidx1,v);
}
if(s.m_activeset[j])
cidx1++;
else
cidx0++;
}
if(s.m_activeset[i])
ridx1++;
else
ridx0++;
}
//--- Now we have TQ2, and we can evaluate TQ1.
//--- In the special case when we have Alpha=0, NFree=0 or NFree=N,
//--- TQ1 is filled by zeros.
s.m_tq1=vector<double>::Zeros(n);
if(s.m_nfree>0 && s.m_nfree<n)
CAblas::RMatrixMVect(s.m_nfree,n-s.m_nfree,s.m_tq2dense,0,s.m_nfree,0,s.m_txc,s.m_nfree,s.m_tq1,0);
//--- And finally, we evaluate TQ0.
v=0.0;
for(i=s.m_nfree; i<n; i++)
for(j=s.m_nfree; j<n; j++)
v+= 0.5*s.m_txc[i]*s.m_tq2dense.Get(i,j)*s.m_txc[j];
s.m_tq0=v;
}
else
{
//--- Alpha=0, diagonal QP
//--- Split variables into two groups - free (F) and constrained (C). Reorder
//--- variables in such way that free vars come first, constrained are last:
//--- x = [xf, xc].
//--- Main quadratic term x'*(tau*D)*x now splits into quadratic and constant
//--- parts:
//--- ( tau*Df ) ( xf )
//--- 0.5*( xf' xc' )*( )*( ) =
//--- ( tau*Dc ) ( xc )
//--- = 0.5*xf'*(tau*Df)*xf + 0.5*xc'(tau*Dc)*xc
//--- We store these parts into temporary variables:
//--- * tau*Df is stored in TQ2Diag
//--- * 0.5*xc'(tau*Dc)*xc is stored into TQ0
s.m_tq0=0.0;
ridx0=0;
for(i=0; i<n; i++)
{
if(!s.m_activeset[i])
{
s.m_tq2diag.Set(ridx0,s.m_tau*s.m_d[i]);
ridx0++;
}
else
s.m_tq0+=0.5*s.m_tau*s.m_d[i]*CMath::Sqr(s.m_xc[i]);
}
s.m_tq1=vector<double>::Zeros(n);
}
}
//--- Re-evaluate TK2/TK1/TK0, if needed
if(s.m_isactivesetchanged || s.m_issecondarytermchanged)
{
//
//--- Split variables into two groups - free (F) and constrained (C). Reorder
//--- variables in such way that free vars come first, constrained are last:
//--- x = [xf, xc].
//
//--- Secondary term theta*(Q*x-r)'*(Q*x-r) now splits into quadratic part,
//--- linear part and constant part:
//--- ( ( xf ) )' ( ( xf ) )
//--- 0.5*theta*( (Qf Qc)'*( ) - r ) * ( (Qf Qc)'*( ) - r ) =
//--- ( ( xc ) ) ( ( xc ) )
//
//--- = 0.5*theta*xf'*(Qf'*Qf)*xf + theta*((Qc*xc-r)'*Qf)*xf +
//--- + theta*(-r'*(Qc*xc-r)-0.5*r'*r+0.5*xc'*Qc'*Qc*xc)
//
//--- We store these parts into temporary variables:
//--- * sqrt(theta)*Qf is stored into TK2
//--- * theta*((Qc*xc-r)'*Qf) is stored into TK1
//--- * theta*(-r'*(Qc*xc-r)-0.5*r'*r+0.5*xc'*Qc'*Qc*xc) is stored into TK0
//
//--- We use several other temporaries to store intermediate results:
//--- * Tmp0 - to store Qc*xc-r
//--- * Tmp1 - to store Qc*xc
//
//--- Generation of TK2/TK1/TK0 is performed as follows:
//--- * we fill TK2/TK1/TK0 (to handle K=0 or Theta=0)
//--- * other steps are performed only for K>0 and Theta>0
//--- * we pass through columns of Q and copy I-th column into left block (Qf) or
//--- right one (Qc) of TK2, depending on presence of X[i] in the active set.
//--- CIdx0 variable contains current position for insertion into upper block,
//--- CIdx1 contains current position for insertion into lower one.
//--- * we calculate Qc*xc-r and store it into Tmp0
//--- * we calculate TK0 and TK1
//--- * we multiply leading part of TK2 which stores Qf by sqrt(theta)
//--- it is important to perform this step AFTER calculation of TK0 and TK1,
//--- because we need original (non-modified) Qf to calculate TK0 and TK1.
//
s.m_tk2=matrix<double>::Zeros(k,n);
s.m_tk1=vector<double>::Zeros(n);
s.m_tk0=0.0;
if(s.m_k>0 && s.m_theta>0.0)
{
//
//--- Split Q into Qf and Qc
//--- Calculate Qc*xc-r, store in Tmp0
//
s.m_tmp0.Resize(k);
s.m_tmp1=vector<double>::Zeros(k);
cidx0=0;
cidx1=nfree;
for(j=0; j<n; j++)
{
if(s.m_activeset[j])
{
s.m_tk2.Col(cidx1,s.m_q.Col(j)+0);
s.m_tmp1+=s.m_q.Col(j)*s.m_txc[cidx1];
cidx1++;
}
else
{
s.m_tk2.Col(cidx0,s.m_q.Col(j)+0);
cidx0++;
}
}
s.m_tmp0=s.m_tmp1-s.m_r+0;
//--- Calculate TK0
v=0.0;
for(i=0; i<k; i++)
v+= s.m_theta*(0.5*CMath::Sqr(s.m_tmp1[i])-s.m_r[i]*s.m_tmp0[i]-0.5*CMath::Sqr(s.m_r[i]));
s.m_tk0=v;
//--- Calculate TK1
if(nfree>0)
{
for(i=0; i<k; i++)
{
v=s.m_theta*s.m_tmp0[i];
for(i_=0; i_<nfree; i_++)
s.m_tk1.Add(i_,v*s.m_tk2.Get(i,i_));
}
}
//--- Calculate TK2
if(nfree>0)
{
v=MathSqrt(s.m_theta);
for(i=0; i<k; i++)
{
for(i_=0; i_<nfree; i_++)
s.m_tk2.Mul(i,i_,v);
}
}
}
}
//--- Re-evaluate TB
if(s.m_isactivesetchanged || s.m_islineartermchanged)
{
ridx0=0;
ridx1=nfree;
for(i=0; i<n; i++)
{
if(s.m_activeset[i])
{
s.m_tb.Set(ridx1,s.m_b[i]);
ridx1++;
}
else
{
s.m_tb.Set(ridx0,s.m_b[i]);
ridx0++;
}
}
}
//--- Compose ECA: either dense ECA or diagonal ECA
if((s.m_isactivesetchanged || s.m_ismaintermchanged) && nfree>0)
{
if(s.m_alpha>0.0)
{
//--- Dense ECA
s.m_ecakind=0;
for(i=0; i<nfree; i++)
{
for(j=i; j<nfree; j++)
s.m_ecadense.Set(i,j,s.m_tq2dense.Get(i,j));
}
if(!CTrFac::SPDMatrixCholeskyRec(s.m_ecadense,0,nfree,true,s.m_tmp0))
{
result=false;
return(result);
}
}
else
{
//--- Diagonal ECA
s.m_ecakind=1;
for(i=0; i<nfree; i++)
{
if(s.m_tq2diag[i]<0.0)
{
result=false;
return(result);
}
s.m_ecadiag.Set(i,MathSqrt(s.m_tq2diag[i]));
}
}
}
//--- Compose EQ
if(s.m_isactivesetchanged || s.m_issecondarytermchanged)
{
for(i=0; i<k; i++)
for(j=0; j<nfree; j++)
s.m_eq.Set(i,j,s.m_tk2.Get(i,j));
}
//--- Calculate ECCM
if((s.m_isactivesetchanged || s.m_ismaintermchanged || s.m_issecondarytermchanged) && s.m_k>0 && s.m_theta>0.0 && nfree>0)
{
//--- Calculate ECCM-Cholesky factor of the "effective" capacitance
//--- matrix CM = I + EQ*inv(EffectiveA)*EQ'.
//--- We calculate CM as follows:
//--- CM = I + EQ*inv(EffectiveA)*EQ'
//--- = I + EQ*ECA^(-1)*ECA^(-T)*EQ'
//--- = I + (EQ*ECA^(-1))*(EQ*ECA^(-1))'
//--- Then we perform Cholesky decomposition of CM.
s.m_tmp2.Resize(k,n);
CAblas::RMatrixCopy(k,nfree,s.m_eq,0,0,s.m_tmp2,0,0);
//--- check
if(!CAp::Assert(s.m_ecakind==0 || s.m_ecakind==1,__FUNCTION__+": unexpected ECAKind"))
return(false);
if(s.m_ecakind==0)
CAblas::RMatrixRightTrsM(k,nfree,s.m_ecadense,0,0,true,false,0,s.m_tmp2,0,0);
if(s.m_ecakind==1)
{
for(i=0; i<k; i++)
for(j=0; j<nfree; j++)
s.m_tmp2.Mul(i,j,1.0/s.m_ecadiag[j]);
}
s.m_eccm=matrix<double>::Identity(k,k);
CAblas::RMatrixSyrk(k,nfree,1.0,s.m_tmp2,0,0,0,1.0,s.m_eccm,0,0,true);
if(!CTrFac::SPDMatrixCholeskyRec(s.m_eccm,0,k,true,s.m_tmp0))
{
result=false;
return(result);
}
}
//--- Compose EB and EC
//--- NOTE: because these quantities are cheap to compute, we do not
//--- use caching here.
for(i=0; i<nfree; i++)
s.m_eb.Set(i,s.m_tq1[i]+s.m_tk1[i]+s.m_tb[i]);
s.m_ec=s.m_tq0+s.m_tk0;
for(i=nfree; i<n; i++)
s.m_ec+=s.m_tb[i]*s.m_txc[i];
//--- Change cache status - everything is cached
s.m_ismaintermchanged=false;
s.m_issecondarytermchanged=false;
s.m_islineartermchanged=false;
s.m_isactivesetchanged=false;
return(result);
}
//+------------------------------------------------------------------+
//| Internal function, solves system Effective_A*x = b. |
//| It should be called after successful completion of CQMRebuild(). |
//| INPUT PARAMETERS: |
//| S - quadratic model, after call to CQMRebuild() |
//| X - right part B, array[S.NFree] |
//| Tmp - temporary array, automatically reallocated if |
//| needed |
//| OUTPUT PARAMETERS: |
//| X - solution, array[S.NFree] |
//| NOTE: when called with zero S.NFree, returns silently |
//| NOTE: this function assumes that EA is non - degenerate |
//+------------------------------------------------------------------+
void CCQModels::CQMSolveEA(CConvexQuadraticModel &s,CRowDouble &x,
CRowDouble &tmp)
{
//--- check
if(!CAp::Assert(s.m_ecakind==0 || s.m_ecakind==1 || (s.m_ecakind==-1 && s.m_nfree==0),__FUNCTION__+": unexpected ECAKind"))
return;
if(s.m_ecakind==0)
{
//--- Dense ECA, use FBLSCholeskySolve() dense m_solver.
CFbls::FblsCholeskySolve(s.m_ecadense,1.0,s.m_nfree,true,x,tmp);
}
if(s.m_ecakind==1)
{
//--- Diagonal ECA
for(int i=0; i<s.m_nfree; i++)
x.Mul(i,1.0/CMath::Sqr(s.m_ecadiag[i]));
}
}
//+------------------------------------------------------------------+
//| This object stores Settings for QQP m_solver. |
//| It must be initialized with QQPLoadDefaults(). |
//| After initialization you may change Settings. |
//+------------------------------------------------------------------+
struct CQQPSettings
{
int m_cgmaxits;
int m_cgminits;
int m_cnmaxupdates;
int m_maxouterits;
int m_sparsesolver;
double m_epsf;
double m_epsg;
double m_epsx;
bool m_cgphase;
bool m_cnphase;
//--- constructor / destructor
CQQPSettings(void) { ZeroMemory(this); }
~CQQPSettings(void) {}
void Copy(const CQQPSettings &obj);
//--- overloading
void operator=(const CQQPSettings &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CQQPSettings::Copy(const CQQPSettings &obj)
{
m_cgmaxits=obj.m_cgmaxits;
m_cgminits=obj.m_cgminits;
m_cnmaxupdates=obj.m_cnmaxupdates;
m_maxouterits=obj.m_maxouterits;
m_sparsesolver=obj.m_sparsesolver;
m_epsf=obj.m_epsf;
m_epsg=obj.m_epsg;
m_epsx=obj.m_epsx;
m_cgphase=obj.m_cgphase;
m_cnphase=obj.m_cnphase;
}
//+------------------------------------------------------------------+
//| This object stores temporaries used by QuickQP m_solver. |
//+------------------------------------------------------------------+
struct CQQPBuffers
{
int m_akind;
int m_cnmodelage;
int m_n;
int m_nfree;
int m_repinneriterationscount;
int m_repncholesky;
int m_repncupdates;
int m_repouteriterationscount;
double m_absamax;
double m_absasum;
double m_absasum2;
bool m_activated[];
bool m_havebndl[];
bool m_havebndu[];
bool m_sparseupper;
bool m_tmpcnb[];
CSparseMatrix m_sparsea;
CSparseMatrix m_sparsecca;
CSparseBuffers m_sbuf;
CSActiveSet m_sas;
CRowInt m_tmpcni;
CRowInt m_yidx;
CRowDouble m_b;
CRowDouble m_bndl;
CRowDouble m_bndu;
CRowDouble m_cgc;
CRowDouble m_cgp;
CRowDouble m_dc;
CRowDouble m_dp;
CRowDouble m_gc;
CRowDouble m_regdiag;
CRowDouble m_regx0;
CRowDouble m_stpbuf;
CRowDouble m_tmp0;
CRowDouble m_tmp1;
CRowDouble m_tmpcn;
CRowDouble m_xf;
CRowDouble m_xp;
CRowDouble m_xs;
CMatrixDouble m_densea;
CMatrixDouble m_densez;
//--- constructor / destructor
CQQPBuffers(void);
~CQQPBuffers(void) {}
//---
void Copy(const CQQPBuffers &obj);
//--- overloading
void operator=(const CQQPBuffers &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CQQPBuffers::CQQPBuffers(void)
{
m_akind=0;
m_cnmodelage=0;
m_n=0;
m_nfree=0;
m_repinneriterationscount=0;
m_repncholesky=0;
m_repncupdates=0;
m_repouteriterationscount=0;
m_absamax=0;
m_absasum=0;
m_absasum2=0;
m_sparseupper=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CQQPBuffers::Copy(const CQQPBuffers &obj)
{
m_akind=obj.m_akind;
m_cnmodelage=obj.m_cnmodelage;
m_n=obj.m_n;
m_nfree=obj.m_nfree;
m_repinneriterationscount=obj.m_repinneriterationscount;
m_repncholesky=obj.m_repncholesky;
m_repncupdates=obj.m_repncupdates;
m_repouteriterationscount=obj.m_repouteriterationscount;
m_absamax=obj.m_absamax;
m_absasum=obj.m_absasum;
m_absasum2=obj.m_absasum2;
ArrayCopy(m_activated,obj.m_activated);
ArrayCopy(m_havebndl,obj.m_havebndl);
ArrayCopy(m_havebndu,obj.m_havebndu);
m_sparseupper=obj.m_sparseupper;
ArrayCopy(m_tmpcnb,obj.m_tmpcnb);
m_sparsea=obj.m_sparsea;
m_sparsecca=obj.m_sparsecca;
m_sbuf=obj.m_sbuf;
m_sas=obj.m_sas;
m_tmpcni=obj.m_tmpcni;
m_yidx=obj.m_yidx;
m_b=obj.m_b;
m_bndl=obj.m_bndl;
m_bndu=obj.m_bndu;
m_cgc=obj.m_cgc;
m_cgp=obj.m_cgp;
m_dc=obj.m_dc;
m_dp=obj.m_dp;
m_gc=obj.m_gc;
m_regdiag=obj.m_regdiag;
m_regx0=obj.m_regx0;
m_stpbuf=obj.m_stpbuf;
m_tmp0=obj.m_tmp0;
m_tmp1=obj.m_tmp1;
m_tmpcn=obj.m_tmpcn;
m_xf=obj.m_xf;
m_xp=obj.m_xp;
m_xs=obj.m_xs;
m_densea=obj.m_densea;
m_densez=obj.m_densez;
}
//+------------------------------------------------------------------+
//| QQP m_solver |
//+------------------------------------------------------------------+
class CQQPSolver
{
public:
static const int m_quickqprestartcg;
static const double m_regz;
static void QQPLoadDefaults(int n,CQQPSettings &s);
static void QQPCopySettings(CQQPSettings &src,CQQPSettings &dst);
static void QQPPreAllocateGrowDense(CQQPBuffers &sstate,int nexpected,int ngrowto);
static void QQPOptimize(CConvexQuadraticModel &cqmac,CSparseMatrix &sparseac,CMatrixDouble &denseac,int akind,bool IsUpper,CRowDouble &bc,CRowDouble &bndlc,CRowDouble &bnduc,CRowDouble &sc,CRowDouble &xoriginc,int nc,CQQPSettings &Settings,CQQPBuffers &sstate,CRowDouble &xs,int &terminationtype);
private:
static double ProjectedTargetFunction(CQQPBuffers &sstate,CRowDouble &x,CRowDouble &d,double stp,CRowDouble &m_tmp0,CRowDouble &m_tmp1);
static void TargetGradient(CQQPBuffers &sstate,CRowDouble &x,CRowDouble &g);
static void QuadraticModel(CQQPBuffers &sstate,CRowDouble &x,CRowDouble &d,CRowDouble &g,double &d1,int &d1est,double &d2,int &d2est,CRowDouble &m_tmp0);
static void FindBestStepAndMove(CQQPBuffers &sstate,CSActiveSet &sas,CRowDouble &d,double stp,bool needact,int cidx,double cval,CRowDouble &addsteps,int addstepscnt,bool &activated[],CRowDouble &m_tmp0,CRowDouble &m_tmp1);
static bool CNewtonBuild(CQQPBuffers &sstate,int sparsesolver,int &ncholesky);
static bool CNewtonUpdate(CQQPBuffers &sstate,CQQPSettings &Settings,int &ncupdates);
static bool CNewtonStep(CQQPBuffers &sstate,CQQPSettings &Settings,CRowDouble &gc);
};
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
const int CQQPSolver::m_quickqprestartcg=50;
const double CQQPSolver::m_regz=1.0E-9;
//+------------------------------------------------------------------+
//| This function initializes QQPSettings structure with default |
//| Settings. |
//| Newly created structure MUST be initialized by default |
//| Settings - or by copy of the already initialized structure. |
//+------------------------------------------------------------------+
void CQQPSolver::QQPLoadDefaults(int n,CQQPSettings &s)
{
s.m_epsg=0.0;
s.m_epsf=0.0;
s.m_epsx=1.0E-6;
s.m_maxouterits=0;
s.m_cgphase=true;
s.m_cnphase=true;
s.m_cgminits=5;
s.m_cgmaxits=MathMax(s.m_cgminits,(int)MathRound(1+0.33*n));
s.m_sparsesolver=0;
s.m_cnmaxupdates=(int)MathRound(1+0.1*n);
}
//+------------------------------------------------------------------+
//| This function initializes QQPSettings structure with copy of |
//| another, already initialized structure. |
//+------------------------------------------------------------------+
void CQQPSolver::QQPCopySettings(CQQPSettings &src,CQQPSettings &dst)
{
dst.m_epsg=src.m_epsg;
dst.m_epsf=src.m_epsf;
dst.m_epsx=src.m_epsx;
dst.m_maxouterits=src.m_maxouterits;
dst.m_cgphase=src.m_cgphase;
dst.m_cnphase=src.m_cnphase;
dst.m_cgminits=src.m_cgminits;
dst.m_cgmaxits=src.m_cgmaxits;
dst.m_sparsesolver=src.m_sparsesolver;
dst.m_cnmaxupdates=src.m_cnmaxupdates;
}
//+------------------------------------------------------------------+
//| This function performs preallocation of internal 2D matrices. If |
//| matrix size is less than expected, we grow to some larger value |
//| (specified by user). |
//| It can be useful in cases when we solve many subsequent QP |
//| problems with increasing sizes - helps to avoid multiple |
//| allocations. |
//| INPUT PARAMETERS: |
//| SState - object which stores temporaries: |
//| * uninitialized object is automatically |
//| initialized |
//| * previously allocated memory is reused as much |
//| as possible |
//| NExpected - if internal buffers have size enough for |
//| NExpected, no preallocation happens. If size is |
//| less than NExpected, buffers are preallocated up|
//| to NGrowTo*NGrowTo |
//| NGrowTo - new size |
//| OUTPUT PARAMETERS: |
//| SState - temporary buffers, some of them are preallocated|
//+------------------------------------------------------------------+
void CQQPSolver::QQPPreAllocateGrowDense(CQQPBuffers &sstate,
int nexpected,
int ngrowto)
{
if(sstate.m_densea.Rows()<nexpected || sstate.m_densea.Cols()<nexpected)
sstate.m_densea.Resize(ngrowto,ngrowto);
if(sstate.m_densez.Rows()<nexpected || sstate.m_densez.Cols()<nexpected)
sstate.m_densez.Resize(ngrowto,ngrowto);
}
//+------------------------------------------------------------------+
//| This function runs QQP m_solver; it returns after optimization |
//| process was completed. Following QP problem is solved: |
//| min(0.5 * (x - x_origin)'*A*(x-x_origin)+b' * (x - x_origin)) |
//| subject to boundary constraints. |
//| IMPORTANT: UNLIKE MANY OTHER SOLVERS, THIS FUNCTION DOES NOT |
//| REQUIRE YOU TO INITIALIZE STATE OBJECT. IT CAN BE |
//| AUTOMATICALLY INITIALIZED DURING SOLUTION PROCESS. |
//| INPUT PARAMETERS: |
//| AC - for dense problems given by CQM model(AKind = 0) |
//| A - term of CQM object contains system matrix. Other |
//| terms are unspecified and should not be referenced.|
//| SparseAC - for sparse problems(AKind = 1) |
//| DenseAC - for traditional dense matrices(AKind = 2) |
//| AKind - matrix term to use: |
//| * 0 for dense CQM(CQMAC) |
//| * 1 for sparse matrix(SparseAC) |
//| * 2 for dense matrix(DenseAC) |
//| IsUpper - which triangle of SparseAC / DenseAC stores matrix-|
//| upper or lower one (for dense matrices this |
//| parameter is not actual). |
//| BC - linear term, array[NC] |
//| BndLC - lower bound, array[NC] |
//| BndUC - upper bound, array[NC] |
//| SC - scale vector, array[NC]: |
//| * I-Th element contains scale of I-Th variable, |
//| * SC[I] > 0 |
//| XOriginC - origin term, array[NC]. Can be zero. |
//| NC - number of variables in the original formulation |
//| (no slack variables). |
//| CLEICC - linear equality / inequality constraints. Present |
//| version of this function does NOT provide publicly |
//| available support for linear constraints. This |
//| feature will be introduced in the future versions |
//| of the function. |
//| NEC, NIC - number of equality / inequality constraints. MUST |
//| BE ZERO IN THE CURRENT VERSION!!! |
//| Settings - QQPSettings object initialized by one of the |
//| initialization functions. |
//| SState - object which stores temporaries: |
//| * uninitialized object is automatically initialized|
//| * previously allocated memory is reused as much as |
//| possible |
//| XS - initial point, array[NC] |
//| OUTPUT PARAMETERS: |
//| XS - last point |
//| TerminationType - termination type: |
//| * |
//| * |
//| * |
//+------------------------------------------------------------------+
void CQQPSolver::QQPOptimize(CConvexQuadraticModel &cqmac,
CSparseMatrix &sparseac,
CMatrixDouble &denseac,
int akind,
bool IsUpper,
CRowDouble &bc,
CRowDouble &bndlc,
CRowDouble &bnduc,
CRowDouble &sc,
CRowDouble &xoriginc,
int nc,
CQQPSettings &Settings,
CQQPBuffers &sstate,
CRowDouble &xs,
int &terminationtype)
{
//--- create variables
int n=0;
int i=0;
int j=0;
int k=0;
double v=0;
double vv=0;
double d2=0;
double d1=0;
int d1est=0;
int d2est=0;
bool needact;
double reststp=0;
double fullstp=0;
double stpmax=0;
double stp=0;
int stpcnt=0;
int cidx=0;
double cval=0;
int cgcnt=0;
int cgmax=0;
int newtcnt=0;
int sparsesolver=0;
double beta=0;
bool b;
double fprev=0;
double fcur=0;
bool problemsolved;
bool isconstrained;
double f0=0;
double f1=0;
int i_=0;
terminationtype=0;
//--- Primary checks
if(!CAp::Assert(akind==0 || akind==1 || akind==2,__FUNCTION__+": incorrect AKind"))
return;
sstate.m_n=nc;
n=sstate.m_n;
terminationtype=0;
sstate.m_repinneriterationscount=0;
sstate.m_repouteriterationscount=0;
sstate.m_repncholesky=0;
sstate.m_repncupdates=0;
//--- Several checks
//--- * matrix size
//--- * scale vector
//--- * consistency of bound constraints
//--- * consistency of Settings
if(akind==1)
{
if(!CAp::Assert(CSparse::SparseGetNRows(sparseac)==n,__FUNCTION__+": rows(SparseAC)<>N"))
return;
if(!CAp::Assert(CSparse::SparseGetNCols(sparseac)==n,__FUNCTION__+": cols(SparseAC)<>N"))
return;
}
if(!CAp::Assert(CApServ::IsFiniteVector(sc,n) && sc.Min()>0.0,__FUNCTION__+": incorrect scale"))
return;
for(i=0; i<n; i++)
{
if(MathIsValidNumber(bndlc[i]) && MathIsValidNumber(bnduc[i]))
if(bndlc[i]>bnduc[i])
{
terminationtype=-3;
return;
}
}
if(!CAp::Assert(Settings.m_cgphase || Settings.m_cnphase,__FUNCTION__+": both phases (CG and Newton) are inactive"))
return;
//--- Allocate data structures
CApServ::BVectorSetLengthAtLeast(sstate.m_havebndl,n);
CApServ::BVectorSetLengthAtLeast(sstate.m_havebndu,n);
sstate.m_bndl.Resize(n);
sstate.m_bndu.Resize(n);
sstate.m_xs.Resize(n);
sstate.m_xf.Resize(n);
sstate.m_xp.Resize(n);
sstate.m_gc.Resize(n);
sstate.m_cgc.Resize(n);
sstate.m_cgp.Resize(n);
sstate.m_dc.Resize(n);
sstate.m_dp.Resize(n);
sstate.m_tmp0.Resize(n);
sstate.m_tmp1.Resize(n);
sstate.m_stpbuf.Resize(15);
CSActiveSets::SASInit(n,sstate.m_sas);
//--- Scale/shift problem coefficients:
//--- min { 0.5*(x-x0)'*A*(x-x0) + b'*(x-x0) }
//--- becomes (after transformation "x = S*y+x0")
//--- min { 0.5*y'*(S*A*S)*y + (S*b)'*y
//--- Modified A_mod=S*A*S and b_mod=S*(b+A*x0) are
//--- stored into SState.DenseA and SState.B.
sstate.m_b=sc*bc+0;
sstate.m_b.Resize(n);
sstate.m_akind=-99;
if(akind==0)
{
//--- Dense QP problem - just copy and scale.
sstate.m_densea.Resize(n,n);
CCQModels::CQMGetA(cqmac,sstate.m_densea);
sstate.m_akind=0;
sstate.m_absamax=0;
sstate.m_absasum=0;
sstate.m_absasum2=0;
for(i=0; i<n; i++)
for(j=0; j<n; j++)
{
v=sc[i]*sstate.m_densea.Get(i,j)*sc[j];
vv=MathAbs(v);
sstate.m_densea.Set(i,j,v);
sstate.m_absamax=MathMax(sstate.m_absamax,vv);
sstate.m_absasum=sstate.m_absasum+vv;
sstate.m_absasum2=sstate.m_absasum2+vv*vv;
}
}
if(akind==1)
{
//--- Sparse QP problem - a bit tricky. Depending on format of the
//--- input we use different strategies for copying matrix:
//--- * SKS matrices are copied to SKS format
//--- * anything else is copied to CRS format
CSparse::SparseCopyToSKSBuf(sparseac,sstate.m_sparsea);
if(IsUpper)
CSparse::SparseTransposeSKS(sstate.m_sparsea);
sstate.m_akind=1;
sstate.m_sparseupper=false;
sstate.m_absamax=0;
sstate.m_absasum=0;
sstate.m_absasum2=0;
for(i=0; i<n; i++)
{
k=sstate.m_sparsea.m_RIdx[i];
for(j=i-sstate.m_sparsea.m_DIdx[i]; j<=i; j++)
{
v=sc[i]*sstate.m_sparsea.m_Vals[k]*sc[j];
vv=MathAbs(v);
sstate.m_sparsea.m_Vals.Set(k,v);
if(i==j)
{
//--- Diagonal terms are counted only once
sstate.m_absamax=MathMax(sstate.m_absamax,vv);
sstate.m_absasum=sstate.m_absasum+vv;
sstate.m_absasum2=sstate.m_absasum2+vv*vv;
}
else
{
//--- Offdiagonal terms are counted twice
sstate.m_absamax=MathMax(sstate.m_absamax,vv);
sstate.m_absasum=sstate.m_absasum+2*vv;
sstate.m_absasum2=sstate.m_absasum2+2*vv*vv;
}
k++;
}
}
}
if(akind==2)
{
//--- Dense QP problem - just copy and scale.
sstate.m_densea.Resize(n,n);
sstate.m_akind=0;
sstate.m_absamax=0;
sstate.m_absasum=0;
sstate.m_absasum2=0;
if(IsUpper)
{
for(i=0; i<n; i++)
{
for(j=i; j<n; j++)
{
v=sc[i]*denseac.Get(i,j)*sc[j];
vv=MathAbs(v);
sstate.m_densea.Set(i,j,v);
sstate.m_densea.Set(j,i,v);
if((double)(i)==v)
k=1;
else
k=2;
sstate.m_absamax=MathMax(sstate.m_absamax,vv);
sstate.m_absasum+=vv*k;
sstate.m_absasum2+=vv*vv*k;
}
}
}
else
{
for(i=0; i<n; i++)
{
for(j=0; j<=i; j++)
{
v=sc[i]*denseac.Get(i,j)*sc[j];
vv=MathAbs(v);
sstate.m_densea.Set(i,j,v);
sstate.m_densea.Set(j,i,v);
if((double)(i)==v)
k=1;
else
k=2;
sstate.m_absamax=MathMax(sstate.m_absamax,vv);
sstate.m_absasum=sstate.m_absasum+vv*k;
sstate.m_absasum2=sstate.m_absasum2+vv*vv*k;
}
}
}
}
//--- check
if(!CAp::Assert(sstate.m_akind>=0,__FUNCTION__+": integrity check failed"))
return;
//--- Load box constraints into State structure.
//--- We apply transformation to variables: y=(x-x_origin)/s,
//--- each of the constraints is appropriately shifted/scaled.
for(i=0; i<n; i++)
{
sstate.m_havebndl[i]=MathIsValidNumber(bndlc[i]);
if(sstate.m_havebndl[i])
sstate.m_bndl.Set(i,(bndlc[i]-xoriginc[i])/sc[i]);
else
{
//--- check
if(!CAp::Assert(AL_NEGINF==bndlc[i],__FUNCTION__+": incorrect lower bound"))
return;
sstate.m_bndl.Set(i,AL_NEGINF);
}
sstate.m_havebndu[i]=MathIsValidNumber(bnduc[i]);
if(sstate.m_havebndu[i])
sstate.m_bndu.Set(i,(bnduc[i]-xoriginc[i])/sc[i]);
else
{
//--- check
if(!CAp::Assert(AL_POSINF==bnduc[i],__FUNCTION__+": incorrect upper bound"))
return;
sstate.m_bndu.Set(i,AL_POSINF);
}
}
//--- Process initial point:
//--- * set it to XS-XOriginC
//--- * make sure that boundary constraints are preserved by transformation
for(i=0; i<n; i++)
{
sstate.m_xs.Set(i,(xs[i]-xoriginc[i])/sc[i]);
if(sstate.m_havebndl[i] && sstate.m_xs[i]<sstate.m_bndl[i])
sstate.m_xs.Set(i,sstate.m_bndl[i]);
if(sstate.m_havebndu[i] && sstate.m_xs[i]>sstate.m_bndu[i])
sstate.m_xs.Set(i,sstate.m_bndu[i]);
if(sstate.m_havebndl[i] && xs[i]==bndlc[i])
sstate.m_xs.Set(i,sstate.m_bndl[i]);
if(sstate.m_havebndu[i] && xs[i]==bnduc[i])
sstate.m_xs.Set(i,sstate.m_bndu[i]);
}
//--- Select sparse direct m_solver
if(akind==1)
{
sparsesolver=Settings.m_sparsesolver;
if(sparsesolver==0)
sparsesolver=1;
if(CSparse::SparseIsSKS(sstate.m_sparsea))
sparsesolver=2;
sparsesolver=2;
//--- check
if(!CAp::Assert(sparsesolver==1 || sparsesolver==2,__FUNCTION__+": incorrect SparseSolver"))
return;
}
else
sparsesolver=0;
//--- For unconstrained problems - try to use fast approach which requires
//--- just one unregularized Cholesky decomposition for solution. If it fails,
//--- switch to general QQP code.
problemsolved=false;
isconstrained=false;
for(i=0; i<n; i++)
isconstrained=(isconstrained || sstate.m_havebndl[i] || sstate.m_havebndu[i]);
if(!isconstrained && Settings.m_cnphase && akind==0)
{
sstate.m_densez=sstate.m_densea;
sstate.m_densez.Resize(n,n);
sstate.m_tmpcn.Resize(n);
sstate.m_repncholesky++;
if(CTrFac::SPDMatrixCholeskyRec(sstate.m_densez,0,n,true,sstate.m_tmpcn))
{
sstate.m_xf=sstate.m_xs;
sstate.m_dc=vector<double>::Zeros(n);
f0=ProjectedTargetFunction(sstate,sstate.m_xf,sstate.m_dc,0.0,sstate.m_tmpcn,sstate.m_tmp1);
for(k=0; k<=3; k++)
{
CAblas::RMatrixMVect(n,n,sstate.m_densea,0,0,0,sstate.m_xf,0,sstate.m_gc,0);
sstate.m_gc+=sstate.m_b;
sstate.m_dc=sstate.m_gc;
sstate.m_dc*=(-1.0);
CFbls::FblsCholeskySolve(sstate.m_densez,1.0,n,true,sstate.m_dc,sstate.m_tmpcn);
f1=ProjectedTargetFunction(sstate,sstate.m_xf,sstate.m_dc,1.0,sstate.m_tmpcn,sstate.m_tmp1);
if(f1>=f0)
break;
sstate.m_xf+=sstate.m_dc;
f0=f1;
}
terminationtype=2;
problemsolved=true;
}
}
//--- Attempt to solve problem with fast approach failed, use generic QQP
if(!problemsolved)
{
//--- Prepare "active set" structure
CSActiveSets::SASSetBC(sstate.m_sas,sstate.m_bndl,sstate.m_bndu);
if(!CSActiveSets::SASStartOptimization(sstate.m_sas,sstate.m_xs))
{
terminationtype=-3;
return;
}
//--- Main loop.
//--- Following variables are used:
//--- * GC stores current gradient (unconstrained)
//--- * CGC stores current gradient (constrained)
//--- * DC stores current search direction
//--- * CGP stores constrained gradient at previous point
//--- (zero on initial entry)
//--- * DP stores previous search direction
//--- (zero on initial entry)
cgmax=Settings.m_cgminits;
sstate.m_repinneriterationscount=0;
sstate.m_repouteriterationscount=0;
while(true)
{
if(Settings.m_maxouterits>0 && sstate.m_repouteriterationscount>=Settings.m_maxouterits)
{
terminationtype=5;
break;
}
if(sstate.m_repouteriterationscount>0)
{
//--- Check EpsF- and EpsX-based stopping criteria.
//--- Because problem was already scaled, we do not scale step before checking its length.
//--- NOTE: these checks are performed only after at least one outer iteration was made.
if(Settings.m_epsf>0.0)
{
//--- NOTE 1: here we rely on the fact that ProjectedTargetFunction() ignore D when Stp=0
//--- NOTE 2: code below handles situation when update increases function value instead
//--- of decreasing it.
fprev=ProjectedTargetFunction(sstate,sstate.m_xp,sstate.m_dc,0.0,sstate.m_tmp0,sstate.m_tmp1);
fcur=ProjectedTargetFunction(sstate,sstate.m_sas.m_xc,sstate.m_dc,0.0,sstate.m_tmp0,sstate.m_tmp1);
if((fprev-fcur)<=(Settings.m_epsf*MathMax(MathAbs(fprev),MathMax(MathAbs(fcur),1.0))))
{
terminationtype=1;
break;
}
}
if(Settings.m_epsx>0.0)
{
v=0.0;
for(i=0; i<n; i++)
{
v+= CMath::Sqr(sstate.m_xp[i]-sstate.m_sas.m_xc[i]);
}
if(MathSqrt(v)<=Settings.m_epsx)
{
terminationtype=2;
break;
}
}
}
sstate.m_repouteriterationscount++;
sstate.m_xp=sstate.m_sas.m_xc;
if(!Settings.m_cgphase)
cgmax=0;
sstate.m_cgp=vector<double>::Zeros(n);
sstate.m_dp=vector<double>::Zeros(n);
for(cgcnt=0; cgcnt<=cgmax-1; cgcnt++)
{
//--- Calculate unconstrained gradient GC for "extended" QP problem
//--- Determine active set, current constrained gradient CGC.
//--- Check gradient-based stopping condition.
//
//--- NOTE: because problem was scaled, we do not have to apply scaling
//--- to gradient before checking stopping condition.
TargetGradient(sstate,sstate.m_sas.m_xc,sstate.m_gc);
CSActiveSets::SASReactivateConstraints(sstate.m_sas,sstate.m_gc);
sstate.m_cgc=sstate.m_gc;
CSActiveSets::SASConstrainedDirection(sstate.m_sas,sstate.m_cgc);
v=CAblasF::RDotV2(n,sstate.m_cgc);
if(MathSqrt(v)<=Settings.m_epsg)
{
terminationtype=4;
break;
}
//--- Prepare search direction DC and explore it.
//--- We try to use CGP/DP to prepare conjugate gradient step,
//--- but we resort to steepest descent step (Beta=0) in case
//--- we are at I-th boundary, but DP[I]<>0.
//--- Such approach allows us to ALWAYS have feasible DC, with
//--- guaranteed compatibility with both feasible area and current
//--- active set.
//--- Automatic CG reset performed every time DP is incompatible
//--- with current active set and/or feasible area. We also
//--- perform reset every QuickQPRestartCG iterations.
sstate.m_dc=sstate.m_cgc;
sstate.m_dc*=(-1.0);
v=0.0;
vv=0.0;
b=false;
for(i=0; i<n; i++)
{
v+= sstate.m_cgc[i]*sstate.m_cgc[i];
vv+=sstate.m_cgp[i]*sstate.m_cgp[i];
b=b || (sstate.m_havebndl[i] && sstate.m_sas.m_xc[i]==sstate.m_bndl[i] && sstate.m_dp[i]!=0.0);
b=b || (sstate.m_havebndu[i] && sstate.m_sas.m_xc[i]==sstate.m_bndu[i] && sstate.m_dp[i]!=0.0);
}
b=b || vv==0.0;
b=b || cgcnt%m_quickqprestartcg==0;
if(!b)
beta=v/vv;
else
beta=0.0;
sstate.m_dc+=sstate.m_dp*beta+0;
CSActiveSets::SASConstrainedDirection(sstate.m_sas,sstate.m_dc);
CSActiveSets::SASExploreDirection(sstate.m_sas,sstate.m_dc,stpmax,cidx,cval);
//--- Build quadratic model of F along descent direction:
//--- F(xc+alpha*D) = D2*alpha^2 + D1*alpha
//--- Terminate algorithm if needed.
//--- NOTE: we do not maintain constant term D0
QuadraticModel(sstate,sstate.m_sas.m_xc,sstate.m_dc,sstate.m_gc,d1,d1est,d2,d2est,sstate.m_tmp0);
if(d1==0.0 && d2==0.0)
{
//--- D1 and D2 are exactly zero, success.
//--- After this if-then we assume that D is non-zero.
terminationtype=4;
break;
}
if(d1est>=0)
{
//--- Numerical noise is too large, it means that we are close
//--- to minimum - and that further improvement is impossible.
//--- After this if-then we assume that D1 is definitely negative
//--- (even under presence of numerical errors).
terminationtype=7;
break;
}
if(d2est<=0 && cidx<0)
{
//--- Function is unbounded from below:
//--- * D1<0 (verified by previous block)
//--- * D2Est<=0, which means that either D2<0 - or it can not
//--- be reliably distinguished from zero.
//--- * step is unconstrained
//--- If these conditions are true, we abnormally terminate QP
//--- algorithm with return code -4
terminationtype=-4;
break;
}
//--- Perform step along DC.
//--- In this block of code we maintain two step length:
//--- * RestStp - restricted step, maximum step length along DC which does
//--- not violate constraints
//--- * FullStp - step length along DC which minimizes quadratic function
//--- without taking constraints into account. If problem is
//--- unbounded from below without constraints, FullStp is
//--- forced to be RestStp.
//--- So, if function is convex (D2>0):
//--- * FullStp = -D1/(2*D2)
//--- * RestStp = restricted FullStp
//--- * 0<=RestStp<=FullStp
//--- If function is non-convex, but bounded from below under constraints:
//--- * RestStp = step length subject to constraints
//--- * FullStp = RestStp
//--- After RestStp and FullStp are initialized, we generate several trial
//--- steps which are different multiples of RestStp and FullStp.
if(d2est>0)
{
//--- check
if(!CAp::Assert(d1<0.0,__FUNCTION__+": internal error"))
return;
fullstp=-(d1/(2*d2));
needact=(fullstp>=stpmax);
if(needact)
{
//--- check
if(!CAp::Assert(sstate.m_stpbuf.Size()>=3,__FUNCTION__+": StpBuf overflow"))
return;
reststp=stpmax;
stp=reststp;
sstate.m_stpbuf.Set(0,reststp*4);
sstate.m_stpbuf.Set(1,fullstp);
sstate.m_stpbuf.Set(2,fullstp/4);
stpcnt=3;
}
else
{
reststp=fullstp;
stp=fullstp;
stpcnt=0;
}
}
else
{
//--- check
if(!CAp::Assert(cidx>=0,__FUNCTION__+": internal error"))
return;
if(!CAp::Assert(sstate.m_stpbuf.Size()>=2,__FUNCTION__+": StpBuf overflow"))
return;
reststp=stpmax;
fullstp=stpmax;
stp=reststp;
needact=true;
sstate.m_stpbuf.Set(0,4*reststp);
stpcnt=1;
}
FindBestStepAndMove(sstate,sstate.m_sas,sstate.m_dc,stp,needact,cidx,cval,sstate.m_stpbuf,stpcnt,sstate.m_activated,sstate.m_tmp0,sstate.m_tmp1);
//--- Update CG information.
sstate.m_dp=sstate.m_dc;
sstate.m_cgp=sstate.m_cgc;
//--- Update iterations counter
sstate.m_repinneriterationscount++;
}
if(terminationtype!=0)
break;
cgmax=Settings.m_cgmaxits;
//--- Generate YIdx - reordering of variables for constrained Newton phase.
//--- Free variables come first, fixed are last ones.
newtcnt=0;
while(true)
{
//--- Skip iteration if constrained Newton is turned off.
if(!Settings.m_cnphase)
break;
//--- At the first iteration - build Cholesky decomposition of Hessian.
//--- At subsequent iterations - refine Hessian by adding new constraints.
//--- Loop is terminated in following cases:
//--- * Hessian is not positive definite subject to current constraints
//--- (termination during initial decomposition)
//--- * there were no new constraints being activated
//--- (termination during update)
//--- * all constraints were activated during last step
//--- (termination during update)
//--- * CNMaxUpdates were performed on matrix
//--- (termination during update)
if(newtcnt==0)
{
//--- Perform initial Newton step. If Cholesky decomposition fails,
//--- increase number of CG iterations to CGMaxIts - it should help
//--- us to find set of constraints which will make matrix positive
//--- definite.
b=CNewtonBuild(sstate,sparsesolver,sstate.m_repncholesky);
if(b)
cgmax=Settings.m_cgminits;
}
else
b=CNewtonUpdate(sstate,Settings,sstate.m_repncupdates);
if(!b)
break;
newtcnt++;
//--- Calculate gradient GC.
TargetGradient(sstate,sstate.m_sas.m_xc,sstate.m_gc);
//--- Bound-constrained Newton step
sstate.m_dc=sstate.m_gc;
if(!CNewtonStep(sstate,Settings,sstate.m_dc))
break;
QuadraticModel(sstate,sstate.m_sas.m_xc,sstate.m_dc,sstate.m_gc,d1,d1est,d2,d2est,sstate.m_tmp0);
if(d1est>=0)
{
//--- We are close to minimum, derivative is nearly zero, break Newton iteration
break;
}
if(d2est>0)
{
//--- Positive definite matrix, we can perform Newton step
//--- check
if(!CAp::Assert(d1<0.0,__FUNCTION__+": internal error"))
return;
fullstp=-(d1/(2*d2));
CSActiveSets::SASExploreDirection(sstate.m_sas,sstate.m_dc,stpmax,cidx,cval);
needact=(fullstp>=stpmax);
if(needact)
{
//--- check
if(!CAp::Assert(sstate.m_stpbuf.Size()>=3,__FUNCTION__+": StpBuf overflow"))
return;
reststp=stpmax;
stp=reststp;
sstate.m_stpbuf.Set(0,reststp*4);
sstate.m_stpbuf.Set(1,fullstp);
sstate.m_stpbuf.Set(2,fullstp/4);
stpcnt=3;
}
else
{
reststp=fullstp;
stp=fullstp;
stpcnt=0;
}
FindBestStepAndMove(sstate,sstate.m_sas,sstate.m_dc,stp,needact,cidx,cval,sstate.m_stpbuf,stpcnt,sstate.m_activated,sstate.m_tmp0,sstate.m_tmp1);
}
else
{
//--- Matrix is semi-definite or indefinite, but regularized
//--- Cholesky succeeded and gave us descent direction in DC.
//--- We will investigate it and try to perform descent step:
//--- * first, we explore direction:
//--- * if it is unbounded, we stop algorithm with
//--- appropriate termination code -4.
//--- * if StpMax=0, we break Newton phase and return to
//--- CG phase - constraint geometry is complicated near
//--- current point, so it is better to use simpler algo.
//--- * second, we check that bounded step decreases function;
//--- if not, we again skip to CG phase
//--- * finally, we use FindBestStep...() function to choose
//--- between bounded step and projection of full-length step
//--- (latter may give additional decrease in
CSActiveSets::SASExploreDirection(sstate.m_sas,sstate.m_dc,stpmax,cidx,cval);
if(cidx<0)
{
//--- Function is unbounded from below:
//--- * D1<0 (verified by previous block)
//--- * D2Est<=0, which means that either D2<0 - or it can not
//--- be reliably distinguished from zero.
//--- * step is unconstrained
//--- If these conditions are true, we abnormally terminate QP
//--- algorithm with return code -4
terminationtype=-4;
break;
}
if(stpmax==0.0)
{
//--- Resort to CG phase.
//--- Increase number of CG iterations.
cgmax=Settings.m_cgmaxits;
break;
}
//--- check
if(!CAp::Assert(stpmax>0.0,__FUNCTION__+": internal error"))
return;
f0=ProjectedTargetFunction(sstate,sstate.m_sas.m_xc,sstate.m_dc,0.0,sstate.m_tmp0,sstate.m_tmp1);
f1=ProjectedTargetFunction(sstate,sstate.m_sas.m_xc,sstate.m_dc,stpmax,sstate.m_tmp0,sstate.m_tmp1);
if(f1>=f0)
{
//--- Descent direction does not actually decrease function value.
//--- Resort to CG phase
//--- Increase number of CG iterations.
cgmax=Settings.m_cgmaxits;
break;
}
//--- check
if(!CAp::Assert(sstate.m_stpbuf.Size()>=3,__FUNCTION__+": StpBuf overflow"))
return;
reststp=stpmax;
stp=reststp;
sstate.m_stpbuf.Set(0,reststp*4);
sstate.m_stpbuf.Set(1,1.00);
sstate.m_stpbuf.Set(2,0.25);
stpcnt=3;
FindBestStepAndMove(sstate,sstate.m_sas,sstate.m_dc,stp,true,cidx,cval,sstate.m_stpbuf,stpcnt,sstate.m_activated,sstate.m_tmp0,sstate.m_tmp1);
}
}
if(terminationtype!=0)
break;
}
CSActiveSets::SASStopOptimization(sstate.m_sas);
sstate.m_xf=sstate.m_sas.m_xc;
}
//--- Stop optimization and unpack results.
//--- Add XOriginC to XS and make sure that boundary constraints are
//--- both (a) satisfied, (b) preserved. Former means that "shifted"
//--- point is feasible, while latter means that point which was exactly
//--- at the boundary before shift will be exactly at the boundary
//--- after shift.
for(i=0; i<n; i++)
{
xs.Set(i,sc[i]*sstate.m_xf[i]+xoriginc[i]);
if(sstate.m_havebndl[i] && xs[i]<bndlc[i])
xs.Set(i,bndlc[i]);
if(sstate.m_havebndu[i] && xs[i]>bnduc[i])
xs.Set(i,bnduc[i]);
if(sstate.m_havebndl[i] && sstate.m_xf[i]==sstate.m_bndl[i])
xs.Set(i,bndlc[i]);
if(sstate.m_havebndu[i] && sstate.m_xf[i]==sstate.m_bndu[i])
xs.Set(i,bnduc[i]);
}
}
//+------------------------------------------------------------------+
//| Target function at point PROJ(X + Stp*D), where PROJ(.) is a |
//| projection into feasible set. |
//| NOTE: if Stp = 0, D is not referenced at all. Thus, there is no |
//| need to fill it by some meaningful values for Stp = 0. |
//| This subroutine uses temporary buffers Tmp0 / 1, which are |
//| automatically resized if needed. |
//+------------------------------------------------------------------+
double CQQPSolver::ProjectedTargetFunction(CQQPBuffers &sstate,
CRowDouble &x,
CRowDouble &d,
double stp,
CRowDouble &m_tmp0,
CRowDouble &m_tmp1)
{
//--- create variables
double result=0;
int n=sstate.m_n;
double v=0;
m_tmp0.Resize(n);
m_tmp1.Resize(n);
//--- Calculate projected point
for(int i=0; i<n; i++)
{
if(stp!=0.0)
v=x[i]+stp*d[i];
else
v=x[i];
if(sstate.m_havebndl[i] && v<sstate.m_bndl[i])
v=sstate.m_bndl[i];
if(sstate.m_havebndu[i] && v>sstate.m_bndu[i])
v=sstate.m_bndu[i];
m_tmp0.Set(i,v);
}
//--- Function value at the Tmp0:
//--- f(x) = 0.5*x'*A*x + b'*x
result=sstate.m_b.Dot(m_tmp0);
if(sstate.m_akind==0)
{
//--- Dense matrix A
result+=0.5*CAblas::RMatrixSyvMVect(n,sstate.m_densea,0,0,true,m_tmp0,0,m_tmp1);
}
else
{
//--- sparse matrix A
//--- check
if(!CAp::Assert(sstate.m_akind==1,__FUNCTION__+": unexpected AKind in ProjectedTargetFunction"))
return(0.0);
result+=0.5*CSparse::SparseVSMV(sstate.m_sparsea,sstate.m_sparseupper,m_tmp0);
}
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Gradient of the target function: |
//| f(x) = 0.5 * x'*A*x + b'*x |
//| which is equal to grad = A * x + b |
//| Here: |
//| * x is array[N] |
//| * A is array[N, N] |
//| * b is array[N] |
//| INPUT PARAMETERS: |
//| SState - structure which stores function terms(not modified)|
//| X - location |
//| G - possibly preallocated buffer |
//| OUTPUT PARAMETERS: |
//| G - array[N], gradient |
//+------------------------------------------------------------------+
void CQQPSolver::TargetGradient(CQQPBuffers &sstate,
CRowDouble &x,
CRowDouble &g)
{
int n=sstate.m_n;
g.Resize(n);
if(sstate.m_akind==0)
{
//--- Dense matrix A
CAblas::RMatrixSymVect(n,1.0,sstate.m_densea,0,0,true,x,0,0.0,g,0);
}
else
{
//--- Sparse matrix A
//--- check
if(!CAp::Assert(sstate.m_akind==1,__FUNCTION__+": unexpected AKind in TargetGradient"))
return;
CSparse::SparseSMV(sstate.m_sparsea,sstate.m_sparseupper,x,g);
}
g+=sstate.m_b;
}
//+------------------------------------------------------------------+
//| First and second derivatives of the "extended" target function |
//| along specified direction. Target function is called "extended" |
//| because of additional slack variables and has form: |
//| f(x)=0.5*x'*A*x+b'*x+penaltyfactor*0.5*(C*x-b)'*(C*x-b) |
//| with gradient grad = A * x + b + penaltyfactor * C'*(C*x-b) |
//| Quadratic model has form |
//| F(x0 + alpha*D) = D2 * alpha ^ 2 + D1 * alpha |
//| INPUT PARAMETERS: |
//| SState - structure which is used to obtain quadratic term |
//| of the model |
//| X - current point, array[N] |
//| D - direction across which derivatives are calculated, |
//| array[N] |
//| G - gradient at current point (pre-calculated by |
//| caller), array[N] |
//| OUTPUT PARAMETERS: |
//| D1 - linear coefficient |
//| D1Est - estimate of D1 sign, accounting for possible |
//| numerical errors: |
//| *>0 means "almost surely positive" |
//| *<0 means "almost surely negative" |
//| *=0 means "pessimistic estimate of numerical |
//| errors in D1 is larger than magnitude of |
//| D1 itself; it is impossible to reliably |
//| distinguish D1 from zero". |
//| D2 - quadratic coefficient |
//| D2Est - estimate of D2 sign, accounting for possible |
//| numerical errors: |
//| *>0 means "almost surely positive" |
//| *<0 means "almost surely negative" |
//| *=0 means "pessimistic estimate of numerical |
//| errors in D2 is larger than magnitude of |
//| D2 itself; it is impossible to reliably |
//| distinguish D2 from zero". |
//+------------------------------------------------------------------+
void CQQPSolver::QuadraticModel(CQQPBuffers &sstate,CRowDouble &x,
CRowDouble &d,CRowDouble &g,
double &d1,int &d1est,double &d2,
int &d2est,CRowDouble &m_tmp0)
{
//--- create variables
int n=sstate.m_n;
double v=0;
double mx=(x.Abs()+0).Max();
double md=(d.Abs()+0).Max();
double mb=(sstate.m_b.Abs()+0).Max();
d1=0;
d1est=0;
d2=0;
d2est=0;
//--- Maximums
//--- D2
if(sstate.m_akind==0)
{
//--- Dense matrix A
d2=0.5*CAblas::RMatrixSyvMVect(n,sstate.m_densea,0,0,true,d,0,m_tmp0);
}
else
{
//--- Sparse matrix A
//--- check
if(!CAp::Assert(sstate.m_akind==1,__FUNCTION__+": unexpected AKind in TargetGradient"))
return;
d2=0.5*CSparse::SparseVSMV(sstate.m_sparsea,sstate.m_sparseupper,d);
}
v=d.Dot(g);
d1=v;
//--- Error estimates
COptServ::EstimateParabolicModel(sstate.m_absasum,sstate.m_absasum2,mx,mb,md,d1,d2,d1est,d2est);
}
//+------------------------------------------------------------------+
//| This function accepts quadratic model of the form |
//| f(x) = 0.5*x'*A*x+b'*x+penaltyfactor*0.5*(C*x-b)'*(C*x-b) |
//| and list of possible steps along direction D. It chooses best |
//| step (one which achieves minimum value of the target function) |
//| and moves current point (given by SAS object) to the new location|
//| Step is bounded subject to boundary constraints. |
//| Candidate steps are divided into two groups: |
//| *"default" step, which is always performed when no candidate |
//| steps LONGER THAN THE DEFAULT ONE is given. This candidate |
//| MUST reduce target function value; it is responsibility of |
//| caller to provide default candidate which reduces target |
//| function. |
//| *"additional candidates", which may be shorter or longer than |
//| the default step. Candidates which are shorter that the |
//| default step are ignored; candidates which are longer than |
//| the "default" step are tested. |
//| The idea is that we ALWAYS try "default" step, and it is |
//| responsibility of the caller to provide us with something which |
//| is worth trying. This step may activate some constraint - that's |
//| why we stopped at "default" step size. However, we may also try |
//| longer steps which may activate additional constraints and |
//| further reduce function value. |
//| INPUT PARAMETERS: |
//| SState - structure which stores model |
//| SAS - active set structure which stores current point in |
//| SAS.XC |
//| D - direction for step |
//| Stp - step length for "default" candidate |
//| NeedAct - whether default candidate activates some constraint|
//| if NeedAct is True, constraint given by CIdc/CVal |
//| is GUARANTEED to be activated in the final point. |
//| CIdx - if NeedAct is True, stores index of the constraint |
//| to activate |
//| CVal - if NeedAct is True, stores constrained value; |
//| SAS.XC[CIdx] is forced to be equal to CVal. |
//| AddSteps - array[AddStepsCnt] of additional steps: |
//| * AddSteps[] <= Stp are ignored |
//| * AddSteps[] > Stp are tried |
//| Activated - possibly preallocated buffer; previously allocated |
//| memory will be reused. |
//| Tmp0 / 1 - possibly preallocated buffers; previously allocated|
//| memory will be reused. |
//| OUTPUT PARAMETERS: |
//| SAS - SAS.XC is set to new point; if there was a |
//| constraint specified by NeedAct/CIdx/CVal, it will |
//| be activated (other constraints may be activated |
//| too, but this one is guaranteed to be active in the|
//| final point). |
//| Activated - elements of this array are set to True, if I-Th |
//| constraint as inactive at previous point, but |
//| become active in the new one. |
//| Situations when we deactivate xi >= 0 and activate xi <= 1 are |
//| considered as activation of previously inactive constraint |
//+------------------------------------------------------------------+
void CQQPSolver::FindBestStepAndMove(CQQPBuffers &sstate,
CSActiveSet &sas,
CRowDouble &d,
double stp,
bool needact,
int cidx,
double cval,
CRowDouble &addsteps,
int addstepscnt,
bool &activated[],
CRowDouble &m_tmp0,
CRowDouble &m_tmp1)
{
//--- create variables
int n=sstate.m_n;
int i=0;
int k=0;
double v=0;
double stpbest=0;
double fbest=0;
double fcand=0;
m_tmp0.Resize(n);
CApServ::BVectorSetLengthAtLeast(activated,n);
//--- Calculate initial step, store to Tmp0
//--- NOTE: Tmp0 is guaranteed to be feasible w.m_r.m_t. boundary constraints
for(i=0; i<n; i++)
{
v=sas.m_xc[i]+stp*d[i];
if(sstate.m_havebndl[i] && v<sstate.m_bndl[i])
v=sstate.m_bndl[i];
if(sstate.m_havebndu[i] && v>sstate.m_bndu[i])
v=sstate.m_bndu[i];
m_tmp0.Set(i,v);
}
if(needact)
m_tmp0.Set(cidx,cval);
//--- Try additional steps, if AddStepsCnt>0
if(addstepscnt>0)
{
//--- Find best step
stpbest=stp;
fbest=ProjectedTargetFunction(sstate,sas.m_xc,d,stpbest,m_tmp0,m_tmp1);
for(k=0; k<=addstepscnt-1; k++)
{
if(addsteps[k]>stp)
{
fcand=ProjectedTargetFunction(sstate,sas.m_xc,d,addsteps[k],m_tmp0,m_tmp1);
if(fcand<fbest)
{
fbest=fcand;
stpbest=addsteps[k];
}
}
}
//--- Prepare best step
//--- NOTE: because only AddSteps[]>Stp were checked,
//--- this step will activate constraint CIdx.
for(i=0; i<n; i++)
{
v=sas.m_xc[i]+stpbest*d[i];
if(sstate.m_havebndl[i] && v<sstate.m_bndl[i])
v=sstate.m_bndl[i];
if(sstate.m_havebndu[i] && v>sstate.m_bndu[i])
v=sstate.m_bndu[i];
m_tmp0.Set(i,v);
}
if(needact)
m_tmp0.Set(cidx,cval);
}
//--- Fill Activated array by information about activated constraints.
//--- Perform step
for(i=0; i<n; i++)
{
activated[i]=false;
v=m_tmp0[i];
if(v==sas.m_xc[i])
continue;
if(sstate.m_havebndl[i] && v==sstate.m_bndl[i])
activated[i]=true;
if(sstate.m_havebndu[i] && v==sstate.m_bndu[i])
activated[i]=true;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
CSActiveSets::SASMoveTo(sas,m_tmp0,needact,cidx,cval);
}
//+------------------------------------------------------------------+
//| This function prepares data for constrained Newton step for |
//| penalized quadratic model of the form |
//| f(x) = 0.5*x'*A*x+b'*x+penaltyfactor*0.5*(C*x-b)'*(C*x-b) |
//| where A can be dense or sparse, and model is considered subject |
//| to equality constraints specified by SState.SAS.XC object. |
//| Constraint is considered active if XC[i] is exactly BndL[i] or |
//| BndU[i], i.e. we ignore internal list of constraints monitored |
//| by SAS object. Our own set of constraints includes all |
//| constraints stored by SAS, but also may include some constraints |
//| which are inactive in SAS. |
//| "Preparation" means that Cholesky decomposition of the effective |
//| system matrix is performed, and we can perform constrained Newton|
//| step. |
//| This function works as black box. It uses fields of SState which |
//| are marked as "Variables for constrained Newton phase", and only |
//| this function and its friends know about these variables. |
//| Everyone else should use: |
//| * CNewtonBuild() to prepare initial Cholesky decomposition for|
//| step |
//| * CNewtonStep() to perform constrained Newton step |
//| * CNewtonUpdate() to update Cholesky matrix after point was |
//| moved and constraints were updated. In some |
//| cases it is possible to efficiently real - |
//| calculate Cholesky decomposition if you know |
//| which constraints were activated. If |
//| efficient real - calculation is impossible, |
//| this function returns False. |
//| INPUT PARAMETERS: |
//| SState - structure which stores model and temporaries |
//| for CN phase; in particular, SAS.XC stores |
//| current point. |
//| SparseSolver - which sparse m_solver to use for sparse model; |
//| ignored for dense QP. Can be: |
//| * 2 - SKS - based Cholesky |
//| NCholesky - counter which is incremented after Cholesky |
//| (successful or failed one) |
//| OUTPUT PARAMETERS: |
//| NCholesky - possibly updated counter |
//| RESULT: |
//| True, if Cholesky decomposition was successfully performed. |
//| False, if a) matrix was semi - definite or indefinite, or |
//| b) particular combination of matrix type(sparse) |
//| and constraints (general linear) is not supported.|
//| NOTE: this function may routinely return False, for indefinite |
//| matrices or for sparse problems with general linear |
//| constraints. You should be able to handle such situations. |
//+------------------------------------------------------------------+
bool CQQPSolver::CNewtonBuild(CQQPBuffers &sstate,int sparsesolver,
int &ncholesky)
{
//--- create variables
int n=sstate.m_n;
int i=0;
int j=0;
int k=0;
double v=0;
bool b;
int ridx0=0;
int ridx1=0;
int nfree=0;
int i_=0;
//--- 1. Set CNModelAge to zero
//--- 2. Generate YIdx - reordering of variables such that free variables
//--- come first and are ordered by ascending, fixed are last ones and
//--- have no particular ordering.
//--- This step is same for dense and sparse problems.
sstate.m_cnmodelage=0;
sstate.m_yidx.Resize(n);
ridx0=0;
ridx1=n-1;
sstate.m_yidx.Fill(-1,0,n);
for(i=0; i<n; i++)
{
//--- check
if(!CAp::Assert(!sstate.m_havebndl[i] || sstate.m_sas.m_xc[i]>=sstate.m_bndl[i],__FUNCTION__+": internal error"))
return(false);
if(!CAp::Assert(!sstate.m_havebndu[i] || sstate.m_sas.m_xc[i]<=sstate.m_bndu[i],__FUNCTION__+": internal error"))
return(false);
b=false;
b=b || (sstate.m_havebndl[i] && sstate.m_sas.m_xc[i]==sstate.m_bndl[i]);
b=b || (sstate.m_havebndu[i] && sstate.m_sas.m_xc[i]==sstate.m_bndu[i]);
if(b)
{
sstate.m_yidx.Set(ridx1,i);
ridx1--;
}
else
{
sstate.m_yidx.Set(ridx0,i);
ridx0++;
}
}
//--- check
if(!CAp::Assert(ridx0==ridx1+1,__FUNCTION__+": internal error"))
return(false);
nfree=ridx0;
sstate.m_nfree=nfree;
if(nfree==0)
return(false);
//--- Constrained Newton matrix: dense version
if(sstate.m_akind==0)
{
sstate.m_densez=sstate.m_densea;
sstate.m_densez.Resize(n,n);
sstate.m_tmpcn.Resize(n);
for(i=1; i<nfree; i++)
{
//--- check
if(!CAp::Assert(sstate.m_yidx[i]>sstate.m_yidx[i-1],__FUNCTION__+": integrity check failed"))
return(false);
}
for(i=0; i<nfree; i++)
{
k=sstate.m_yidx[i];
for(j=i; j<nfree; j++)
sstate.m_densez.Set(i,j,sstate.m_densez.Get(k,sstate.m_yidx[j]));
}
sstate.m_regdiag.Resize(n);
for(i=0; i<nfree; i++)
{
v=0.0;
for(j=0; j<i; j++)
v+= MathAbs(sstate.m_densez.Get(j,i));
for(j=i; j<nfree; j++)
v+= MathAbs(sstate.m_densez.Get(i,j));
if(v==0.0)
v=1.0;
sstate.m_regdiag.Set(i,m_regz*v);
}
for(i=0; i<nfree; i++)
sstate.m_densez.Add(i,i,sstate.m_regdiag[i]);
ncholesky++;
if(!CTrFac::SPDMatrixCholeskyRec(sstate.m_densez,0,nfree,true,sstate.m_tmpcn))
return(false);
for(i=nfree-1; i>=0; i--)
{
for(i_=i; i_<nfree; i_++)
sstate.m_tmpcn.Set(i_,sstate.m_densez.Get(i,i_));
k=sstate.m_yidx[i];
for(j=k; j<n; j++)
sstate.m_densez.Set(k,j,0);
for(j=i; j<nfree; j++)
sstate.m_densez.Set(k,sstate.m_yidx[j],sstate.m_tmpcn[j]);
}
for(i=nfree; i<n; i++)
{
k=sstate.m_yidx[i];
sstate.m_densez.Set(k,k,1.0);
for(j=k+1; j<n; j++)
sstate.m_densez.Set(k,j,0);
}
//--- return result
return(true);
}
//--- Constrained Newton matrix: sparse version
if(sstate.m_akind==1)
{
//--- check
if(!CAp::Assert(sparsesolver==2,__FUNCTION__+": internal error"))
return(false);
//--- Copy sparse A to Z and fill rows/columns corresponding to active
//--- constraints by zeros. Diagonal elements corresponding to active
//--- constraints are filled by unit values.
CSparse::SparseCopyToSKSBuf(sstate.m_sparsea,sstate.m_sparsecca);
sstate.m_tmpcn=vector<double>::Zeros(n);
for(i=nfree; i<n; i++)
sstate.m_tmpcn.Set(sstate.m_yidx[i],1);
for(i=0; i<n; i++)
{
k=sstate.m_sparsecca.m_RIdx[i];
for(j=i-sstate.m_sparsecca.m_DIdx[i]; j<=i; j++)
{
if(sstate.m_tmpcn[i]!=0.0 || sstate.m_tmpcn[j]!=0.0)
{
//--- I-th or J-th variable is in active set (constrained)
if(i==j)
sstate.m_sparsecca.m_Vals.Set(k,1.0);
else
sstate.m_sparsecca.m_Vals.Set(k,0.0);
}
k++;
}
}
//--- Perform sparse Cholesky
ncholesky++;
if(!CTrFac::SparseCholeskySkyLine(sstate.m_sparsecca,n,sstate.m_sparseupper))
return(false);
//--- return result
return(true);
}
//--- Unexpected :)
CAp::Assert(false,__FUNCTION__+": internal error");
return(false);
}
//+------------------------------------------------------------------+
//| This function updates equality-constrained Cholesky matrix after |
//| activation of the new equality constraints. Matrix being updated |
//| is quadratic term of the function below |
//| f(x) = 0.5*x'*A*x+b'*x+penaltyfactor*0.5*(C*x-b)'*(C*x-b) |
//| where A can be dense or CSparse:: |
//| This function uses YIdx[] array(set by CNewtonBuild() function) |
//| to distinguish between active and inactive constraints. |
//| This function works as black box. It uses fields of SState which |
//| are marked as "Variables for constrained Newton phase", and only |
//| this function and its friends know about these variables. |
//| Everyone else should use: |
//| * CNewtonBuild() to prepare initial Cholesky decomposition for|
//| step |
//| * CNewtonStep() to perform constrained Newton step |
//| * CNewtonUpdate() to update Cholesky matrix after point was |
//| moved and constraints were updated. In some |
//| cases it is possible to efficiently real - |
//| calculate Cholesky decomposition if you know |
//| which constraints were activated. If |
//| efficient real - calculation is impossible, |
//| this function returns False. |
//| INPUT PARAMETERS: |
//| SState - structure which stores model and temporaries for|
//| CN phase; in particular, SAS.XC stores current |
//| point. |
//| Settings - QQPSettings object which was initialized by |
//| appropriate construction function. |
//| NCUpdates - counter which is incremented after each update |
//| (one update means one variable being fixed) |
//| OUTPUT PARAMETERS: |
//| NCUpdates - possibly updated counter |
//| RESULT: |
//| True, if Cholesky decomposition was successfully performed. |
//| False, if a) model age was too high, or b) particular |
//| combination of matrix type(sparse) and constraints|
//| (general linear) is not supported |
//| NOTE: this function may routinely return False. You should be |
//| able to handle such situations. |
//+------------------------------------------------------------------+
bool CQQPSolver::CNewtonUpdate(CQQPBuffers &sstate,CQQPSettings &Settings,
int &ncupdates)
{
//--- return result
int n=sstate.m_n;
int nfree=sstate.m_nfree;
int ntofix=0;
bool b;
int ridx0=0;
int ridx1=0;
int i=0;
int k=0;
//--- Cholesky updates for sparse problems are not supported
if(sstate.m_akind==1)
return(false);
//--- Determine variables to fix and move them to YIdx[NFree-NToFix:NFree-1]
//--- Exit if CNModelAge increased too much.
sstate.m_tmpcni.Resize(n);
ridx0=0;
ridx1=nfree-1;
for(i=0; i<nfree; i++)
sstate.m_tmpcni.Set(i,-1);
for(k=0; k<nfree; k++)
{
i=sstate.m_yidx[k];
//--- check
if(!CAp::Assert(!sstate.m_havebndl[i] || sstate.m_sas.m_xc[i]>=sstate.m_bndl[i],__FUNCTION__+": internal error"))
return(false);
if(!CAp::Assert(!sstate.m_havebndu[i] || sstate.m_sas.m_xc[i]<=sstate.m_bndu[i],__FUNCTION__+": internal error"))
return(false);
b=false;
b=b || (sstate.m_havebndl[i] && sstate.m_sas.m_xc[i]==sstate.m_bndl[i]);
b=b || (sstate.m_havebndu[i] && sstate.m_sas.m_xc[i]==sstate.m_bndu[i]);
if(b)
{
sstate.m_tmpcni.Set(ridx1,i);
ridx1--;
}
else
{
sstate.m_tmpcni.Set(ridx0,i);
ridx0++;
}
}
//--- check
if(!CAp::Assert(ridx0==ridx1+1,__FUNCTION__+": internal error"))
return(false);
ntofix=nfree-ridx0;
if(ntofix==0 || ntofix==nfree)
return(false);
if(sstate.m_cnmodelage+ntofix>Settings.m_cnmaxupdates)
return(false);
for(i=0; i<nfree; i++)
{
sstate.m_yidx.Set(i,sstate.m_tmpcni[i]);
}
//--- Constrained Newton matrix: dense version.
if(sstate.m_akind==0)
{
//--- Update Cholesky matrix with SPDMatrixCholeskyUpdateFixBuf()
CApServ::BVectorSetLengthAtLeast(sstate.m_tmpcnb,n);
ArrayInitialize(sstate.m_tmpcnb,false);
for(i=nfree-ntofix; i<nfree; i++)
sstate.m_tmpcnb[sstate.m_yidx[i]]=true;
CTrFac::SPDMatrixCholeskyUpdateFixBuf(sstate.m_densez,n,true,sstate.m_tmpcnb,sstate.m_tmpcn);
//--- Update information stored in State and exit
sstate.m_nfree=nfree-ntofix;
sstate.m_cnmodelage=sstate.m_cnmodelage+ntofix;
ncupdates=ncupdates+ntofix;
//--- return result
return(true);
}
//--- Unexpected :)
CAp::Assert(false,__FUNCTION__+": internal error");
return(false);
}
//+------------------------------------------------------------------+
//| This function prepares equality - constrained Newton step using |
//| previously calculated constrained Cholesky matrix of the problem |
//| f(x) = 0.5*x'*A*x+b'*x+penaltyfactor*0.5*(C*x-b)'*(C*x-b) |
//| where A can be dense or CSparse:: |
//| As input, this function accepts gradient at the current location.|
//| As output, it returns step vector (replaces gradient). |
//| This function works as black box. It uses fields of SState which |
//| are marked as "Variables for constrained Newton phase", and only |
//| this function and its friends know about these variables. |
//| Everyone else should use: |
//| * CNewtonBuild() to prepare initial Cholesky decomposition for|
//| step |
//| * CNewtonStep() to perform constrained Newton step |
//| * CNewtonUpdate() to update Cholesky matrix after point was |
//| moved and constraints were updated. In some |
//| cases it is possible to efficiently real - |
//| calculate Cholesky decomposition if you know |
//| which constraints were activated. If |
//| efficient real - calculation is impossible, |
//| this function returns False. |
//| INPUT PARAMETERS: |
//| SState - structure which stores model and temporaries for|
//| CN phase; in particular, SAS.XC stores current |
//| point. |
//| Settings - QQPSettings object which was initialized by |
//| appropriate construction function. |
//| GC - array[N], gradient of the target function |
//| OUTPUT PARAMETERS: |
//| GC - array[N], step vector(on success) |
//| RESULT: |
//| True, if step was successfully calculated. |
//| False, if step calculation failed: |
//| a) gradient was exactly zero, |
//| b) gradient norm was smaller than EpsG (stopping |
//| condition) |
//| c) all variables were equality - constrained |
//| NOTE: this function may routinely return False. You should be |
//| able to handle such situations. |
//+------------------------------------------------------------------+
bool CQQPSolver::CNewtonStep(CQQPBuffers &sstate,CQQPSettings &Settings,
CRowDouble &gc)
{
//--- create variables
int n=sstate.m_n;
int nfree=sstate.m_nfree;
double v=0;
for(int i=nfree; i<n; i++)
gc.Set(sstate.m_yidx[i],0.0);
v=gc.Dot(gc);
if(MathSqrt(v)<=Settings.m_epsg)
return(false);
gc*=(-1.0);
if(sstate.m_akind==0)
{
//--- Dense Newton step.
//--- Use straightforward Cholesky m_solver.
CFbls::FblsCholeskySolve(sstate.m_densez,1.0,n,true,gc,sstate.m_tmpcn);
return(true);
}
if(sstate.m_akind==1)
{
//--- Sparse Newton step.
//--- We have T*T' = L*L' = U'*U (depending on specific triangle stored in SparseCCA).
if(sstate.m_sparseupper)
{
CSparse::SparseTRSV(sstate.m_sparsecca,sstate.m_sparseupper,false,1,gc);
CSparse::SparseTRSV(sstate.m_sparsecca,sstate.m_sparseupper,false,0,gc);
}
else
{
CSparse::SparseTRSV(sstate.m_sparsecca,sstate.m_sparseupper,false,0,gc);
CSparse::SparseTRSV(sstate.m_sparsecca,sstate.m_sparseupper,false,1,gc);
}
return(true);
}
//--- return result
CAp::Assert(false,__FUNCTION__+": internal error");
return(false);
}
//+------------------------------------------------------------------+
//| This object stores Settings for DENSE - AUL m_solver. |
//| It must be initialized with QPDENSEAULLoadDefaults(). |
//| After initialization you may change Settings. |
//+------------------------------------------------------------------+
struct CQPDenseAULSettings
{
int m_outerits;
double m_epsx;
double m_rho;
//--- constructor / destructor
CQPDenseAULSettings(void) { ZeroMemory(this); }
~CQPDenseAULSettings(void) {}
//---
void Copy(const CQPDenseAULSettings &obj);
//--- overloading
void operator=(const CQPDenseAULSettings &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CQPDenseAULSettings::Copy(const CQPDenseAULSettings &obj)
{
m_outerits=obj.m_outerits;
m_epsx=obj.m_epsx;
m_rho=obj.m_rho;
}
//+------------------------------------------------------------------+
//| This object stores temporaries used by Dense - AUL m_solver. |
//+------------------------------------------------------------------+
struct CQPDenseAULBuffers
{
int m_repinneriterationscount;
int m_repncholesky;
int m_repnmv;
int m_repnwrk0;
int m_repnwrk1;
int m_repnwrkchanges;
int m_repnwrkf;
int m_repouteriterationscount;
bool m_sclsfthasbndl[];
bool m_sclsfthasbndu[];
CSparseMatrix m_dummysparse;
CRowInt m_cidx;
CRowInt m_nicnact;
CRowDouble m_cscales;
CRowDouble m_d;
CRowDouble m_deltax;
CRowDouble m_exb;
CRowDouble m_exbndl;
CRowDouble m_exbndu;
CRowDouble m_exscale;
CRowDouble m_exxc;
CRowDouble m_exxorigin;
CRowDouble m_modelg;
CRowDouble m_nicerr;
CRowDouble m_nulc;
CRowDouble m_nulcest;
CRowDouble m_qrrightpart;
CRowDouble m_qrsv0;
CRowDouble m_qrsvx1;
CRowDouble m_qrtau;
CRowDouble m_sclsftb;
CRowDouble m_sclsftbndl;
CRowDouble m_sclsftbndu;
CRowDouble m_sclsftxc;
CRowDouble m_tmp0;
CRowDouble m_tmpg;
CQQPSettings m_qqpsettingsuser;
CQQPBuffers m_qqpbuf;
CMatrixDouble m_exa;
CMatrixDouble m_qrkkt;
CMatrixDouble m_sclsfta;
CMatrixDouble m_sclsftcleic;
CMatrixDouble m_tmp2;
CConvexQuadraticModel m_dummycqm;
//--- constructor / destructor
CQPDenseAULBuffers(void);
~CQPDenseAULBuffers(void) {}
//---
void Copy(const CQPDenseAULBuffers &obj);
//--- overloading
void operator=(const CQPDenseAULBuffers &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CQPDenseAULBuffers::CQPDenseAULBuffers(void)
{
m_repinneriterationscount=0;
m_repncholesky=0;
m_repnmv=0;
m_repnwrk0=0;
m_repnwrk1=0;
m_repnwrkchanges=0;
m_repnwrkf=0;
m_repouteriterationscount=0;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CQPDenseAULBuffers::Copy(const CQPDenseAULBuffers &obj)
{
m_repinneriterationscount=obj.m_repinneriterationscount;
m_repncholesky=obj.m_repncholesky;
m_repnmv=obj.m_repnmv;
m_repnwrk0=obj.m_repnwrk0;
m_repnwrk1=obj.m_repnwrk1;
m_repnwrkchanges=obj.m_repnwrkchanges;
m_repnwrkf=obj.m_repnwrkf;
m_repouteriterationscount=obj.m_repouteriterationscount;
ArrayCopy(m_sclsfthasbndl,obj.m_sclsfthasbndl);
ArrayCopy(m_sclsfthasbndu,obj.m_sclsfthasbndu);
m_dummysparse=obj.m_dummysparse;
m_cidx=obj.m_cidx;
m_nicnact=obj.m_nicnact;
m_cscales=obj.m_cscales;
m_d=obj.m_d;
m_deltax=obj.m_deltax;
m_exb=obj.m_exb;
m_exbndl=obj.m_exbndl;
m_exbndu=obj.m_exbndu;
m_exscale=obj.m_exscale;
m_exxc=obj.m_exxc;
m_exxorigin=obj.m_exxorigin;
m_modelg=obj.m_modelg;
m_nicerr=obj.m_nicerr;
m_nulc=obj.m_nulc;
m_nulcest=obj.m_nulcest;
m_qrrightpart=obj.m_qrrightpart;
m_qrsv0=obj.m_qrsv0;
m_qrsvx1=obj.m_qrsvx1;
m_qrtau=obj.m_qrtau;
m_sclsftb=obj.m_sclsftb;
m_sclsftbndl=obj.m_sclsftbndl;
m_sclsftbndu=obj.m_sclsftbndu;
m_sclsftxc=obj.m_sclsftxc;
m_tmp0=obj.m_tmp0;
m_tmpg=obj.m_tmpg;
m_qqpsettingsuser=obj.m_qqpsettingsuser;
m_qqpbuf=obj.m_qqpbuf;
m_exa=obj.m_exa;
m_qrkkt=obj.m_qrkkt;
m_sclsfta=obj.m_sclsfta;
m_sclsftcleic=obj.m_sclsftcleic;
m_tmp2=obj.m_tmp2;
m_dummycqm=obj.m_dummycqm;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CQPDenseAULSolver
{
public:
//--- constants
static const double m_evictionlevel;
static const double m_expansionratio;
static void QPDenseAULLoadDefaults(int nmain,CQPDenseAULSettings &s);
static void QPDenseAULOptimize(CConvexQuadraticModel &a,CSparseMatrix &sparsea,int akind,bool sparseaupper,CRowDouble &b,CRowDouble &bndl,CRowDouble &bndu,CRowDouble &s,CRowDouble &xorigin,int nn,CMatrixDouble &cleic,int dnec,int dnic,CSparseMatrix &scleic,int snec,int snic,bool renormlc,CQPDenseAULSettings &Settings,CQPDenseAULBuffers &State,CRowDouble &xs,CRowDouble &lagbc,CRowDouble &laglc,int &terminationtype);
private:
static void GenerateExModel(CMatrixDouble &sclsfta,CRowDouble &sclsftb,int nmain,CRowDouble &sclsftbndl,bool &sclsfthasbndl[],CRowDouble &sclsftbndu,bool &sclsfthasbndu[],CMatrixDouble &sclsftcleic,int sclsftnec,int sclsftnic,CRowDouble &nulc,double rho,CMatrixDouble &exa,CRowDouble &exb,CRowDouble &exbndl,CRowDouble &exbndu,CMatrixDouble &tmp2);
static void GenerateExInitialPoint(CRowDouble &sclsftxc,int nmain,int nslack,CRowDouble &exxc);
static void UpdateLagrangeMultipliers(CMatrixDouble &sclsfta,CRowDouble &sclsftb,int nmain,CRowDouble &sclsftbndl,bool &sclsfthasbndl[],CRowDouble &sclsftbndu,bool &sclsfthasbndu[],CMatrixDouble &sclsftcleic,int sclsftnec,int sclsftnic,CRowDouble &exxc,CRowDouble &nulcest,CQPDenseAULBuffers &buffers);
static void ScaleShiftOriginalPproblem(CConvexQuadraticModel &a,CSparseMatrix &sparsea,int akind,bool sparseaupper,CRowDouble &b,CRowDouble &bndl,CRowDouble &bndu,CRowDouble &s,CRowDouble &xorigin,int nmain,CMatrixDouble &cleic,int dnec,int dnic,CSparseMatrix &scleic,int snec,int snic,bool renormlc,CQPDenseAULBuffers &State,CRowDouble &xs);
static double NormalizeQuadraticTerm(CMatrixDouble &a,CRowDouble &b,int n,CMatrixDouble &cleic,int nec,int nic,bool usecleic,CMatrixDouble &tmp2);
static void SelectInitialWorkingSet(CMatrixDouble &a,int nmain,CMatrixDouble &cleic,int nec,int nic,CRowDouble &m_tmp0,CMatrixDouble &tmp2,int &nicwork,bool &allowwseviction);
};
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
const double CQPDenseAULSolver::m_evictionlevel=-0.01;
const double CQPDenseAULSolver::m_expansionratio=0.20;
//+------------------------------------------------------------------+
//| This function initializes QPDENSEAULSettings structure with |
//| default Settings. |
//| Newly created structure MUST be initialized by default Settings -|
//| or by copy of the already initialized structure. |
//+------------------------------------------------------------------+
void CQPDenseAULSolver::QPDenseAULLoadDefaults(int nmain,
CQPDenseAULSettings &s)
{
s.m_epsx=1.0E-6;
s.m_outerits=5;
s.m_rho=100.0;
}
//+------------------------------------------------------------------+
//| This function runs Dense-AUL m_solver; it returns after |
//| optimization process was completed. Following QP problem is |
//| solved: |
//| min(0.5 * (x - x_origin)'*A*(x-x_origin)+b' * (x - x_origin)) |
//| subject to combination of box and general linear dense / sparse |
//| constraints. |
//| INPUT PARAMETERS: |
//| DenseA - for dense problems(AKind = 0), A-term of CQM |
//| object contains system matrix. Other terms are |
//| unspecified and should not be referenced. |
//| SparseA - for sparse problems(AKind = 1), CRS format |
//| AKind - sparse matrix format: |
//| * 0 for dense matrix |
//| * 1 for sparse matrix |
//| SparseUpper - which triangle of SparseAC stores matrix - upper|
//| or lower one (for dense matrices this parameter |
//| is not actual). |
//| B - linear term, array[N] |
//| BndL - lower bound, array[N] |
//| BndU - upper bound, array[N] |
//| S - scale vector, array[NC]: |
//| * I-Th element contains scale of I-Th variable, |
//| * SC[I] > 0 |
//| XOrigin - origin term, array[NC]. Can be zero. |
//| N - number of variables in the original formulation|
//| (no slack variables). |
//| CLEIC - dense linear equality / inequality constraints. |
//| Equality constraints come first. |
//| NEC, NIC - number of dense equality/inequality constraints.|
//| SCLEIC - sparse linear equality / inequality constraints.|
//| Equality constraints come first. |
//| SNEC, SNIC - number of sparse equality/inequality constraints|
//| RenormLC - whether constraints should be renormalized |
//| (recommended) or used "as is". |
//| Settings - QPDENSEAULSettings object initialized by one of |
//| the initialization functions. |
//| State - object which stores temporaries |
//| XS - initial point, array[NC] |
//| OUTPUT PARAMETERS: |
//| XS - last point |
//| TerminationType - termination type: |
//| * |
//| * |
//| * |
//+------------------------------------------------------------------+
void CQPDenseAULSolver::QPDenseAULOptimize(CConvexQuadraticModel &a,
CSparseMatrix &sparsea,
int akind,bool sparseaupper,
CRowDouble &b,
CRowDouble &bndl,
CRowDouble &bndu,
CRowDouble &s,
CRowDouble &xorigin,
int nn,CMatrixDouble &cleic,
int dnec,int dnic,
CSparseMatrix &scleic,
int snec,int snic,bool renormlc,
CQPDenseAULSettings &Settings,
CQPDenseAULBuffers &State,
CRowDouble &xs,CRowDouble &lagbc,
CRowDouble &laglc,
int &terminationtype)
{
//--- create variables
int i=0;
int j=0;
int k=0;
double v=0;
double vv=0;
double rho=Settings.m_rho;
double epsx=Settings.m_epsx;
int outeridx=0;
int nmain=nn;
int nslack=dnic+snic;
int ntotal=nmain+nslack;
int nectotal=dnec+snec;
int nictotal=dnic+snic;
int ktotal=dnec+dnic+snec+snic;
double maxrho=1.0E12;
double feaserr=0;
double feaserrprev=0;
double requestedfeasdecrease=0.33;
int goodcounter=0;
int stagnationcounter=0;
int nicwork=0;
int kwork=0;
int nwork=0;
bool allowwseviction;
bool workingsetextended;
double targetscale=0;
int i_=0;
terminationtype=0;
if(epsx<=0.0)
epsx=1.0E-9;
//--- Integrity checks
if(snec+snic>0)
{
if(!CAp::Assert(scleic.m_MatrixType==1,__FUNCTION__+": unexpected sparse matrix format"))
return;
if(!CAp::Assert(scleic.m_M==snec+snic,__FUNCTION__+": unexpected sparse matrix size"))
return;
if(!CAp::Assert(scleic.m_N==nmain+1,__FUNCTION__+": unexpected sparse matrix size"))
return;
}
//--- Prepare
State.m_repinneriterationscount=0;
State.m_repouteriterationscount=0;
State.m_repncholesky=0;
State.m_repnmv=0;
State.m_repnwrkchanges=0;
State.m_repnwrk0=0;
State.m_repnwrk1=0;
State.m_repnwrkf=0;
terminationtype=0;
State.m_cidx.Resize(ktotal);
State.m_nulc.Resize(ktotal);
State.m_nulcest.Resize(ktotal);
State.m_exb.Resize(ntotal);
State.m_exxc.Resize(ntotal);
State.m_exxorigin.Resize(ntotal);
State.m_exbndl.Resize(ntotal);
State.m_exbndu.Resize(ntotal);
State.m_exscale.Resize(ntotal);
State.m_tmp0.Resize(ntotal);
State.m_nicerr.Resize(nictotal);
State.m_nicnact.Resize(nictotal);
//--- Allocate Lagrange multipliers, fill by default values (zeros)
lagbc=vector<double>::Zeros(nmain);
laglc=vector<double>::Zeros(ktotal);
//--- Prepare scaled/shifted model in dense format - input parameters
//--- are converted and stored in State.SclSftA/B/HasBndL/HasBndU/BndL/BndU/CLEIC/XC/CScales
ScaleShiftOriginalPproblem(a,sparsea,akind,sparseaupper,b,bndl,bndu,s,xorigin,nmain,cleic,dnec,dnic,scleic,snec,snic,renormlc,State,xs);
//--- Normalize model in such way that norm(A)~1 (very roughly)
//--- We have two lower bounds for sigma_max(A):
//--- * first estimate is provided by Frobenius norm, it is equal to ANorm/NMain
//--- * second estimate is provided by max(CAC)
//--- We select largest one of these estimates, because using just one
//--- of them is prone to different failure modes. Then, we divide A and B
//--- by this estimate.
targetscale=NormalizeQuadraticTerm(State.m_sclsfta,State.m_sclsftb,nmain,State.m_sclsftcleic,nectotal,nictotal,renormlc,State.m_tmp2);
//--- Select working set of inequality constraints.
//--- Although it is possible to process all inequality constraints
//--- at once, in one large batch, some QP problems have NIC>>N constraints,
//--- but only minor fraction of them is inactive in the solution.
//--- Because algorithm running time is O((N+NEC+NIC)^3), we can
//--- save a lot of time if we process only those inequality constraints
//--- which need activation. Generally, NEC<N, and only O(N) inequality
//--- constraints are active in the solution.
//--- We can do so by solving problem first without general inequality
//--- constraints at all (box and general equality constraints are added),
//--- and by iteratively adding more and more inequality constraints in
//--- order to get feasible solution. Such set of inequality constraints
//--- is called "working set".
//--- NOTE: such approach works reliably only for convex QP problems; non-convex
//--- QP problem can be unbounded when some constraints are dropped.
//--- NOTE: we can also remove some constraints from working set, but eviction
//--- can be performed only limited amount of times (at most once); if
//--- constraint is added to working set second time, it is never removed.
//--- NOTE: we do not perform constraint eviction on non-convex problems
SelectInitialWorkingSet(State.m_sclsfta,nmain,State.m_sclsftcleic,nectotal,nictotal,State.m_tmp0,State.m_tmp2,nicwork,allowwseviction);
kwork=nectotal+nicwork;
nwork=nmain+nicwork;
State.m_repnwrk0=nicwork;
for(i=0; i<nicwork; i++)
State.m_nicnact.Set(i,1);
for(i=nicwork; i<=nictotal-1; i++)
State.m_nicnact.Set(i,0);
//--- Perform outer iteration
for(i=0; i<ktotal; i++)
{
State.m_cidx.Set(i,i);
State.m_nulc.Set(i,0);
}
for(i=0; i<ntotal; i++)
{
State.m_exscale.Set(i,1.0);
State.m_exxorigin.Set(i,0.0);
}
GenerateExInitialPoint(State.m_sclsftxc,nmain,nslack,State.m_exxc);
goodcounter=0;
stagnationcounter=0;
feaserr=CMath::m_maxrealnumber;
for(outeridx=0; outeridx<Settings.m_outerits; outeridx++)
{
//--- Repeat loop until working set stabilizes.
do
{
//--- Preallocate space for ExA and for QQP m_solver; we do not allocate
//--- array[NTotal,NTotal] from the start because NTotal can be much
//--- larger than NMain for problems with large amount of inequality
//--- constraints, and we usually need NWork=O(NMain).
//--- NOTE: for the sake of simplicity, 1-dimensional arrays were
//--- preallocated to the maximum size required (NTotal).
if(State.m_exa.Rows()<nwork || State.m_exa.Cols()<nwork)
{
i=nwork+nwork/3+1;
State.m_exa.Resize(i,i);
}
CQQPSolver::QQPPreAllocateGrowDense(State.m_qqpbuf,nwork,i);
//--- Generate penalized quadratic model
GenerateExModel(State.m_sclsfta,State.m_sclsftb,nmain,State.m_sclsftbndl,State.m_sclsfthasbndl,State.m_sclsftbndu,State.m_sclsfthasbndu,State.m_sclsftcleic,nectotal,nicwork,State.m_nulc,rho,State.m_exa,State.m_exb,State.m_exbndl,State.m_exbndu,State.m_tmp2);
//--- Solve extended QP problem subject to current working set of general
//--- inequality constraints.
CQQPSolver::QQPLoadDefaults(nwork,State.m_qqpsettingsuser);
State.m_qqpsettingsuser.m_maxouterits=50;
State.m_qqpsettingsuser.m_epsg=0.0;
State.m_qqpsettingsuser.m_epsf=0.0;
State.m_qqpsettingsuser.m_epsx=0.01*epsx;
State.m_qqpsettingsuser.m_cnphase=true;
CQQPSolver::QQPOptimize(State.m_dummycqm,State.m_dummysparse,State.m_exa,2,true,State.m_exb,State.m_exbndl,State.m_exbndu,State.m_exscale,State.m_exxorigin,nwork,State.m_qqpsettingsuser,State.m_qqpbuf,State.m_exxc,k);
State.m_repncholesky+=State.m_qqpbuf.m_repncholesky;
//--- Evaluate violation of constraints
for(i=0; i<nictotal; i++)
{
v=0.0;
for(i_=0; i_<nmain; i_++)
v+=State.m_sclsftcleic.Get(nectotal+i,i_)*State.m_exxc[i_];
v-=State.m_sclsftcleic.Get(nectotal+i,nmain);
State.m_nicerr.Set(i,v);
}
//--- Working set expansion:
//--- * select limited amount of most violated constraints
//--- * perform permutation of non-work constraints such that
//--- candidate constraint is first the list (update XC and NuLC)
//--- * increase working set size by 1
//--- * increase activation count for new constraint by 1 (this count
//--- is used later by working set eviction phase)
//--- * repeat
//--- NOTE: we use selection sort algorithm because its O(NAdded*NWork) cost
//--- is still comparable to the cost of constraints evaluation
workingsetextended=false;
i=0;
while((double)(i)<(1+m_expansionratio*nmain) && nicwork<nictotal)
{
//--- Select most violated constraint
k=nicwork;
for(j=nicwork; j<nictotal; j++)
{
if(State.m_nicerr[j]>State.m_nicerr[k])
k=j;
}
//--- If violation is positive, add it
if(State.m_nicerr[k]>0.0)
{
State.m_sclsftcleic.SwapRows(nectotal+nicwork,nectotal+k);
State.m_nicerr.Swap(nicwork,k);
State.m_nicnact.Swap(nicwork,k);
State.m_cidx.Swap(nectotal+nicwork,nectotal+k);
State.m_cscales.Swap(nectotal+nicwork,nectotal+k);
State.m_exxc.Set(nmain+nicwork,0.0);
State.m_nulc.Set(nectotal+nicwork,0.0);
State.m_nicnact.Add(nicwork,1);
nicwork++;
nwork++;
kwork++;
i++;
workingsetextended=true;
}
else
break;
}
//--- Working set eviction:
//--- * select constraints which are (1) far away from the
//--- boundary, AND (2) has less than two activation attempts
//--- (if constraint is regularly activated/deactivated, we keep
//--- it in the working set no matter what)
//--- * remove such constraints from the working set one by one
if(allowwseviction)
{
for(k=nicwork-1; k>=0; k--)
{
if(State.m_nicerr[k]<m_evictionlevel && State.m_nicnact[k]<=1)
{
State.m_sclsftcleic.SwapRows(nectotal+nicwork-1,nectotal+k);
State.m_cidx.Swap(nectotal+nicwork-1,nectotal+k);
State.m_cscales.Swap(nectotal+nicwork-1,nectotal+k);
State.m_nicerr.Swap(nicwork-1,k);
State.m_nicnact.Swap(nicwork-1,k);
State.m_exxc.Swap(nmain+nicwork-1,nmain+k);
State.m_nulc.Swap(nectotal+nicwork-1,nectotal+k);
nicwork--;
nwork--;
kwork--;
}
}
}
//--- Report working set statistics
if(State.m_repnwrk1==0)
State.m_repnwrk1=nicwork;
State.m_repnwrkf=nicwork;
if(workingsetextended)
State.m_repnwrkchanges++;
}
while(workingsetextended);
//--- Estimate Lagrange multipliers using alternative algorithm
for(i_=0; i_<kwork; i_++)
State.m_nulcest.Set(i_,State.m_nulc[i_]);
UpdateLagrangeMultipliers(State.m_sclsfta,State.m_sclsftb,nmain,State.m_sclsftbndl,State.m_sclsfthasbndl,State.m_sclsftbndu,State.m_sclsfthasbndu,State.m_sclsftcleic,nectotal,nicwork,State.m_exxc,State.m_nulcest,State);
//--- Update XC and Lagrange multipliers
feaserrprev=feaserr;
feaserr=0;
for(i=0; i<kwork; i++)
{
//--- Calculate I-th feasibility error in V using formula for distance
//--- between point and line (here we calculate actual distance between
//--- XN and hyperplane Ci'*XN=Bi, which is different from error Ci'*XN-Bi).
v=0;
vv=0;
for(j=0; j<nmain; j++)
{
v+= State.m_sclsftcleic.Get(i,j)*State.m_exxc[j];
vv+=CMath::Sqr(State.m_sclsftcleic.Get(i,j));
}
if(i>=nectotal)
{
v+= State.m_exxc[nmain+(i-nectotal)];
vv++;
}
v-=State.m_sclsftcleic.Get(i,nmain);
vv=CApServ::Coalesce(vv,1);
v=v/MathSqrt(vv);
//--- Calculate magnitude of Lagrangian update (and Lagrangian parameters themselves)
feaserr+=CMath::Sqr(v);
State.m_nulc.Set(i,State.m_nulcest[i]);
}
feaserr=MathSqrt(feaserr);
if(feaserr<epsx)
goodcounter++;
else
goodcounter=0;
if(feaserr>(feaserrprev*requestedfeasdecrease))
stagnationcounter++;
else
stagnationcounter=0;
if(goodcounter>=2)
break;
if(stagnationcounter>=2)
rho=MathMin(rho*10.0,maxrho);
else
rho=MathMin(rho*1.41,maxrho);
}
//--- Convert Lagrange multipliers from internal format to one expected
//--- by caller:
//--- * reorder multipliers for linear constraints
//--- * compute residual from gradient+linearconstraints
//--- * compute multipliers for box constraints from residual
//--- * rescale everything
for(i=0; i<nectotal+nicwork; i++)
if(State.m_cscales[i]!=0)
laglc.Set(State.m_cidx[i],-(State.m_nulc[i]*targetscale/State.m_cscales[i]));
else
laglc.Set(State.m_cidx[i],AL_NaN);
State.m_tmpg.Resize(nmain);
for(i=0; i<nmain; i++)
{
v=State.m_sclsftb[i];
for(j=0; j<nmain; j++)
v+= State.m_sclsfta.Get(i,j)*State.m_exxc[j];
State.m_tmpg.Set(i,v);
}
CAblas::RMatrixGemVect(nmain,nectotal+nicwork,-1.0,State.m_sclsftcleic,0,0,1,State.m_nulc,0,1.0,State.m_tmpg,0);
for(i=0; i<nmain; i++)
{
if((State.m_sclsfthasbndl[i] && State.m_exxc[i]==State.m_sclsftbndl[i]) || (State.m_sclsfthasbndu[i] && State.m_exxc[i]==State.m_sclsftbndu[i]))
lagbc.Set(i,-State.m_tmpg[i]);
}
for(i=0; i<nmain; i++)
lagbc.Mul(i,targetscale/s[i]);
//--- Unpack results.
//--- Add XOrigin to XC and make sure that boundary constraints are
//--- satisfied.
for(i=0; i<nmain; i++)
{
//--- Unscale/unshift
xs.Set(i,s[i]*State.m_exxc[i]+xorigin[i]);
//--- Make sure that point is feasible w.m_r.m_t. box constraints.
//--- Enforce box constraints which were active in the scaled/shifted solution.
if(State.m_sclsfthasbndl[i])
{
if(xs[i]<bndl[i])
xs.Set(i,bndl[i]);
if(State.m_exxc[i]==State.m_sclsftbndl[i])
xs.Set(i,bndl[i]);
}
if(State.m_sclsfthasbndu[i])
{
if(xs[i]>bndu[i])
xs.Set(i,bndu[i]);
if(State.m_exxc[i]==State.m_sclsftbndu[i])
xs.Set(i,bndu[i]);
}
}
terminationtype=2;
}
//+------------------------------------------------------------------+
//| This function generates box - constrained QP problem, which is |
//| penalized and augmented formulation of original linearly |
//| constrained problem |
//+------------------------------------------------------------------+
void CQPDenseAULSolver::GenerateExModel(CMatrixDouble &sclsfta,
CRowDouble &sclsftb,
int nmain,
CRowDouble &sclsftbndl,
bool &sclsfthasbndl[],
CRowDouble &sclsftbndu,
bool &sclsfthasbndu[],
CMatrixDouble &sclsftcleic,
int sclsftnec,
int sclsftnic,
CRowDouble &nulc,
double rho,
CMatrixDouble &exa,
CRowDouble &exb,
CRowDouble &exbndl,
CRowDouble &exbndu,
CMatrixDouble &tmp2)
{
//--- create variables
int nslack=sclsftnic;
int ntotal=nmain+nslack;
int i=0;
int j=0;
double v=0;
int i_=0;
//--- Integrity check for properly preallocated storage
if(!CAp::Assert(exa.Rows()>=ntotal && exa.Cols()>=ntotal,__FUNCTION__+" - integrity check failed"))
return;
if(!CAp::Assert(exb.Size()>=ntotal && exbndl.Size()>=ntotal && exbndu.Size()>=ntotal,__FUNCTION__+" - integrity check failed"))
return;
//--- Primary quadratic term
for(i=0; i<ntotal; i++)
for(j=i; j<ntotal; j++)
exa.Set(i,j,0);
for(i=0; i<nmain; i++)
for(j=i; j<nmain; j++)
exa.Set(i,j,sclsfta.Get(i,j));
//--- Primary linear term
for(i=0; i<ntotal; i++)
exb.Set(i,0);
for(i=0; i<nmain; i++)
exb.Set(i,sclsftb[i]);
//--- Box constraints - move primary, add slack
for(i=0; i<nmain; i++)
{
if(sclsfthasbndl[i])
exbndl.Set(i,sclsftbndl[i]);
else
exbndl.Set(i,AL_NEGINF);
if(sclsfthasbndu[i])
exbndu.Set(i,sclsftbndu[i]);
else
exbndu.Set(i,AL_POSINF);
}
for(i=nmain; i<ntotal; i++)
{
exbndl.Set(i,0);
exbndu.Set(i,AL_POSINF);
}
//--- Handle equality constraints:
//--- * modify quadratic term
//--- * modify linear term
//--- * add Lagrangian term
tmp2.Resize(sclsftnec+sclsftnic,ntotal);
for(i=0; i<sclsftnec+sclsftnic; i++)
{
//--- Given constraint row ci and right hand side ri,
//--- I-th quadratic constraint adds penalty term
//--- 0.5*Rho*(ci'*x-ri)^2 =
//--- = 0.5*Rho*(ci'*x-ri)^T*(ci'*x-ri) =
//--- = 0.5*Rho*(x'*ci-ri')*(ci'*x-ri) =
//--- = 0.5*Rho*(x'*ci*ci'*x - ri'*ci'*x - x'*ci*ri + ri'*ri )
//--- = 0.5*Rho*(x'*(ci*ci')*x - 2*ri*(ci'*x) + ri^2 )
//--- Thus, quadratic term is updated by
//--- 0.5*Rho*(ci*ci')
//--- (with actual update to ExA being performed without 0.5
//--- multiplier because entire matrix is post-multipliead by 0.5)
//--- and linear term receives update
//--- -Rho*ri*ci
//--- Similaryly, lagrangian term is -NUi*(ci'*x-ri),
//--- so linear term is updated by
//--- -NUi*ci
//--- Because our model does not take into account constant term,
//--- we calculate just quadratic and linear terms.
for(i_=0; i_<nmain; i_++)
tmp2.Set(i,i_,sclsftcleic.Get(i,i_));
for(j=nmain; j<ntotal; j++)
tmp2.Set(i,j,0);
if(i>=sclsftnec)
tmp2.Set(i,nmain+i-sclsftnec,1.0);
v=-(rho*sclsftcleic.Get(i,nmain));
for(i_=0; i_<ntotal; i_++)
exb.Add(i_,v*tmp2.Get(i,i_));
v=-nulc[i];
for(i_=0; i_<ntotal; i_++)
exb.Add(i_,v*tmp2.Get(i,i_));
}
CAblas::RMatrixSyrk(ntotal,sclsftnec+sclsftnic,rho,tmp2,0,0,2,1.0,exa,0,0,true);
}
//+------------------------------------------------------------------+
//| This function generates initial point for "extended" box - |
//| constrained QP problem. |
//+------------------------------------------------------------------+
void CQPDenseAULSolver::GenerateExInitialPoint(CRowDouble &sclsftxc,
int nmain,
int nslack,
CRowDouble &exxc)
{
int ntotal=nmain+nslack;
exxc=sclsftxc;
exxc.Resize(ntotal);
for(int i=nmain; i<ntotal; i++)
exxc.Set(i,0);
}
//+------------------------------------------------------------------+
//| This function estimates Lagrange multipliers for scaled - shifted|
//| QP problem (here "scaled-shifted" means that we performed |
//| variable scaling and subtracted origin) given by quadratic |
//| term A, linear term B, box constraints and linear constraint |
//| matrix. |
//| It is assumed that all linear constraints are equality ones, |
//| with first NEC ones being constraints without slack variables, |
//| and next NIC ones having slack variables. The only inequality |
//| constraints we have are box ones, with first NMain ones being |
//| "general" box constraints, and next NIC ones being non-negativity|
//| constraints(not specified explicitly). |
//| We also make use of the current point XC, which is used to |
//| determine active box constraints. |
//| Actual QP problem size is NMain + NIC, but some parameters have |
//| lower dimensionality. |
//| Parameters sizes are: |
//| * A is assumed to be array[NMain, NMain] |
//| * B is assumed to be array[NMain] |
//| * BndL, BndU are array[NMain] |
//| * CLEIC is array[NEC + NIC, NMain + 1] (last item in a row |
//| containts right part) |
//| * ExXC is array[NMain + NIC], holds current point |
//| * NuLCEst is array[NEC + NIC], holds initial values of Lagrange|
//| coeffs |
//| On exit NuLCEst is updated with new estimate of Lagrange |
//| multipliers. |
//+------------------------------------------------------------------+
void CQPDenseAULSolver::UpdateLagrangeMultipliers(CMatrixDouble &sclsfta,
CRowDouble &sclsftb,
int nmain,
CRowDouble &sclsftbndl,
bool &sclsfthasbndl[],
CRowDouble &sclsftbndu,
bool &sclsfthasbndu[],
CMatrixDouble &sclsftcleic,
int sclsftnec,
int sclsftnic,
CRowDouble &exxc,
CRowDouble &nulcest,
CQPDenseAULBuffers &buffers)
{
//--- create variables
int nslack=sclsftnic;
int ntotal=nmain+nslack;
int ktotal=sclsftnec+sclsftnic;
int nqrrows=0;
int nqrcols=0;
int i=0;
int j=0;
double lambdareg=0;
double mxdiag=0;
double v=0;
bool isactive;
int i_=0;
//--- Given current point ExXC, we can determine active and inactive
//--- constraints. After we drop inactive inequality constraints, we
//--- have equality-only constrained QP problem, with mix of general
//--- linear equality constraints and "simple" constraints Xi=Ci.
//--- Problem min(0.5*x'*A*x + b'*x) s.m_t. C*x=d (general linear
//--- constraints) can be solved by explicitly writing out Lagrange
//--- equations:
//--- [ A C' ] [ X ] [ -b]
//--- [ ] [ ] = [ ]
//--- [ C ] [ L ] [ d ]
//--- or
//--- [ X ]
//--- A1* [ ] = b1
//--- [ L ]
//--- where X stands for solution itself, and L stands for Lagrange
//--- multipliers. It can be easily solved with direct linear m_solver.
//--- However, such formulation does not account for "simple" equality
//--- constraints on variables. It is possible to include "simple"
//--- constraints into "general" ones (i.e. append (0 ... 0 -1 0 ... 0)'
//--- to the constraint matrix), but it will increase problem
//--- size.
//--- Another approach is to use initial values of X and L (X0 and L0)
//--- as starting point, and to solve for "offset" from (X0, L0):
//--- [ X0+X1 ]
//--- A1*[ ] = b1
//--- [ L0+L1 ]
//--- or
//--- [ X1 ] [ X0 ]
//--- A1*[ ] = b1 - A1*[ ]
//--- [ L1 ] [ L0 ]
//--- In such formulation components of X1 which correspond to active
//--- constraints on variables are "frozen" at value 0 (because we have
//--- equality constraint, offset from constrained value have to be zero).
//--- Thus, we can rewrite corresponding columns of A1 with zeros - and
//--- use this space to store (0 ... 0 -1 0 ... 0)', which is used to
//--- account for Lagrange multipliers for "simple" constraints.
nqrcols=ntotal+ktotal;
nqrrows=nqrcols;
buffers.m_qrsv0=exxc;
buffers.m_qrsv0.Resize(nqrcols);
buffers.m_qrsvx1.Resize(nqrcols);
for(i=0; i<ktotal; i++)
buffers.m_qrsv0.Set(ntotal+i,nulcest[i]);
lambdareg=1.0E-8;
while(true)
{
//--- Initialize matrix A1 and right part b1 with zeros
buffers.m_qrkkt=matrix<double>::Zeros(nqrcols+nqrcols,nqrcols+1);
buffers.m_qrrightpart=vector<double>::Zeros(nqrcols+nqrcols);
//--- Append quadratic term (note: we implicitly add NSlack zeros to
//--- A and b).
mxdiag=0;
for(i=0; i<nmain; i++)
{
for(j=0; j<nmain; j++)
buffers.m_qrkkt.Set(i,j,sclsfta.Get(i,j));
buffers.m_qrrightpart.Set(i,-sclsftb[i]);
mxdiag=MathMax(mxdiag,MathAbs(sclsfta.Get(i,i)));
}
mxdiag=CApServ::Coalesce(mxdiag,1);
//--- Append general linear constraints
for(i=0; i<ktotal; i++)
{
for(j=0; j<nmain; j++)
{
buffers.m_qrkkt.Set(ntotal+i,j,-sclsftcleic.Get(i,j));
buffers.m_qrkkt.Set(j,ntotal+i,-sclsftcleic.Get(i,j));
}
if(i>=sclsftnec)
{
buffers.m_qrkkt.Set(ntotal+i,nmain+(i-sclsftnec),-1);
buffers.m_qrkkt.Set(nmain+(i-sclsftnec),ntotal+i,-1);
}
buffers.m_qrrightpart.Set(ntotal+i,-sclsftcleic.Get(i,nmain));
}
//--- Append regularizer to the bottom of the matrix
//--- (it will be factored in during QR decomposition)
if(lambdareg>0.0)
{
nqrrows=nqrcols+nqrcols;
for(i=0; i<nqrcols; i++)
buffers.m_qrkkt.Set(nqrcols+i,i,lambdareg*mxdiag);
}
//--- Subtract reference point (X0,L0) from the system
for(i=0; i<nqrcols; i++)
{
v=0.0;
for(i_=0; i_<nqrcols; i_++)
v+=buffers.m_qrkkt.Get(i,i_)*buffers.m_qrsv0[i_];
buffers.m_qrrightpart.Add(i,-v);
}
//--- Handle active "simple" equality constraints
for(i=0; i<ntotal; i++)
{
isactive=false;
if(i<nmain && ((sclsfthasbndl[i] && exxc[i]==sclsftbndl[i]) || (sclsfthasbndu[i] && exxc[i]==sclsftbndu[i])))
isactive=true;
if(i>=nmain && exxc[i]==0.0)
isactive=true;
if(!isactive)
continue;
for(j=0; j<nqrrows; j++)
buffers.m_qrkkt.Set(j,i,0);
buffers.m_qrkkt.Set(i,i,-1);
}
//--- Solve via QR decomposition:
//--- * append right part to the system matrix
//--- * perform QR decomposition of the extended matrix (right part is implicitly
//--- multiplied by Q during decomposition; believe me, it works!)
//--- * check condition number, increase regularization value if necessary and retry
//--- * solve triangular system, break iteration
for(i=0; i<nqrrows; i++)
buffers.m_qrkkt.Set(i,nqrcols,buffers.m_qrrightpart[i]);
COrtFac::RMatrixQR(buffers.m_qrkkt,nqrrows,nqrcols+1,buffers.m_qrtau);
if(CRCond::RMatrixTrRCond1(buffers.m_qrkkt,nqrcols,true,false)<=(1000.0*CMath::m_machineepsilon))
{
lambdareg=CApServ::Coalesce(10*lambdareg,1.0E-13);
continue;
}
for(i=nqrcols-1; i>=0; i--)
{
v=buffers.m_qrkkt.Get(i,nqrcols);
for(j=i+1; j<nqrcols; j++)
v-=buffers.m_qrkkt.Get(i,j)*buffers.m_qrsvx1[j];
buffers.m_qrsvx1.Set(i,v/buffers.m_qrkkt.Get(i,i));
}
break;
}
//--- Update Lagrange coefficients
for(i=0; i<ktotal; i++)
nulcest.Set(i,buffers.m_qrsv0[ntotal+i]+buffers.m_qrsvx1[ntotal+i]);
}
//+------------------------------------------------------------------+
//| This function generates scaled (by S) and shifted (by XC) |
//| reformulation of the original problem. |
//| INPUT PARAMETERS: |
//| DenseA - for dense problems(AKind = 0), A - term of CQM |
//| object contains system matrix. Other terms are |
//| unspecified and should not be referenced. |
//| SparseA - for sparse problems(AKind = 1), CRS format |
//| AKind - sparse matrix format: |
//| * 0 for dense matrix |
//| * 1 for sparse matrix |
//| SparseUpper - which triangle of SparseAC stores matrix - upper|
//| or lower one (for dense matrices this parameter |
//| is not actual). |
//| B - linear term, array[N] |
//| BndL - lower bound, array[N] |
//| BndU - upper bound, array[N] |
//| S - scale vector, array[NC]: |
//| * I-Th element contains scale of I-Th variable, |
//| * SC[I] > 0 |
//| XOrigin - origin term, array[NC]. Can be zero. |
//| N - number of variables in the original formulation |
//| (no slack variables). |
//| CLEIC - dense linear equality/inequality constraints. |
//| Equality constraints come first. |
//| NEC, NIC - number of dense equality/inequality constraints.|
//| SCLEIC - sparse linear equality / inequality constraints.|
//| Equality constraints come first. |
//| SNEC, SNIC - number of sparse equality/inequality constraints|
//| RenormLC - whether constraints should be renormalized |
//| (recommended) or used "as is". |
//| Settings - QPDENSEAULSettings object initialized by one of |
//| the initialization functions. |
//| State - object which stores temporaries |
//| XS - initial point, array[NC] |
//| On output, following fields of the State structure are modified: |
//| * SclSftA - array[NMain, NMain], quadratic term, both |
//| triangles |
//| * SclSftB - array[NMain], linear term |
//| * SclSftXC - array[NMain], initial point |
//| * SclSftHasBndL, |
//| SclSftHasBndU, |
//| SclSftBndL, |
//| SclSftBndU - array[NMain], lower / upper bounds |
//| * SclSftCLEIC- array[KTotal, NMain + 1], general linear |
//| constraints |
//| NOTE: State.Tmp2 is used to store temporary array[NMain, NMain] |
//+------------------------------------------------------------------+
void CQPDenseAULSolver::ScaleShiftOriginalPproblem(CConvexQuadraticModel &a,
CSparseMatrix &sparsea,
int akind,
bool sparseaupper,
CRowDouble &b,
CRowDouble &bndl,
CRowDouble &bndu,
CRowDouble &s,
CRowDouble &xorigin,
int nmain,
CMatrixDouble &cleic,
int dnec,
int dnic,
CSparseMatrix &scleic,
int snec,
int snic,
bool renormlc,
CQPDenseAULBuffers &State,
CRowDouble &xs)
{
//--- create variables
int i=0;
int j=0;
int k=0;
int j0=0;
int j1=0;
double v=0;
double vv=0;
int ktotal=0;
//--- check
if(!CAp::Assert(akind==0 || akind==1,__FUNCTION__+": unexpected AKind"))
return;
ktotal=dnec+dnic+snec+snic;
CApServ::BVectorSetLengthAtLeast(State.m_sclsfthasbndl,nmain);
CApServ::BVectorSetLengthAtLeast(State.m_sclsfthasbndu,nmain);
State.m_sclsfta=matrix<double>::Zeros(nmain,nmain);
State.m_sclsftb.Resize(nmain);
State.m_sclsftxc.Resize(nmain);
State.m_sclsftbndl.Resize(nmain);
State.m_sclsftbndu.Resize(nmain);
State.m_sclsftcleic.Resize(ktotal,nmain+1);
State.m_cscales.Resize(ktotal);
if(akind==0)
{
//--- Extract dense A and scale
CCQModels::CQMGetA(a,State.m_tmp2);
for(i=0; i<nmain; i++)
{
for(j=i; j<nmain; j++)
{
v=State.m_tmp2.Get(i,j)*s[i]*s[j];
State.m_sclsfta.Set(i,j,v);
State.m_sclsfta.Set(j,i,v);
}
}
}
if(akind==1)
{
//--- Extract sparse A and scale
//--- check
if(!CAp::Assert(sparsea.m_MatrixType==1,__FUNCTION__+": unexpected sparse matrix format"))
return;
if(!CAp::Assert(sparsea.m_M==nmain,__FUNCTION__+": unexpected sparse matrix size"))
return;
if(!CAp::Assert(sparsea.m_N==nmain,__FUNCTION__+": unexpected sparse matrix size"))
return;
if(sparseaupper)
{
for(i=0; i<nmain; i++)
{
if(sparsea.m_DIdx[i]!=sparsea.m_UIdx[i])
State.m_sclsfta.Set(i,i,sparsea.m_Vals[sparsea.m_DIdx[i]]*s[i]*s[i]);
j0=sparsea.m_UIdx[i];
j1=sparsea.m_RIdx[i+1]-1;
for(j=j0; j<=j1; j++)
{
k=sparsea.m_Idx[j];
v=sparsea.m_Vals[j]*s[i]*s[k];
State.m_sclsfta.Set(i,k,v);
State.m_sclsfta.Set(k,i,v);
}
}
}
else
{
for(i=0; i<nmain; i++)
{
if(sparsea.m_DIdx[i]!=sparsea.m_UIdx[i])
State.m_sclsfta.Set(i,i,sparsea.m_Vals[sparsea.m_DIdx[i]]*s[i]*s[i]);
j0=sparsea.m_RIdx[i];
j1=sparsea.m_DIdx[i]-1;
for(j=j0; j<=j1; j++)
{
k=sparsea.m_Idx[j];
v=sparsea.m_Vals[j]*s[i]*s[k];
State.m_sclsfta.Set(i,k,v);
State.m_sclsfta.Set(k,i,v);
}
}
}
}
for(i=0; i<nmain; i++)
{
State.m_sclsfthasbndl[i]=MathIsValidNumber(bndl[i]);
State.m_sclsfthasbndu[i]=MathIsValidNumber(bndu[i]);
State.m_sclsftb.Set(i,b[i]*s[i]);
State.m_sclsftxc.Set(i,(xs[i]-xorigin[i])/s[i]);
State.m_sclsftbndl.Set(i,bndl[i]);
State.m_sclsftbndu.Set(i,bndu[i]);
}
CLPQPServ::ScaleShiftBCInplace(s,xorigin,State.m_sclsftbndl,State.m_sclsftbndu,nmain);
for(i=0; i<=ktotal-1; i++)
for(j=0; j<=nmain; j++)
State.m_sclsftcleic.Set(i,j,0);
for(i=0; i<dnec; i++)
{
for(j=0; j<nmain; j++)
{
v=cleic.Get(i,j)*s[j];
State.m_sclsftcleic.Set(i,j,v);
}
State.m_sclsftcleic.Set(i,nmain,cleic.Get(i,nmain));
}
for(i=0; i<dnic; i++)
{
for(j=0; j<nmain; j++)
{
v=cleic.Get(dnec+i,j)*s[j];
State.m_sclsftcleic.Set(dnec+snec+i,j,v);
}
State.m_sclsftcleic.Set(dnec+snec+i,nmain,cleic.Get(dnec+i,nmain));
}
for(i=0; i<snec; i++)
{
//--- Because constraints are sparse, everything is a bit tricky -
//--- it is possible that N-th element of the row is zero and not
//--- stored; it is also possible that entire row is empty.
j0=scleic.m_RIdx[i];
j1=scleic.m_RIdx[i+1]-1;
if(j1>=j0 && scleic.m_Idx[j1]==nmain)
{
State.m_sclsftcleic.Set(dnec+i,nmain,scleic.m_Vals[j1]);
j1--;
}
for(j=j0; j<=j1; j++)
{
k=scleic.m_Idx[j];
v=scleic.m_Vals[j]*s[k];
State.m_sclsftcleic.Set(dnec+i,k,v);
}
}
for(i=0; i<snic; i++)
{
//--- Because constraints are sparse, everything is a bit tricky -
//--- it is possible that N-th element of the row is zero and not
//--- stored; it is also possible that entire row is empty.
j0=scleic.m_RIdx[snec+i];
j1=scleic.m_RIdx[snec+i+1]-1;
if(j1>=j0 && scleic.m_Idx[j1]==nmain)
{
State.m_sclsftcleic.Set(dnec+snec+dnic+i,nmain,scleic.m_Vals[j1]);
j1--;
}
for(j=j0; j<=j1; j++)
{
k=scleic.m_Idx[j];
v=scleic.m_Vals[j]*s[k];
State.m_sclsftcleic.Set(dnec+snec+dnic+i,k,v);
}
}
if(renormlc && ktotal>0)
{
//--- Normalize linear constraints in such way that they have unit norm
//--- (after variable scaling)
for(i=0; i<ktotal; i++)
{
vv=0.0;
for(j=0; j<nmain; j++)
{
v=State.m_sclsftcleic.Get(i,j);
vv+=v*v;
}
vv=MathSqrt(vv);
State.m_cscales.Set(i,vv);
if(vv>0.0)
{
vv=1/vv;
for(j=0; j<=nmain; j++)
State.m_sclsftcleic.Mul(i,j,vv);
}
}
}
else
{
//--- Load unit scales
for(i=0; i<ktotal; i++)
State.m_cscales.Set(i,1.0);
}
for(i=0; i<ktotal; i++)
{
//--- Apply XOrigin
v=0.0;
for(j=0; j<nmain; j++)
v+= State.m_sclsftcleic.Get(i,j)*(xorigin[j]/s[j]);
State.m_sclsftcleic.Add(i,nmain,-v);
}
}
//+------------------------------------------------------------------+
//| Normalize model in such way that norm(A)~1(very roughly) |
//| We have two lower bounds for sigma_max(A): |
//| * first estimate is provided by Frobenius norm, it is equal |
//| to ANorm / NMain |
//| * second estimate is provided by max(CAC) |
//| We select largest one of these estimates, because using just one |
//| of them is prone to different failure modes. Then, we divide A |
//| and B by this estimate. |
//| INPUT PARAMETERS: |
//| A - array[N, N], quadratic term, full triangle is given|
//| B - array[N], linear term |
//| N - problem size |
//| CLEIC - array[NEC + NIC, N + 1], linear equality/inequality|
//| constraints |
//| NEC - number of equality constraints |
//| NIC - number of inequality constraints |
//| UseCLEIC - additional normalization of A in such way that |
//| CLEIC * A * CLEIC'~1: |
//| * if False, CLEIC is ignored |
//| * if True, CLEIC rows MUST have unit norm (we |
//| check it) |
//| Tmp2 - additional buffer, possibly preallocated |
//| OUTPUT PARAMETERS: |
//| A, B - appropriately rescaled by 1 / SCL |
//| RESULT: multiplier SCL |
//+------------------------------------------------------------------+
double CQPDenseAULSolver::NormalizeQuadraticTerm(CMatrixDouble &a,
CRowDouble &b,
int n,
CMatrixDouble &cleic,
int nec,
int nic,
bool usecleic,
CMatrixDouble &tmp2)
{
//--- create variables
double result=0;
int i=0;
int j=0;
double anorm=0;
double maxcac=0;
double v=0;
double vv=0;
int nmain=n;
int ktotal=nec+nic;
for(i=0; i<nmain; i++)
for(j=0; j<nmain; j++)
anorm=anorm+CMath::Sqr(a.Get(i,j));
anorm=MathSqrt(anorm);
if(usecleic && ktotal>0)
{
//--- Calculate max(|diag(C*A*C')|), where C is constraint matrix
tmp2.Resize(ktotal,nmain);
CAblas::RMatrixGemm(ktotal,nmain,nmain,1.0,cleic,0,0,0,a,0,0,0,0.0,tmp2,0,0);
maxcac=0.0;
for(i=0; i<ktotal; i++)
{
v=0;
vv=0;
for(j=0; j<nmain; j++)
{
v+= tmp2.Get(i,j)*cleic.Get(i,j);
vv+=CMath::Sqr(cleic.Get(i,j));
}
//--- check
if(!CAp::Assert(MathAbs(vv-1)<1.0E-9 || vv==0.0,__FUNCTION__+": integrity check failed"))
return(result);
maxcac=MathMax(maxcac,MathAbs(v));
}
}
else
maxcac=0;
result=CApServ::Coalesce(MathMax(maxcac,anorm/nmain),1);
v=1/result;
for(i=0; i<nmain; i++)
for(j=0; j<nmain; j++)
a.Mul(i,j,v);
for(i=0; i<nmain; i++)
b.Mul(i,v);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function selects initial working set of general inequality |
//| constraints for QP problem: |
//| * for non - convex QP problems - NICWork = NIC is returned |
//| * otherwise - NICWork = 0 is returned (we |
//| have to determine working set |
//| iteratively) |
//| INPUT PARAMETERS: |
//| A - array[NMain], quadratic term, full matrix is |
//| stored |
//| NMain - number of variables in the "original" QP problem|
//| CLEIC - array[NEC + NIC, NMain + 1], constraint matrix |
//| NEC - number of equality constraints |
//| NIC - number of inequality constraints |
//| OUTPUT PARAMETERS: |
//| NICWork - recommended size of working set; in current |
//| version either all (NICWork = NIC) or none |
//| (NICWork = 0) constraints are included. |
//| AllowWSEviction - whether problem properties allow eviction |
//| of constraints from working set or not. |
//| Non-convex problems do not allow eviction, |
//| convex ones do. |
//+------------------------------------------------------------------+
void CQPDenseAULSolver::SelectInitialWorkingSet(CMatrixDouble &a,
int nmain,
CMatrixDouble &cleic,
int nec,
int nic,
CRowDouble &m_tmp0,
CMatrixDouble &tmp2,
int &nicwork,
bool &allowwseviction)
{
nicwork=0;
allowwseviction=false;
tmp2=a;
tmp2.Resize(nmain,nmain);
m_tmp0.Resize(nmain);
if(!CTrFac::SPDMatrixCholeskyRec(tmp2,0,nmain,true,m_tmp0))
{
//--- Matrix is indefinite.
//--- We have to select full working set, otherwise algorithm may fail
//--- because problem with reduced working set can be unbounded from below.
nicwork=nic;
allowwseviction=false;
}
else
{
//--- Positive definite matrix.
//--- We can select zero initial working set and expand it later.
nicwork=0;
allowwseviction=true;
}
}
//+------------------------------------------------------------------+
//| This object stores Settings for QPBLEIC m_solver. |
//| It must be initialized with QPBLEICLoadDefaults(). |
//| After initialization you may change Settings. |
//+------------------------------------------------------------------+
struct CQPBLEICSettings
{
int m_maxits;
double m_epsf;
double m_epsg;
double m_epsx;
//--- constructor / destructor
CQPBLEICSettings(void) { ZeroMemory(this); }
~CQPBLEICSettings(void) {}
//--- copy
void Copy(const CQPBLEICSettings &obj);
//--- overloading
void operator=(const CQPBLEICSettings &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CQPBLEICSettings::Copy(const CQPBLEICSettings &obj)
{
m_maxits=obj.m_maxits;
m_epsf=obj.m_epsf;
m_epsg=obj.m_epsg;
m_epsx=obj.m_epsx;
}
//+------------------------------------------------------------------+
//| This object stores temporaries used by QuickQP m_solver. |
//+------------------------------------------------------------------+
struct CQPBLEICbuffers
{
int m_repinneriterationscount;
int m_repouteriterationscount;
CRowInt m_tmpi;
CRowDouble m_tmp0;
CRowDouble m_tmp1;
CMinBLEICState m_solver;
CMinBLEICReport m_solverrep;
//--- constructor / destructor
CQPBLEICbuffers(void);
~CQPBLEICbuffers(void) {}
//--- copy
void Copy(const CQPBLEICbuffers &obj);
//--- overloading
void operator=(const CQPBLEICbuffers &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CQPBLEICbuffers::CQPBLEICbuffers(void)
{
m_repinneriterationscount=0;
m_repouteriterationscount=0;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CQPBLEICbuffers::Copy(const CQPBLEICbuffers &obj)
{
m_repinneriterationscount=obj.m_repinneriterationscount;
m_repouteriterationscount=obj.m_repouteriterationscount;
m_tmpi=obj.m_tmpi;
m_tmp0=obj.m_tmp0;
m_tmp1=obj.m_tmp1;
m_solver=obj.m_solver;
m_solverrep=obj.m_solverrep;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CQPBLEICSolver
{
public:
static void QPBLEICLoadDefaults(int nmain,CQPBLEICSettings &s);
static void QPBLEICCopySettings(CQPBLEICSettings &src,CQPBLEICSettings &dst);
static void QPBLEICOptimize(CConvexQuadraticModel &a,CSparseMatrix &sparsea,int akind,bool sparseaupper,double absasum,double absasum2,CRowDouble &b,CRowDouble &bndl,CRowDouble &bndu,CRowDouble &s,CRowDouble &xorigin,int n,CMatrixDouble &cleic,int nec,int nic,CQPBLEICSettings &Settings,CQPBLEICbuffers &sstate,bool &firstcall,CRowDouble &xs,int &terminationtype);
};
//+------------------------------------------------------------------+
//| This function initializes QPBLEICSettings structure with default |
//| Settings. |
//| Newly created structure MUST be initialized by default Settings -|
//| or by copy of the already initialized structure. |
//+------------------------------------------------------------------+
void CQPBLEICSolver::QPBLEICLoadDefaults(int nmain,
CQPBLEICSettings &s)
{
s.m_epsg=0.0;
s.m_epsf=0.0;
s.m_epsx=1.0E-6;
s.m_maxits=0;
}
//+------------------------------------------------------------------+
//| This function initializes QPBLEICSettings structure with copy |
//| of another, already initialized structure. |
//+------------------------------------------------------------------+
void CQPBLEICSolver::QPBLEICCopySettings(CQPBLEICSettings &src,
CQPBLEICSettings &dst)
{
dst.m_epsg=src.m_epsg;
dst.m_epsf=src.m_epsf;
dst.m_epsx=src.m_epsx;
dst.m_maxits=src.m_maxits;
}
//+------------------------------------------------------------------+
//| This function runs QPBLEIC m_solver; it returns after |
//| optimization process was completed. Following QP problem is |
//| solved: |
//| min(0.5 * (x - x_origin)'*A*(x-x_origin)+b' * (x - x_origin)) |
//| subject to boundary constraints. |
//| INPUT PARAMETERS: |
//| AC - for dense problems(AKind = 0), A - term of CQM |
//| object contains system matrix. Other terms are |
//| unspecified and should not be referenced. |
//| SparseAC - for sparse problems(AKind = 1) |
//| AKind - sparse matrix format: |
//| * 0 for dense matrix |
//| * 1 for sparse matrix |
//| SparseUpper - which triangle of SparseAC stores matrix - upper |
//| or lower one (for dense matrices this parameter is |
//| not actual). |
//| AbsASum - SUM( | A.Get(i,j) |) |
//| AbsASum2 - SUM(A.Get(i,j) ^ 2) |
//| BC - linear term, array[NC] |
//| BndLC - lower bound, array[NC] |
//| BndUC - upper bound, array[NC] |
//| SC - scale vector, array[NC]: |
//| * I-th element contains scale of I-th variable, |
//| * SC[I] > 0 |
//| XOriginC - origin term, array[NC]. Can be zero. |
//| NC - number of variables in the original formulation |
//| (no slack variables). |
//| CLEICC - linear equality / inequality constraints. Present |
//| version of this function does NOT provide publicly|
//| available support for linear constraints. This |
//| feature will be introduced in the future versions |
//| of the function. |
//| NEC, NIC - number of equality/inequality constraints. MUST BE |
//| ZERO IN THE CURRENT VERSION!!! |
//| Settings - QPBLEICSettings object initialized by one of the |
//| initialization functions. |
//| SState - object which stores temporaries: |
//| * if uninitialized object was passed, FirstCall |
//| parameter MUST be set to True; object will be |
//| automatically initialized by the function, and |
//| FirstCall will be set to False. |
//| * if FirstCall = False, it is assumed that this |
//| parameter was already initialized by previous |
//| call to this function with same problem |
//| dimensions(variable count N). |
//| FirstCall- whether it is first call of this function for this |
//| specific instance of SState, with this number of |
//| variables N specified. |
//| XS - initial point, array[NC] |
//| OUTPUT PARAMETERS: |
//| XS - last point |
//| FirstCall- uncondtionally set to False |
//| TerminationType - termination type: |
//| * |
//| * |
//| * |
//+------------------------------------------------------------------+
void CQPBLEICSolver::QPBLEICOptimize(CConvexQuadraticModel &a,
CSparseMatrix &sparsea,
int akind,bool sparseaupper,
double absasum,double absasum2,
CRowDouble &b,CRowDouble &bndl,
CRowDouble &bndu,CRowDouble &s,
CRowDouble &xorigin,int n,
CMatrixDouble &cleic,int nec,
int nic,CQPBLEICSettings &Settings,
CQPBLEICbuffers &sstate,
bool &firstcall,CRowDouble &xs,
int &terminationtype)
{
//--- create variables
int i=0;
double d2=0;
double d1=0;
double d0=0;
double v=0;
double v0=0;
double v1=0;
double md=0;
double mx=0;
double mb=0;
int d1est=0;
int d2est=0;
int i_=0;
terminationtype=0;
//--- check
if(!CAp::Assert(akind==0 || akind==1,__FUNCTION__+": unexpected AKind"))
return;
sstate.m_repinneriterationscount=0;
sstate.m_repouteriterationscount=0;
terminationtype=0;
//--- Prepare m_solver object, if needed
if(firstcall)
{
CMinBLEIC::MinBLEICCreate(n,xs,sstate.m_solver);
firstcall=false;
}
//--- Prepare max(|B|)
mb=0.0;
for(i=0; i<n; i++)
mb=MathMax(mb,MathAbs(b[i]));
//--- Temporaries
sstate.m_tmpi.Resize(nec+nic);
sstate.m_tmp0.Resize(n);
sstate.m_tmp1.Resize(n);
sstate.m_tmpi.Fill(0,0,nec);
sstate.m_tmpi.Fill(-1,nec,nic);
CMinBLEIC::MinBLEICSetLC(sstate.m_solver,cleic,sstate.m_tmpi,nec+nic);
CMinBLEIC::MinBLEICSetBC(sstate.m_solver,bndl,bndu);
CMinBLEIC::MinBLEICSetDRep(sstate.m_solver,true);
CMinBLEIC::MinBLEICSetCond(sstate.m_solver,CMath::m_minrealnumber,0.0,0.0,Settings.m_maxits);
CMinBLEIC::MinBLEICSetScale(sstate.m_solver,s);
CMinBLEIC::MinBLEICSetPrecScale(sstate.m_solver);
CMinBLEIC::MinBLEICRestartFrom(sstate.m_solver,xs);
while(CMinBLEIC::MinBLEICIteration(sstate.m_solver))
{
//--- Line search started
if(sstate.m_solver.m_lsstart)
{
//--- Iteration counters:
//--- * inner iterations count is increased on every line search
//--- * outer iterations count is increased only at steepest descent line search
sstate.m_repinneriterationscount++;
if(sstate.m_solver.m_steepestdescentstep)
sstate.m_repouteriterationscount++;
//--- Build quadratic model of F along descent direction:
//--- F(x+alpha*d) = D2*alpha^2 + D1*alpha + D0
//--- Calculate estimates of linear and quadratic term
//--- (term magnitude is compared with magnitude of numerical errors)
d0=sstate.m_solver.m_f;
d1=sstate.m_solver.m_d.Dot(sstate.m_solver.m_g);
d2=0;
switch(akind)
{
case 0:
d2=CCQModels::CQMXTADX2(a,sstate.m_solver.m_d,sstate.m_tmp0);
break;
case 1:
CSparse::SparseSMV(sparsea,sparseaupper,sstate.m_solver.m_d,sstate.m_tmp0);
d2=sstate.m_solver.m_d.Dot(sstate.m_tmp0);
d2/=2;
break;
}
mx=0.0;
md=0.0;
for(i=0; i<n; i++)
{
mx=MathMax(mx,MathAbs(sstate.m_solver.m_x[i]));
md=MathMax(md,MathAbs(sstate.m_solver.m_d[i]));
}
COptServ::EstimateParabolicModel(absasum,absasum2,mx,mb,md,d1,d2,d1est,d2est);
//--- Tests for "normal" convergence.
//--- This line search may be started from steepest descent
//--- stage (stage 2) or from L-BFGS stage (stage 3) of the
//--- BLEIC algorithm. Depending on stage type, different
//--- checks are performed.
//--- Say, L-BFGS stage is an equality-constrained refinement
//--- stage of BLEIC. This stage refines current iterate
//--- under "frozen" equality constraints. We can terminate
//--- iterations at this stage only when we encounter
//--- unconstrained direction of negative curvature. In all
//--- other cases (say, when constrained gradient is zero)
//--- we should not terminate algorithm because everything may
//--- change after de-activating presently active constraints.
//--- Tests for convergence are performed only at "steepest descent" stage
//--- of the BLEIC algorithm, and only when function is non-concave
//--- (D2 is positive or approximately zero) along direction D.
//--- NOTE: we do not test iteration count (MaxIts) here, because
//--- this stopping condition is tested by BLEIC itself.
if(sstate.m_solver.m_steepestdescentstep && d2est>=0)
{
if(d1est>=0)
{
//--- "Emergency" stopping condition: D is non-descent direction.
//--- Sometimes it is possible because of numerical noise in the
//--- target function.
terminationtype=4;
xs=sstate.m_solver.m_x;
break;
}
if(d2est>0)
{
//--- Stopping condition #4 - gradient norm is small:
//--- 1. rescale State.Solver.D and State.Solver.G according to
//--- current scaling, store results to Tmp0 and Tmp1.
//--- 2. Normalize Tmp0 (scaled direction vector).
//--- 3. compute directional derivative (in scaled variables),
//--- which is equal to DOTPRODUCT(Tmp0,Tmp1).
sstate.m_tmp0=sstate.m_solver.m_d/s+0;
sstate.m_tmp1=sstate.m_solver.m_g*s+0;
v=sstate.m_tmp0.Dot(sstate.m_tmp0);
//--- check
if(!CAp::Assert(v>0.0,__FUNCTION__+": inernal errror (scaled direction is zero)"))
return;
v=1/MathSqrt(v);
sstate.m_tmp0*=v;
v=sstate.m_tmp0.Dot(sstate.m_tmp1);
if(MathAbs(v)<=Settings.m_epsg)
{
terminationtype=4;
xs=sstate.m_solver.m_x;
break;
}
//--- Stopping condition #1 - relative function improvement is small:
//--- 1. calculate steepest descent step: V = -D1/(2*D2)
//--- 2. calculate function change: V1= D2*V^2 + D1*V
//--- 3. stop if function change is small enough
v=-(d1/(2*d2));
v1=d2*v*v+d1*v;
if(MathAbs(v1)<=Settings.m_epsf*MathMax(d0,1.0))
{
terminationtype=1;
xs=sstate.m_solver.m_x;
break;
}
//--- Stopping condition #2 - scaled step is small:
//--- 1. calculate step multiplier V0 (step itself is D*V0)
//--- 2. calculate scaled step length V
//--- 3. stop if step is small enough
v0=-(d1/(2*d2));
v=0;
for(i=0; i<n; i++)
v+=CMath::Sqr(v0*sstate.m_solver.m_d[i]/s[i]);
if(MathSqrt(v)<=Settings.m_epsx)
{
terminationtype=2;
xs=sstate.m_solver.m_x;
break;
}
}
}
//--- Test for unconstrained direction of negative curvature
if((d2est<0 || (d2est==0 && d1est<0)) && !sstate.m_solver.m_boundedstep)
{
//--- Function is unbounded from below:
//--- * function will decrease along D, i.e. either:
//--- * D2<0
//--- * D2=0 and D1<0
//--- * step is unconstrained
//--- If these conditions are true, we abnormally terminate QP
//--- algorithm with return code -4 (we can do so at any stage
//--- of BLEIC - whether it is L-BFGS or steepest descent one).
terminationtype=-4;
xs=sstate.m_solver.m_x;
break;
}
//--- Suggest new step (only if D1 is negative far away from zero,
//--- D2 is positive far away from zero).
if(d1est<0 && d2est>0)
sstate.m_solver.m_stp=CApServ::SafeMinPosRV(-d1,2*d2,sstate.m_solver.m_curstpmax);
}
//--- Gradient evaluation
if(sstate.m_solver.m_needfg)
{
sstate.m_tmp0=sstate.m_solver.m_x-xorigin+0;
switch(akind)
{
case 0:
CCQModels::CQMADX(a,sstate.m_tmp0,sstate.m_tmp1);
break;
case 1:
CSparse::SparseSMV(sparsea,sparseaupper,sstate.m_tmp0,sstate.m_tmp1);
break;
}
v0=sstate.m_tmp0.Dot(sstate.m_tmp1);
v1=sstate.m_tmp0.Dot(b);
sstate.m_solver.m_f=0.5*v0+v1;
sstate.m_solver.m_g=sstate.m_tmp1+b+0;
}
}
if(terminationtype==0)
{
//--- BLEIC optimizer was terminated by one of its inner stopping
//--- conditions. Usually it is iteration counter (if such
//--- stopping condition was specified by user).
CMinBLEIC::MinBLEICResultsBuf(sstate.m_solver,xs,sstate.m_solverrep);
terminationtype=sstate.m_solverrep.m_terminationtype;
}
else
{
//--- BLEIC optimizer was terminated in "emergency" mode by QP
//--- m_solver.
//--- NOTE: such termination is "emergency" only when viewed from
//--- BLEIC's position. QP m_solver sees such termination as
//--- routine one, triggered by QP's stopping criteria.
CMinBLEIC::MinBLEICEmergencyTermination(sstate.m_solver);
}
}
//+------------------------------------------------------------------+
//| This object stores nonlinear optimizer State. |
//| You should use functions provided by MinQP subpackage to work |
//| with this object |
//+------------------------------------------------------------------+
class CMinQPState
{
public:
//--- variables
int m_akind;
int m_algokind;
int m_mdense;
int m_msparse;
int m_n;
int m_repinneriterationscount;
int m_repncholesky;
int m_repnmv;
int m_repouteriterationscount;
int m_repterminationtype;
int m_stype;
double m_absamax;
double m_absasum2;
double m_absasum;
double m_veps;
bool m_dbgskipconstraintnormalization;
bool m_havex;
bool m_qpbleicfirstcall;
bool m_sparseaupper;
//--- objects
CVIPMState m_vsolver;
CQQPSettings m_qqpsettingsuser;
CQQPBuffers m_qqpbuf;
CQPDenseAULSettings m_qpdenseaulsettingsuser;
CQPDenseAULBuffers m_qpdenseaulbuf;
CQPBLEICbuffers m_qpbleicbuf;
CQPBLEICSettings m_qpbleicsettingsuser;
CConvexQuadraticModel m_a;
//--- arrays
bool m_havebndl[];
bool m_havebndu[];
CRowInt m_elagidx;
CRowDouble m_b;
CRowDouble m_bndl;
CRowDouble m_bndu;
CRowDouble m_cl;
CRowDouble m_cu;
CRowDouble m_effectives;
CRowDouble m_elaglc;
CRowDouble m_elagmlt;
CRowDouble m_replagbc;
CRowDouble m_replaglc;
CRowDouble m_s;
CRowDouble m_startx;
CRowDouble m_tmp0;
CRowDouble m_wrkbndl;
CRowDouble m_wrkbndu;
CRowDouble m_wrkcl;
CRowDouble m_wrkcu;
CRowDouble m_xorigin;
CRowDouble m_xs;
//--- matrix
CSparseMatrix m_dummysparse;
CSparseMatrix m_sparsea;
CSparseMatrix m_sparsec;
CSparseMatrix m_wrksparsec;
CMatrixDouble m_densec;
CMatrixDouble m_dummyr2;
CMatrixDouble m_ecleic;
CMatrixDouble m_tmpr2;
CMatrixDouble m_wrkdensec;
//--- constructor, destructor
CMinQPState(void);
~CMinQPState(void) {}
//--- copy
void Copy(const CMinQPState &obj);
//--- overloading
void operator=(const CMinQPState &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinQPState::CMinQPState(void)
{
m_akind=0;
m_algokind=0;
m_mdense=0;
m_msparse=0;
m_n=0;
m_repinneriterationscount=0;
m_repncholesky=0;
m_repnmv=0;
m_repouteriterationscount=0;
m_repterminationtype=0;
m_stype=0;
m_absamax=0;
m_absasum2=0;
m_absasum=0;
m_veps=0;
m_dbgskipconstraintnormalization=false;
m_havex=false;
m_qpbleicfirstcall=false;
m_sparseaupper=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinQPState::Copy(const CMinQPState &obj)
{
m_akind=obj.m_akind;
m_algokind=obj.m_algokind;
m_mdense=obj.m_mdense;
m_msparse=obj.m_msparse;
m_n=obj.m_n;
m_repinneriterationscount=obj.m_repinneriterationscount;
m_repncholesky=obj.m_repncholesky;
m_repnmv=obj.m_repnmv;
m_repouteriterationscount=obj.m_repouteriterationscount;
m_repterminationtype=obj.m_repterminationtype;
m_stype=obj.m_stype;
m_absamax=obj.m_absamax;
m_absasum2=obj.m_absasum2;
m_absasum=obj.m_absasum;
m_veps=obj.m_veps;
m_dbgskipconstraintnormalization=obj.m_dbgskipconstraintnormalization;
ArrayCopy(m_havebndl,obj.m_havebndl);
ArrayCopy(m_havebndu,obj.m_havebndu);
m_havex=obj.m_havex;
m_qpbleicfirstcall=obj.m_qpbleicfirstcall;
m_sparseaupper=obj.m_sparseaupper;
m_vsolver=obj.m_vsolver;
m_dummysparse=obj.m_dummysparse;
m_sparsea=obj.m_sparsea;
m_sparsec=obj.m_sparsec;
m_wrksparsec=obj.m_wrksparsec;
m_elagidx=obj.m_elagidx;
m_b=obj.m_b;
m_bndl=obj.m_bndl;
m_bndu=obj.m_bndu;
m_cl=obj.m_cl;
m_cu=obj.m_cu;
m_effectives=obj.m_effectives;
m_elaglc=obj.m_elaglc;
m_elagmlt=obj.m_elagmlt;
m_replagbc=obj.m_replagbc;
m_replaglc=obj.m_replaglc;
m_s=obj.m_s;
m_startx=obj.m_startx;
m_tmp0=obj.m_tmp0;
m_wrkbndl=obj.m_wrkbndl;
m_wrkbndu=obj.m_wrkbndu;
m_wrkcl=obj.m_wrkcl;
m_wrkcu=obj.m_wrkcu;
m_xorigin=obj.m_xorigin;
m_xs=obj.m_xs;
m_qqpsettingsuser=obj.m_qqpsettingsuser;
m_qqpbuf=obj.m_qqpbuf;
m_qpdenseaulsettingsuser=obj.m_qpdenseaulsettingsuser;
m_qpdenseaulbuf=obj.m_qpdenseaulbuf;
m_qpbleicbuf=obj.m_qpbleicbuf;
m_qpbleicsettingsuser=obj.m_qpbleicsettingsuser;
m_densec=obj.m_densec;
m_dummyr2=obj.m_dummyr2;
m_ecleic=obj.m_ecleic;
m_tmpr2=obj.m_tmpr2;
m_wrkdensec=obj.m_wrkdensec;
m_a=obj.m_a;
}
//+------------------------------------------------------------------+
//| This object stores nonlinear optimizer State. |
//| You should use functions provided by MinQP subpackage to work |
//| with this object |
//+------------------------------------------------------------------+
class CMinQPStateShell
{
private:
CMinQPState m_innerobj;
public:
//--- constructors, destructor
CMinQPStateShell(void) {}
CMinQPStateShell(CMinQPState &obj) { m_innerobj.Copy(obj); }
~CMinQPStateShell(void) {}
//--- method
CMinQPState *GetInnerObj(void) { return(GetPointer(m_innerobj)); }
};
//+------------------------------------------------------------------+
//| This structure stores optimization report: |
//| * InnerIterationsCount number of inner iterations |
//| * OuterIterationsCount number of outer iterations |
//| * NCholesky number of Cholesky decomposition |
//| * NMV number of matrix-vector products |
//| (only products calculated as part of |
//| iterative process are counted) |
//| * TerminationType completion code (see below) |
//| Completion codes: |
//| * -5 inappropriate m_solver was used: |
//| * Cholesky m_solver for semidefinite or indefinite problems|
//| * Cholesky m_solver for problems with non-boundary |
//| constraints |
//| * -3 inconsistent constraints (or, maybe, feasible point is |
//| too hard to find). If you are sure that constraints are |
//| feasible, try to restart optimizer with better initial |
//| approximation. |
//| * 4 successful completion |
//| * 5 MaxIts steps was taken |
//| * 7 stopping conditions are too stringent, |
//| further improvement is impossible, |
//| X contains best point found so far. |
//+------------------------------------------------------------------+
class CMinQPReport
{
public:
//--- variables
int m_inneriterationscount;
int m_outeriterationscount;
int m_nmv;
int m_ncholesky;
int m_terminationtype;
//--- arrays
CRowDouble m_lagbc;
CRowDouble m_laglc;
//--- constructor, destructor
CMinQPReport(void);
~CMinQPReport(void) {}
//--- copy
void Copy(const CMinQPReport &obj);
//--- overloading
void operator=(const CMinQPReport &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinQPReport::CMinQPReport(void)
{
m_inneriterationscount=0;
m_outeriterationscount=0;
m_nmv=0;
m_ncholesky=0;
m_terminationtype=0;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinQPReport::Copy(const CMinQPReport &obj)
{
//--- copy variables
m_inneriterationscount=obj.m_inneriterationscount;
m_outeriterationscount=obj.m_outeriterationscount;
m_nmv=obj.m_nmv;
m_ncholesky=obj.m_ncholesky;
m_terminationtype=obj.m_terminationtype;
//--- copy arrays
m_lagbc=obj.m_lagbc;
m_laglc=obj.m_laglc;
}
//+------------------------------------------------------------------+
//| This structure stores optimization report: |
//| * InnerIterationsCount number of inner iterations |
//| * OuterIterationsCount number of outer iterations |
//| * NCholesky number of Cholesky decomposition |
//| * NMV number of matrix-vector products |
//| (only products calculated as part of |
//| iterative process are counted) |
//| * TerminationType completion code (see below) |
//| Completion codes: |
//| * -5 inappropriate m_solver was used: |
//| * Cholesky m_solver for semidefinite or indefinite problems|
//| * Cholesky m_solver for problems with non-boundary |
//| constraints |
//| * -3 inconsistent constraints (or, maybe, feasible point is |
//| too hard to find). If you are sure that constraints are |
//| feasible, try to restart optimizer with better initial |
//| approximation. |
//| * 4 successful completion |
//| * 5 MaxIts steps was taken |
//| * 7 stopping conditions are too stringent, |
//| further improvement is impossible, |
//| X contains best point found so far. |
//+------------------------------------------------------------------+
class CMinQPReportShell
{
private:
CMinQPReport m_innerobj;
public:
//--- constructors, destructor
CMinQPReportShell(void) {}
CMinQPReportShell(CMinQPReport &obj) { m_innerobj.Copy(obj); }
~CMinQPReportShell(void) {}
//--- methods
int GetInnerIterationsCount(void);
void SetInnerIterationsCount(const int i);
int GetOuterIterationsCount(void);
void SetOuterIterationsCount(const int i);
int GetNMV(void);
void SetNMV(const int i);
int GetNCholesky(void);
void SetNCholesky(const int i);
int GetTerminationType(void);
void SetTerminationType(const int i);
CMinQPReport *GetInnerObj(void);
};
//+------------------------------------------------------------------+
//| Returns the value of the variable inneriterationscount |
//+------------------------------------------------------------------+
int CMinQPReportShell::GetInnerIterationsCount(void)
{
return(m_innerobj.m_inneriterationscount);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable inneriterationscount |
//+------------------------------------------------------------------+
void CMinQPReportShell::SetInnerIterationsCount(const int i)
{
m_innerobj.m_inneriterationscount=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable outeriterationscount |
//+------------------------------------------------------------------+
int CMinQPReportShell::GetOuterIterationsCount(void)
{
return(m_innerobj.m_outeriterationscount);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable outeriterationscount |
//+------------------------------------------------------------------+
void CMinQPReportShell::SetOuterIterationsCount(const int i)
{
m_innerobj.m_outeriterationscount=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable nmv |
//+------------------------------------------------------------------+
int CMinQPReportShell::GetNMV(void)
{
return(m_innerobj.m_nmv);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable nmv |
//+------------------------------------------------------------------+
void CMinQPReportShell::SetNMV(const int i)
{
m_innerobj.m_nmv=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable ncholesky |
//+------------------------------------------------------------------+
int CMinQPReportShell::GetNCholesky(void)
{
return(m_innerobj.m_ncholesky);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable ncholesky |
//+------------------------------------------------------------------+
void CMinQPReportShell::SetNCholesky(const int i)
{
m_innerobj.m_ncholesky=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable terminationtype |
//+------------------------------------------------------------------+
int CMinQPReportShell::GetTerminationType(void)
{
return(m_innerobj.m_terminationtype);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable terminationtype |
//+------------------------------------------------------------------+
void CMinQPReportShell::SetTerminationType(const int i)
{
m_innerobj.m_terminationtype=i;
}
//+------------------------------------------------------------------+
//| Return object of class |
//+------------------------------------------------------------------+
CMinQPReport* CMinQPReportShell::GetInnerObj(void)
{
return(GetPointer(m_innerobj));
}
//+------------------------------------------------------------------+
//| Constrained quadratic programming |
//+------------------------------------------------------------------+
class CMinQP
{
public:
static void MinQPCreate(const int n,CMinQPState &State);
static void MinQPSetLinearTerm(CMinQPState &State,double &b[]);
static void MinQPSetLinearTerm(CMinQPState &State,CRowDouble &b);
static void MinQPSetQuadraticTerm(CMinQPState &State,CMatrixDouble &a,const bool IsUpper);
static void MinQPSetQuadraticTermSparse(CMinQPState &State,CSparseMatrix &a,bool IsUpper);
static void MinQPSetStartingPoint(CMinQPState &State,double &x[]);
static void MinQPSetStartingPoint(CMinQPState &State,CRowDouble &x);
static void MinQPSetOrigin(CMinQPState &State,double &xorigin[]);
static void MinQPSetOrigin(CMinQPState &State,CRowDouble &xorigin);
static void MinQPSetScale(CMinQPState &State,CRowDouble &s);
static void MinQPSetScaleAutoDiag(CMinQPState &State);
static void MinQPSetAlgoBLEIC(CMinQPState &State,double epsg,double epsf,double epsx,int m_maxits);
static void MinQPSetAlgoDenseAUL(CMinQPState &State,double epsx,double rho,int itscnt);
static void MinQPSetAlgoDenseIPM(CMinQPState &State,double eps);
static void MinQPSetAlgoSparseIPM(CMinQPState &State,double eps);
static void MinQPSetAlgoQuickQP(CMinQPState &State,double epsg,double epsf,double epsx,int maxouterits,bool usenewton);
static void MinQPSetAlgoCholesky(CMinQPState &State);
static void MinQPSetBC(CMinQPState &State,double &bndl[],double &bndu[]);
static void MinQPSetBC(CMinQPState &State,CRowDouble &bndl,CRowDouble &bndu);
static void MinQPSetBCAll(CMinQPState &State,double bndl,double bndu);
static void MinQPSetBCI(CMinQPState &State,int i,double bndl,double bndu);
static void MinQPSetLC(CMinQPState &State,CMatrixDouble &c,CRowInt &ct,int k);
static void MinQPSetLCSparse(CMinQPState &State,CSparseMatrix &c,CRowInt &ct,int k);
static void MinQPSetLCMixed(CMinQPState &State,CSparseMatrix &sparsec,CRowInt &sparsect,int sparsek,CMatrixDouble &densec,CRowInt &densect,int densek);
static void MinQPSetLCMixedLegacy(CMinQPState &State,CMatrixDouble &densec,CRowInt &densect,int densek,CSparseMatrix &sparsec,CRowInt &sparsect,int sparsek);
static void MinQPSetLC2Dense(CMinQPState &State,CMatrixDouble &a,CRowDouble &al,CRowDouble &au,int k);
static void MinQPSetLC2(CMinQPState &State,CSparseMatrix &a,CRowDouble &al,CRowDouble &au,int k);
static void MinQPSetLC2Mixed(CMinQPState &State,CSparseMatrix &sparsea,int ksparse,CMatrixDouble &densea,int kdense,CRowDouble &al,CRowDouble &au);
static void MinQPAddLC2Dense(CMinQPState &State,CRowDouble &a,double al,double au);
static void MinQPAddLC2(CMinQPState &State,CRowInt &idxa,CRowDouble &vala,int nnz,double al,double au);
static void MinQPAddLC2SparseFromDense(CMinQPState &State,CRowDouble &da,double al,double au);
static void MinQPOptimize(CMinQPState &State);
static void MinQPResults(CMinQPState &State,double &x[],CMinQPReport &rep);
static void MinQPResults(CMinQPState &State,CRowDouble &x,CMinQPReport &rep);
static void MinQPResultsBuf(CMinQPState &State,double &x[],CMinQPReport &rep);
static void MinQPResultsBuf(CMinQPState &State,CRowDouble &x,CMinQPReport &rep);
static void MinQPSetLinearTermFast(CMinQPState &State,double &b[]);
static void MinQPSetLinearTermFast(CMinQPState &State,CRowDouble &b);
static void MinQPSetQuadraticTermFast(CMinQPState &State,CMatrixDouble &a,const bool IsUpper,const double s);
static void MinQPRewriteDiagonal(CMinQPState &State,double &s[]);
static void MinQPRewriteDiagonal(CMinQPState &State,CRowDouble &s);
static void MinQPSetStartingPointFast(CMinQPState &State,double &x[]);
static void MinQPSetStartingPointFast(CMinQPState &State,CRowDouble &x);
static void MinQPSetOriginFast(CMinQPState &State,double &xorigin[]);
static void MinQPSetOriginFast(CMinQPState &State,CRowDouble &xorigin);
};
//+------------------------------------------------------------------+
//| CONSTRAINED QUADRATIC PROGRAMMING |
//| The subroutine creates QP optimizer. After initial creation, it |
//| contains default optimization problem with zero quadratic and |
//| linear terms and no constraints. You should set quadratic/linear |
//| terms with calls to functions provided by MinQP subpackage. |
//| INPUT PARAMETERS: |
//| N - problem size |
//| OUTPUT PARAMETERS: |
//| State - optimizer with zero quadratic/linear terms |
//| and no constraints |
//+------------------------------------------------------------------+
void CMinQP::MinQPCreate(const int n,CMinQPState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
//--- initialize QP m_solver
State.m_n=n;
State.m_mdense=0;
State.m_msparse=0;
State.m_repterminationtype=0;
State.m_absamax=1;
State.m_absasum=1;
State.m_absasum2=1;
State.m_akind=0;
State.m_sparseaupper=false;
CCQModels::CQMInit(n,State.m_a);
//--- allocation
ArrayResize(State.m_havebndl,n);
ArrayResize(State.m_havebndu,n);
State.m_b=vector<double>::Zeros(n);
State.m_bndl=vector<double>::Full(n,AL_NEGINF);
State.m_bndu=vector<double>::Full(n,AL_POSINF);
State.m_s=vector<double>::Ones(n);
State.m_startx=vector<double>::Zeros(n);
State.m_xorigin=vector<double>::Zeros(n);
State.m_xs.Resize(n);
State.m_replagbc=vector<double>::Zeros(n);
//--- initialization
ArrayInitialize(State.m_havebndl,false);
ArrayInitialize(State.m_havebndu,false);
State.m_stype=0;
State.m_havex=false;
//--- function call
MinQPSetAlgoBLEIC(State,0.0,0.0,0.0,0);
CQQPSolver::QQPLoadDefaults(n,State.m_qqpsettingsuser);
CQPBLEICSolver::QPBLEICLoadDefaults(n,State.m_qpbleicsettingsuser);
CQPDenseAULSolver::QPDenseAULLoadDefaults(n,State.m_qpdenseaulsettingsuser);
State.m_qpbleicfirstcall=true;
State.m_dbgskipconstraintnormalization=false;
State.m_veps=0.0;
}
//+------------------------------------------------------------------+
//| This function sets linear term for QP m_solver. |
//| By default, linear term is zero. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| B - linear term, array[N]. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetLinearTerm(CMinQPState &State,double &b[])
{
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(b)>=n,__FUNCTION__+": Length(B)<N"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(b,n),__FUNCTION__+": B contains infinite or NaN elements"))
return;
//--- function call
MinQPSetLinearTermFast(State,b);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetLinearTerm(CMinQPState &State,CRowDouble &b)
{
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(b)>=n,__FUNCTION__+": Length(B)<N"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(b,n),__FUNCTION__+": B contains infinite or NaN elements"))
return;
//--- function call
MinQPSetLinearTermFast(State,b);
}
//+------------------------------------------------------------------+
//| This function sets quadratic term for QP m_solver. |
//| By default quadratic term is zero. |
//| IMPORTANT: this m_solver minimizes following function: |
//| f(x) = 0.5*x'*A*x + b'*x. |
//| Note that quadratic term has 0.5 before it. So if you want to |
//| minimize |
//| f(x) = x^2 + x |
//| you should rewrite your problem as follows: |
//| f(x) = 0.5*(2*x^2) + x |
//| and your matrix A will be equal to [[2.0]], not to [[1.0]] |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| A - matrix, array[N,N] |
//| IsUpper - (optional) storage type: |
//| * if True, symmetric matrix A is given by its |
//| upper triangle, and the lower triangle isn?t |
//| used |
//| * if False, symmetric matrix A is given by its |
//| lower triangle, and the upper triangle isn?t |
//| used |
//| * if not given, both lower and upper triangles |
//| must be filled. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetQuadraticTerm(CMinQPState &State,CMatrixDouble &a,
const bool IsUpper)
{
int n=State.m_n;
//--- check
if(!CAp::Assert((int)CAp::Rows(a)>=n,__FUNCTION__+": Rows(A)<N"))
return;
//--- check
if(!CAp::Assert((int)CAp::Cols(a)>=n,__FUNCTION__+": Cols(A)<N"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteRTrMatrix(a,n,IsUpper),__FUNCTION__+": A contains infinite or NaN elements"))
return;
//--- function call
MinQPSetQuadraticTermFast(State,a,IsUpper,0.0);
}
//+------------------------------------------------------------------+
//| This function sets sparse quadratic term for QP solver. By |
//| default, quadratic term is zero. This function overrides previous|
//| calls to MinQPSetQuadraticTerm() or MinQPSetQuadraticTermSparse()|
//| NOTE: dense solvers like DENSE-AUL-QP or DENSE-IPM-QP will |
//| convert this matrix to dense storage anyway. |
//| IMPORTANT: This solver minimizes following function: |
//| f(x) = 0.5*x'*A*x + b'*x. |
//| Note that quadratic term has 0.5 before it. So if you |
//| want to minimize |
//| f(x) = x^2 + x |
//| you should rewrite your problem as follows: |
//| f(x) = 0.5*(2*x^2) + x |
//| and your matrix A will be equal to [[2.0]], not to |
//| [[1.0]] |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| A - matrix, array[N,N] |
//| IsUpper - (optional) storage type: |
//| * if True, symmetric matrix A is given by its upper|
//| triangle, and the lower triangle isn't used |
//| * if False, symmetric matrix A is given by its |
//| lower triangle, and the upper triangle isn't used|
//| * if not given, both lower and upper triangles must|
//| be filled. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetQuadraticTermSparse(CMinQPState &State,
CSparseMatrix &a,
bool IsUpper)
{
//--- create variables
int n=State.m_n;
int t0=0;
int t1=0;
int i=0;
int j=0;
double v=0;
//--- check
if(!CAp::Assert(CSparse::SparseGetNRows(a)==n,__FUNCTION__+": Rows(A)<>N"))
return;
if(!CAp::Assert(CSparse::SparseGetNCols(a)==n,__FUNCTION__+": Cols(A)<>N"))
return;
//--- function call
CSparse::SparseCopyToCRSBuf(a,State.m_sparsea);
State.m_sparseaupper=IsUpper;
State.m_akind=1;
//-- Estimate norm of A
//-- (it will be used later in the quadratic penalty function)
State.m_absamax=0;
State.m_absasum=0;
State.m_absasum2=0;
t0=0;
t1=0;
while(CSparse::SparseEnumerate(a,t0,t1,i,j,v))
{
if(i==j)
{
//-- Diagonal terms are counted only once
State.m_absamax=MathMax(State.m_absamax,v);
State.m_absasum=State.m_absasum+v;
State.m_absasum2=State.m_absasum2+v*v;
}
if((j>i && IsUpper) || (j<i && !IsUpper))
{
//-- Offdiagonal terms are counted twice
State.m_absamax=MathMax(State.m_absamax,v);
State.m_absasum=State.m_absasum+2*v;
State.m_absasum2=State.m_absasum2+2*v*v;
}
}
}
//+------------------------------------------------------------------+
//| This function sets starting point for QP m_solver. It is useful to |
//| have good initial approximation to the solution, because it will |
//| increase speed of convergence and identification of active |
//| constraints. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| X - starting point, array[N]. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetStartingPoint(CMinQPState &State,double &x[])
{
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(B)<N"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN elements"))
return;
//--- function call
MinQPSetStartingPointFast(State,x);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetStartingPoint(CMinQPState &State,CRowDouble &x)
{
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(B)<N"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN elements"))
return;
//--- function call
MinQPSetStartingPointFast(State,x);
}
//+------------------------------------------------------------------+
//| This function sets origin for QP m_solver. By default, following |
//| QP program is solved: |
//| min(0.5*x'*A*x+b'*x) |
//| This function allows to solve different problem: |
//| min(0.5*(x-x_origin)'*A*(x-x_origin)+b'*(x-x_origin)) |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| XOrigin - origin, array[N]. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetOrigin(CMinQPState &State,double &xorigin[])
{
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(xorigin)>=n,__FUNCTION__+": Length(B)<N"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(xorigin,n),__FUNCTION__+": B contains infinite or NaN elements"))
return;
//--- function call
MinQPSetOriginFast(State,xorigin);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetOrigin(CMinQPState &State,CRowDouble &xorigin)
{
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(xorigin)>=n,__FUNCTION__+": Length(B)<N"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(xorigin,n),__FUNCTION__+": B contains infinite or NaN elements"))
return;
//--- function call
MinQPSetOriginFast(State,xorigin);
}
//+------------------------------------------------------------------+
//| This function sets scaling coefficients. |
//| ALGLIB optimizers use scaling matrices to test stopping |
//| conditions (step size and gradient are scaled before comparison |
//| with tolerances) and as preconditioner. |
//| Scale of the I-th variable is a translation invariant measure of:|
//| a) "how large" the variable is |
//| b) how large the step should be to make significant changes in |
//| the function |
//| If you do not know how to choose scales of your variables, you |
//| can: |
//| * read www.alglib.net/optimization/scaling.php article |
//| * use minqpsetscaleautodiag(), which calculates scale using |
//| diagonal ofthe quadratic term: S is set to 1/sqrt(diag(A)), |
//| which works well sometimes. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| S - array[N], non-zero scaling coefficients S[i] may |
//| be negative, sign doesn't matter. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetScale(CMinQPState &State,
CRowDouble &s)
{
//--- check
if(!CAp::Assert(s.Size()>=State.m_n,__FUNCTION__+": Length(S)<N"))
return;
for(int i=0; i<State.m_n; i++)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(s[i]),__FUNCTION__+": S contains infinite or NAN elements"))
return;
if(!CAp::Assert(s[i]!=0.0,__FUNCTION__+": S contains zero elements"))
return;
}
//--- copy
State.m_s=s.Abs()+0;
State.m_stype=0;
}
//+------------------------------------------------------------------+
//| This function sets automatic evaluation of variable scaling. |
//| IMPORTANT: this function works only for matrices with positive |
//| diagonal elements! Zero or negative elements will |
//| result in -9 error code being returned. Specify scale |
//| vector manually with MinQPSetScale() in such cases. |
//| ALGLIB optimizers use scaling matrices to test |
//| stopping conditions (step size and gradient are scaled|
//| before comparison with tolerances) and as |
//| preconditioner. |
//| The best way to set scaling is to manually specify variable |
//| scales. However, sometimes you just need quick-and-dirty |
//| solution - either when you perform fast prototyping, or when you |
//| know your problem well and you are 100 % sure that this quick |
//| solution is robust enough in your case. |
//| One such solution is to evaluate scale of I-th variable |
//| as 1 / Sqrt(A[i, i]), where A[i, i] is an I-th diagonal |
//| element of the quadratic term. |
//| Such approach works well sometimes, but you have to be careful |
//| here. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetScaleAutoDiag(CMinQPState &State)
{
State.m_stype=1;
}
//+------------------------------------------------------------------+
//| This function tells solver to use BLEIC-based algorithm and sets |
//| stopping criteria for the algorithm. |
//| This algorithm is intended for large-scale problems, possibly |
//| nonconvex, with small number of general linear constraints. |
//| Feasible initial point is essential for good performance. |
//| IMPORTANT: when DENSE-IPM (or DENSE-AUL for nonconvex problems) |
//| solvers are applicable, their performance is much |
//| better than that of BLEIC-QP. |
//| We recommend you to use BLEIC only when other solvers |
//| can not be used. |
//| ALGORITHM FEATURES: |
//| * supports dense and sparse QP problems |
//| * supports box and general linear equality / inequality |
//| constraints |
//| * can solve all types of problems (convex, semidefinite, |
//| nonconvex) as long as they are bounded from below under |
//| constraints. |
//| Say, it is possible to solve "min{-x^2} subject to -1<=x<=+1". |
//| Of course, global minimum is found only for positive definite and|
//| semidefinite problems. As for indefinite ones-only local minimum |
//| is found. |
//| ALGORITHM OUTLINE : |
//| * BLEIC-QP solver is just a driver function for MinBLEIC |
//| solver; it solves quadratic programming problem as general |
//| linearly constrained optimization problem, which is solved |
//| by means of BLEIC solver(part of ALGLIB, active set method). |
//| ALGORITHM LIMITATIONS: |
//| * This algorithm is inefficient on problems with hundreds and |
//| thousands of general inequality constraints and infeasible |
//| initial point. Initial feasibility detection stage may take |
//| too long on such constraint sets. Consider using DENSE-IPM |
//| or DENSE-AUL instead. |
//| * unlike QuickQP solver, this algorithm does not perform Newton|
//| steps and does not use Level 3 BLAS. Being general-purpose |
//| active set method, it can activate constraints only |
//| one-by-one. Thus, its performance is lower than that of |
//| QuickQP. |
//| * its precision is also a bit inferior to that of QuickQP. |
//| BLEIC-QP performs only LBFGS steps(no Newton steps), which |
//| are good at detecting neighborhood of the solution, buy needs|
//| many iterations to find solution with more than 6 digits of |
//| precision. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| EpsG - >= 0, The subroutine finishes its work if the |
//| condition | v | < EpsG is satisfied, where: |
//| * | . | means Euclidian norm |
//| * v - scaled constrained gradient vector, |
//| v[i] = g[i] * s[i] |
//| * g - gradient |
//| * s - scaling coefficients set by |
//| MinQPSetScale() |
//| EpsF - >= 0, The subroutine finishes its work if |
//| exploratory steepest descent step on k+1-th |
//| iteration satisfies following condition: |
//| |F(k+1) - F(k)| <= EpsF * max{ |F(k)|, |F(k+1)|, 1}|
//| EpsX - >= 0, The subroutine finishes its work if |
//| exploratory steepest descent step on k+1-th |
//| iteration satisfies following condition: |
//| * | . | means Euclidian norm |
//| * v - scaled step vector, v[i] = dx[i] / s[i] |
//| * dx - step vector, dx = X(k + 1) - X(k) |
//| * s - scaling coefficients set by |
//| MinQPSetScale() |
//| MaxIts - maximum number of iterations. If MaxIts = 0, the |
//| number of iterations is unlimited. |
//| NOTE: this algorithm uses LBFGS iterations, which are relatively |
//| cheap, but improve function value only a bit. So you will |
//| need many iterations to converge - from 0.1 * N to 10 * N, |
//| depending on problem's condition number. |
//| IT IS VERY IMPORTANT TO CALL MinQPSetScale() WHEN YOU USE THIS |
//| ALGORITHM BECAUSE ITS STOPPING CRITERIA ARE SCALE - DEPENDENT! |
//| Passing EpsG = 0, EpsF = 0 and EpsX = 0 and MaxIts = 0 |
//| (simultaneously) will lead to automatic stopping criterion |
//| selection (presently it is small step length, but it may change |
//| in the future versions of ALGLIB). |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetAlgoBLEIC(CMinQPState &State,
double epsg,
double epsf,
double epsx,
int m_maxits)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(epsg),__FUNCTION__+": EpsG is not finite number"))
return;
if(!CAp::Assert(epsg>=0.0,__FUNCTION__+": negative EpsG"))
return;
if(!CAp::Assert(MathIsValidNumber(epsf),__FUNCTION__+": EpsF is not finite number"))
return;
if(!CAp::Assert(epsf>=0.0,__FUNCTION__+": negative EpsF"))
return;
if(!CAp::Assert(MathIsValidNumber(epsx),__FUNCTION__+": EpsX is not finite number"))
return;
if(!CAp::Assert(epsx>=0.0,__FUNCTION__+": negative EpsX"))
return;
if(!CAp::Assert(m_maxits>=0,__FUNCTION__+": negative MaxIts!"))
return;
State.m_algokind=2;
if(epsg==0.0 && epsf==0.0 && epsx==0.0 && m_maxits==0)
epsx=1.0E-6;
State.m_qpbleicsettingsuser.m_epsg=epsg;
State.m_qpbleicsettingsuser.m_epsf=epsf;
State.m_qpbleicsettingsuser.m_epsx=epsx;
State.m_qpbleicsettingsuser.m_maxits=m_maxits;
}
//+------------------------------------------------------------------+
//| This function tells QP solver to use DENSE-AUL algorithm and sets|
//| stopping criteria for the algorithm. |
//| This algorithm is intended for non-convex problems with moderate |
//| (up to several thousands) variable count and arbitrary number of |
//| constraints which are either(a) effectively convexified under |
//| constraints or (b) have unique solution even with nonconvex |
//| target. |
//| IMPORTANT: when DENSE-IPM solver is applicable, its performance |
//| is usually much better than that of DENSE-AUL. We |
//| recommend you to use DENSE-AUL only when other solvers|
//| can not be used. |
//| ALGORITHM FEATURES: |
//| * supports box and dense / sparse general linear equality / |
//| inequality constraints |
//| * convergence is theoretically proved for positive - definite |
//| (convex) QP problems. Semidefinite and non-convex problems |
//| can be solved as long as they are bounded from below under |
//| constraints, although without theoretical guarantees. |
//| ALGORITHM OUTLINE: |
//| * this algorithm is an augmented Lagrangian method with dense |
//| preconditioner(hence its name). |
//| * it performs several outer iterations in order to refine |
//| values of the Lagrange multipliers. Single outer iteration is|
//| a solution of some optimization problem: first it performs |
//| dense Cholesky factorization of the Hessian in order to build|
//| preconditioner (adaptive regularization is applied to enforce|
//| positive definiteness), and then it uses L-BFGS optimizer to |
//| solve optimization problem. |
//| * typically you need about 5-10 outer iterations to converge |
//| to solution |
//| ALGORITHM LIMITATIONS: |
//| * because dense Cholesky driver is used, this algorithm has |
//| O(N^2) memory requirements and O(OuterIterations*N^3) minimum|
//| running time. From the practical point of view, it limits its|
//| applicability by several thousands of variables. |
//| From the other side, variables count is the most limiting factor,|
//| and dependence on constraint count is much more lower. Assuming |
//| that constraint matrix is sparse, it may handle tens of thousands|
//| of general linear constraints. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| EpsX - >= 0, stopping criteria for inner optimizer. Inner |
//| iterations are stopped when step length (with|
//| variable scaling being applied) is less than |
//| EpsX. See MinQPSetScale() for more |
//| information on variable scaling. |
//| Rho - penalty coefficient, Rho > 0: |
//| * large enough that algorithm converges with |
//| desired precision. |
//| * not TOO large to prevent ill-conditioning |
//| * recommended values are 100, 1000 or 10000 |
//| ItsCnt - number of outer iterations: |
//| * recommended values: 10 - 15 (although in most |
//| cases it converges within 5 iterations, you may |
//| need a few more to be sure). |
//| * ItsCnt = 0 means that small number of outer |
//| iterations is automatically chosen (10 iterations|
//| in current version). |
//| * ItsCnt = 1 means that AUL algorithm performs just|
//| as usual penalty method. |
//| * ItsCnt > 1 means that AUL algorithm performs |
//| specified number of outer iterations |
//| IT IS VERY IMPORTANT TO CALL MinQPSetScale() WHEN YOU USE THIS |
//| ALGORITHM BECAUSE ITS CONVERGENCE PROPERTIES AND STOPPING |
//| CRITERIA ARE SCALE - DEPENDENT! |
//| NOTE: Passing EpsX = 0 will lead to automatic step length |
//| selection (specific step length chosen may change in the |
//| future versions of ALGLIB, so it is better to specify step |
//| length explicitly). |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetAlgoDenseAUL(CMinQPState &State,
double epsx,
double rho,
int itscnt)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(epsx),__FUNCTION__+": EpsX is not finite number"))
return;
if(!CAp::Assert(epsx>=0.0,__FUNCTION__+": negative EpsX"))
return;
if(!CAp::Assert(MathIsValidNumber(rho),__FUNCTION__+": Rho is not finite number"))
return;
if(!CAp::Assert(rho>0.0,__FUNCTION__+": non-positive Rho"))
return;
if(!CAp::Assert(itscnt>=0,__FUNCTION__+": negative ItsCnt!"))
return;
State.m_algokind=4;
if(epsx==0.0)
epsx=1.0E-8;
if(itscnt==0)
itscnt=10;
State.m_qpdenseaulsettingsuser.m_epsx=epsx;
State.m_qpdenseaulsettingsuser.m_outerits=itscnt;
State.m_qpdenseaulsettingsuser.m_rho=rho;
}
//+------------------------------------------------------------------+
//| This function tells QP solver to use DENSE-IPM QP algorithm and |
//| sets stopping criteria for the algorithm. |
//| This algorithm is intended for convex and semidefinite problems |
//| with moderate (up to several thousands) variable count and |
//| arbitrary number of constraints. |
//| IMPORTANT: this algorithm won't work for nonconvex problems, use |
//| DENSE-AUL or BLEIC-QP instead. If you try to run |
//| DENSE-IPM on problem with indefinite matrix (matrix |
//| having at least one negative eigenvalue) then |
//| depending on circumstances it may either(a) stall at |
//| some arbitrary point, or (b) throw exception on |
//| failure of Cholesky decomposition. |
//| ALGORITHM FEATURES: |
//| * supports box and dense / sparse general linear equality / |
//| inequality constraints |
//| ALGORITHM OUTLINE: |
//| * this algorithm is our implementation of interior point method|
//| as formulated by R.J.Vanderbei, with minor modifications to |
//| the algorithm (damped Newton directions are extensively used)|
//| * like all interior point methods, this algorithm tends to |
//| converge in roughly same number of iterations (between 15 and|
//| 50) independently from the problem dimensionality |
//| ALGORITHM LIMITATIONS: |
//| * because dense Cholesky driver is used, for N-dimensional |
//| problem with M dense constaints this algorithm has |
//| O(N^2 + N*M) memory requirements and O(N^3 + N*M^2) running |
//| time. Having sparse constraints with Z nonzeros per row |
//| relaxes storage and running time down to O(N^2 + M*Z) and |
//| O(N^3 + N*Z^2) From the practical point of view, it limits |
//| its applicability by several thousands of variables. From the|
//| other side, variables count is the most limiting factor, and |
//| dependence on constraint count is much more lower. Assuming |
//| that constraint matrix is sparse, it may handle tens of |
//| thousands of general linear constraints. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| Eps - >= 0, stopping criteria. The algorithm stops when |
//| primal and dual infeasiblities as well as |
//| complementarity gap are less than Eps. |
//| IT IS VERY IMPORTANT TO CALL minqpsetscale() WHEN YOU USE THIS |
//| ALGORITHM BECAUSE ITS CONVERGENCE PROPERTIES AND STOPPING |
//| CRITERIA ARE SCALE - DEPENDENT! |
//| NOTE: Passing EpsX = 0 will lead to automatic selection of small |
//| epsilon. |
//| ===== TRACING IPM SOLVER ======================================= |
//| IPM solver supports advanced tracing capabilities. You can trace |
//| algorithm output by specifying following trace symbols (case- |
//| insensitive) by means of trace_file() call: |
//| * 'IPM' - for basic trace of algorithm steps and decisions|
//| Only short scalars(function values and deltas) |
//| are printed. N-dimensional quantities like |
//| search directions are NOT printed. |
//| * 'IPM.DETAILED' - for output of points being visited and |
//| search directions. This symbol also implicitly |
//| defines 'IPM'. You can control output format by |
//| additionally specifying: |
//| * nothing to output in 6-digit exponential |
//| format |
//| * 'PREC.E15' to output in 15-digit exponential |
//| format |
//| * 'PREC.F6' to output in 6-digit fixed-point |
//| format |
//| By default trace is disabled and adds no overhead to the |
//| optimization process. However, specifying any of the symbols adds|
//| some formatting and output - related overhead. |
//| You may specify multiple symbols by separating them with commas: |
//| > |
//| >CAlglib::Trace_File("IPM,PREC.F6", "path/to/trace.log") |
//| > |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetAlgoDenseIPM(CMinQPState &State,
double eps)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(eps),__FUNCTION__+": Eps is not finite number"))
return;
if(!CAp::Assert(eps>=0.0,__FUNCTION__+": negative Eps"))
return;
//--- change value
State.m_algokind=5;
State.m_veps=eps;
}
//+------------------------------------------------------------------+
//| This function tells QP solver to use SPARSE-IPM QP algorithm |
//| and sets stopping criteria for the algorithm. |
//| This algorithm is intended for convex and semidefinite |
//| problems with large variable and constraint count and sparse |
//| quadratic term and constraints. It is possible to have some |
//| limited set of dense linear constraints - they will be handled |
//| separately by dense BLAS - but the more dense constraints you |
//| have, the more time solver needs. |
//| IMPORTANT: internally this solver performs large and sparse(N+M) |
//| x(N+M) triangular factorization. So it expects both |
//| quadratic term and constraints to be highly CSparse |
//| However, its running time is influenced by BOTH fill|
//| factor and sparsity pattern. |
//| Generally we expect that no more than few nonzero elements per |
//| row are present. However different sparsity patterns may result |
//| in completely different running times even given same fill |
//| factor. |
//| In many cases this algorithm outperforms DENSE-IPM by order of |
//| magnitude. However, in some cases you may get better results |
//| with DENSE-IPM even when solving sparse task. |
//| IMPORTANT: this algorithm won't work for nonconvex problems, use |
//| DENSE-AUL or BLEIC - QP instead. If you try to run |
//| DENSE-IPM on problem with indefinite matrix (matrix |
//| having at least one negative eigenvalue) then |
//| depending on circumstances it may either(a) stall at |
//| some arbitrary point, or (b) throw exception on |
//| failure of Cholesky decomposition. |
//| ALGORITHM FEATURES: |
//| * supports box and dense/sparse general linear equality/ |
//| inequality constraints |
//| * specializes on large-scale sparse problems |
//| ALGORITHM OUTLINE: |
//| * this algorithm is our implementation of interior point |
//| method as formulated by R.J.Vanderbei, with minor |
//| modifications to the algorithm (damped Newton directions are|
//| extensively used) |
//| * like all interior point methods, this algorithm tends to |
//| converge in roughly same number of iterations(between 15 and |
//| 50) independently from the problem dimensionality |
//| ALGORITHM LIMITATIONS: |
//| * this algorithm may handle moderate number of dense |
//| constraints, usually no more than a thousand of dense ones |
//| without losing its efficiency. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| Eps - >= 0, stopping criteria. The algorithm stops when |
//| primal and dual infeasiblities as well as |
//| complementarity gap are less than Eps. |
//| IT IS VERY IMPORTANT TO CALL minqpsetscale() WHEN YOU USE THIS |
//| ALGORITHM BECAUSE ITS CONVERGENCE PROPERTIES AND STOPPING |
//| CRITERIA ARE SCALE - DEPENDENT! |
//| NOTE: Passing EpsX = 0 will lead to automatic selection of small |
//| epsilon. |
//| ===== TRACING IPM SOLVER ======================================= |
//| IPM solver supports advanced tracing capabilities. You can trace |
//| algorithm output by specifying following trace symbols |
//| (case-insensitive) by means of trace_file() call: |
//| * 'IPM' - for basic trace of algorithm steps and decisions. |
//| Only short scalars(function values and deltas) are |
//| printed. N-dimensional quantities like search |
//| directions are NOT printed. |
//| * 'IPM.DETAILED' - for output of points being visited and |
//| search directions. This symbol also implicitly |
//| defines 'IPM'. You can control output format by |
//| additionally specifying: |
//| * nothing to output in 6-digit exponential |
//| format |
//| * 'PREC.E15' to output in 15-digit exponential |
//| format |
//| * 'PREC.F6' to output in 6-digit fixed-point |
//| format |
//| By default trace is disabled and adds no overhead to the |
//| optimization process. However, specifying any of the symbols adds|
//| some formatting and output - related overhead. |
//| You may specify multiple symbols by separating them with commas: |
//| > |
//| >CAlglib::Trace_File("IPM,PREC.F6", "path/to/trace.log") |
//| > |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetAlgoSparseIPM(CMinQPState &State,
double eps)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(eps),__FUNCTION__+": Eps is not finite number"))
return;
if(!CAp::Assert(eps>=0.0,__FUNCTION__+": negative Eps"))
return;
State.m_algokind=6;
State.m_veps=eps;
}
//+------------------------------------------------------------------+
//| This function tells solver to use QuickQP algorithm: special |
//| extra-fast algorithm for problems with box-only constrants. It |
//| may solve non-convex problems as long as they are bounded from |
//| below under constraints. |
//| ALGORITHM FEATURES: |
//| * several times faster than DENSE-IPM when running on box-only |
//| problem |
//| * utilizes accelerated methods for activation of constraints. |
//| * supports dense and sparse QP problems |
//| * supports ONLY box constraints; general linear constraints are|
//| NOT supported by this solver |
//| * can solve all types of problems (convex, semidefinite, |
//| nonconvex) as long as they are bounded from below under |
//| constraints. Say, it is possible to solve "min{-x^2} subject |
//| to -1<=x<=+1". In convex/semidefinite case global minimum is|
//| returned, in nonconvex case-algorithm returns one of the |
//| local minimums. |
//| ALGORITHM OUTLINE: |
//| * algorithm performs two kinds of iterations: constrained CG |
//| iterations and constrained Newton iterations |
//| * initially it performs small number of constrained CG |
//| iterations, which can efficiently activate/deactivate |
//| multiple constraints |
//| * after CG phase algorithm tries to calculate Cholesky |
//| decomposition and to perform several constrained Newton |
//| steps. If Cholesky decomposition failed(matrix is |
//| indefinite even under constraints), we perform more CG |
//| iterations until we converge to such set of constraints that |
//| system matrix becomes positive definite. Constrained Newton |
//| steps greatly increase convergence speed and precision. |
//| * algorithm interleaves CG and Newton iterations which allows |
//| to handle indefinite matrices (CG phase) and quickly converge|
//| after final set of constraints is found (Newton phase). |
//| Combination of CG and Newton phases is called "outer |
//| iteration". |
//| * it is possible to turn off Newton phase (beneficial for |
//| semidefinite problems - Cholesky decomposition will fail too |
//| often) |
//| ALGORITHM LIMITATIONS: |
//| * algorithm does not support general linear constraints; only|
//| box ones are supported |
//| * Cholesky decomposition for sparse problems is performed with |
//| Skyline Cholesky solver, which is intended for low-profile |
//| matrices. No profile-reducing reordering of variables is |
//| performed in this version of ALGLIB. |
//| * problems with near-zero negative eigenvalues (or exacty zero |
//| ones) may experience about 2-3x performance penalty. The |
//| reason is that Cholesky decomposition can not be performed |
//| until we identify directions of zero and negative curvature |
//| and activate corresponding boundary constraints- but we need |
//| a lot of trial and errors because these directions are hard |
//| to notice in the matrix spectrum. In this case you may turn |
//| off Newton phase of algorithm. Large negative eigenvalues |
//| are not an issue, so highly non-convex problems can be |
//| solved very efficiently. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| EpsG - >= 0. The subroutine finishes its work if the |
//| condition |v| < EpsG is satisfied, where: |
//| * | . | means Euclidian norm |
//| * v - scaled constrained gradient vector, |
//| v[i] = g[i] * s[i] |
//| * g - gradient |
//| * s - scaling coefficients set by MinQPSetScale() |
//| EpsF - >= 0. The subroutine finishes its work if |
//| exploratory steepest descent step on k+1-th |
//| iteration satisfies following condition: |
//| |F(k+1) - F(k)| <= EpsF*max{|F(k)|, |F(k+1)|, 1} |
//| EpsX - >= 0. The subroutine finishes its work if |
//| exploratory steepest descent step on k+1-th |
//| iteration satisfies following condition: |
//| * | . | means Euclidian norm |
//| * v - scaled step vector, v[i] = dx[i] / s[i] |
//| * dx - step vector, dx = X(k + 1) - X(k) |
//| * s - scaling coefficients set by MinQPSetScale() |
//| MaxOuterIts - maximum number of OUTER iterations. One outer |
//| iteration includes some amount of CG iterations |
//| (from 5 to ~N) and one or several (usually small |
//| amount) Newton steps. Thus, one outer iteration has|
//| high cost, but can greatly reduce funcation value. |
//| Use 0 if you do not want to limit number of outer |
//| iterations. |
//| UseNewton - use Newton phase or not: |
//| * Newton phase improves performance of positive |
//| definite dense problems (about 2 times |
//| improvement can be observed) |
//| * can result in some performance penalty on |
//| semidefinite or slightly negative definite |
//| problems - each Newton phase will bring no |
//| improvement (Cholesky failure), but still will |
//| require computational time. |
//| * if you doubt, you can turn off this phase - |
//| optimizer will retain its most of its high speed.|
//| IT IS VERY IMPORTANT TO CALL MinQPSetScale() WHEN YOU USE THIS |
//| ALGORITHM BECAUSE ITS STOPPING CRITERIA ARE SCALE - DEPENDENT! |
//| Passing EpsG = 0, EpsF = 0 and EpsX = 0 and MaxIts = 0 |
//| (simultaneously) will lead to automatic stopping criterion |
//| selection (presently it is small step length, but it may change |
//| in the future versions of ALGLIB). |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetAlgoQuickQP(CMinQPState &State,
double epsg,
double epsf,
double epsx,
int maxouterits,
bool usenewton)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(epsg),__FUNCTION__+": EpsG is not finite number"))
return;
if(!CAp::Assert(epsg>=0.0,__FUNCTION__+": negative EpsG"))
return;
if(!CAp::Assert(MathIsValidNumber(epsf),__FUNCTION__+": EpsF is not finite number"))
return;
if(!CAp::Assert(epsf>=0.0,__FUNCTION__+": negative EpsF"))
return;
if(!CAp::Assert(MathIsValidNumber(epsx),__FUNCTION__+": EpsX is not finite number"))
return;
if(!CAp::Assert(epsx>=0.0,__FUNCTION__+": negative EpsX"))
return;
if(!CAp::Assert(maxouterits>=0,__FUNCTION__+": negative MaxOuterIts!"))
return;
State.m_algokind=3;
if(epsg==0.0 && epsf==0.0 && epsx==0.0 && maxouterits==0)
epsx=1.0E-6;
State.m_qqpsettingsuser.m_maxouterits=maxouterits;
State.m_qqpsettingsuser.m_epsg=epsg;
State.m_qqpsettingsuser.m_epsf=epsf;
State.m_qqpsettingsuser.m_epsx=epsx;
State.m_qqpsettingsuser.m_cnphase=usenewton;
}
//+------------------------------------------------------------------+
//| This function sets boundary constraints for QP m_solver |
//| Boundary constraints are inactive by default (after initial |
//| creation). After being set, they are preserved until explicitly |
//| turned off with another SetBC() call. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| BndL - lower bounds, array[N]. |
//| If some (all) variables are unbounded, you may |
//| specify very small number or -INF (latter is |
//| recommended because it will allow m_solver to use |
//| better algorithm). |
//| BndU - upper bounds, array[N]. |
//| If some (all) variables are unbounded, you may |
//| specify very large number or +INF (latter is |
//| recommended because it will allow m_solver to use |
//| better algorithm). |
//| NOTE: it is possible to specify BndL[i]=BndU[i]. In this case |
//| I-th variable will be "frozen" at X[i]=BndL[i]=BndU[i]. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetBC(CMinQPState &State,double &bndl[],double &bndu[])
{
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(bndl)>=n,__FUNCTION__+": Length(BndL)<N"))
return;
//--- check
if(!CAp::Assert(CAp::Len(bndu)>=n,__FUNCTION__+": Length(BndU)<N"))
return;
//--- change values
for(int i=0; i<n; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(bndl[i]) || AL_NEGINF==bndl[i],__FUNCTION__+": BndL contains NAN or +INF"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(bndu[i]) || AL_POSINF==bndu[i],__FUNCTION__+": BndU contains NAN or -INF"))
return;
//--- change values
State.m_havebndl[i]=MathIsValidNumber(bndl[i]);
State.m_havebndu[i]=MathIsValidNumber(bndu[i]);
}
State.m_bndl=bndl;
State.m_bndu=bndu;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetBC(CMinQPState &State,CRowDouble &bndl,CRowDouble &bndu)
{
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(bndl)>=n,__FUNCTION__+": Length(BndL)<N"))
return;
//--- check
if(!CAp::Assert(CAp::Len(bndu)>=n,__FUNCTION__+": Length(BndU)<N"))
return;
//--- change values
for(int i=0; i<n; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(bndl[i]) || AL_NEGINF==bndl[i],__FUNCTION__+": BndL contains NAN or +INF"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(bndu[i]) || AL_POSINF==bndu[i],__FUNCTION__+": BndU contains NAN or -INF"))
return;
//--- change values
State.m_havebndl[i]=MathIsValidNumber(bndl[i]);
State.m_havebndu[i]=MathIsValidNumber(bndu[i]);
}
State.m_bndl=bndl;
State.m_bndu=bndu;
}
//+------------------------------------------------------------------+
//| This function sets box constraints for QP solver (all variables |
//| at once, same constraints for all variables) |
//| Box constraints are inactive by default (after initial creation)|
//| After being set, they are preserved until explicitly overwritten |
//| with another MinQPSetBC() or MinQPSetBCAll() call, or partially |
//| overwritten with MinQPSetBCI() call. |
//| Following types of constraints are supported: |
//| DESCRIPTION CONSTRAINT HOW TO SPECIFY |
//| fixed variable x[i]=Bnd BndL=BndU |
//| lower bound BndL<=x[i] BndU=+INF |
//| upper bound x[i]<=BndU BndL=-INF |
//| range BndL<=x[i]<=BndU ... |
//| free variable - BndL=-INF, BndU+INF|
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| BndL - lower bound, same for all variables |
//| BndU - upper bound, same for all variables |
//| NOTE: infinite values can be specified by means of AL_NEGINF and |
//| AL_POSINF. |
//| NOTE: you may replace infinities by very small/very large values,|
//| but it is not recommended because large numbers may |
//| introduce large numerical errors in the algorithm. |
//| NOTE: BndL>BndU will result in QP problem being recognized as |
//| infeasible. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetBCAll(CMinQPState &State,
double bndl,
double bndu)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(bndl) || AL_NEGINF==bndl,__FUNCTION__+": BndL is NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(bndu) || AL_POSINF==bndu,__FUNCTION__+": BndU is NAN or -INF"))
return;
State.m_bndl.Fill(bndl);
State.m_bndu.Fill(bndu);
ArrayInitialize(State.m_havebndl,MathIsValidNumber(bndl));
ArrayInitialize(State.m_havebndu,MathIsValidNumber(bndu));
}
//+------------------------------------------------------------------+
//| This function sets box constraints for I-th variable (other |
//| variables are not modified). |
//| Following types of constraints are supported: |
//| DESCRIPTION CONSTRAINT HOW TO SPECIFY |
//| fixed variable x[i] = Bnd BndL = BndU |
//| lower bound BndL <= x[i] BndU = +INF |
//| upper bound x[i] <= BndU BndL = -INF |
//| range BndL <= x[i] <= BndU ... |
//| free variable - BndL -INF, BndU +INF|
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| BndL - lower bound |
//| BndU - upper bound |
//| NOTE: infinite values can be specified by means of AL_POSINF and |
//| AL_POSINF. |
//| NOTE: you may replace infinities by very small/very large values,|
//| but it is not recommended because large numbers may |
//| introduce large numerical errors in the algorithm. |
//| NOTE: BndL>BndU will result in QP problem being recognized as |
//| infeasible. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetBCI(CMinQPState &State,
int i,
double bndl,
double bndu)
{
//--- check
if(!CAp::Assert(i>=0 && i<State.m_n,__FUNCTION__+": I is outside of [0,N)"))
return;
if(!CAp::Assert(MathIsValidNumber(bndl) || AL_NEGINF==bndl,__FUNCTION__+": BndL is NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(bndu) || AL_POSINF==bndu,__FUNCTION__+": BndU is NAN or -INF"))
return;
//--- change values
State.m_bndl.Set(i,bndl);
State.m_bndu.Set(i,bndu);
State.m_havebndl[i]=MathIsValidNumber(bndl);
State.m_havebndu[i]=MathIsValidNumber(bndu);
}
//+------------------------------------------------------------------+
//| This function sets dense linear constraints for QP optimizer. |
//| This function overrides results of previous calls to MinQPSetLC()|
//| MinQPSetLCSparse() and MinQPSetLCMixed(). After call to this |
//| function all non-box constraints are dropped, and you have only |
//| those constraints which were specified in the present call. |
//| If you want to specify mixed(with dense and sparse terms) linear |
//| constraints, you should call MinQPSetLCMixed(). |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinQPCreate |
//| call. |
//| C - linear constraints, array[K, N + 1]. Each row of C |
//| represents one constraint, either equality or |
//| inequality (see below): |
//| * first N elements correspond to coefficients, |
//| * last element corresponds to the right part. |
//| All elements of C(including right part) must be |
//| finite. |
//| CT - type of constraints, array[K]: |
//| * if CT[i] > 0, then I-th constraint is |
//| C[i, *] * x >= C[i, n + 1] |
//| * if CT[i] = 0, then I-th constraint is |
//| C[i, *] * x = C[i, n + 1] |
//| * if CT[i] < 0, then I-th constraint is |
//| C[i, *] * x <= C[i, n + 1] |
//| K - number of equality/inequality constraints, K >= 0: |
//| * if given, only leading K elements of C/CT are |
//| used |
//| * if not given, automatically determined from sizes|
//| of C/CT |
//| NOTE 1: linear (non-bound) constraints are satisfied only |
//| approximately - there always exists some violation due |
//| to numerical errors and algorithmic limitations |
//| (BLEIC-QP solver is most precise, AUL-QP solver is less |
//| precise). |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetLC(CMinQPState &State,
CMatrixDouble &c,
CRowInt &ct,
int k)
{
CSparseMatrix dummyc;
CRowInt dummyct;
MinQPSetLCMixed(State,dummyc,dummyct,0,c,ct,k);
}
//+------------------------------------------------------------------+
//| This function sets sparse linear constraints for QP optimizer. |
//| This function overrides results of previous calls to MinQPSetLC()|
//| MinQPSetLCSparse() and MinQPSetLCMixed(). After call to this |
//| function all non-box constraints are dropped, and you have only |
//| those constraints which were specified in the present call. |
//| If you want to specify mixed(with dense and sparse terms) linear |
//| constraints, you should call MinQPSetLCMixed(). |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinQPCreate |
//| call. |
//| C - linear constraints, sparse matrix with dimensions |
//| at least [K, N + 1]. If matrix has larger size, |
//| only leading Kx(N + 1) rectangle is used. Each row |
//| of C represents one constraint, either equality or |
//| inequality(see below) : |
//| * first N elements correspond to coefficients, |
//| * last element corresponds to the right part. |
//| All elements of C(including right part) must be |
//| finite. |
//| CT - type of constraints, array[K]: |
//| * if CT[i] > 0, then I-th constraint is |
//| C[i, *] * x >= C[i, n + 1] |
//| * if CT[i] = 0, then I-th constraint is |
//| C[i, *] * x = C[i, n + 1] |
//| * if CT[i] < 0, then I-th constraint is |
//| C[i, *] * x <= C[i, n + 1] |
//| K - number of equality/inequality constraints, K >= 0: |
//| * if given, only leading K elements of C/CT are |
//| used |
//| * if not given, automatically determined from sizes|
//| of C/CT |
//| NOTE 1: linear (non-bound) constraints are satisfied only |
//| approximately - there always exists some violation due |
//| to numerical errors and algorithmic limitations |
//| (BLEIC-QP solver is most precise, AUL-QP solver is less |
//| precise). |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetLCSparse(CMinQPState &State,
CSparseMatrix &c,
CRowInt &ct,
int k)
{
CMatrixDouble dummyc;
CRowInt dummyct;
MinQPSetLCMixed(State,c,ct,k,dummyc,dummyct,0);
}
//+------------------------------------------------------------------+
//| This function sets mixed linear constraints, which include a set |
//| of dense rows, and a set of sparse rows. |
//| This function overrides results of previous calls to MinQPSetLC()|
//| MinQPSetLCSparse() and MinQPSetLCMixed(). After call to this |
//| function all non-box constraints are dropped, and you have only |
//| those constraints which were specified in the present call. |
//| If you want to specify mixed(with dense and sparse terms) linear |
//| constraints, you should call MinQPSetLCMixed(). |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinQPCreate |
//| call. |
//| SparseC - linear constraints, sparse matrix with dimensions |
//| EXACTLY EQUAL TO [SparseK, N + 1]. Each row of C |
//| represents one constraint, either equality or |
//| inequality (see below): |
//| * first N elements correspond to coefficients, |
//| * last element corresponds to the right part. |
//| All elements of C(including right part) must be |
//| finite. |
//| SparseCT - type of sparse constraints, array[K]: |
//| * if SparseCT[i] > 0, then I-th constraint is |
//| SparseC[i, *] * x >= SparseC[i, n + 1] |
//| * if SparseCT[i] = 0, then I-th constraint is |
//| SparseC[i, *] * x = SparseC[i, n + 1] |
//| * if SparseCT[i] < 0, then I-th constraint is |
//| SparseC[i, *] * x <= SparseC[i, n + 1] |
//| SparseK - number of sparse equality/inequality constraints, |
//| K >= 0 |
//| DenseC - dense linear constraints, array[K, N + 1]. Each row|
//| of DenseC represents one constraint, either |
//| equality or inequality(see below): |
//| * first N elements correspond to coefficients, |
//| * last element corresponds to the right part. |
//| All elements of DenseC (including right part) must |
//| be finite. |
//| DenseCT - type of constraints, array[K]: |
//| * if DenseCT[i] > 0, then I-th constraint is |
//| DenseC[i, *] * x >= DenseC[i, n + 1] |
//| * if DenseCT[i] = 0, then I-th constraint is |
//| DenseC[i, *] * x = DenseC[i, n + 1] |
//| * if DenseCT[i] < 0, then I-th constraint is |
//| DenseC[i, *] * x <= DenseC[i, n + 1] |
//| DenseK - number of equality/inequality constraints, |
//| DenseK >= 0 |
//| NOTE 1: linear(non-box) constraints are satisfied only |
//| approximately - there always exists some violation due |
//| to numerical errors and algorithmic limitations |
//| (BLEIC-QP solver is most precise, AUL-QP solver is less |
//| precise). |
//| NOTE 2: due to backward compatibility reasons SparseC can be |
//| larger than [SparseK, N + 1]. In this case only leading |
//| [SparseK, N+1] submatrix will be used. However, the rest |
//| of ALGLIB has more strict requirements on the input size,|
//| so we recommend you to pass sparse term whose size |
//| exactly matches algorithm expectations. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetLCMixed(CMinQPState &State,CSparseMatrix &sparsec,
CRowInt &sparsect,int sparsek,
CMatrixDouble &densec,CRowInt &densect,
int densek)
{
//--- create variables
int n=State.m_n;
int i=0;
int j=0;
int j0=0;
double v=0;
CRowInt srcidx;
CRowInt dstidx;
CRowDouble s;
CRowInt rs;
CRowInt eoffs;
CRowInt roffs;
CRowDouble v2;
CRowInt eidx;
CRowDouble eval;
int t0=0;
int t1=0;
int nnz=0;
//-- First, check for errors in the inputs
if(!CAp::Assert(densek>=0,__FUNCTION__+": K<0"))
return;
if(!CAp::Assert(densek==0 || densec.Cols()>n,__FUNCTION__+": Cols(C)<N+1"))
return;
if(!CAp::Assert(densec.Rows()>=densek,__FUNCTION__+": Rows(DenseC)<DenseK"))
return;
if(!CAp::Assert(CAp::Len(densect)>=densek,__FUNCTION__+": Length(DenseCT)<DenseK"))
return;
if(!CAp::Assert(CApServ::IsFiniteMatrix(densec,densek,n+1),__FUNCTION__+": C contains infinite or NaN values!"))
return;
if(!CAp::Assert(sparsek>=0,__FUNCTION__+": SparseK<0"))
return;
if(!CAp::Assert(sparsek==0 || CSparse::SparseGetNCols(sparsec)>=n+1,__FUNCTION__+": Cols(SparseC)<N+1"))
return;
if(!CAp::Assert(sparsek==0 || CSparse::SparseGetNRows(sparsec)>=sparsek,__FUNCTION__+": Rows(SparseC)<SparseK"))
return;
if(!CAp::Assert(CAp::Len(sparsect)>=sparsek,__FUNCTION__+": Length(SparseCT)<SparseK"))
return;
//-- Allocate place for Lagrange multipliers, fill by zero
State.m_replaglc=vector<double>::Zeros(densek+sparsek);
//-- Init
State.m_cl=vector<double>::Zeros(densek+sparsek);
State.m_cu=vector<double>::Zeros(densek+sparsek);
State.m_mdense=densek;
State.m_msparse=sparsek;
if(sparsek>0)
{
//-- Evaluate row sizes for new storage
rs.Resize(sparsek);
rs.Fill(0);
t0=0;
t1=0;
nnz=0;
while(CSparse::SparseEnumerate(sparsec,t0,t1,i,j,v))
{
if(i>sparsek-1 || j>n-1)
continue;
//--- check
if(!CAp::Assert(MathIsValidNumber(v),__FUNCTION__+": C contains infinite or NAN values"))
return;
nnz++;
rs.Add(i,1);
}
//-- Prepare new sparse CRS storage, copy leading SparseK*N submatrix into the storage
for(i=0; i<sparsek; i++)
{
State.m_cl.Set(i,0);
State.m_cu.Set(i,0);
}
State.m_sparsec.m_M=sparsek;
State.m_sparsec.m_N=n;
State.m_sparsec.m_RIdx.Resize(sparsek+1);
State.m_sparsec.m_Idx.Resize(nnz);
State.m_sparsec.m_Vals.Resize(nnz);
eoffs.Resize(sparsek+1);
State.m_sparsec.m_RIdx.Set(0,0);
eoffs.Set(0,0);
for(i=1; i<=sparsek; i++)
{
State.m_sparsec.m_RIdx.Set(i,State.m_sparsec.m_RIdx[i-1]+rs[i-1]);
eoffs.Set(i,State.m_sparsec.m_RIdx[i]);
}
t0=0;
t1=0;
while(CSparse::SparseEnumerate(sparsec,t0,t1,i,j,v))
{
if(i>sparsek-1 || j>n)
continue;
if(j<n)
{
//-- Copy left part of constraint
j0=eoffs[i];
State.m_sparsec.m_Idx.Set(j0,j);
State.m_sparsec.m_Vals.Set(j0,v);
eoffs.Set(i,j0+1);
}
else
{
//-- Handle right part of the constraint
State.m_cl.Set(i,v);
State.m_cu.Set(i,v);
}
}
for(i=0; i<sparsek; i++)
{
//--- check
if(!CAp::Assert(eoffs[i]==State.m_sparsec.m_RIdx[i+1],__FUNCTION__+": critical integrity check failed (sparse copying)"))
return;
}
CSparse::SparseCreateCRSInplace(State.m_sparsec);
for(i=0; i<sparsek; i++)
{
if(sparsect[i]>0)
State.m_cu.Set(i,AL_POSINF);
if(sparsect[i]<0)
State.m_cl.Set(i,AL_NEGINF);
}
}
if(densek>0)
{
//-- Copy dense constraints
State.m_densec=densec;
State.m_densec.Resize(densek,n);
for(i=0; i<densek; i++)
{
if(densect[i]>0)
{
State.m_cl.Set(sparsek+i,densec.Get(i,n));
State.m_cu.Set(sparsek+i,AL_POSINF);
continue;
}
if(densect[i]<0)
{
State.m_cl.Set(sparsek+i,AL_NEGINF);
State.m_cu.Set(sparsek+i,densec.Get(i,n));
continue;
}
State.m_cl.Set(sparsek+i,densec.Get(i,n));
State.m_cu.Set(sparsek+i,densec.Get(i,n));
}
}
}
//+------------------------------------------------------------------+
//| This function provides legacy API for specification of mixed |
//| dense / sparse linear constraints. |
//| New conventions used by ALGLIB since release 3.16.0 State that |
//| set of sparse constraints comes first, followed by set of |
//| dense ones. This convention is essential when you talk about |
//| things like order of Lagrange multipliers. |
//| However, legacy API accepted mixed constraints in reverse order. |
//| This function is here to simplify situation with code relying on |
//| legacy API. It simply accepts constraints in one order (old) and |
//| passes them to new API, now in correct order. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetLCMixedLegacy(CMinQPState &State,
CMatrixDouble &densec,
CRowInt &densect,
int densek,
CSparseMatrix &sparsec,
CRowInt &sparsect,
int sparsek)
{
MinQPSetLCMixed(State,sparsec,sparsect,sparsek,densec,densect,densek);
}
//+------------------------------------------------------------------+
//| This function sets two-sided linear constraints AL <= A*x <= AU |
//| with dense constraint matrix A. |
//| NOTE: knowing that constraint matrix is dense helps some QP |
//| solvers (especially modern IPM method) to utilize efficient dense|
//| Level 3 BLAS for dense parts of the problem. If your problem has |
//| both dense and sparse constraints, you can use MinQPSetLC2Mixed()|
//| function, which will result in dense algebra being applied to |
//| dense terms, and sparse sparse linear algebra applied to sparse |
//| terms. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinQPCreate() |
//| call. |
//| A - linear constraints, array[K, N]. Each row of A |
//| represents one constraint. One-sided inequality |
//| constraints, two-sided inequality constraints, |
//| equality constraints are supported (see below) |
//| AL, AU - lower and upper bounds, array[K]; |
//| * AL[i] = AU[i] => equality constraint Ai * x |
//| * AL[i]<AU.Set(i,> two-sided constraint |
//| AL[i] <= Ai*x <= AU[i] |
//| * AL[i] = -INF => one-sided constraint |
//| Ai*x <= AU[i] |
//| * AU[i] = +INF => one-sided constraint |
//| AL[i] <= Ai*x |
//| * AL[i] = -INF, AU[i] = +INF => constraint is |
//| ignored |
//| K - number of equality/inequality constraints, K >= 0; |
//| if not given, inferred from sizes of A, AL, AU. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetLC2Dense(CMinQPState &State,CMatrixDouble &a,
CRowDouble &al,CRowDouble &au,int k)
{
MinQPSetLC2Mixed(State,State.m_dummysparse,0,a,k,al,au);
}
//+------------------------------------------------------------------+
//| This function sets two-sided linear constraints AL <= A*x <= AU |
//| with sparse constraining matrix A. Recommended for large-scale |
//| problems. |
//| This function overwrites linear (non-box) constraints set by |
//| previous calls(if such calls were made). |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinQPCreate() |
//| call. |
//| A - sparse matrix with size [K, N](exactly!). Each row |
//| of A represents one general linear constraint. A |
//| can be stored in any sparse storage format. |
//| AL, AU - lower and upper bounds, array[K]; |
//| * AL[i] = AU[i] => equality constraint Ai*x |
//| * AL[i]<AU.Set(i,> two-sided constraint |
//| AL[i] <= Ai*x <= AU[i] |
//| * AL[i] = -INF => one-sided constraint |
//| Ai*x <= AU[i] |
//| * AU[i] = +INF => one-sided constraint |
//| AL[i] <= Ai*x |
//| * AL[i] = -INF, AU[i] = +INF => constraint is |
//| ignored |
//| K - number of equality/inequality constraints, K >= 0. |
//| If K = 0 is specified, A, AL, AU are ignored. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetLC2(CMinQPState &State,CSparseMatrix &a,
CRowDouble &al,CRowDouble &au,int k)
{
MinQPSetLC2Mixed(State,a,k,State.m_dummyr2,0,al,au);
}
//+------------------------------------------------------------------+
//| This function sets two-sided linear constraints AL <= A*x <= AU |
//| with mixed constraining matrix A including sparse part (first |
//| SparseK rows) and dense part(last DenseK rows). Recommended for |
//| large-scale problems. |
//| This function overwrites linear (non-box) constraints set by |
//| previous calls (if such calls were made). |
//| This function may be useful if constraint matrix includes large |
//| number of both types of rows - dense and CSparse If you have just|
//| a few sparse rows, you may represent them in dense format without|
//| losing performance. Similarly, if you have just a few dense rows,|
//| you may store them in sparse format with almost same performance.|
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinQPCreate() |
//| call. |
//| SparseA - sparse matrix with size [K, N](exactly!). Each row |
//| of A represents one general linear constraint. A |
//| can be stored in any sparse storage format. |
//| SparseK - number of sparse constraints, SparseK >= 0 |
//| DenseA - linear constraints, array[K, N], set of dense |
//| constraints. Each row of A represents one general |
//| linear constraint. |
//| DenseK - number of dense constraints, DenseK >= 0 |
//| AL, AU - lower and upper bounds, array[SparseK + DenseK], |
//| with former SparseK elements corresponding to |
//| sparse constraints, and latter DenseK elements |
//| corresponding to dense constraints; |
//| * AL[i] = AU[i] => equality constraint Ai*x |
//| * AL[i]<AU.Set(i,> two-sided constraint |
//| AL[i] <= Ai*x <= AU[i] |
//| * AL[i] = -INF => one-sided constraint |
//| Ai*x <= AU[i] |
//| * AU[i] = +INF => one-sided constraint |
//| AL[i] <= Ai*x |
//| * AL[i] = -INF, AU[i] = +INF => constraint is |
//| ignored |
//| K - number of equality/inequality constraints, K >= 0.|
//| If K = 0 is specified, A, AL, AU are ignored. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetLC2Mixed(CMinQPState &State,CSparseMatrix &sparsea,
int ksparse,CMatrixDouble &densea,
int kdense,CRowDouble &al,CRowDouble &au)
{
//--- create variables
int n=State.m_n;
int m=kdense+ksparse;
//-- Check input arguments
if(!CAp::Assert(ksparse>=0,__FUNCTION__+": KSparse<0"))
return;
if(!CAp::Assert(ksparse==0 || CSparse::SparseGetNCols(sparsea)==n,__FUNCTION__+": Cols(SparseA)<>N"))
return;
if(!CAp::Assert(ksparse==0 || CSparse::SparseGetNRows(sparsea)==ksparse,__FUNCTION__+": Rows(SparseA)<>K"))
return;
if(!CAp::Assert(kdense>=0,__FUNCTION__+": KDense<0"))
return;
if(!CAp::Assert(kdense==0 || CAp::Cols(densea)>=n,__FUNCTION__+": Cols(DenseA)<N"))
return;
if(!CAp::Assert(kdense==0 || CAp::Rows(densea)>=kdense,__FUNCTION__+": Rows(DenseA)<K"))
return;
if(!CAp::Assert(CApServ::IsFiniteMatrix(densea,kdense,n),__FUNCTION__+": DenseA contains infinite or NaN values!"))
return;
if(!CAp::Assert(CAp::Len(al)>=kdense+ksparse,__FUNCTION__+": Length(AL)<K"))
return;
if(!CAp::Assert(CAp::Len(au)>=kdense+ksparse,__FUNCTION__+": Length(AU)<K"))
return;
for(int i=0; i<m; i++)
{
if(!CAp::Assert(MathIsValidNumber(al[i]) || IsNegInf(al[i]),__FUNCTION__+": AL contains NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(au[i]) || IsPosInf(au[i]),__FUNCTION__+": AU contains NAN or -INF"))
return;
}
//-- Allocate place for Lagrange multipliers, fill by zero
State.m_replaglc=vector<double>::Zeros(kdense+ksparse);
//-- Quick exit if needed
if(m==0)
{
State.m_mdense=0;
State.m_msparse=0;
return;
}
//-- Prepare
State.m_cl=al;
State.m_cu=au;
State.m_cl.Resize(m);
State.m_cu.Resize(m);
State.m_mdense=kdense;
State.m_msparse=ksparse;
//-- Copy dense and sparse terms
if(ksparse>0)
{
CSparse::SparseCopyToCRSBuf(sparsea,State.m_sparsec);
}
if(kdense>0)
{
State.m_densec=densea;
State.m_densec.Resize(kdense,n);
}
}
//+------------------------------------------------------------------+
//| This function appends two-sided linear constraint AL <= A*x <= AU|
//| to the matrix of currently present dense constraints. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinQPCreate() |
//| call. |
//| A - linear constraint coefficient, array[N], right side|
//| is NOT included. |
//| AL, AU - lower and upper bounds; |
//| * AL = AU => equality constraint Ai*x |
//| * AL < AU => two-sided constraint |
//| AL <= Ai*x <= AU |
//| * AL = -INF => one-sided constraint |
//| Ai*x <= AU |
//| * AU = +INF => one-sided constraint |
//| AL <= Ai*x |
//| * AL = -INF, AU = +INF => constraint is |
//| ignored |
//+------------------------------------------------------------------+
void CMinQP::MinQPAddLC2Dense(CMinQPState &State,
CRowDouble &a,
double al,
double au)
{
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(a)>=n,__FUNCTION__+": Length(A)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(a,n),__FUNCTION__+": A contains infinite or NaN values!"))
return;
if(!CAp::Assert(MathIsValidNumber(al) || IsNegInf(al),__FUNCTION__+": AL is NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(au) || IsPosInf(au),__FUNCTION__+": AU is NAN or -INF"))
return;
State.m_cl.Resize(State.m_msparse+State.m_mdense+1);
State.m_cu.Resize(State.m_msparse+State.m_mdense+1);
State.m_replaglc.Resize(State.m_msparse+State.m_mdense+1);
State.m_densec.Resize(State.m_mdense+1,n);
State.m_densec.Row(State.m_mdense,a);
State.m_cl.Set(State.m_msparse+State.m_mdense,al);
State.m_cu.Set(State.m_msparse+State.m_mdense,au);
State.m_replaglc.Set(State.m_msparse+State.m_mdense,0.0);
State.m_mdense++;
}
//+------------------------------------------------------------------+
//| This function appends two-sided linear constraint AL <= A*x <= AU|
//| to the list of currently present sparse constraints. |
//| Constraint is passed in compressed format: as list of non-zero |
//| entries of coefficient vector A. Such approach is more efficient |
//| than dense storage for highly sparse constraint vectors. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinQPCreate() |
//| call. |
//| IdxA - array[NNZ], indexes of non-zero elements of A: |
//| * can be unsorted |
//| * can include duplicate indexes (corresponding |
//| entries of ValA[] will be summed) |
//| ValA - array[NNZ], values of non-zero elements of A |
//| NNZ - number of non-zero coefficients in A |
//| AL, AU - lower and upper bounds; |
//| * AL = AU => equality constraint A*x |
//| * AL<AU => two-sided constraint AL <= A*x <= AU |
//| * AL = -INF => one-sided constraint A*x <= AU |
//| * AU = +INF => one-sided constraint AL <= A*x |
//| * AL = -INF, AU = +INF => constraint is ignored |
//+------------------------------------------------------------------+
void CMinQP::MinQPAddLC2(CMinQPState &State,CRowInt &idxa,
CRowDouble &vala,int nnz,double al,
double au)
{
//--- create variables
int n=State.m_n;
int i=0;
int j=0;
int k=0;
int offs=0;
int offsdst=0;
int didx=0;
int uidx=0;
//-- Check inputs
if(!CAp::Assert(nnz>=0,__FUNCTION__+": NNZ<0"))
return;
if(!CAp::Assert(CAp::Len(idxa)>=nnz,__FUNCTION__+": Length(IdxA)<NNZ"))
return;
if(!CAp::Assert(CAp::Len(vala)>=nnz,__FUNCTION__+": Length(ValA)<NNZ"))
return;
for(i=0; i<nnz; i++)
if(!CAp::Assert(idxa[i]>=0 && idxa[i]<n,__FUNCTION__+": IdxA contains indexes outside of [0,N) range"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(vala,nnz),__FUNCTION__+": ValA contains infinite or NaN values!"))
return;
if(!CAp::Assert(MathIsValidNumber(al) || IsNegInf(al),__FUNCTION__+": AL is NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(au) || IsPosInf(au),__FUNCTION__+": AU is NAN or -INF"))
return;
//-- If M=0, it means that A is uninitialized.
//-- Prepare sparse matrix structure
if(State.m_msparse==0)
{
State.m_sparsec.m_MatrixType=1;
State.m_sparsec.m_M=0;
State.m_sparsec.m_N=n;
State.m_sparsec.m_NInitialized=0;
State.m_sparsec.m_RIdx.Resize(1);
State.m_sparsec.m_RIdx.Set(0,0);
}
//--- check
if(!CAp::Assert(State.m_sparsec.m_MatrixType==1 && State.m_sparsec.m_M==State.m_msparse,__FUNCTION__+": integrity check failed!"))
return;
//-- Reallocate inequality bounds
State.m_cl.Resize(State.m_msparse+State.m_mdense+1);
State.m_cu.Resize(State.m_msparse+State.m_mdense+1);
State.m_replaglc.Resize(State.m_msparse+State.m_mdense+1);
for(i=State.m_msparse+State.m_mdense; i>=State.m_msparse+1; i--)
{
State.m_cl.Set(i,State.m_cl[i-1]);
State.m_cu.Set(i,State.m_cu[i-1]);
State.m_replaglc.Set(i,State.m_replaglc[i-1]);
}
State.m_cl.Set(State.m_msparse,al);
State.m_cu.Set(State.m_msparse,au);
State.m_replaglc.Set(State.m_msparse,0.0);
//-- Reallocate sparse storage
offs=State.m_sparsec.m_RIdx[State.m_msparse];
State.m_sparsec.m_Idx.Resize(offs+nnz);
State.m_sparsec.m_Vals.Resize(offs+nnz);
State.m_sparsec.m_DIdx.Resize(State.m_msparse+1);
State.m_sparsec.m_UIdx.Resize(State.m_msparse+1);
State.m_sparsec.m_RIdx.Resize(State.m_msparse+2);
//-- If NNZ=0, perform quick and simple row append.
if(nnz==0)
{
State.m_sparsec.m_DIdx.Set(State.m_msparse,State.m_sparsec.m_RIdx[State.m_msparse]);
State.m_sparsec.m_UIdx.Set(State.m_msparse,State.m_sparsec.m_RIdx[State.m_msparse]);
State.m_sparsec.m_RIdx.Set(State.m_msparse+1,State.m_sparsec.m_RIdx[State.m_msparse]);
State.m_sparsec.m_M++;
State.m_msparse++;
return;
}
//-- Now we are sure that SparseC contains properly initialized sparse
//-- matrix (or some appropriate dummy for M=0) and we have NNZ>0
//-- (no need to care about degenerate cases).
//-- Append rows to SparseC:
//-- * append data
//-- * sort in place
//-- * merge duplicate indexes
//-- * compute DIdx and UIdx
for(i=0; i<nnz; i++)
{
State.m_sparsec.m_Idx.Set(offs+i,idxa[i]);
State.m_sparsec.m_Vals.Set(offs+i,vala[i]);
}
CTSort::TagSortMiddleIR(State.m_sparsec.m_Idx,State.m_sparsec.m_Vals,offs,nnz);
offsdst=offs;
for(i=1; i<nnz; i++)
{
if(State.m_sparsec.m_Idx[offsdst]!=State.m_sparsec.m_Idx[offs+i])
{
offsdst++;
State.m_sparsec.m_Idx.Set(offsdst,State.m_sparsec.m_Idx[offs+i]);
State.m_sparsec.m_Vals.Set(offsdst,State.m_sparsec.m_Vals[offs+i]);
}
else
State.m_sparsec.m_Vals.Add(offsdst,State.m_sparsec.m_Vals[offs+i]);
}
nnz=offsdst-offs+1;
uidx=-1;
didx=-1;
for(j=offs; j<=offsdst; j++)
{
k=State.m_sparsec.m_Idx[j];
if(k==State.m_msparse)
didx=j;
else
if(k>State.m_msparse && uidx==-1)
{
uidx=j;
break;
}
}
if(uidx==-1)
uidx=offsdst+1;
if(didx==-1)
didx=uidx;
State.m_sparsec.m_DIdx.Set(State.m_msparse,didx);
State.m_sparsec.m_UIdx.Set(State.m_msparse,uidx);
State.m_sparsec.m_RIdx.Set(State.m_msparse+1,offsdst+1);
State.m_sparsec.m_NInitialized=State.m_sparsec.m_RIdx[State.m_msparse+1];
State.m_sparsec.m_M++;
State.m_msparse++;
}
//+------------------------------------------------------------------+
//| This function appends two-sided linear constraint AL <= A*x <= AU|
//| to the list of currently present sparse constraints. |
//| Constraint vector A is passed as a dense array which is |
//| internally sparsified by this function. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinQPCreate() |
//| call. |
//| DA - array[N], constraint vector |
//| AL, AU - lower and upper bounds; |
//| * AL = AU => equality constraint A*x |
//| * AL < AU => two-sided constraint AL<=A*x<=AU |
//| * AL = -INF => one-sided constraint A*x <= AU |
//| * AU = +INF => one-sided constraint AL <= A*x |
//| * AL = -INF, AU = +INF => constraint is ignored |
//+------------------------------------------------------------------+
void CMinQP::MinQPAddLC2SparseFromDense(CMinQPState &State,
CRowDouble &da,
double al,
double au)
{
//--- create variables
int n=State.m_n;
int i=0;
int j=0;
int k=0;
int nzi=0;
int offs=0;
int nnz=0;
int didx=0;
int uidx=0;
//-- Check inputs
if(!CAp::Assert(CAp::Len(da)>=n,__FUNCTION__+": Length(DA)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(da,n),__FUNCTION__+": DA contains infinities/NANs"))
return;
if(!CAp::Assert(MathIsValidNumber(al) || IsNegInf(al),__FUNCTION__+": AL is NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(au) || IsPosInf(au),__FUNCTION__+": AU is NAN or -INF"))
return;
//-- If M=0, it means that A is uninitialized.
//-- Prepare sparse matrix structure
if(State.m_msparse==0)
{
State.m_sparsec.m_MatrixType=1;
State.m_sparsec.m_M=0;
State.m_sparsec.m_N=n;
State.m_sparsec.m_NInitialized=0;
State.m_sparsec.m_RIdx.Resize(1);
State.m_sparsec.m_RIdx.Set(0,0);
}
//--- check
if(!CAp::Assert(State.m_sparsec.m_MatrixType==1 && State.m_sparsec.m_M==State.m_msparse,__FUNCTION__+": integrity check failed!"))
return;
//-- Reallocate inequality bounds
State.m_cl.Resize(State.m_msparse+State.m_mdense+1);
State.m_cu.Resize(State.m_msparse+State.m_mdense+1);
State.m_replaglc.Resize(State.m_msparse+State.m_mdense+1);
for(i=State.m_msparse+State.m_mdense; i>=State.m_msparse+1; i--)
{
State.m_cl.Set(i,State.m_cl[i-1]);
State.m_cu.Set(i,State.m_cu[i-1]);
State.m_replaglc.Set(i,State.m_replaglc[i-1]);
}
State.m_cl.Set(State.m_msparse,al);
State.m_cu.Set(State.m_msparse,au);
State.m_replaglc.Set(State.m_msparse,0.0);
//-- Determine nonzeros count.
//-- Reallocate sparse storage.
nnz=0;
for(i=0; i<n; i++)
{
if(!(da[i]==0.0))
nnz++;
}
offs=State.m_sparsec.m_RIdx[State.m_msparse];
State.m_sparsec.m_Idx.Resize(offs+nnz);
State.m_sparsec.m_Vals.Resize(offs+nnz);
State.m_sparsec.m_DIdx.Resize(State.m_msparse+1);
State.m_sparsec.m_UIdx.Resize(State.m_msparse+1);
State.m_sparsec.m_RIdx.Resize(State.m_msparse+2);
//-- If NNZ=0, perform quick and simple row append.
if(nnz==0)
{
State.m_sparsec.m_DIdx.Set(State.m_msparse,State.m_sparsec.m_RIdx[State.m_msparse]);
State.m_sparsec.m_UIdx.Set(State.m_msparse,State.m_sparsec.m_RIdx[State.m_msparse]);
State.m_sparsec.m_RIdx.Set(State.m_msparse+1,State.m_sparsec.m_RIdx[State.m_msparse]);
State.m_sparsec.m_M++;
State.m_msparse++;
return;
}
//-- Now we are sure that SparseC contains properly initialized sparse
//-- matrix (or some appropriate dummy for M=0) and we have NNZ>0
//-- (no need to care about degenerate cases).
//-- Append rows to SparseC:
//-- * append data
//-- * compute DIdx and UIdx
nzi=0;
for(i=0; i<n; i++)
{
if(!(da[i]==0.0))
{
State.m_sparsec.m_Idx.Set(offs+nzi,i);
State.m_sparsec.m_Vals.Set(offs+nzi,da[i]);
nzi++;
}
}
uidx=-1;
didx=-1;
for(j=offs; j<=offs+nnz-1; j++)
{
k=State.m_sparsec.m_Idx[j];
if(k==State.m_msparse)
didx=j;
else
{
if(k>State.m_msparse && uidx==-1)
{
uidx=j;
break;
}
}
}
if(uidx==-1)
{
uidx=offs+nnz;
}
if(didx==-1)
{
didx=uidx;
}
State.m_sparsec.m_DIdx.Set(State.m_msparse,didx);
State.m_sparsec.m_UIdx.Set(State.m_msparse,uidx);
State.m_sparsec.m_RIdx.Set(State.m_msparse+1,offs+nnz);
State.m_sparsec.m_NInitialized=State.m_sparsec.m_RIdx[State.m_msparse+1];
State.m_sparsec.m_M++;
State.m_msparse++;
}
//+------------------------------------------------------------------+
//| This function solves quadratic programming problem. |
//| You should call it after setting m_solver options with |
//| MinQPSet...() calls. |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| You should use MinQPResults() function to access results after |
//| calls to this function. |
//+------------------------------------------------------------------+
void CMinQP::MinQPOptimize(CMinQPState &State)
{
//--- create variables
int i=0;
int j=0;
int j0=0;
int j1=0;
int k=0;
int nbc=0;
int nlc=0;
int nactive=0;
int nfree=0;
int neq=0;
int nineq=0;
int curecpos=0;
int curicpos=neq;
double f=0;
double fprev=0;
double v=0;
double x=0;
int i_=0;
//--- initialization
int n=State.m_n;
int m=State.m_mdense+State.m_msparse;
State.m_repterminationtype=-5;
State.m_repinneriterationscount=0;
State.m_repouteriterationscount=0;
State.m_repncholesky=0;
State.m_repnmv=0;
//-- Zero-fill Lagrange multipliers (their initial value)
State.m_replagbc=vector<double>::Zeros(n);
State.m_replaglc=vector<double>::Zeros(m);
//-- Initial point:
//-- * if we have starting point in StartX, we just have to bound it
//-- * if we do not have StartX, deduce initial point from boundary constraints
if(State.m_havex)
{
for(i=0; i<n; i++)
{
x=State.m_startx[i];
if(State.m_havebndl[i] && x<State.m_bndl[i])
State.m_xs.Set(i,State.m_bndl[i]);
else
if(State.m_havebndu[i] && x>State.m_bndu[i])
State.m_xs.Set(i,State.m_bndu[i]);
else
State.m_xs.Set(i,x);
}
}
else
{
for(i=0; i<n; i++)
{
if(State.m_havebndl[i] && State.m_havebndu[i])
{
State.m_xs.Set(i,0.5*(State.m_bndl[i]+State.m_bndu[i]));
continue;
}
if(State.m_havebndl[i])
{
State.m_xs.Set(i,State.m_bndl[i]);
continue;
}
if(State.m_havebndu[i])
{
State.m_xs.Set(i,State.m_bndu[i]);
continue;
}
State.m_xs.Set(i,0);
}
}
//--- check correctness of constraints
for(i=0; i<n; i++)
{
//--- check
if(State.m_havebndl[i] && State.m_havebndu[i])
{
//--- check
if(State.m_bndl[i]>State.m_bndu[i])
{
State.m_repterminationtype=-3;
return;
}
}
}
//--- count number of bound and linear constraints
nbc=0;
nlc=0;
for(i=0; i<n; i++)
{
//--- check
if(State.m_havebndl[i])
nbc++;
//--- check
if(State.m_havebndu[i])
nbc++;
}
//-- Effective scale
State.m_effectives=vector<double>::Zeros(n);
switch(State.m_stype)
{
case 0:
//-- User scale (or default one)
State.m_effectives=State.m_s;
break;
case 1:
//-- Diagonal is used for scaling:
//-- * unpack
//-- * convert to scale, return error on failure
if(State.m_akind==0)
{
//-- Unpack CQM structure
CCQModels::CQMGetDiagA(State.m_a,State.m_effectives);
}
else
{
if(State.m_akind==1)
{
for(i=0; i<n; i++)
State.m_effectives.Set(i,CSparse::SparseGet(State.m_sparsea,i,i));
}
else
{
//--- check
if(!CAp::Assert(false,__FUNCTION__+": integrity check failed"))
return;
}
}
//---
if(State.m_effectives.Min()<=0.0)
{
State.m_repterminationtype=-9;
return;
}
State.m_effectives=State.m_effectives.Pow(-0.5)+0;
break;
default:
//--- check
if(!CAp::Assert(false,__FUNCTION__+": integrity check failed"))
return;
}
//-- Solvers which can not handle new two-sided constraints need them to be
//-- converted into legacy equality/inequality one-sided format
if(State.m_algokind==2 || State.m_algokind==4)
{
//-- Scan constraint left/right sides, count equality ones and one/two-sided inequality ones
neq=0;
nineq=0;
for(i=0; i<m; i++)
{
if(MathIsValidNumber(State.m_cl[i]) && MathIsValidNumber(State.m_cu[i]) && State.m_cl[i]==State.m_cu[i])
{
neq++;
continue;
}
if(MathIsValidNumber(State.m_cl[i]))
nineq++;
if(MathIsValidNumber(State.m_cu[i]))
nineq++;
}
//-- Perform conversion
State.m_ecleic=matrix<double>::Zeros(neq+nineq,n+1);
State.m_elagmlt=vector<double>::Zeros(neq+nineq);
State.m_elagidx.Resize(neq+nineq);
State.m_elagidx.Fill(0);
curecpos=0;
curicpos=neq;
for(i=0; i<m ; i++)
{
if(MathIsValidNumber(State.m_cl[i]) && MathIsValidNumber(State.m_cu[i]) && State.m_cl[i]==State.m_cu[i])
{
//-- Offload equality constraint
if(i<State.m_msparse)
{
State.m_ecleic.Row(curecpos,vector<double>::Zeros(n+1));
j0=State.m_sparsec.m_RIdx[i];
j1=State.m_sparsec.m_RIdx[i+1]-1;
for(j=j0; j<=j1; j++)
State.m_ecleic.Set(curecpos,State.m_sparsec.m_Idx[j],State.m_sparsec.m_Vals[j]);
}
else
{
for(j=0; j<n; j++)
State.m_ecleic.Set(curecpos,j,State.m_densec.Get(i-State.m_msparse,j));
}
State.m_ecleic.Set(curecpos,n,State.m_cu[i]);
State.m_elagidx.Set(curecpos,i);
State.m_elagmlt.Set(curecpos,1.0);
curecpos++;
continue;
}
if(MathIsValidNumber(State.m_cl[i]))
{
//-- Offload inequality constraint of the form CL<=C*x, convert it to -C*x<=-CL
if(i<State.m_msparse)
{
State.m_ecleic.Row(curicpos,vector<double>::Zeros(n+1));
j0=State.m_sparsec.m_RIdx[i];
j1=State.m_sparsec.m_RIdx[i+1]-1;
for(j=j0; j<=j1; j++)
State.m_ecleic.Set(curicpos,State.m_sparsec.m_Idx[j],-State.m_sparsec.m_Vals[j]);
}
else
{
for(j=0; j<n; j++)
State.m_ecleic.Set(curicpos,j,-State.m_densec.Get(i-State.m_msparse,j));
}
State.m_ecleic.Set(curicpos,n,-State.m_cl[i]);
State.m_elagidx.Set(curicpos,i);
State.m_elagmlt.Set(curicpos,-1.0);
curicpos++;
}
if(MathIsValidNumber(State.m_cu[i]))
{
//-- Offload inequality constraint of the form C*x<=CU
if(i<State.m_msparse)
{
State.m_ecleic.Row(curicpos,vector<double>::Zeros(n+1));
j0=State.m_sparsec.m_RIdx[i];
j1=State.m_sparsec.m_RIdx[i+1]-1;
for(j=j0; j<=j1; j++)
State.m_ecleic.Set(curicpos,State.m_sparsec.m_Idx[j],State.m_sparsec.m_Vals[j]);
}
else
{
for(j=0; j<n; j++)
State.m_ecleic.Set(curicpos,j,State.m_densec.Get(i-State.m_msparse,j));
}
State.m_ecleic.Set(curicpos,n,State.m_cu[i]);
State.m_elagidx.Set(curicpos,i);
State.m_elagmlt.Set(curicpos,1.0);
curicpos++;
}
}
//--- check
if(!CAp::Assert(curecpos==neq && curicpos==neq+nineq,__FUNCTION__+": critical integrity check failed (ECLEIC conversion)"))
return;
//-- Run solvers
if(State.m_algokind==2)
{
CQPBLEICSolver::QPBLEICOptimize(State.m_a,State.m_sparsea,State.m_akind,State.m_sparseaupper,State.m_absasum,State.m_absasum2,State.m_b,State.m_bndl,State.m_bndu,State.m_effectives,State.m_xorigin,n,State.m_ecleic,neq,nineq,State.m_qpbleicsettingsuser,State.m_qpbleicbuf,State.m_qpbleicfirstcall,State.m_xs,State.m_repterminationtype);
State.m_repinneriterationscount=State.m_qpbleicbuf.m_repinneriterationscount;
State.m_repouteriterationscount=State.m_qpbleicbuf.m_repouteriterationscount;
return;
}
if(State.m_algokind==4)
{
CQPDenseAULSolver::QPDenseAULOptimize(State.m_a,State.m_sparsea,State.m_akind,State.m_sparseaupper,State.m_b,State.m_bndl,State.m_bndu,State.m_effectives,State.m_xorigin,n,State.m_ecleic,neq,nineq,State.m_dummysparse,0,0,!State.m_dbgskipconstraintnormalization,State.m_qpdenseaulsettingsuser,State.m_qpdenseaulbuf,State.m_xs,State.m_replagbc,State.m_elaglc,State.m_repterminationtype);
for(i=0; i<neq+nineq; i++)
State.m_replaglc.Add(State.m_elagidx[i],State.m_elaglc[i]*State.m_elagmlt[i]);
State.m_repinneriterationscount=State.m_qpdenseaulbuf.m_repinneriterationscount;
State.m_repouteriterationscount=State.m_qpdenseaulbuf.m_repouteriterationscount;
State.m_repncholesky=State.m_qpdenseaulbuf.m_repncholesky;
return;
}
//--- check
if(!CAp::Assert(false,__FUNCTION__+": integrity check failed - unknown solver"))
return;
}
//-- QuickQP solver
if(State.m_algokind==3)
{
if(State.m_mdense+State.m_msparse>0)
{
State.m_repterminationtype=-5;
return;
}
CQQPSolver::QQPOptimize(State.m_a,State.m_sparsea,State.m_dummyr2,State.m_akind,State.m_sparseaupper,State.m_b,State.m_bndl,State.m_bndu,State.m_effectives,State.m_xorigin,n,State.m_qqpsettingsuser,State.m_qqpbuf,State.m_xs,State.m_repterminationtype);
State.m_repinneriterationscount=State.m_qqpbuf.m_repinneriterationscount;
State.m_repouteriterationscount=State.m_qqpbuf.m_repouteriterationscount;
State.m_repncholesky=State.m_qqpbuf.m_repncholesky;
return;
}
//-- QP-DENSE-IPM and QP-SPARSE-IPM solvers
if(State.m_algokind==5 || State.m_algokind==6)
{
//-- Prepare working versions of constraints; these versions may be modified
//-- when we detect that some bounds are irrelevant.
State.m_wrkbndl=State.m_bndl;
State.m_wrkbndu=State.m_bndu;
if(State.m_msparse>0)
CSparse::SparseCopyBuf(State.m_sparsec,State.m_wrksparsec);
if(State.m_mdense>0)
CAblasF::RCopyAllocM(State.m_mdense,n,State.m_densec,State.m_wrkdensec);
CAblasF::RCopyAllocV(m,State.m_cl,State.m_wrkcl);
CAblasF::RCopyAllocV(m,State.m_cu,State.m_wrkcu);
//-- Solve
//--- check
if(!CAp::Assert(State.m_akind==0 || State.m_akind==1,__FUNCTION__+": unexpected AKind"))
return;
if(State.m_akind==0)
CCQModels::CQMGetA(State.m_a,State.m_tmpr2);
if(State.m_algokind==5)
CVIPMSolver::VIPMInitDense(State.m_vsolver,State.m_effectives,State.m_xorigin,n);
if(State.m_algokind==6)
CVIPMSolver::VIPMInitSparse(State.m_vsolver,State.m_effectives,State.m_xorigin,n);
CVIPMSolver::VIPMSetQuadraticLinear(State.m_vsolver,State.m_tmpr2,State.m_sparsea,State.m_akind,State.m_sparseaupper,State.m_b);
CVIPMSolver::VIPMSetConstraints(State.m_vsolver,State.m_wrkbndl,State.m_wrkbndu,State.m_wrksparsec,State.m_msparse,State.m_wrkdensec,State.m_mdense,State.m_wrkcl,State.m_wrkcu);
CVIPMSolver::VIPMSetCond(State.m_vsolver,State.m_veps,State.m_veps,State.m_veps);
CVIPMSolver::VIPMOptimize(State.m_vsolver,true,State.m_xs,State.m_replagbc,State.m_replaglc,State.m_repterminationtype);
State.m_repinneriterationscount=State.m_vsolver.m_repiterationscount;
State.m_repouteriterationscount=State.m_repinneriterationscount;
State.m_repncholesky=State.m_vsolver.m_repncholesky;
return;
}
//-- Integrity check failed - unknown solver
if(!CAp::Assert(false,__FUNCTION__+": integrity check failed - unknown solver"))
return;
}
//+------------------------------------------------------------------+
//| QP solver results |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| X - array[0..m_n-1], solution |
//| Rep - optimization report. You should check Rep. |
//| TerminationType, which contains completion code, |
//| and you may check another fields which contain |
//| another information about algorithm functioning. |
//+------------------------------------------------------------------+
void CMinQP::MinQPResults(CMinQPState &State,double &x[],CMinQPReport &rep)
{
//--- reset memory
ArrayResize(x,0);
//--- function call
MinQPResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinQP::MinQPResults(CMinQPState &State,CRowDouble &x,CMinQPReport &rep)
{
//--- reset memory
x.Resize(0);
//--- function call
MinQPResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| QP results |
//| Buffered implementation of MinQPResults() which uses |
//| pre-allocated buffer to store X[]. If buffer size is too small, |
//| it resizes buffer. It is intended to be used in the inner cycles |
//| of performance critical algorithms where array reallocation |
//| penalty is too large to be ignored. |
//+------------------------------------------------------------------+
void CMinQP::MinQPResultsBuf(CMinQPState &State,double &x[],
CMinQPReport &rep)
{
//--- check
if(!CAp::Assert(State.m_xs.Size()>=State.m_n,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(State.m_replagbc.Size()>=State.m_n,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(State.m_replaglc.Size()>=State.m_mdense+State.m_msparse,__FUNCTION__+": integrity check failed"))
return;
//--- copy
State.m_xs.ToArray(x);
rep.m_lagbc=State.m_replagbc;
rep.m_laglc=State.m_replaglc;
ArrayResize(x,State.m_n);
rep.m_lagbc.Resize(State.m_n);
rep.m_laglc.Resize(State.m_mdense+State.m_msparse);
//--- change values
rep.m_inneriterationscount=State.m_repinneriterationscount;
rep.m_outeriterationscount=State.m_repouteriterationscount;
rep.m_nmv=State.m_repnmv;
rep.m_terminationtype=State.m_repterminationtype;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinQP::MinQPResultsBuf(CMinQPState &State,CRowDouble &x,
CMinQPReport &rep)
{
//--- check
if(!CAp::Assert(State.m_xs.Size()>=State.m_n,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(State.m_replagbc.Size()>=State.m_n,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(State.m_replaglc.Size()>=State.m_mdense+State.m_msparse,__FUNCTION__+": integrity check failed"))
return;
//--- copy
x=State.m_xs;
rep.m_lagbc=State.m_replagbc;
rep.m_laglc=State.m_replaglc;
x.Resize(State.m_n);
rep.m_lagbc.Resize(State.m_n);
rep.m_laglc.Resize(State.m_mdense+State.m_msparse);
//--- change values
rep.m_inneriterationscount=State.m_repinneriterationscount;
rep.m_outeriterationscount=State.m_repouteriterationscount;
rep.m_nmv=State.m_repnmv;
rep.m_terminationtype=State.m_repterminationtype;
}
//+------------------------------------------------------------------+
//| Fast version of MinQPSetLinearTerm(), which doesn't check its |
//| arguments. For internal use only. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetLinearTermFast(CMinQPState &State,double &b[])
{
//--- initialization
State.m_b=b;
State.m_b.Resize(State.m_n);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetLinearTermFast(CMinQPState &State,CRowDouble &b)
{
//--- initialization
State.m_b=b;
State.m_b.Resize(State.m_n);
}
//+------------------------------------------------------------------+
//| Fast version of MinQPSetQuadraticTerm(), which doesn't check its |
//| arguments. |
//| It accepts additional parameter - shift S, which allows to |
//| "shift" matrix A by adding s*I to A. S must be positive (although|
//| it is not checked). |
//| For internal use only. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetQuadraticTermFast(CMinQPState &State,CMatrixDouble &a,
const bool IsUpper,const double s)
{
//--- create variables
int n=State.m_n;
int i=0;
int j=0;
double v=0;
int j0=0;
int j1=0;
State.m_akind=0;
//--- function call
CCQModels::CQMSetA(State.m_a,a,IsUpper,1.0);
if(s>0.0)
{
State.m_tmp0=a.Diag()+s;
State.m_tmp0.Resize(n);
CCQModels::CQMRewriteDenseDiagonal(State.m_a,State.m_tmp0);
}
//-- Estimate norm of A
//-- (it will be used later in the quadratic penalty function)
State.m_absamax=0;
State.m_absasum=0;
State.m_absasum2=0;
for(i=0; i<n; i++)
{
if(IsUpper)
{
j0=i;
j1=n-1;
}
else
{
j0=0;
j1=i;
}
for(j=j0; j<=j1; j++)
{
v=MathAbs(a.Get(i,j));
State.m_absamax=MathMax(State.m_absamax,v);
State.m_absasum=State.m_absasum+v;
State.m_absasum2=State.m_absasum2+v*v;
}
}
}
//+------------------------------------------------------------------+
//| Interna lfunction which allows to rewrite diagonal of quadratic |
//| term. For internal use only. |
//| This function can be used only when you have dense A and already |
//| made MinQPSetQuadraticTerm(Fast) call. |
//+------------------------------------------------------------------+
void CMinQP::MinQPRewriteDiagonal(CMinQPState &State,double &s[])
{
CRowDouble S=s;
MinQPRewriteDiagonal(State,S);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinQP::MinQPRewriteDiagonal(CMinQPState &State,CRowDouble &s)
{
CCQModels::CQMRewriteDenseDiagonal(State.m_a,s);
}
//+------------------------------------------------------------------+
//| Fast version of MinQPSetStartingPoint(), which doesn't check its |
//| arguments. For internal use only. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetStartingPointFast(CMinQPState &State,double &x[])
{
//--- initialization
State.m_startx=x;
State.m_startx.Resize(State.m_n);
State.m_havex=true;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetStartingPointFast(CMinQPState &State,CRowDouble &x)
{
//--- initialization
State.m_startx=x;
State.m_startx.Resize(State.m_n);
State.m_havex=true;
}
//+------------------------------------------------------------------+
//| Fast version of MinQPSetOrigin(), which doesn't check its |
//| arguments. For internal use only. |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetOriginFast(CMinQPState &State,double &xorigin[])
{
//--- initialization
State.m_xorigin=xorigin;
State.m_xorigin.Resize(State.m_n);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinQP::MinQPSetOriginFast(CMinQPState &State,CRowDouble &xorigin)
{
//--- initialization
State.m_xorigin=xorigin;
State.m_xorigin.Resize(State.m_n);
}
//+------------------------------------------------------------------+
//| Step finder for Levenberg-Marquardt optimizer. |
//| Internal object used by MinLM unit. |
//| This structure should be initialized with MinLMStepFinderInit(). |
//| Each step search session should start with MinLMStepFinderStart()|
//| call, followed by a sequence of MinLMStepFinderIteration() calls.|
//+------------------------------------------------------------------+
struct CMinLMStepFinder
{
int m_m;
int m_maxmodelage;
int m_modelage;
int m_n;
double m_actualdecrease;
double m_epsx;
double m_f;
double m_fbase;
double m_predicteddecrease;
double m_stpmax;
bool m_hasfi;
bool m_havebndl[];
bool m_havebndu[];
bool m_needf;
bool m_needfi;
RCommState m_rstate;
CSparseMatrix m_tmpsp;
CRowInt m_tmpct;
CRowDouble m_bndl;
CRowDouble m_bndu;
CRowDouble m_choleskybuf;
CRowDouble m_fi;
CRowDouble m_fibase;
CRowDouble m_modeldiag;
CRowDouble m_s;
CRowDouble m_tmp0;
CRowDouble m_x;
CRowDouble m_xbase;
CRowDouble m_xdir;
CMinQPState m_qpstate;
CMinQPReport m_qprep;
//--- constructor / destructor
CMinLMStepFinder(void);
~CMinLMStepFinder(void) {}
//---
void Copy(const CMinLMStepFinder &obj);
//--- overloading
void operator=(const CMinLMStepFinder &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinLMStepFinder::CMinLMStepFinder(void)
{
m_m=0;
m_maxmodelage=0;
m_modelage=0;
m_n=0;
m_actualdecrease=0;
m_epsx=0;
m_f=0;
m_fbase=0;
m_predicteddecrease=0;
m_stpmax=0;
m_hasfi=false;
m_needf=false;
m_needfi=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinLMStepFinder::Copy(const CMinLMStepFinder &obj)
{
m_m=obj.m_m;
m_maxmodelage=obj.m_maxmodelage;
m_modelage=obj.m_modelage;
m_n=obj.m_n;
m_actualdecrease=obj.m_actualdecrease;
m_epsx=obj.m_epsx;
m_f=obj.m_f;
m_fbase=obj.m_fbase;
m_predicteddecrease=obj.m_predicteddecrease;
m_stpmax=obj.m_stpmax;
m_hasfi=obj.m_hasfi;
ArrayCopy(m_havebndl,obj.m_havebndl);
ArrayCopy(m_havebndu,obj.m_havebndu);
m_needf=obj.m_needf;
m_needfi=obj.m_needfi;
m_rstate=obj.m_rstate;
m_tmpsp=obj.m_tmpsp;
m_tmpct=obj.m_tmpct;
m_bndl=obj.m_bndl;
m_bndu=obj.m_bndu;
m_choleskybuf=obj.m_choleskybuf;
m_fi=obj.m_fi;
m_fibase=obj.m_fibase;
m_modeldiag=obj.m_modeldiag;
m_s=obj.m_s;
m_tmp0=obj.m_tmp0;
m_x=obj.m_x;
m_xbase=obj.m_xbase;
m_xdir=obj.m_xdir;
m_qpstate=obj.m_qpstate;
m_qprep=obj.m_qprep;
}
//+------------------------------------------------------------------+
//| Levenberg-Marquardt optimizer. |
//| This structure should be created using one of the |
//| MinLMCreate() functions. You should not access its fields |
//| directly; use ALGLIB functions to work with it. |
//+------------------------------------------------------------------+
class CMinLMState
{
public:
//--- variables
int m_algomode;
int m_m;
int m_maxits;
int m_maxmodelage;
int m_modelage;
int m_n;
int m_nec;
int m_nic;
int m_repiterationscount;
int m_repncholesky;
int m_repnfunc;
int m_repngrad;
int m_repnhess;
int m_repnjac;
int m_repterminationtype;
double m_actualdecrease;
double m_diffstep;
double m_epsx;
double m_f;
double m_fbase;
double m_lambdav;
double m_nu;
double m_predicteddecrease;
double m_stpmax;
double m_teststep;
double m_xm1;
double m_xp1;
bool m_deltafready;
bool m_deltaxready;
bool m_hasf;
bool m_hasfi;
bool m_hasg;
bool m_makeadditers;
bool m_needf;
bool m_needfg;
bool m_needfgh;
bool m_needfi;
bool m_needfij;
bool m_userterminationneeded;
bool m_xrep;
bool m_xupdated;
//--- objects
RCommState m_rstate;
CSmoothnessMonitor m_smonitor;
CMinQPState m_qpstate;
CMinQPReport m_qprep;
CMinLMStepFinder m_finderstate;
CMinLBFGSState m_internalstate;
CMinLBFGSReport m_internalrep;
//--- arrays
bool m_havebndl[];
bool m_havebndu[];
CRowDouble m_bndl;
CRowDouble m_bndu;
CRowDouble m_choleskybuf;
CRowDouble m_deltaf;
CRowDouble m_deltax;
CRowDouble m_fc1;
CRowDouble m_fi;
CRowDouble m_fibase;
CRowDouble m_fm1;
CRowDouble m_fp1;
CRowDouble m_g;
CRowDouble m_gbase;
CRowDouble m_gc1;
CRowDouble m_gm1;
CRowDouble m_gp1;
CRowDouble m_lastscaleused;
CRowDouble m_s;
CRowDouble m_tmp0;
CRowDouble m_x;
CRowDouble m_xbase;
CRowDouble m_xdir;
CRowDouble m_xnew;
//--- matrix
CMatrixDouble m_cleic;
CMatrixDouble m_h;
CMatrixDouble m_j;
CMatrixDouble m_quadraticmodel;
//--- constructor, destructor
CMinLMState(void);
~CMinLMState(void) {}
//--- copy
void Copy(const CMinLMState &obj);
//--- overloading
void operator=(const CMinLMState &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinLMState::CMinLMState(void)
{
m_algomode=0;
m_m=0;
m_maxits=0;
m_maxmodelage=0;
m_modelage=0;
m_n=0;
m_nec=0;
m_nic=0;
m_repiterationscount=0;
m_repncholesky=0;
m_repnfunc=0;
m_repngrad=0;
m_repnhess=0;
m_repnjac=0;
m_repterminationtype=0;
m_actualdecrease=0;
m_diffstep=0;
m_epsx=0;
m_f=0;
m_fbase=0;
m_lambdav=0;
m_nu=0;
m_predicteddecrease=0;
m_stpmax=0;
m_teststep=0;
m_xm1=0;
m_xp1=0;
m_deltafready=false;
m_deltaxready=false;
m_hasf=false;
m_hasfi=false;
m_hasg=false;
m_makeadditers=false;
m_needf=false;
m_needfg=false;
m_needfgh=false;
m_needfi=false;
m_needfij=false;
m_userterminationneeded=false;
m_xrep=false;
m_xupdated=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinLMState::Copy(const CMinLMState &obj)
{
m_algomode=obj.m_algomode;
m_m=obj.m_m;
m_maxits=obj.m_maxits;
m_maxmodelage=obj.m_maxmodelage;
m_modelage=obj.m_modelage;
m_n=obj.m_n;
m_nec=obj.m_nec;
m_nic=obj.m_nic;
m_repiterationscount=obj.m_repiterationscount;
m_repncholesky=obj.m_repncholesky;
m_repnfunc=obj.m_repnfunc;
m_repngrad=obj.m_repngrad;
m_repnhess=obj.m_repnhess;
m_repnjac=obj.m_repnjac;
m_repterminationtype=obj.m_repterminationtype;
m_actualdecrease=obj.m_actualdecrease;
m_diffstep=obj.m_diffstep;
m_epsx=obj.m_epsx;
m_f=obj.m_f;
m_fbase=obj.m_fbase;
m_lambdav=obj.m_lambdav;
m_nu=obj.m_nu;
m_predicteddecrease=obj.m_predicteddecrease;
m_stpmax=obj.m_stpmax;
m_teststep=obj.m_teststep;
m_xm1=obj.m_xm1;
m_xp1=obj.m_xp1;
m_deltafready=obj.m_deltafready;
m_deltaxready=obj.m_deltaxready;
m_hasf=obj.m_hasf;
m_hasfi=obj.m_hasfi;
m_hasg=obj.m_hasg;
ArrayCopy(m_havebndl,obj.m_havebndl);
ArrayCopy(m_havebndu,obj.m_havebndu);
m_makeadditers=obj.m_makeadditers;
m_needf=obj.m_needf;
m_needfg=obj.m_needfg;
m_needfgh=obj.m_needfgh;
m_needfi=obj.m_needfi;
m_needfij=obj.m_needfij;
m_userterminationneeded=obj.m_userterminationneeded;
m_xrep=obj.m_xrep;
m_xupdated=obj.m_xupdated;
m_rstate=obj.m_rstate;
m_smonitor=obj.m_smonitor;
m_bndl=obj.m_bndl;
m_bndu=obj.m_bndu;
m_choleskybuf=obj.m_choleskybuf;
m_deltaf=obj.m_deltaf;
m_deltax=obj.m_deltax;
m_fc1=obj.m_fc1;
m_fi=obj.m_fi;
m_fibase=obj.m_fibase;
m_fm1=obj.m_fm1;
m_fp1=obj.m_fp1;
m_g=obj.m_g;
m_gbase=obj.m_gbase;
m_gc1=obj.m_gc1;
m_gm1=obj.m_gm1;
m_gp1=obj.m_gp1;
m_lastscaleused=obj.m_lastscaleused;
m_s=obj.m_s;
m_tmp0=obj.m_tmp0;
m_x=obj.m_x;
m_xbase=obj.m_xbase;
m_xdir=obj.m_xdir;
m_xnew=obj.m_xnew;
m_qpstate=obj.m_qpstate;
m_qprep=obj.m_qprep;
m_finderstate=obj.m_finderstate;
m_internalstate=obj.m_internalstate;
m_internalrep=obj.m_internalrep;
m_cleic=obj.m_cleic;
m_h=obj.m_h;
m_j=obj.m_j;
m_quadraticmodel=obj.m_quadraticmodel;
}
//+------------------------------------------------------------------+
//| Levenberg-Marquardt optimizer. |
//| This structure should be created using one of the |
//| MinLMCreate() functions. You should not access its fields |
//| directly; use ALGLIB functions to work with it. |
//+------------------------------------------------------------------+
class CMinLMStateShell
{
private:
CMinLMState m_innerobj;
public:
//--- constructors, destructor
CMinLMStateShell(void) {}
CMinLMStateShell(CMinLMState &obj) { m_innerobj.Copy(obj); }
~CMinLMStateShell(void) {}
//--- methods
bool GetNeedF(void);
void SetNeedF(const bool b);
bool GetNeedFG(void);
void SetNeedFG(const bool b);
bool GetNeedFGH(void);
void SetNeedFGH(const bool b);
bool GetNeedFI(void);
void SetNeedFI(const bool b);
bool GetNeedFIJ(void);
void SetNeedFIJ(const bool b);
bool GetXUpdated(void);
void SetXUpdated(const bool b);
double GetF(void);
void SetF(const double d);
CMinLMState *GetInnerObj(void);
};
//+------------------------------------------------------------------+
//| Returns the value of the variable needf |
//+------------------------------------------------------------------+
bool CMinLMStateShell::GetNeedF(void)
{
return(m_innerobj.m_needf);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable needf |
//+------------------------------------------------------------------+
void CMinLMStateShell::SetNeedF(const bool b)
{
m_innerobj.m_needf=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable needfg |
//+------------------------------------------------------------------+
bool CMinLMStateShell::GetNeedFG(void)
{
return(m_innerobj.m_needfg);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable needfg |
//+------------------------------------------------------------------+
void CMinLMStateShell::SetNeedFG(const bool b)
{
m_innerobj.m_needfg=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable needfgh |
//+------------------------------------------------------------------+
bool CMinLMStateShell::GetNeedFGH(void)
{
return(m_innerobj.m_needfgh);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable needfgh |
//+------------------------------------------------------------------+
void CMinLMStateShell::SetNeedFGH(const bool b)
{
m_innerobj.m_needfgh=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable needfi |
//+------------------------------------------------------------------+
bool CMinLMStateShell::GetNeedFI(void)
{
return(m_innerobj.m_needfi);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable needfi |
//+------------------------------------------------------------------+
void CMinLMStateShell::SetNeedFI(const bool b)
{
m_innerobj.m_needfi=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable needfij |
//+------------------------------------------------------------------+
bool CMinLMStateShell::GetNeedFIJ(void)
{
return(m_innerobj.m_needfij);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable needfij |
//+------------------------------------------------------------------+
void CMinLMStateShell::SetNeedFIJ(const bool b)
{
m_innerobj.m_needfij=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable xupdated |
//+------------------------------------------------------------------+
bool CMinLMStateShell::GetXUpdated(void)
{
return(m_innerobj.m_xupdated);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable xupdated |
//+------------------------------------------------------------------+
void CMinLMStateShell::SetXUpdated(const bool b)
{
m_innerobj.m_xupdated=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable f |
//+------------------------------------------------------------------+
double CMinLMStateShell::GetF(void)
{
return(m_innerobj.m_f);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable f |
//+------------------------------------------------------------------+
void CMinLMStateShell::SetF(const double d)
{
m_innerobj.m_f=d;
}
//+------------------------------------------------------------------+
//| Return object of class |
//+------------------------------------------------------------------+
CMinLMState* CMinLMStateShell::GetInnerObj(void)
{
return(GetPointer(m_innerobj));
}
//+------------------------------------------------------------------+
//| Optimization report, filled by MinLMResults() function |
//| FIELDS: |
//| * TerminationType, completetion code: |
//| * -9 derivative correctness check failed; |
//| see Rep.WrongNum, Rep.WrongI, Rep.WrongJ for |
//| more information. |
//| * 1 relative function improvement is no more than |
//| EpsF. |
//| * 2 relative step is no more than EpsX. |
//| * 4 gradient is no more than EpsG. |
//| * 5 MaxIts steps was taken |
//| * 7 stopping conditions are too stringent, |
//| further improvement is impossible |
//| * IterationsCount, contains iterations count |
//| * NFunc, number of function calculations |
//| * NJac, number of Jacobi matrix calculations |
//| * NGrad, number of gradient calculations |
//| * NHess, number of Hessian calculations |
//| * NCholesky, number of Cholesky decomposition calculations |
//+------------------------------------------------------------------+
class CMinLMReport
{
public:
//--- variables
int m_iterationscount;
int m_terminationtype;
int m_nfunc;
int m_njac;
int m_ngrad;
int m_nhess;
int m_ncholesky;
//--- constructor, destructor
CMinLMReport(void) { ZeroMemory(this); }
~CMinLMReport(void) {}
//--- copy
void Copy(const CMinLMReport &obj);
//--- overloading
void operator=(const CMinLMReport &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinLMReport::Copy(const CMinLMReport &obj)
{
//--- copy variables
m_iterationscount=obj.m_iterationscount;
m_terminationtype=obj.m_terminationtype;
m_nfunc=obj.m_nfunc;
m_njac=obj.m_njac;
m_ngrad=obj.m_ngrad;
m_nhess=obj.m_nhess;
m_ncholesky=obj.m_ncholesky;
}
//+------------------------------------------------------------------+
//| Optimization report, filled by MinLMResults() function |
//| FIELDS: |
//| * TerminationType, completetion code: |
//| * -9 derivative correctness check failed; |
//| see Rep.WrongNum, Rep.WrongI, Rep.WrongJ for |
//| more information. |
//| * 1 relative function improvement is no more than |
//| EpsF. |
//| * 2 relative step is no more than EpsX. |
//| * 4 gradient is no more than EpsG. |
//| * 5 MaxIts steps was taken |
//| * 7 stopping conditions are too stringent, |
//| further improvement is impossible |
//| * IterationsCount, contains iterations count |
//| * NFunc, number of function calculations |
//| * NJac, number of Jacobi matrix calculations |
//| * NGrad, number of gradient calculations |
//| * NHess, number of Hessian calculations |
//| * NCholesky, number of Cholesky decomposition calculations |
//+------------------------------------------------------------------+
class CMinLMReportShell
{
private:
CMinLMReport m_innerobj;
public:
//--- constructors, destructor
CMinLMReportShell(void) {}
CMinLMReportShell(CMinLMReport &obj) { m_innerobj.Copy(obj); }
~CMinLMReportShell(void) {}
//--- methods
int GetIterationsCount(void);
void SetIterationsCount(const int i);
int GetTerminationType(void);
void SetTerminationType(const int i);
int GetNFunc(void);
void SetNFunc(const int i);
int GetNJAC(void);
void SetNJAC(const int i);
int GetNGrad(void);
void SetNGrad(const int i);
int GetNHess(void);
void SetNHess(const int i);
int GetNCholesky(void);
void SetNCholesky(const int i);
CMinLMReport *GetInnerObj(void);
};
//+------------------------------------------------------------------+
//| Returns the value of the variable iterationscount |
//+------------------------------------------------------------------+
int CMinLMReportShell::GetIterationsCount(void)
{
return(m_innerobj.m_iterationscount);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable iterationscount |
//+------------------------------------------------------------------+
void CMinLMReportShell::SetIterationsCount(const int i)
{
m_innerobj.m_iterationscount=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable terminationtype |
//+------------------------------------------------------------------+
int CMinLMReportShell::GetTerminationType(void)
{
return(m_innerobj.m_terminationtype);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable terminationtype |
//+------------------------------------------------------------------+
void CMinLMReportShell::SetTerminationType(const int i)
{
m_innerobj.m_terminationtype=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable nfunc |
//+------------------------------------------------------------------+
int CMinLMReportShell::GetNFunc(void)
{
return(m_innerobj.m_nfunc);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable nfunc |
//+------------------------------------------------------------------+
void CMinLMReportShell::SetNFunc(const int i)
{
m_innerobj.m_nfunc=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable njac |
//+------------------------------------------------------------------+
int CMinLMReportShell::GetNJAC(void)
{
return(m_innerobj.m_njac);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable njac |
//+------------------------------------------------------------------+
void CMinLMReportShell::SetNJAC(const int i)
{
m_innerobj.m_njac=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable ngrad |
//+------------------------------------------------------------------+
int CMinLMReportShell::GetNGrad(void)
{
return(m_innerobj.m_ngrad);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable ngrad |
//+------------------------------------------------------------------+
void CMinLMReportShell::SetNGrad(const int i)
{
m_innerobj.m_ngrad=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable nhess |
//+------------------------------------------------------------------+
int CMinLMReportShell::GetNHess(void)
{
return(m_innerobj.m_nhess);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable nhess |
//+------------------------------------------------------------------+
void CMinLMReportShell::SetNHess(const int i)
{
m_innerobj.m_nhess=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable ncholesky |
//+------------------------------------------------------------------+
int CMinLMReportShell::GetNCholesky(void)
{
return(m_innerobj.m_ncholesky);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable ncholesky |
//+------------------------------------------------------------------+
void CMinLMReportShell::SetNCholesky(const int i)
{
m_innerobj.m_ncholesky=i;
}
//+------------------------------------------------------------------+
//| Return object of class |
//+------------------------------------------------------------------+
CMinLMReport *CMinLMReportShell::GetInnerObj(void)
{
return(GetPointer(m_innerobj));
}
//+------------------------------------------------------------------+
//| Levenberg-Marquardt method |
//+------------------------------------------------------------------+
class CMinLM
{
public:
//--- class constants
static const double m_lambdaup;
static const double m_lambdadown;
static const double m_suspiciousnu;
static const int m_smallmodelage;
static const int m_additers;
//--- public methods
static void MinLMCreateVJ(const int n,const int m,double &x[],CMinLMState &State);
static void MinLMCreateVJ(const int n,const int m,CRowDouble &x,CMinLMState &State);
static void MinLMCreateV(const int n,const int m,double &x[],const double diffstep,CMinLMState &State);
static void MinLMCreateV(const int n,const int m,CRowDouble &x,const double diffstep,CMinLMState &State);
static void MinLMCreateFGH(const int n,double &x[],CMinLMState &State);
static void MinLMCreateFGH(const int n,CRowDouble &x,CMinLMState &State);
static void MinLMSetCond(CMinLMState &State,double epsx,const int m_maxits);
static void MinLMSetXRep(CMinLMState &State,const bool needxrep);
static void MinLMSetStpMax(CMinLMState &State,const double stpmax);
static void MinLMSetScale(CMinLMState &State,double &s[]);
static void MinLMSetScale(CMinLMState &State,CRowDouble &s);
static void MinLMSetBC(CMinLMState &State,double &bndl[],double &bndu[]);
static void MinLMSetBC(CMinLMState &State,CRowDouble &bndl,CRowDouble &bndu);
static void MinLMSetLC(CMinLMState &State,CMatrixDouble &c,CRowInt &ct,int k);
static void MinLMSetAccType(CMinLMState &State,int acctype);
static void MinLMOptGuardGradient(CMinLMState &State,double teststep);
static void MinLMOptGuardResults(CMinLMState &State,COptGuardReport &rep);
static void MinLMResults(CMinLMState &State,double &x[],CMinLMReport &rep);
static void MinLMResults(CMinLMState &State,CRowDouble &x,CMinLMReport &rep);
static void MinLMResultsBuf(CMinLMState &State,double &x[],CMinLMReport &rep);
static void MinLMResultsBuf(CMinLMState &State,CRowDouble &x,CMinLMReport &rep);
static void MinLMRestartFrom(CMinLMState &State,double &x[]);
static void MinLMRestartFrom(CMinLMState &State,CRowDouble &x);
static void MinLMRequestTermination(CMinLMState &State);
static void MinLMCreateVGJ(const int n,const int m,double &x[],CMinLMState &State);
static void MinLMCreateVGJ(const int n,const int m,CRowDouble &x,CMinLMState &State);
static void MinLMCreateFGJ(const int n,const int m,double &x[],CMinLMState &State);
static void MinLMCreateFGJ(const int n,const int m,CRowDouble &x,CMinLMState &State);
static void MinLMCreateFJ(const int n,const int m,double &x[],CMinLMState &State);
static void MinLMCreateFJ(const int n,const int m,CRowDouble &x,CMinLMState &State);
static bool MinLMIteration(CMinLMState &State);
private:
static void LMPRepare(const int n,const int m,bool havegrad,CMinLMState &State);
static void ClearRequestFields(CMinLMState &State);
static bool IncreaseLambda(double &lambdav,double &nu);
static void DecreaseLambda(double &lambdav,double &nu);
static int CheckDecrease(CMatrixDouble &quadraticmodel,CRowDouble &gbase,double fbase,int n,CRowDouble &deltax,double fnew,double &lambdav,double &nu);
static bool MinLMStepFinderInit(CMinLMStepFinder &State,int n,int m,int maxmodelage,bool hasfi,CRowDouble &xbase,CRowDouble &bndl,CRowDouble &bndu,CMatrixDouble &cleic,int nec,int nic,CRowDouble &s,double stpmax,double epsx);
static void MinLMStepFinderStart(CMinLMStepFinder &State,CMatrixDouble &quadraticmodel,CRowDouble &gbase,double fbase,CRowDouble &xbase,CRowDouble &fibase,int modelage);
static bool MinLMStepFinderIteration(CMinLMStepFinder &State,double &lambdav,double &nu,CRowDouble &xnew,CRowDouble &deltax,bool &deltaxready,CRowDouble &deltaf,bool &deltafready,int &iflag,double &fnew,int &ncholesky);
};
//+------------------------------------------------------------------+
//| Initialize constants |
//+------------------------------------------------------------------+
const double CMinLM::m_lambdaup=2.0;
const double CMinLM::m_lambdadown=0.33;
const double CMinLM::m_suspiciousnu=16;
const int CMinLM::m_smallmodelage=3;
const int CMinLM::m_additers=5;
//+------------------------------------------------------------------+
//| IMPROVED LEVENBERG-MARQUARDT METHOD FOR |
//| NON-LINEAR LEAST SQUARES OPTIMIZATION |
//| DESCRIPTION: |
//| This function is used to find minimum of function which is |
//| represented as sum of squares: |
//| F(x) = f[0]^2(x[0],...,x[n-1]) + ... + |
//| + f[m-1]^2(x[0],...,x[n-1]) |
//| using value of function vector f[] and Jacobian of f[]. |
//| REQUIREMENTS: |
//| This algorithm will request following information during its |
//| operation: |
//| * function vector f[] at given point X |
//| * function vector f[] and Jacobian of f[] (simultaneously) at |
//| given point |
//| There are several overloaded versions of MinLMOptimize() |
//| function which correspond to different LM-like optimization |
//| algorithms provided by this unit. You should choose version which|
//| accepts fvec() and jac() callbacks. First one is used to |
//| calculate f[] at given point, second one calculates f[] and |
//| Jacobian df[i]/dx[j]. |
//| You can try to initialize MinLMState structure with VJ function |
//| and then use incorrect version of MinLMOptimize() (for example,|
//| version which works with general form function and does not |
//| provide Jacobian), but it will lead to exception being thrown |
//| after first attempt to calculate Jacobian. |
//| USAGE: |
//| 1. User initializes algorithm State with MinLMCreateVJ() call |
//| 2. User tunes m_solver parameters with MinLMSetCond(), |
//| MinLMSetStpMax() and other functions |
//| 3. User calls MinLMOptimize() function which takes algorithm |
//| State and callback functions. |
//| 4. User calls MinLMResults() to get solution |
//| 5. Optionally, user may call MinLMRestartFrom() to solve another |
//| problem with same N/M but another starting point and/or |
//| another function. MinLMRestartFrom() allows to reuse already |
//| initialized structure. |
//| INPUT PARAMETERS: |
//| N - dimension, N>1 |
//| * if given, only leading N elements of X are |
//| used |
//| * if not given, automatically determined from |
//| size of X |
//| M - number of functions f[i] |
//| X - initial solution, array[0..m_n-1] |
//| OUTPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NOTES: |
//| 1. you may tune stopping conditions with MinLMSetCond() function |
//| 2. if target function contains exp() or other fast growing |
//| functions, and optimization algorithm makes too large steps |
//| which leads to overflow, use MinLMSetStpMax() function to |
//| bound algorithm's steps. |
//+------------------------------------------------------------------+
void CMinLM::MinLMCreateVJ(const int n,const int m,double &x[],
CMinLMState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1!"))
return;
//--- check
if(!CAp::Assert(m>=1,__FUNCTION__+": M<1!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- initialize,check parameters
State.m_teststep=0;
State.m_n=n;
State.m_m=m;
State.m_algomode=1;
State.m_hasf=false;
State.m_hasfi=true;
State.m_hasg=false;
//--- second stage of initialization
LMPRepare(n,m,false,State);
MinLMSetAccType(State,0);
MinLMSetCond(State,0,0);
MinLMSetXRep(State,false);
MinLMSetStpMax(State,0);
MinLMRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLM::MinLMCreateVJ(const int n,const int m,CRowDouble &x,
CMinLMState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1!"))
return;
//--- check
if(!CAp::Assert(m>=1,__FUNCTION__+": M<1!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- initialize,check parameters
State.m_teststep=0;
State.m_n=n;
State.m_m=m;
State.m_algomode=1;
State.m_hasf=false;
State.m_hasfi=true;
State.m_hasg=false;
//--- second stage of initialization
LMPRepare(n,m,false,State);
MinLMSetAccType(State,0);
MinLMSetCond(State,0,0);
MinLMSetXRep(State,false);
MinLMSetStpMax(State,0);
MinLMRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| IMPROVED LEVENBERG-MARQUARDT METHOD FOR |
//| NON-LINEAR LEAST SQUARES OPTIMIZATION |
//| DESCRIPTION: |
//| This function is used to find minimum of function which is |
//| represented as sum of squares: |
//| F(x) = f[0]^2(x[0],...,x[n-1]) + ... + |
//| + f[m-1]^2(x[0],...,x[n-1]) |
//| using value of function vector f[] only. Finite differences are |
//| used to calculate Jacobian. |
//| REQUIREMENTS: |
//| This algorithm will request following information during its |
//| operation: |
//| * function vector f[] at given point X |
//| There are several overloaded versions of MinLMOptimize() function|
//| which correspond to different LM-like optimization algorithms |
//| provided by this unit. You should choose version which accepts |
//| fvec() callback. |
//| You can try to initialize MinLMState structure with VJ function |
//| and then use incorrect version of MinLMOptimize() (for example, |
//| version which works with general form function and does not |
//| accept function vector), but it will lead to exception being |
//| thrown after first attempt to calculate Jacobian. |
//| USAGE: |
//| 1. User initializes algorithm State with MinLMCreateV() call |
//| 2. User tunes m_solver parameters with MinLMSetCond(), |
//| MinLMSetStpMax() and other functions |
//| 3. User calls MinLMOptimize() function which takes algorithm |
//| State and callback functions. |
//| 4. User calls MinLMResults() to get solution |
//| 5. Optionally, user may call MinLMRestartFrom() to solve another |
//| problem with same N/M but another starting point and/or |
//| another function. MinLMRestartFrom() allows to reuse already |
//| initialized structure. |
//| INPUT PARAMETERS: |
//| N - dimension, N>1 |
//| * if given, only leading N elements of X are |
//| used |
//| * if not given, automatically determined from |
//| size of X |
//| M - number of functions f[i] |
//| X - initial solution, array[0..m_n-1] |
//| DiffStep- differentiation step, >0 |
//| OUTPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| See also MinLMIteration, MinLMResults. |
//| NOTES: |
//| 1. you may tune stopping conditions with MinLMSetCond() function |
//| 2. if target function contains exp() or other fast growing |
//| functions, and optimization algorithm makes too large steps |
//| which leads to overflow, use MinLMSetStpMax() function to |
//| bound algorithm's steps. |
//+------------------------------------------------------------------+
void CMinLM::MinLMCreateV(const int n,const int m,double &x[],
const double diffstep,CMinLMState &State)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(diffstep),__FUNCTION__+": DiffStep is not finite!"))
return;
//--- check
if(!CAp::Assert(diffstep>0.0,__FUNCTION__+": DiffStep<=0!"))
return;
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1!"))
return;
//--- check
if(!CAp::Assert(m>=1,__FUNCTION__+": M<1!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- initialize
State.m_teststep=0;
State.m_n=n;
State.m_m=m;
State.m_algomode=0;
State.m_hasf=false;
State.m_hasfi=true;
State.m_hasg=false;
State.m_diffstep=diffstep;
//--- second stage of initialization
LMPRepare(n,m,false,State);
MinLMSetAccType(State,1);
MinLMSetCond(State,0,0);
MinLMSetXRep(State,false);
MinLMSetStpMax(State,0);
MinLMRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLM::MinLMCreateV(const int n,const int m,CRowDouble &x,
const double diffstep,CMinLMState &State)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(diffstep),__FUNCTION__+": DiffStep is not finite!"))
return;
//--- check
if(!CAp::Assert(diffstep>0.0,__FUNCTION__+": DiffStep<=0!"))
return;
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1!"))
return;
//--- check
if(!CAp::Assert(m>=1,__FUNCTION__+": M<1!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- initialize
State.m_teststep=0;
State.m_n=n;
State.m_m=m;
State.m_algomode=0;
State.m_hasf=false;
State.m_hasfi=true;
State.m_hasg=false;
State.m_diffstep=diffstep;
//--- second stage of initialization
LMPRepare(n,m,false,State);
MinLMSetAccType(State,1);
MinLMSetCond(State,0,0);
MinLMSetXRep(State,false);
MinLMSetStpMax(State,0);
MinLMRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| LEVENBERG-MARQUARDT-LIKE METHOD FOR NON-LINEAR OPTIMIZATION |
//| DESCRIPTION: |
//| This function is used to find minimum of general form (not |
//| "sum-of-squares") function |
//| F = F(x[0], ..., x[n-1]) |
//| using its gradient and Hessian. Levenberg-Marquardt modification |
//| with L-BFGS pre-optimization and internal pre-conditioned L-BFGS |
//| optimization after each Levenberg-Marquardt step is used. |
//| REQUIREMENTS: |
//| This algorithm will request following information during its |
//| operation: |
//| * function value F at given point X |
//| * F and gradient G (simultaneously) at given point X |
//| * F, G and Hessian H (simultaneously) at given point X |
//| There are several overloaded versions of MinLMOptimize() |
//| function which correspond to different LM-like optimization |
//| algorithms provided by this unit. You should choose version which|
//| accepts func(), grad() and hess() function pointers. First |
//| pointer is used to calculate F at given point, second one |
//| calculates F(x) and grad F(x), third one calculates F(x), grad |
//| F(x), hess F(x). |
//| You can try to initialize MinLMState structure with FGH-function |
//| and then use incorrect version of MinLMOptimize() (for example, |
//| version which does not provide Hessian matrix), but it will lead |
//| to exception being thrown after first attempt to calculate |
//| Hessian. |
//| USAGE: |
//| 1. User initializes algorithm State with MinLMCreateFGH() call |
//| 2. User tunes m_solver parameters with MinLMSetCond(), |
//| MinLMSetStpMax() and other functions |
//| 3. User calls MinLMOptimize() function which takes algorithm |
//| State and pointers (delegates, etc.) to callback functions. |
//| 4. User calls MinLMResults() to get solution |
//| 5. Optionally, user may call MinLMRestartFrom() to solve another |
//| problem with same N but another starting point and/or another |
//| function. MinLMRestartFrom() allows to reuse already |
//| initialized structure. |
//| INPUT PARAMETERS: |
//| N - dimension, N>1 |
//| * if given, only leading N elements of X are |
//| used |
//| * if not given, automatically determined from |
//| size of X |
//| X - initial solution, array[0..m_n-1] |
//| OUTPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NOTES: |
//| 1. you may tune stopping conditions with MinLMSetCond() function |
//| 2. if target function contains exp() or other fast growing |
//| functions, and optimization algorithm makes too large steps |
//| which leads to overflow, use MinLMSetStpMax() function to |
//| bound algorithm's steps. |
//+------------------------------------------------------------------+
void CMinLM::MinLMCreateFGH(const int n,double &x[],CMinLMState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- initialize
State.m_teststep=0;
State.m_n=n;
State.m_m=0;
State.m_algomode=2;
State.m_hasf=true;
State.m_hasfi=false;
State.m_hasg=true;
//--- init2
LMPRepare(n,0,true,State);
MinLMSetAccType(State,2);
MinLMSetCond(State,0,0);
MinLMSetXRep(State,false);
MinLMSetStpMax(State,0);
MinLMRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLM::MinLMCreateFGH(const int n,CRowDouble &x,CMinLMState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- initialize
State.m_teststep=0;
State.m_n=n;
State.m_m=0;
State.m_algomode=2;
State.m_hasf=true;
State.m_hasfi=false;
State.m_hasg=true;
//--- init2
LMPRepare(n,0,true,State);
MinLMSetAccType(State,2);
MinLMSetCond(State,0,0);
MinLMSetXRep(State,false);
MinLMSetStpMax(State,0);
MinLMRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| This function sets stopping conditions for Levenberg-Marquardt |
//| optimization algorithm. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| EpsX - >=0 |
//| The subroutine finishes its work if on k+1-th |
//| iteration the condition |v|<=EpsX is fulfilled, |
//| where: |
//| * |.| means Euclidian norm |
//| * v - scaled step vector, v[i]=dx[i]/s[i] |
//| * dx - ste pvector, dx=X(k+1)-X(k) |
//| * s - scaling coefficients set by MinLMSetScale()|
//| MaxIts - maximum number of iterations. If MaxIts=0, the |
//| number of iterations is unlimited. Only |
//| Levenberg-Marquardt iterations are counted |
//| (L-BFGS/CG iterations are NOT counted because |
//| their cost is very low compared to that of LM). |
//| Passing EpsG=0, EpsF=0, EpsX=0 and MaxIts=0 (simultaneously) will|
//| lead to automatic stopping criterion selection (small EpsX). |
//+------------------------------------------------------------------+
void CMinLM::MinLMSetCond(CMinLMState &State,double epsx,
const int m_maxits)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(epsx),__FUNCTION__+": EpsX is not finite number!"))
return;
//--- check
if(!CAp::Assert(epsx>=0.0,__FUNCTION__+": negative EpsX!"))
return;
//--- check
if(!CAp::Assert(m_maxits>=0,__FUNCTION__+": negative MaxIts!"))
return;
//--- check
if(epsx==0.0 && m_maxits==0)
epsx=1.0E-9;
//--- change values
State.m_epsx=epsx;
State.m_maxits=m_maxits;
}
//+------------------------------------------------------------------+
//| This function turns on/off reporting. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NeedXRep- whether iteration reports are needed or not |
//| If NeedXRep is True, algorithm will call rep() callback function |
//| if it is provided to MinLMOptimize(). Both Levenberg-Marquardt |
//| and internal L-BFGS iterations are reported. |
//+------------------------------------------------------------------+
void CMinLM::MinLMSetXRep(CMinLMState &State,const bool needxrep)
{
State.m_xrep=needxrep;
}
//+------------------------------------------------------------------+
//| This function sets maximum step length |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| StpMax - maximum step length, >=0. Set StpMax to 0.0, if |
//| you don't want to limit step length. |
//| Use this subroutine when you optimize target function which |
//| contains exp() or other fast growing functions, and optimization |
//| algorithm makes too large steps which leads to overflow. This |
//| function allows us to reject steps that are too large (and |
//| therefore expose us to the possible overflow) without actually |
//| calculating function value at the x+stp*d. |
//| NOTE: non-zero StpMax leads to moderate performance degradation |
//| because intermediate step of preconditioned L-BFGS optimization |
//| is incompatible with limits on step size. |
//+------------------------------------------------------------------+
void CMinLM::MinLMSetStpMax(CMinLMState &State,const double stpmax)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(stpmax),__FUNCTION__+": StpMax is not finite!"))
return;
//--- check
if(!CAp::Assert(stpmax>=0.0,__FUNCTION__+": StpMax<0!"))
return;
//--- change value
State.m_stpmax=stpmax;
}
//+------------------------------------------------------------------+
//| This function sets scaling coefficients for LM optimizer. |
//| ALGLIB optimizers use scaling matrices to test stopping |
//| conditions (step size and gradient are scaled before comparison |
//| with tolerances). Scale of the I-th variable is a translation |
//| invariant measure of: |
//| a) "how large" the variable is |
//| b) how large the step should be to make significant changes in |
//| the function |
//| Generally, scale is NOT considered to be a form of |
//| preconditioner. But LM optimizer is unique in that it uses |
//| scaling matrix both in the stopping condition tests and as |
//| Marquardt damping factor. |
//| Proper scaling is very important for the algorithm performance. |
//| It is less important for the quality of results, but still has |
//| some influence (it is easier to converge when variables are |
//| properly scaled, so premature stopping is possible when very |
//| badly scalled variables are combined with relaxed stopping |
//| conditions). |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| S - array[N], non-zero scaling coefficients |
//| S[i] may be negative, sign doesn't matter. |
//+------------------------------------------------------------------+
void CMinLM::MinLMSetScale(CMinLMState &State,double &s[])
{
//--- check
if(!CAp::Assert(CAp::Len(s)>=State.m_n,__FUNCTION__+": Length(S)<N"))
return;
for(int i=0; i<State.m_n; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(s[i]),__FUNCTION__+": S contains infinite or NAN elements"))
return;
//--- check
if(!CAp::Assert(s[i]!=0.0,__FUNCTION__+": S contains zero elements"))
return;
//--- change value
State.m_s.Set(i,MathAbs(s[i]));
}
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLM::MinLMSetScale(CMinLMState &State,CRowDouble &s)
{
//--- check
if(!CAp::Assert(CAp::Len(s)>=State.m_n,__FUNCTION__+": Length(S)<N"))
return;
for(int i=0; i<State.m_n; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(s[i]),__FUNCTION__+": S contains infinite or NAN elements"))
return;
//--- check
if(!CAp::Assert(s[i]!=0.0,__FUNCTION__+": S contains zero elements"))
return;
//--- change value
State.m_s.Set(i,MathAbs(s[i]));
}
}
//+------------------------------------------------------------------+
//| This function sets boundary constraints for LM optimizer |
//| Boundary constraints are inactive by default (after initial |
//| creation). They are preserved until explicitly turned off with |
//| another SetBC() call. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| BndL - lower bounds, array[N]. |
//| If some (all) variables are unbounded, you may |
//| specify very small number or -INF (latter is |
//| recommended because it will allow m_solver to use |
//| better algorithm). |
//| BndU - upper bounds, array[N]. |
//| If some (all) variables are unbounded, you may |
//| specify very large number or +INF (latter is |
//| recommended because it will allow m_solver to use |
//| better algorithm). |
//| NOTE 1: it is possible to specify BndL[i]=BndU[i]. In this case |
//| I-th variable will be "frozen" at X[i]=BndL[i]=BndU[i]. |
//| NOTE 2: this m_solver has following useful properties: |
//| * bound constraints are always satisfied exactly |
//| * function is evaluated only INSIDE area specified by bound |
//| constraints or at its boundary |
//+------------------------------------------------------------------+
void CMinLM::MinLMSetBC(CMinLMState &State,double &bndl[],double &bndu[])
{
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(bndl)>=n,__FUNCTION__+": Length(BndL)<N"))
return;
//--- check
if(!CAp::Assert(CAp::Len(bndu)>=n,__FUNCTION__+": Length(BndU)<N"))
return;
for(int i=0; i<n; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(bndl[i]) || AL_NEGINF==bndl[i],"MinLMSetBC: BndL contains NAN or +INF"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(bndu[i]) || AL_POSINF==bndu[i],"MinLMSetBC: BndU contains NAN or -INF"))
return;
//--- change values
State.m_bndl.Set(i,bndl[i]);
State.m_bndu.Set(i,bndu[i]);
State.m_havebndl[i]=MathIsValidNumber(bndl[i]);
State.m_havebndu[i]=MathIsValidNumber(bndu[i]);
}
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLM::MinLMSetBC(CMinLMState &State,CRowDouble &bndl,
CRowDouble &bndu)
{
//--- initialization
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(bndl)>=n,__FUNCTION__+": Length(BndL)<N"))
return;
//--- check
if(!CAp::Assert(CAp::Len(bndu)>=n,__FUNCTION__+": Length(BndU)<N"))
return;
for(int i=0; i<n; i++)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(bndl[i]) || AL_NEGINF==bndl[i],"MinLMSetBC: BndL contains NAN or +INF"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(bndu[i]) || AL_POSINF==bndu[i],"MinLMSetBC: BndU contains NAN or -INF"))
return;
//--- change values
State.m_bndl.Set(i,bndl[i]);
State.m_bndu.Set(i,bndu[i]);
State.m_havebndl[i]=MathIsValidNumber(bndl[i]);
State.m_havebndu[i]=MathIsValidNumber(bndu[i]);
}
}
//+------------------------------------------------------------------+
//| This function sets general linear constraints for LM optimizer |
//| Linear constraints are inactive by default (after initial |
//| creation). They are preserved until explicitly turned off with |
//| another minlmsetlc() call. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| C - linear constraints, array[K,N+1]. |
//| Each row of C represents one constraint, either |
//| equality or inequality (see below): |
//| * first N elements correspond to coefficients, |
//| * last element corresponds to the right part. |
//| All elements of C (including right part) must be |
//| finite. |
//| CT - type of constraints, array[K]: |
//| * if CT[i]>0, then I-th constraint is C[i,*]*x >= C[i,n+1] |
//| * if CT[i]=0, then I-th constraint is C[i,*]*x = C[i,n+1] |
//| * if CT[i]<0, then I-th constraint is C[i,*]*x <= C[i,n+1] |
//| K - number of equality/inequality constraints, K>=0: |
//| * if given, only leading K elements of C/CT are used |
//| * if not given, automatically determined from sizes of C/CT |
//| IMPORTANT: if you have linear constraints, it is strongly |
//| recommended to set scale of variables with |
//| MinLMSetScale(). QP solver which is used to calculate |
//| linearly constrained steps heavily relies on good |
//| scaling of input problems. |
//| IMPORTANT: solvers created with MinLMCreateFGH() do not support |
//| linear constraints. |
//| NOTE: linear (non-bound) constraints are satisfied only |
//| approximately - there always exists some violation due to |
//| numerical errors and algorithmic limitations. |
//| NOTE: general linear constraints add significant overhead to |
//| solution process. Although solver performs roughly same |
//| amount of iterations (when compared with similar box-only |
//| constrained problem), each iteration now involves solution |
//| of linearly constrained QP subproblem, which requires ~3-5 |
//| times more Cholesky decompositions. Thus, if you can |
//| reformulate your problem in such way this it has only box |
//| constraints, it may be beneficial to do so. |
//+------------------------------------------------------------------+
void CMinLM::MinLMSetLC(CMinLMState &State,CMatrixDouble &c,
CRowInt &ct,int k)
{
//--- create variables
int n=State.m_n;
int i=0;
int i_=0;
//--- First, check for errors in the inputs
if(!CAp::Assert(k>=0,__FUNCTION__+": K<0"))
return;
if(!CAp::Assert(c.Cols()>=n+1 || k==0,__FUNCTION__+": Cols(C)<N+1"))
return;
if(!CAp::Assert(c.Rows()>=k,__FUNCTION__+": Rows(C)<K"))
return;
if(!CAp::Assert(ct.Size()>=k,__FUNCTION__+": Length(CT)<K"))
return;
if(!CAp::Assert(CApServ::IsFiniteMatrix(c,k,n+1),__FUNCTION__+": C contains infinite or NaN values!"))
return;
//--- Handle zero K
if(k==0)
{
State.m_nec=0;
State.m_nic=0;
return;
}
//--- Equality constraints are stored first, in the upper
//--- NEC rows of State.CLEIC matrix. Inequality constraints
//--- are stored in the next NIC rows.
//--- NOTE: we convert inequality constraints to the form
//--- A*x<=b before copying them.
State.m_cleic.Resize(k,n+1);
State.m_nec=0;
State.m_nic=0;
for(i=0; i<k; i++)
{
if(ct[i]==0)
{
State.m_cleic.Row(State.m_nec,c[i]+0);
State.m_nec++;
}
}
for(i=0; i<k; i++)
{
if(ct[i]==0)
continue;
if(ct[i]>0)
State.m_cleic.Row(State.m_nec+State.m_nic,c[i]*(-1.0));
else
State.m_cleic.Row(State.m_nec+State.m_nic,c[i]+0);
State.m_nic++;
}
}
//+------------------------------------------------------------------+
//| This function is used to change acceleration Settings |
//| You can choose between three acceleration strategies: |
//| * AccType=0, no acceleration. |
//| * AccType=1, secant updates are used to update quadratic model |
//| after each iteration. After fixed number of iterations (or |
//| after model breakdown) we recalculate quadratic model using |
//| analytic Jacobian or finite differences. Number of secant-based|
//| iterations depends on optimization Settings: about 3 |
//| iterations - when we have analytic Jacobian, up to 2*N |
//| iterations - when we use finite differences to calculate |
//| Jacobian. |
//| AccType=1 is recommended when Jacobian calculation cost is |
//| prohibitive high (several Mx1 function vector calculations |
//| followed by several NxN Cholesky factorizations are faster than |
//| calculation of one M*N Jacobian). It should also be used when we|
//| have no Jacobian, because finite difference approximation takes |
//| too much time to compute. |
//| Table below list optimization protocols (XYZ protocol corresponds|
//| to MinLMCreateXYZ) and acceleration types they support (and use |
//| by default). |
//| ACCELERATION TYPES SUPPORTED BY OPTIMIZATION PROTOCOLS: |
//| protocol 0 1 comment |
//| V + + |
//| VJ + + |
//| FGH + |
//| DAFAULT VALUES: |
//| protocol 0 1 comment |
//| V x without acceleration it is so slooooooooow |
//| VJ x |
//| FGH x |
//| NOTE: this function should be called before optimization. |
//| Attempt to call it during algorithm iterations may result in |
//| unexpected behavior. |
//| NOTE: attempt to call this function with unsupported |
//| protocol/acceleration combination will result in exception being |
//| thrown. |
//+------------------------------------------------------------------+
void CMinLM::MinLMSetAccType(CMinLMState &State,int acctype)
{
//--- check
if(!CAp::Assert((acctype==0 || acctype==1) || acctype==2,__FUNCTION__+": incorrect AccType!"))
return;
//--- check
if(acctype==2)
acctype=0;
//--- check
if(acctype==0)
{
State.m_maxmodelage=0;
State.m_makeadditers=false;
//--- exit the function
return;
}
//--- check
if(acctype==1)
{
//--- check
if(!CAp::Assert(State.m_hasfi,__FUNCTION__+": AccType=1 is incompatible with current protocol!"))
return;
//--- check
if(State.m_algomode==0)
State.m_maxmodelage=2*State.m_n;
else
State.m_maxmodelage=m_smallmodelage;
State.m_makeadditers=false;
//--- exit the function
return;
}
}
//+------------------------------------------------------------------+
//| This function activates/deactivates verification of the |
//| user-supplied analytic Jacobian. |
//| Upon activation of this option OptGuard integrity checker |
//| performs numerical differentiation of your target function vector|
//| at the initial point (note: future versions may also perform |
//| check at the final point) and compares numerical Jacobian with|
//| analytic one provided by you. |
//| If difference is too large, an error flag is set and optimization|
//| session continues. After optimization session is over, you can |
//| retrieve the report which stores both Jacobians, and specific|
//| components highlighted as suspicious by the OptGuard. |
//| The OptGuard report can be retrieved with MinLMOptGuardResults().|
//| IMPORTANT: gradient check is a high-overhead option which will |
//| cost you about 3*N additional function evaluations. In|
//| many cases it may cost as much as the rest of the |
//| optimization session. |
//| YOU SHOULD NOT USE IT IN THE PRODUCTION CODE UNLESS YOU WANT TO |
//| CHECK DERIVATIVES PROVIDED BY SOME THIRD PARTY. |
//| NOTE: unlike previous incarnation of the gradient checking code, |
//| OptGuard does NOT interrupt optimization even if it |
//| discovers bad gradient. |
//| INPUT PARAMETERS: |
//| State - structure used to store algorithm State |
//| TestStep - verification step used for numerical |
//| differentiation: |
//| * TestStep=0 turns verification off |
//| * TestStep>0 activates verification. You should |
//| carefully choose TestStep. Value which is too |
//| large (so large that function behavior is non-|
//| cubic at this scale) will lead to false alarms. |
//| Too short step will result in rounding errors |
//| dominating numerical derivative. |
//| You may use different step for different parameters by means of |
//| setting scale with MinLMSetScale(). |
//| === EXPLANATION ================================================ |
//| In order to verify gradient algorithm performs following steps: |
//| * two trial steps are made to X[i]-TestStep*S[i] and |
//| X[i]+TestStep*S[i], where X[i] is i-th component of the |
//| initial point and S[i] is a scale of i-th parameter |
//| * F(X) is evaluated at these trial points |
//| * we perform one more evaluation in the middle point of the |
//| interval |
//| * we build cubic model using function values and derivatives|
//| at trial points and we compare its prediction with actual |
//| value in the middle point |
//+------------------------------------------------------------------+
void CMinLM::MinLMOptGuardGradient(CMinLMState &State,
double teststep)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(teststep),__FUNCTION__+": TestStep contains NaN or INF"))
return;
if(!CAp::Assert(teststep>=0.0,__FUNCTION__+": invalid argument TestStep(TestStep<0)"))
return;
State.m_teststep=teststep;
}
//+------------------------------------------------------------------+
//| Results of OptGuard integrity check, should be called after |
//| optimization session is over. |
//| OptGuard checks analytic Jacobian against reference value |
//| obtained by numerical differentiation with user-specified step. |
//| NOTE: other optimizers perform additional OptGuard checks for |
//| things like C0/C1-continuity violations. However, LM |
//| optimizer can check only for incorrect Jacobian. |
//| The reason is that unlike line search methods LM optimizer does |
//| not perform extensive evaluations along the line. Thus, we simply|
//| do not have enough data to catch C0/C1-violations. |
//| This check is activated with MinLMOptGuardGradient() function. |
//| Following flags are set when these errors are suspected: |
//| * rep.badgradsuspected, and additionally: |
//| * rep.badgradfidx for specific function (Jacobian row) |
//| suspected |
//| * rep.badgradvidx for specific variable (Jacobian column) |
//| suspected |
//| * rep.badgradxbase, a point where gradient/Jacobian is |
//| tested |
//| * rep.badgraduser, user-provided gradient/Jacobian |
//| * rep.badgradnum, reference gradient/Jacobian obtained via |
//| numerical differentiation |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| Rep - OptGuard report |
//+------------------------------------------------------------------+
void CMinLM::MinLMOptGuardResults(CMinLMState &State,
COptGuardReport &rep)
{
COptServ::SmoothnessMonitorExportReport(State.m_smonitor,rep);
}
//+------------------------------------------------------------------+
//| Levenberg-Marquardt algorithm results |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| X - array[0..m_n-1], solution |
//| Rep - optimization report; |
//| see comments for this structure for more Info. |
//+------------------------------------------------------------------+
void CMinLM::MinLMResults(CMinLMState &State,double &x[],
CMinLMReport &rep)
{
//--- reset memory
ArrayResize(x,0);
//--- function call
MinLMResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLM::MinLMResults(CMinLMState &State,CRowDouble &x,
CMinLMReport &rep)
{
//--- reset memory
x.Resize(0);
//--- function call
MinLMResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| Levenberg-Marquardt algorithm results |
//| Buffered implementation of MinLMResults(), which uses |
//| pre-allocated buffer to store X[]. If buffer size is too small, |
//| it resizes buffer. It is intended to be used in the inner cycles |
//| of performance critical algorithms where array reallocation |
//| penalty is too large to be ignored. |
//+------------------------------------------------------------------+
void CMinLM::MinLMResultsBuf(CMinLMState &State,double &x[],
CMinLMReport &rep)
{
//--- copy
State.m_x.ToArray(x);
//--- change values
rep.m_iterationscount=State.m_repiterationscount;
rep.m_terminationtype=State.m_repterminationtype;
rep.m_nfunc=State.m_repnfunc;
rep.m_njac=State.m_repnjac;
rep.m_ngrad=State.m_repngrad;
rep.m_nhess=State.m_repnhess;
rep.m_ncholesky=State.m_repncholesky;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLM::MinLMResultsBuf(CMinLMState &State,CRowDouble &x,
CMinLMReport &rep)
{
//--- copy
x=State.m_x;
//--- change values
rep.m_iterationscount=State.m_repiterationscount;
rep.m_terminationtype=State.m_repterminationtype;
rep.m_nfunc=State.m_repnfunc;
rep.m_njac=State.m_repnjac;
rep.m_ngrad=State.m_repngrad;
rep.m_nhess=State.m_repnhess;
rep.m_ncholesky=State.m_repncholesky;
}
//+------------------------------------------------------------------+
//| This subroutine restarts LM algorithm from new point. All |
//| optimization parameters are left unchanged. |
//| This function allows to solve multiple optimization problems |
//| (which must have same number of dimensions) without object |
//| reallocation penalty. |
//| INPUT PARAMETERS: |
//| State - structure used for reverse communication |
//| previously allocated with MinLMCreateXXX call. |
//| X - new starting point. |
//+------------------------------------------------------------------+
void CMinLM::MinLMRestartFrom(CMinLMState &State,double &x[])
{
//--- check
if(!CAp::Assert(CAp::Len(x)>=State.m_n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,State.m_n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- copy
State.m_xbase=x;
State.m_xbase.Resize(State.m_n);
//--- allocation
State.m_rstate.ia.Resize(5);
ArrayResizeAL(State.m_rstate.ba,1);
State.m_rstate.ra.Resize(4);
State.m_rstate.stage=-1;
//--- function call
ClearRequestFields(State);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLM::MinLMRestartFrom(CMinLMState &State,CRowDouble &x)
{
//--- check
if(!CAp::Assert(CAp::Len(x)>=State.m_n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,State.m_n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- copy
State.m_xbase=x;
State.m_xbase.Resize(State.m_n);
//--- allocation
State.m_rstate.ia.Resize(5);
ArrayResizeAL(State.m_rstate.ba,1);
State.m_rstate.ra.Resize(4);
State.m_rstate.stage=-1;
//--- function call
ClearRequestFields(State);
}
//+------------------------------------------------------------------+
//| This subroutine submits request for termination of running |
//| optimizer. It should be called from user-supplied callback when |
//| user decides that it is time to "smoothly" terminate optimization|
//| process. As result, optimizer stops at point which was "current |
//| accepted" when termination request was submitted and returns |
//| error code 8 (successful termination). |
//| INPUT PARAMETERS: |
//| State - optimizer structure |
//| NOTE: after request for termination optimizer may perform several|
//| additional calls to user-supplied callbacks. It does NOT |
//| guarantee to stop immediately - it just guarantees that |
//| these additional calls will be discarded later. |
//| NOTE: calling this function on optimizer which is NOT running |
//| will have no effect. |
//| NOTE: multiple calls to this function are possible. First call is|
//| counted, subsequent calls are silently ignored. |
//+------------------------------------------------------------------+
void CMinLM::MinLMRequestTermination(CMinLMState &State)
{
State.m_userterminationneeded=true;
}
//+------------------------------------------------------------------+
//| This is obsolete function. |
//| Since ALGLIB 3.3 it is equivalent to MinLMCreateVJ(). |
//+------------------------------------------------------------------+
void CMinLM::MinLMCreateVGJ(const int n,const int m,double &x[],
CMinLMState &State)
{
MinLMCreateVJ(n,m,x,State);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLM::MinLMCreateVGJ(const int n,const int m,CRowDouble &x,
CMinLMState &State)
{
MinLMCreateVJ(n,m,x,State);
}
//+------------------------------------------------------------------+
//| This is obsolete function. |
//| Since ALGLIB 3.3 it is equivalent to MinLMCreateFJ(). |
//+------------------------------------------------------------------+
void CMinLM::MinLMCreateFGJ(const int n,const int m,double &x[],
CMinLMState &State)
{
MinLMCreateFJ(n,m,x,State);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLM::MinLMCreateFGJ(const int n,const int m,CRowDouble &x,
CMinLMState &State)
{
MinLMCreateFJ(n,m,x,State);
}
//+------------------------------------------------------------------+
//| This function is considered obsolete since ALGLIB 3.1.0 and is |
//| present for backward compatibility only. We recommend to use |
//| MinLMCreateVJ, which provides similar, but more consistent and |
//| feature-rich interface. |
//+------------------------------------------------------------------+
void CMinLM::MinLMCreateFJ(const int n,const int m,double &x[],
CMinLMState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1!"))
return;
//--- check
if(!CAp::Assert(m>=1,__FUNCTION__+": M<1!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- initialize
State.m_teststep=0;
State.m_n=n;
State.m_m=m;
State.m_algomode=1;
State.m_hasf=true;
State.m_hasfi=false;
State.m_hasg=false;
//--- init 2
LMPRepare(n,m,true,State);
//--- function call
MinLMSetAccType(State,0);
//--- function call
MinLMSetCond(State,0,0);
//--- function call
MinLMSetXRep(State,false);
//--- function call
MinLMSetStpMax(State,0);
//--- function call
MinLMRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLM::MinLMCreateFJ(const int n,const int m,CRowDouble &x,
CMinLMState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1!"))
return;
//--- check
if(!CAp::Assert(m>=1,__FUNCTION__+": M<1!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- initialize
State.m_teststep=0;
State.m_n=n;
State.m_m=m;
State.m_algomode=1;
State.m_hasf=true;
State.m_hasfi=false;
State.m_hasg=false;
//--- init 2
LMPRepare(n,m,true,State);
//--- function call
MinLMSetAccType(State,0);
//--- function call
MinLMSetCond(State,0,0);
//--- function call
MinLMSetXRep(State,false);
//--- function call
MinLMSetStpMax(State,0);
//--- function call
MinLMRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| Prepare internal structures (except for RComm). |
//| Note: M must be zero for FGH mode, non-zero for V/VJ/FJ/FGJ mode.|
//+------------------------------------------------------------------+
void CMinLM::LMPRepare(const int n,const int m,bool havegrad,
CMinLMState &State)
{
COptServ::SmoothnessMonitorInit(State.m_smonitor,State.m_s,0,0,false);
if(n<=0 || m<0)
return;
if(havegrad)
State.m_g.Resize(n);
if(m!=0)
{
State.m_j.Resize(m,n);
State.m_fi.Resize(m);
State.m_fibase.Resize(m);
State.m_deltaf.Resize(m);
State.m_fm1.Resize(m);
State.m_fp1.Resize(m);
State.m_fc1.Resize(m);
State.m_gm1.Resize(m);
State.m_gp1.Resize(m);
State.m_gc1.Resize(m);
}
else
State.m_h.Resize(n,n);
State.m_x=vector<double>::Zeros(n);
State.m_deltax.Resize(n);
State.m_quadraticmodel.Resize(n,n);
State.m_xbase.Resize(n);
State.m_gbase.Resize(n);
State.m_xdir.Resize(n);
State.m_tmp0.Resize(n);
//--- prepare internal L-BFGS
CMinLBFGS::MinLBFGSCreate(n,MathMin(m_additers,n),State.m_x,State.m_internalstate);
CMinLBFGS::MinLBFGSSetCond(State.m_internalstate,0.0,0.0,0.0,MathMin(m_additers,n));
//--- Prepare internal QP solver
CMinQP::MinQPCreate(n,State.m_qpstate);
CMinQP::MinQPSetAlgoQuickQP(State.m_qpstate,0.0,0.0,CApServ::Coalesce(0.01*State.m_epsx,1.0E-12),10,true);
//--- Prepare boundary constraints
State.m_bndl=vector<double>::Full(n,AL_NEGINF);
State.m_bndu=vector<double>::Full(n,AL_POSINF);
ArrayResize(State.m_havebndl,n);
ArrayResize(State.m_havebndu,n);
ArrayInitialize(State.m_havebndl,false);
ArrayInitialize(State.m_havebndu,false);
//--- Prepare scaling matrix
State.m_s=vector<double>::Ones(n);
State.m_lastscaleused=vector<double>::Ones(n);
//--- Prepare linear constraints
State.m_nec=0;
State.m_nic=0;
}
//+------------------------------------------------------------------+
//| Clears request fileds (to be sure that we don't forgot to clear |
//| something) |
//+------------------------------------------------------------------+
void CMinLM::ClearRequestFields(CMinLMState &State)
{
//--- change values
State.m_needf=false;
State.m_needfg=false;
State.m_needfgh=false;
State.m_needfij=false;
State.m_needfi=false;
State.m_xupdated=false;
}
//+------------------------------------------------------------------+
//| Increases lambda, returns False when there is a danger of |
//| overflow |
//+------------------------------------------------------------------+
bool CMinLM::IncreaseLambda(double &lambdav,double &nu)
{
//--- create variables
double lnlambda=MathLog(lambdav);
double lnlambdaup=MathLog(m_lambdaup);
double lnnu=MathLog(nu);
double lnmax=MathLog(CMath::m_maxrealnumber);
//--- check
if(lnlambda+lnlambdaup+lnnu>0.25*lnmax)
return(false);
//--- check
if(lnnu+MathLog(2)>lnmax)
return(false);
//--- change values
lambdav=lambdav*m_lambdaup*nu;
nu=nu*2;
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Decreases lambda, but leaves it unchanged when there is danger of|
//| underflow. |
//+------------------------------------------------------------------+
void CMinLM::DecreaseLambda(double &lambdav,double &nu)
{
//--- initialization
nu=1;
//--- check
if(MathLog(lambdav)+MathLog(m_lambdadown)<MathLog(CMath::m_minrealnumber))
lambdav=CMath::m_minrealnumber;
else
lambdav=lambdav*m_lambdadown;
}
//+------------------------------------------------------------------+
//| This function compares actual decrease vs predicted decrease and |
//| updates LambdaV/Nu accordingly. |
//| INPUT PARAMETERS: |
//| QuadraticModel - array[N,N], full Hessian matrix of quadratic |
//| model at deltaX=0 |
//| GBase - array[N], gradient at deltaX=0 |
//| FBase - F(deltaX=0) |
//| N - size |
//| DeltaX - step vector |
//| FNew - new function value |
//| LambdaV - lambda-value, updated on exit |
//| Nu - Nu-multiplier, updated on exit |
//| On exit it returns: |
//| * Result=0 - if we have to continue iterations |
//| * Result<>0 - if termination with completion code Result is|
//| requested |
//+------------------------------------------------------------------+
int CMinLM::CheckDecrease(CMatrixDouble &quadraticmodel,
CRowDouble &gbase,
double fbase,
int n,
CRowDouble &deltax,
double fnew,
double &lambdav,
double &nu)
{
//--- create variables
int result=0;
int i=0;
double v=0;
double t=0;
double predicteddecrease=0;
double actualdecrease=0;
for(i=0; i<n; i++)
{
v=deltax.Dot(quadraticmodel[i]+0);
t+=deltax[i]*(gbase[i]+0.5 *v);
}
predicteddecrease=-t;
actualdecrease=-(fnew-fbase);
if(predicteddecrease<=0.0)
{
result=7;
return(result);
}
v=actualdecrease/predicteddecrease;
if(v<0.1)
if(!IncreaseLambda(lambdav,nu))
{
//--- Lambda is too large, we have to break iterations.
result=7;
return(result);
}
if(v>0.5)
DecreaseLambda(lambdav,nu);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function initializes step finder object with problem |
//| statement; model parameters specified during this call should not|
//| (and can not) change during object lifetime (although it is |
//| possible to re-initialize object with different Settings). |
//| This function reuses internally allocated objects as much as |
//| possible. |
//| In addition to initializing step finder, this function enforces |
//| feasibility in initial point X passed to this function. It is |
//| important that LM iteration starts from feasible point and |
//| performs feasible steps; |
//| RETURN VALUE: |
//| True for successful initialization |
//| False for inconsistent constraints; you should not use step |
//| finder if it returned False. |
//+------------------------------------------------------------------+
bool CMinLM::MinLMStepFinderInit(CMinLMStepFinder &State,
int n,
int m,
int maxmodelage,
bool hasfi,
CRowDouble &xbase,
CRowDouble &bndl,
CRowDouble &bndu,
CMatrixDouble &cleic,
int nec,
int nic,
CRowDouble &s,
double stpmax,
double epsx)
{
//--- create variable
int i=0;
//--- initialization
State.m_n=n;
State.m_m=m;
State.m_maxmodelage=maxmodelage;
State.m_hasfi=hasfi;
State.m_stpmax=stpmax;
State.m_epsx=epsx;
//--- Allocate temporaries, create QP solver, select QP algorithm
ArrayResize(State.m_havebndl,n);
ArrayResize(State.m_havebndu,n);
State.m_bndl.Resize(n);
State.m_bndu.Resize(n);
State.m_s=s;
State.m_x.Resize(n);
State.m_xbase.Resize(n);
State.m_tmp0.Resize(n);
State.m_modeldiag.Resize(n);
State.m_tmpct.Resize(nec+nic);
State.m_xdir.Resize(n);
if(hasfi)
{
State.m_fi.Resize(m);
State.m_fibase.Resize(m);
}
for(i=0; i<n; i++)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(bndl[i]) || IsNegInf(bndl[i]),__FUNCTION__+": integrity check failed"))
return(false);
if(!CAp::Assert(MathIsValidNumber(bndu[i]) || IsPosInf(bndu[i]),__FUNCTION__+": integrity check failed"))
return(false);
State.m_havebndl[i]=MathIsValidNumber(bndl[i]);
State.m_havebndu[i]=MathIsValidNumber(bndu[i]);
State.m_bndl.Set(i,bndl[i]);
State.m_bndu.Set(i,bndu[i]);
}
State.m_tmpct.Fill(0,0,nec);
State.m_tmpct.Fill(-1,nec,nic);
CMinQP::MinQPCreate(n,State.m_qpstate);
if(nec+nic==0)
CMinQP::MinQPSetAlgoQuickQP(State.m_qpstate,0.0,0.0,CApServ::Coalesce(0.01*epsx,1.0E-12),10,true);
else
CMinQP::MinQPSetAlgoDenseAUL(State.m_qpstate,CApServ::Coalesce(0.01*epsx,1.0E-12),100,10);
CMinQP::MinQPSetBC(State.m_qpstate,bndl,bndu);
CMinQP::MinQPSetLC(State.m_qpstate,cleic,State.m_tmpct,nec+nic);
CMinQP::MinQPSetScale(State.m_qpstate,s);
//--- Check feasibility of constraints:
//--- * check/enforce box constraints (straightforward)
//--- * prepare QP subproblem which return us a feasible point
for(i=0; i<n; i++)
{
if(State.m_havebndl[i] && State.m_havebndu[i] && State.m_bndl[i]>State.m_bndu[i])
return(false);
if(State.m_havebndl[i] && xbase[i]<State.m_bndl[i])
xbase.Set(i,State.m_bndl[i]);
if(State.m_havebndu[i] && xbase[i]>State.m_bndu[i])
xbase.Set(i,State.m_bndu[i]);
}
if(nec+nic>0)
{
//--- Well, we have linear constraints... let's use heavy machinery.
//--- We will modify QP solver State below, but everything will be
//--- restored in MinLMStepFinderStart().
CSparse::SparseCreate(n,n,n,State.m_tmpsp);
for(i=0; i<n; i++)
CSparse::SparseSet(State.m_tmpsp,i,i,0.5);
State.m_tmp0=vector<double>::Zeros(n);
CMinQP::MinQPSetStartingPointFast(State.m_qpstate,xbase);
CMinQP::MinQPSetOriginFast(State.m_qpstate,xbase);
CMinQP::MinQPSetLinearTermFast(State.m_qpstate,State.m_tmp0);
CMinQP::MinQPSetQuadraticTermSparse(State.m_qpstate,State.m_tmpsp,true);
CMinQP::MinQPOptimize(State.m_qpstate);
CMinQP::MinQPResultsBuf(State.m_qpstate,xbase,State.m_qprep);
}
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| This function prepares LM step search session. |
//+------------------------------------------------------------------+
void CMinLM::MinLMStepFinderStart(CMinLMStepFinder &State,
CMatrixDouble &quadraticmodel,
CRowDouble &gbase,
double fbase,
CRowDouble &xbase,
CRowDouble &fibase,
int modelage)
{
int n=State.m_n;
//--- allocated
State.m_rstate.ia.Resize(3);
ArrayResize(State.m_rstate.ba,1);
State.m_rstate.ra.Resize(1);
//--- initialization
State.m_rstate.stage=-1;
State.m_modelage=modelage;
State.m_fbase=fbase;
if(State.m_hasfi)
{
State.m_fibase=fibase;
}
State.m_xbase=xbase;
State.m_modeldiag=quadraticmodel.Diag()+0;
CMinQP::MinQPSetStartingPointFast(State.m_qpstate,xbase);
CMinQP::MinQPSetOriginFast(State.m_qpstate,xbase);
CMinQP::MinQPSetLinearTermFast(State.m_qpstate,gbase);
CMinQP::MinQPSetQuadraticTermFast(State.m_qpstate,quadraticmodel,true,0.0);
}
//+------------------------------------------------------------------+
//| This function runs LM step search session. |
//| Find value of Levenberg-Marquardt damping parameter which: |
//| * leads to positive definite damped model |
//| * within bounds specified by StpMax |
//| * generates step which decreases function value |
//| After this block IFlag is set to: |
//| * -8, if infinities/NANs were detected in function values/ |
//| gradient |
//| * -3, if constraints are infeasible |
//| * -2, if model update is needed (either Lambda growth is too |
//| large or step is too short, but we can't rely on model |
//| and stop iterations) |
//| * -1, if model is fresh, Lambda have grown too large, |
//| termination is needed |
//| * 0, if everything is OK, continue iterations |
//| * >0 - successful completion (step size is small enough) |
//| State.Nu can have any value on enter, but after exit it is set |
//| to 1.0 |
//+------------------------------------------------------------------+
bool CMinLM::MinLMStepFinderIteration(CMinLMStepFinder &State,
double &lambdav,
double &nu,
CRowDouble &xnew,
CRowDouble &deltax,
bool &deltaxready,
CRowDouble &deltaf,
bool &deltafready,
int &iflag,
double &fnew,
int &ncholesky)
{
int i=0;
bool bflag=false;
double v=0;
int n=0;
int m=0;
//--- Reverse communication preparations
//--- I know it looks ugly, but it works the same way
//--- anywhere from C++ to Python.
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(State.m_rstate.stage>=0)
{
i=State.m_rstate.ia[0];
n=State.m_rstate.ia[1];
m=State.m_rstate.ia[2];
bflag=State.m_rstate.ba[0];
v=State.m_rstate.ra[0];
}
else
{
i=-838;
n=939;
m=-526;
bflag=true;
v=-541;
}
//--- check
switch(State.m_rstate.stage)
{
case 0:
State.m_needfi=false;
v=State.m_fi.Dot(State.m_fi);
fnew=v;
deltaf=State.m_fi-State.m_fibase+0;
deltafready=true;
if(!MathIsValidNumber(fnew))
{
//--- Integrity check failed, break!
iflag=-8;
nu=1;
return(false);
}
if(fnew>=State.m_fbase)
{
//--- Increase lambda and continue
if(!IncreaseLambda(lambdav,nu))
{
iflag=-1;
nu=1;
return(false);
}
break;
}
//--- We've found our step!
iflag=0;
nu=1;
return(false);
break;
case 1:
State.m_needf=false;
fnew=State.m_f;
if(!MathIsValidNumber(fnew))
{
//--- Integrity check failed, break!
iflag=-8;
nu=1;
return(false);
}
if(fnew>=State.m_fbase)
{
//--- Increase lambda and continue
if(!IncreaseLambda(lambdav,nu))
{
iflag=-1;
nu=1;
return(false);
}
break;
}
//--- We've found our step!
iflag=0;
nu=1;
return(false);
break;
default:
iflag=-99;
n=State.m_n;
m=State.m_m;
break;
}
//--- Routine body
while(true)
{
deltaxready=false;
deltafready=false;
//--- Do we need model update?
if(State.m_modelage>0 && nu>=m_suspiciousnu)
{
iflag=-2;
nu=1;
return(false);
}
//
//--- Setup quadratic solver and solve quadratic programming problem.
//--- After problem is solved we'll try to bound step by StpMax
//--- (Lambda will be increased if step size is too large).
//
//--- We use BFlag variable to indicate that we have to increase Lambda.
//--- If it is False, we will try to increase Lambda and move to new iteration.
//
bflag=true;
State.m_tmp0=State.m_modeldiag.ToVector()+State.m_s.Pow(-2)*lambdav;
CMinQP::MinQPRewriteDiagonal(State.m_qpstate,State.m_tmp0);
CMinQP::MinQPOptimize(State.m_qpstate);
CMinQP::MinQPResultsBuf(State.m_qpstate,xnew,State.m_qprep);
ncholesky+=State.m_qprep.m_ncholesky;
if(State.m_qprep.m_terminationtype==-3)
{
//--- Infeasible constraints
iflag=-3;
nu=1;
return(false);
}
if(State.m_qprep.m_terminationtype==-4 || State.m_qprep.m_terminationtype==-5)
{
//--- Unconstrained direction of negative curvature was detected
if(!IncreaseLambda(lambdav,nu))
{
iflag=-1;
nu=1;
return(false);
}
continue;
}
//--- check
if(!CAp::Assert(State.m_qprep.m_terminationtype>0,__FUNCTION__+": unexpected completion code from QP solver"))
return(false);
State.m_xdir=xnew-State.m_xbase+0;
v=MathPow(State.m_xdir/State.m_s+0,2).Sum();
if(MathIsValidNumber(v))
{
v=MathSqrt(v);
if(State.m_stpmax>0.0 && v>State.m_stpmax)
bflag=false;
}
else
bflag=false;
if(!bflag)
{
//--- Solution failed:
//--- try to increase lambda to make matrix positive definite and continue.
if(!IncreaseLambda(lambdav,nu))
{
iflag=-1;
nu=1;
return(false);
}
continue;
}
//--- Step in State.XDir and it is bounded by StpMax.
//--- We should check stopping conditions on step size here.
//--- DeltaX, which is used for secant updates, is initialized here.
//--- This code is a bit tricky because sometimes XDir<>0, but
//--- it is so small that XDir+XBase==XBase (in finite precision
//--- arithmetics). So we set DeltaX to XBase, then
//--- add XDir, and then subtract XBase to get exact value of
//--- DeltaX.
//--- Step length is estimated using DeltaX.
//--- NOTE: stopping conditions are tested
//--- for fresh models only (ModelAge=0)
deltax=xnew-State.m_xbase+0;
deltaxready=true;
v=MathPow(deltax/State.m_s+0,2.0).Sum();
v=MathSqrt(v);
if(v<=State.m_epsx)
{
if(State.m_modelage==0)
{
//--- Step is too short, model is fresh and we can rely on it.
//--- Terminating.
iflag=2;
nu=1;
return(false);
}
else
{
//--- Step is suspiciously short, but model is not fresh
//--- and we can't rely on it.
iflag=-2;
nu=1;
return(false);
}
}
//--- Let's evaluate new step:
//--- a) if we have Fi vector, we evaluate it using rcomm, and
//--- then we manually calculate State.F as sum of squares of Fi[]
//--- b) if we have F value, we just evaluate it through rcomm interface
//--- We prefer (a) because we may need Fi vector for additional
//--- iterations
State.m_x=xnew;
State.m_needf=false;
State.m_needfi=false;
if(!State.m_hasfi)
{
State.m_needf=true;
State.m_rstate.stage=1;
}
else
{
State.m_needfi=true;
State.m_rstate.stage=0;
}
break;
}
//--- Saving State
State.m_rstate.ba[0]=bflag;
State.m_rstate.ia.Set(0,i);
State.m_rstate.ia.Set(1,n);
State.m_rstate.ia.Set(2,m);
State.m_rstate.ra.Set(0,v);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| NOTES: |
//| 1. Depending on function used to create State structure, this |
//| algorithm may accept Jacobian and/or Hessian and/or gradient. |
//| According to the said above, there ase several versions of |
//| this function, which accept different sets of callbacks. |
//| This flexibility opens way to subtle errors - you may create |
//| State with MinLMCreateFGH() (optimization using Hessian), but |
//| call function which does not accept Hessian. So when |
//| algorithm will request Hessian, there will be no callback to |
//| call. In this case exception will be thrown. |
//| Be careful to avoid such errors because there is no way to |
//| find them at compile time - you can see them at runtime only. |
//+------------------------------------------------------------------+
bool CMinLM::MinLMIteration(CMinLMState &State)
{
//--- create variables
int n=0;
int m=0;
bool bflag=false;
int iflag=0;
double v=0;
double s=0;
double t=0;
double fnew=0;
int i=0;
int k=0;
int i_=0;
int label=-1;
//--- Reverse communication preparations
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(State.m_rstate.stage>=0)
{
n=State.m_rstate.ia[0];
m=State.m_rstate.ia[1];
iflag=State.m_rstate.ia[2];
i=State.m_rstate.ia[3];
k=State.m_rstate.ia[4];
bflag=State.m_rstate.ba[0];
v=State.m_rstate.ra[0];
s=State.m_rstate.ra[1];
t=State.m_rstate.ra[2];
fnew=State.m_rstate.ra[3];
}
else
{
n=359;
m=-58;
iflag=-919;
i=-909;
k=81;
bflag=true;
v=74;
s=-788;
t=809;
fnew=205;
}
//--- check stage
switch(State.m_rstate.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
case 3:
label=3;
break;
case 4:
label=4;
break;
case 5:
label=5;
break;
case 6:
label=6;
break;
case 7:
label=7;
break;
case 8:
label=8;
break;
case 9:
label=9;
break;
case 10:
label=10;
break;
case 11:
label=11;
break;
case 12:
label=12;
break;
case 13:
label=13;
break;
case 14:
label=14;
break;
case 15:
label=15;
break;
case 16:
label=16;
break;
case 17:
label=17;
break;
case 18:
label=18;
break;
case 19:
label=19;
break;
case 20:
label=20;
break;
case 21:
label=21;
break;
case 22:
label=22;
break;
case 23:
label=23;
break;
case 24:
label=24;
break;
case 25:
label=25;
break;
case 26:
label=26;
break;
case 27:
label=27;
break;
default:
//--- Routine body
//--- prepare
n=State.m_n;
m=State.m_m;
State.m_repiterationscount=0;
State.m_repterminationtype=0;
State.m_repnfunc=0;
State.m_repnjac=0;
State.m_repngrad=0;
State.m_repnhess=0;
State.m_repncholesky=0;
State.m_userterminationneeded=false;
if(m>0)
COptServ::SmoothnessMonitorInit(State.m_smonitor,State.m_s,n,m,false);
State.m_lastscaleused=State.m_s;
//--- Prepare LM step finder and enforce/check feasibility of constraints
if(!MinLMStepFinderInit(State.m_finderstate,n,m,State.m_maxmodelage,State.m_hasfi,State.m_xbase,State.m_bndl,State.m_bndu,State.m_cleic,State.m_nec,State.m_nic,State.m_s,State.m_stpmax,State.m_epsx))
{
State.m_repterminationtype=-3;
return(false);
}
//--- set constraints for obsolete QP solver
CMinQP::MinQPSetBC(State.m_qpstate,State.m_bndl,State.m_bndu);
//--- Check correctness of the analytic Jacobian
ClearRequestFields(State);
if(!(State.m_algomode==1 && State.m_teststep>0.0))
{
label=28;
break;
}
//--- check
if(!CAp::Assert(m>0,__FUNCTION__+": integrity check failed"))
return(false);
label=30;
break;
}
//--- main loop
while(label>=0)
switch(label)
{
case 30:
if(!COptServ::SmoothnessMonitorCheckGradientATX0(State.m_smonitor,State.m_xbase,State.m_s,State.m_bndl,State.m_bndu,true,State.m_teststep))
{
label=31;
break;
}
State.m_x=State.m_smonitor.m_x;
State.m_needfij=true;
State.m_rstate.stage=0;
label=-1;
break;
case 0:
State.m_needfij=false;
State.m_smonitor.m_fi=State.m_fi;
State.m_smonitor.m_j=State.m_j;
label=30;
break;
case 31:
case 28:
//--- Initial report of current point
//--- Note 1: we rewrite State.X twice because
//--- user may accidentally change it after first call.
//--- Note 2: we set NeedF or NeedFI depending on what
//--- information about function we have.
if(!State.m_xrep)
{
label=32;
break;
}
State.m_x=State.m_xbase;
ClearRequestFields(State);
if(!State.m_hasf)
{
label=34;
break;
}
State.m_needf=true;
State.m_rstate.stage=1;
label=-1;
break;
case 1:
State.m_needf=false;
label=35;
break;
case 34:
//--- check
if(!CAp::Assert(State.m_hasfi,__FUNCTION__+": internal error 2!"))
return(false);
State.m_needfi=true;
State.m_rstate.stage=2;
label=-1;
break;
case 2:
State.m_needfi=false;
v=CAblasF::RDotV2(m,State.m_fi);
State.m_f=v;
case 35:
State.m_repnfunc=State.m_repnfunc+1;
State.m_x=State.m_xbase;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=3;
label=-1;
break;
case 3:
State.m_xupdated=false;
case 32:
if(State.m_userterminationneeded)
{
//--- User requested termination
State.m_x=State.m_xbase;
State.m_repterminationtype=8;
return(false);
}
//--- Prepare control variables
State.m_nu=1;
State.m_lambdav=-CMath::m_maxrealnumber;
State.m_modelage=State.m_maxmodelage+1;
State.m_deltaxready=false;
State.m_deltafready=false;
if(State.m_algomode==2)
{
label=36;
break;
}
//--- Jacobian-based optimization mode
//--- Main cycle.
//--- We move through it until either:
//--- * one of the stopping conditions is met
//--- * we decide that stopping conditions are too stringent
//--- and break from cycle
case 38:
//--- First, we have to prepare quadratic model for our function.
//--- We use BFlag to ensure that model is prepared;
//--- if it is false at the end of this block, something went wrong.
//--- We may either calculate brand new model or update old one.
//--- Before this block we have:
//--- * State.XBase - current position.
//--- * State.DeltaX - if DeltaXReady is True
//--- * State.DeltaF - if DeltaFReady is True
//--- After this block is over, we will have:
//--- * State.XBase - base point (unchanged)
//--- * State.FBase - F(XBase)
//--- * State.GBase - linear term
//--- * State.QuadraticModel - quadratic term
//--- * State.LambdaV - current estimate for lambda
//--- We also clear DeltaXReady/DeltaFReady flags
//--- after initialization is done.
//--- check
if(!CAp::Assert(State.m_algomode==0 || State.m_algomode==1,__FUNCTION__+": integrity check failed"))
return(false);
if(!(State.m_modelage>State.m_maxmodelage || !(State.m_deltaxready && State.m_deltafready)))
{
label=40;
break;
}
//--- Refresh model (using either finite differences or analytic Jacobian)
if(State.m_algomode!=0)
{
label=42;
break;
}
//--- Optimization using F values only.
//--- Use finite differences to estimate Jacobian.
//--- check
if(!CAp::Assert(State.m_hasfi,__FUNCTION__+": internal error when estimating Jacobian (no f[])"))
return(false);
k=0;
case 44:
if(k>n-1)
{
label=46;
break;
}
//--- We guard X[k] from leaving [BndL,BndU].
//--- In case BndL=BndU, we assume that derivative in this direction is zero.
State.m_x=State.m_xbase;
State.m_x.Add(k,-State.m_s[k]*State.m_diffstep);
if(State.m_havebndl[k])
State.m_x.Set(k,MathMax(State.m_x[k],State.m_bndl[k]));
if(State.m_havebndu[k])
State.m_x.Set(k,MathMin(State.m_x[k],State.m_bndu[k]));
State.m_xm1=State.m_x[k];
ClearRequestFields(State);
State.m_needfi=true;
State.m_rstate.stage=4;
label=-1;
break;
case 4:
State.m_repnfunc++;
State.m_fm1=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Add(k,State.m_s[k]*State.m_diffstep);
if(State.m_havebndl[k])
State.m_x.Set(k,MathMax(State.m_x[k],State.m_bndl[k]));
if(State.m_havebndu[k])
State.m_x.Set(k,MathMin(State.m_x[k],State.m_bndu[k]));
State.m_xp1=State.m_x[k];
ClearRequestFields(State);
State.m_needfi=true;
State.m_rstate.stage=5;
label=-1;
break;
case 5:
State.m_repnfunc++;
State.m_fp1=State.m_fi;
v=State.m_xp1-State.m_xm1;
if(v!=0.0)
{
v=1/v;
State.m_j.Col(k,(State.m_fp1-State.m_fm1)*v);
}
else
State.m_j.Col(k,vector<double>::Zeros(m));
k++;
label=44;
break;
case 46:
//--- Calculate F(XBase)
State.m_x=State.m_xbase;
ClearRequestFields(State);
State.m_needfi=true;
State.m_rstate.stage=6;
label=-1;
break;
case 6:
State.m_needfi=false;
State.m_repnfunc++;
State.m_repnjac++;
//--- New model
State.m_modelage=0;
label=41;
break;
case 42:
//--- Obtain f[] and Jacobian
State.m_x=State.m_xbase;
ClearRequestFields(State);
State.m_needfij=true;
State.m_rstate.stage=7;
label=-1;
break;
case 7:
State.m_needfij=false;
State.m_repnfunc++;
State.m_repnjac++;
//--- New model
State.m_modelage=0;
case 43:
label=41;
break;
case 40:
//--- State.J contains Jacobian or its current approximation;
//--- refresh it using secant updates:
//--- f(x0+dx) = f(x0) + J*dx,
//--- J_new = J_old + u*h'
//--- h = x_new-x_old
//--- u = (f_new - f_old - J_old*h)/(h'h)
//--- We can explicitly generate h and u, but it is
//--- preferential to do in-place calculations. Only
//--- I-th row of J_old is needed to calculate u[I],
//--- so we can update J row by row in one pass.
//--- NOTE: we expect that State.XBase contains new point,
//--- State.FBase contains old point, State.DeltaX and
//--- State.DeltaY contain updates from last step.
//--- check
if(!CAp::Assert(State.m_deltaxready && State.m_deltafready,__FUNCTION__+": uninitialized DeltaX/DeltaF"))
return(false);
t=CAblasF::RDotV2(n,State.m_deltax);
//--- check
if(!CAp::Assert(t!=0.0,__FUNCTION__+": internal error (T=0)"))
return(false);
for(i=0; i<m; i++)
{
v=CAblasF::RDotVR(n,State.m_deltax,State.m_j,i);
v=(State.m_deltaf[i]-v)/t;
State.m_j.Row(i,State.m_j[i]+State.m_deltax*v);
}
State.m_fi=State.m_fibase+State.m_deltaf+0;
//--- Increase model age
State.m_modelage++;
case 41:
CAblas::RMatrixGemm(n,n,m,2.0,State.m_j,0,0,1,State.m_j,0,0,0,0.0,State.m_quadraticmodel,0,0);
CAblas::RMatrixMVect(n,m,State.m_j,0,0,1,State.m_fi,0,State.m_gbase,0);
State.m_gbase*=2.0;
v=CAblasF::RDotV2(m,State.m_fi);
State.m_fbase=v;
State.m_fibase=State.m_fi;
State.m_deltaxready=false;
State.m_deltafready=false;
//--- Perform integrity check (presense of NAN/INF)
for(i=0; i<n; i++)
v=0.1*v+State.m_gbase[i];
if(!CMath::IsFinite(v))
{
//--- Break!
State.m_repterminationtype=-8;
return(false);
}
//--- If Lambda is not initialized, initialize it using quadratic model
if(State.m_lambdav<0.0)
{
State.m_lambdav=0;
for(i=0; i<n; i++)
State.m_lambdav=MathMax(State.m_lambdav,MathAbs(State.m_quadraticmodel.Get(i,i))*CMath::Sqr(State.m_s[i]));
State.m_lambdav=0.001*State.m_lambdav;
if(State.m_lambdav==0.0)
State.m_lambdav=1;
}
//--- Find value of Levenberg-Marquardt damping parameter which:
//--- * leads to positive definite damped model
//--- * within bounds specified by StpMax
//--- * generates step which decreases function value
//--- After this block IFlag is set to:
//--- * -8, if internal integrity control detected NAN/INF in function values
//--- * -3, if constraints are infeasible
//--- * -2, if model update is needed (either Lambda growth is too large
//--- or step is too short, but we can't rely on model and stop iterations)
//--- * -1, if model is fresh, Lambda have grown too large, termination is needed
//--- * 0, if everything is OK, continue iterations
//--- * >0, successful termination, step is less than EpsX
//--- State.Nu can have any value on enter, but after exit it is set to 1.0
iflag=-99;
MinLMStepFinderStart(State.m_finderstate,State.m_quadraticmodel,State.m_gbase,State.m_fbase,State.m_xbase,State.m_fibase,State.m_modelage);
case 47:
if(!MinLMStepFinderIteration(State.m_finderstate,State.m_lambdav,State.m_nu,State.m_xnew,State.m_deltax,State.m_deltaxready,State.m_deltaf,State.m_deltafready,iflag,fnew,State.m_repncholesky))
{
label=48;
break;
}
//--- check
if(!CAp::Assert(State.m_hasfi || State.m_hasf,__FUNCTION__+": internal error 2!"))
return(false);
State.m_repnfunc++;
ClearRequestFields(State);
if(!State.m_finderstate.m_needfi)
{
label=49;
break;
}
//--- check
if(!CAp::Assert(State.m_hasfi,__FUNCTION__+": internal error 2!"))
return(false);
State.m_x=State.m_finderstate.m_x;
State.m_needfi=true;
State.m_rstate.stage=8;
label=-1;
break;
case 8:
State.m_needfi=false;
State.m_finderstate.m_fi=State.m_fi;
label=47;
break;
case 49:
if(!State.m_finderstate.m_needf)
{
label=51;
break;
}
//--- check
if(!CAp::Assert(State.m_hasf,__FUNCTION__+": internal error 2!"))
return(false);
State.m_x=State.m_finderstate.m_x;
State.m_needf=true;
State.m_rstate.stage=9;
label=-1;
break;
case 9:
State.m_needf=false;
State.m_finderstate.m_f=State.m_f;
label=47;
break;
case 51:
CAp::Assert(false,__FUNCTION__+": internal error 2!");
return(false);
case 48:
if(State.m_userterminationneeded)
{
//--- User requested termination
State.m_x=State.m_xbase;
State.m_repterminationtype=8;
return(false);
}
State.m_nu=1;
//--- check
if(!CAp::Assert((iflag>=-3 && iflag<=0) || iflag==-8 || iflag>0,__FUNCTION__+": internal integrity check failed!"))
return(false);
if(iflag==-3)
{
State.m_repterminationtype=-3;
return(false);
}
if(iflag==-2)
{
State.m_modelage=State.m_maxmodelage+1;
label=38;
break;
}
if(iflag!=-1)
{
label=53;
break;
}
//--- Stopping conditions are too stringent
State.m_repterminationtype=7;
if(!State.m_xrep)
{
label=55;
break;
}
State.m_x=State.m_xbase;
State.m_f=State.m_fbase;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=10;
label=-1;
break;
case 10:
State.m_xupdated=false;
case 55:
return(false);
case 53:
if(!(iflag==-8 || iflag>0))
{
label=57;
break;
}
//--- Either:
//--- * Integrity check failed - infinities or NANs
//--- * successful termination (step size is small enough)
State.m_repterminationtype=iflag;
if(!State.m_xrep)
{
label=59;
break;
}
State.m_x=State.m_xbase;
State.m_f=State.m_fbase;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=11;
label=-1;
break;
case 11:
State.m_xupdated=false;
case 59:
return(false);
case 57:
State.m_f=fnew;
//--- Levenberg-Marquardt step is ready.
//--- Compare predicted vs. actual decrease and decide what to do with lambda.
//--- NOTE: we expect that State.DeltaX contains direction of step,
//--- State.F contains function value at new point.
//--- check
if(!CAp::Assert(State.m_deltaxready,__FUNCTION__+": deltaX is not ready"))
return(false);
iflag=CheckDecrease(State.m_quadraticmodel,State.m_gbase,State.m_fbase,n,State.m_deltax,State.m_f,State.m_lambdav,State.m_nu);
if(iflag==0)
{
label=61;
break;
}
State.m_repterminationtype=iflag;
if(!State.m_xrep)
{
label=63;
break;
}
State.m_x=State.m_xbase;
State.m_f=State.m_fbase;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=12;
label=-1;
break;
case 12:
State.m_xupdated=false;
case 63:
return(false);
case 61:
//--- Accept step, report it and
//--- test stopping conditions on iterations count and function decrease.
//--- NOTE: we expect that State.DeltaX contains direction of step,
//--- State.F contains function value at new point.
//--- NOTE2: we should update XBase ONLY. In the beginning of the next
//--- iteration we expect that State.FIBase is NOT updated and
//--- contains old value of a function vector.
State.m_xbase=State.m_xnew;
if(!State.m_xrep)
{
label=65;
break;
}
State.m_x=State.m_xbase;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=13;
label=-1;
break;
case 13:
State.m_xupdated=false;
case 65:
State.m_repiterationscount++;
if(State.m_repiterationscount>=State.m_maxits && State.m_maxits>0)
State.m_repterminationtype=5;
if(State.m_repterminationtype<=0)
{
label=67;
break;
}
if(!State.m_xrep)
{
label=69;
break;
}
//--- Report: XBase contains new point, F contains function value at new point
State.m_x=State.m_xbase;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=14;
label=-1;
break;
case 14:
State.m_xupdated=false;
case 69:
return(false);
case 67:
State.m_modelage=State.m_modelage+1;
label=38;
break;
case 39:
//--- Lambda is too large, we have to break iterations.
State.m_repterminationtype=7;
if(!State.m_xrep)
{
label=71;
break;
}
State.m_x=State.m_xbase;
State.m_f=State.m_fbase;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=15;
label=-1;
break;
case 15:
State.m_xupdated=false;
case 71:
label=37;
break;
case 36:
//--- Legacy Hessian-based mode
//--- Main cycle.
//--- We move through it until either:
//--- * one of the stopping conditions is met
//--- * we decide that stopping conditions are too stringent
//--- and break from cycle
if(State.m_nec+State.m_nic>0)
{
//--- FGH solver does not support general linear constraints
State.m_repterminationtype=-5;
return(false);
}
case 73:
//--- First, we have to prepare quadratic model for our function.
//--- We use BFlag to ensure that model is prepared;
//--- if it is false at the end of this block, something went wrong.
//--- We may either calculate brand new model or update old one.
//--- Before this block we have:
//--- * State.XBase - current position.
//--- * State.DeltaX - if DeltaXReady is True
//--- * State.DeltaF - if DeltaFReady is True
//--- After this block is over, we will have:
//--- * State.XBase - base point (unchanged)
//--- * State.FBase - F(XBase)
//--- * State.GBase - linear term
//--- * State.QuadraticModel - quadratic term
//--- * State.LambdaV - current estimate for lambda
//--- We also clear DeltaXReady/DeltaFReady flags
//--- after initialization is done.
bflag=false;
if(!(State.m_algomode==0 || State.m_algomode==1))
{
label=75;
break;
}
//--- Calculate f[] and Jacobian
if(!(State.m_modelage>State.m_maxmodelage || !(State.m_deltaxready && State.m_deltafready)))
{
label=77;
break;
}
//--- Refresh model (using either finite differences or analytic Jacobian)
if(State.m_algomode!=0)
{
label=79;
break;
}
//--- Optimization using F values only.
//--- Use finite differences to estimate Jacobian.
//--- check
if(!CAp::Assert(State.m_hasfi,__FUNCTION__+": internal error when estimating Jacobian (no f[])"))
return(false);
k=0;
case 81:
if(k>n-1)
{
label=83;
break;
}
//--- We guard X[k] from leaving [BndL,BndU].
//--- In case BndL=BndU, we assume that derivative in this direction is zero.
State.m_x=State.m_xbase;
State.m_x.Add(k,-State.m_s[k]*State.m_diffstep);
if(State.m_havebndl[k])
State.m_x.Set(k,MathMax(State.m_x[k],State.m_bndl[k]));
if(State.m_havebndu[k])
State.m_x.Set(k,MathMin(State.m_x[k],State.m_bndu[k]));
State.m_xm1=State.m_x[k];
ClearRequestFields(State);
State.m_needfi=true;
State.m_rstate.stage=16;
label=-1;
break;
case 16:
State.m_repnfunc++;
State.m_fm1=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Add(k,State.m_s[k]*State.m_diffstep);
if(State.m_havebndl[k])
State.m_x.Set(k,MathMax(State.m_x[k],State.m_bndl[k]));
if(State.m_havebndu[k])
State.m_x.Set(k,MathMin(State.m_x[k],State.m_bndu[k]));
State.m_xp1=State.m_x[k];
ClearRequestFields(State);
State.m_needfi=true;
State.m_rstate.stage=17;
label=-1;
break;
case 17:
State.m_repnfunc++;
State.m_fp1=State.m_fi;
v=State.m_xp1-State.m_xm1;
if(v!=0.0)
{
v=1/v;
State.m_j.Col(k,(State.m_fp1-State.m_fm1)*v);
}
else
State.m_j.Col(k,vector<double>::Zeros(m));
k++;
label=81;
break;
case 83:
//--- Calculate F(XBase)
State.m_x=State.m_xbase;
ClearRequestFields(State);
State.m_needfi=true;
State.m_rstate.stage=18;
label=-1;
break;
case 18:
State.m_needfi=false;
State.m_repnfunc++;
State.m_repnjac++;
//--- New model
State.m_modelage=0;
label=80;
break;
case 79:
//--- Obtain f[] and Jacobian
State.m_x=State.m_xbase;
ClearRequestFields(State);
State.m_needfij=true;
State.m_rstate.stage=19;
label=-1;
break;
case 19:
State.m_needfij=false;
State.m_repnfunc++;
State.m_repnjac++;
//--- New model
State.m_modelage=0;
case 80:
label=78;
break;
case 77:
//--- State.J contains Jacobian or its current approximation;
//--- refresh it using secant updates:
//--- f(x0+dx) = f(x0) + J*dx,
//--- J_new = J_old + u*h'
//--- h = x_new-x_old
//--- u = (f_new - f_old - J_old*h)/(h'h)
//--- We can explicitly generate h and u, but it is
//--- preferential to do in-place calculations. Only
//--- I-th row of J_old is needed to calculate u[I],
//--- so we can update J row by row in one pass.
//--- NOTE: we expect that State.XBase contains new point,
//--- State.FBase contains old point, State.DeltaX and
//--- State.DeltaY contain updates from last step.
//--- check
if(!CAp::Assert(State.m_deltaxready && State.m_deltafready,__FUNCTION__+": uninitialized DeltaX/DeltaF"))
return(false);
t=CAblasF::RDotV2(n,State.m_deltax);
//--- check
if(!CAp::Assert(t!=0.0,__FUNCTION__+": internal error (T=0)"))
return(false);
for(i=0; i<m; i++)
{
v=CAblasF::RDotVR(n,State.m_deltax,State.m_j,i);
v=(State.m_deltaf[i]-v)/t;
State.m_j.Row(i,State.m_j[i]+State.m_deltax*v);
}
State.m_fi=State.m_fibase+State.m_deltaf+0;
//--- Increase model age
State.m_modelage++;
case 78:
//--- Generate quadratic model:
//--- f(xbase+dx) =
//--- = (f0 + J*dx)'(f0 + J*dx)
//--- = f0^2 + dx'J'f0 + f0*J*dx + dx'J'J*dx
//--- = f0^2 + 2*f0*J*dx + dx'J'J*dx
//--- Note that we calculate 2*(J'J) instead of J'J because
//--- our quadratic model is based on Tailor decomposition,
//--- i.e. it has 0.5 before quadratic term.
CAblas::RMatrixGemm(n,n,m,2.0,State.m_j,0,0,1,State.m_j,0,0,0,0.0,State.m_quadraticmodel,0,0);
CAblas::RMatrixMVect(n,m,State.m_j,0,0,1,State.m_fi,0,State.m_gbase,0);
State.m_gbase*=2;
v=CAblasF::RDotV2(m,State.m_fi);
State.m_fbase=v;
State.m_fibase=State.m_fi;
//--- set control variables
bflag=true;
case 75:
if(State.m_algomode!=2)
{
label=84;
break;
}
//--- check
if(!CAp::Assert(!State.m_hasfi,__FUNCTION__+": internal error (HasFI is True in Hessian-based mode)"))
return(false);
//--- Obtain F, G, H
State.m_x=State.m_xbase;
ClearRequestFields(State);
State.m_needfgh=true;
State.m_rstate.stage=20;
label=-1;
break;
case 20:
State.m_needfgh=false;
State.m_repnfunc++;
State.m_repngrad++;
State.m_repnhess++;
CAblas::RMatrixCopy(n,n,State.m_h,0,0,State.m_quadraticmodel,0,0);
State.m_gbase=State.m_g;
State.m_fbase=State.m_f;
//--- set control variables
bflag=true;
State.m_modelage=0;
case 84:
//--- check
if(!CAp::Assert(bflag,__FUNCTION__+": internal integrity check failed!"))
return(false);
State.m_deltaxready=false;
State.m_deltafready=false;
//--- Perform integrity check (presense of NAN/INF)
v=State.m_fbase;
for(i=0; i<n; i++)
v=0.1*v+State.m_gbase[i];
if(!CMath::IsFinite(v))
{
//--- Break!
State.m_repterminationtype=-8;
return(false);
}
//--- If Lambda is not initialized, initialize it using quadratic model
if(State.m_lambdav<0.0)
{
State.m_lambdav=0;
for(i=0; i<n; i++)
State.m_lambdav=MathMax(State.m_lambdav,MathAbs(State.m_quadraticmodel.Get(i,i))*CMath::Sqr(State.m_s[i]));
State.m_lambdav=0.001*State.m_lambdav;
if(State.m_lambdav==0.0)
State.m_lambdav=1;
}
//--- Find value of Levenberg-Marquardt damping parameter which:
//--- * leads to positive definite damped model
//--- * within bounds specified by StpMax
//--- * generates step which decreases function value
//--- After this block IFlag is set to:
//--- * -3, if constraints are infeasible
//--- * -2, if model update is needed (either Lambda growth is too large
//--- or step is too short, but we can't rely on model and stop iterations)
//--- * -1, if model is fresh, Lambda have grown too large, termination is needed
//--- * 0, if everything is OK, continue iterations
//--- State.Nu can have any value on enter, but after exit it is set to 1.0
iflag=-99;
case 86:
//--- Do we need model update?
if(State.m_modelage>0 && State.m_nu>=m_suspiciousnu)
{
iflag=-2;
label=87;
break;
}
//--- Setup quadratic solver and solve quadratic programming problem.
//--- After problem is solved we'll try to bound step by StpMax
//--- (Lambda will be increased if step size is too large).
//--- We use BFlag variable to indicate that we have to increase Lambda.
//--- If it is False, we will try to increase Lambda and move to new iteration.
bflag=true;
CMinQP::MinQPSetStartingPointFast(State.m_qpstate,State.m_xbase);
CMinQP::MinQPSetOriginFast(State.m_qpstate,State.m_xbase);
CMinQP::MinQPSetLinearTermFast(State.m_qpstate,State.m_gbase);
CMinQP::MinQPSetQuadraticTermFast(State.m_qpstate,State.m_quadraticmodel,true,0.0);
State.m_tmp0=State.m_quadraticmodel.Diag()+State.m_s.Pow(-2.0)*State.m_lambdav;
CMinQP::MinQPRewriteDiagonal(State.m_qpstate,State.m_tmp0);
CMinQP::MinQPOptimize(State.m_qpstate);
CMinQP::MinQPResultsBuf(State.m_qpstate,State.m_xdir,State.m_qprep);
if(State.m_qprep.m_terminationtype>0)
{
//--- successful solution of QP problem
State.m_xdir-=State.m_xbase;
v=CAblasF::RDotV2(n,State.m_xdir);
if(CMath::IsFinite(v))
{
v=MathSqrt(v);
if(State.m_stpmax>0.0 && v>State.m_stpmax)
bflag=false;
}
else
bflag=false;
}
else
{
//--- Either problem is non-convex (increase LambdaV) or constraints are inconsistent
//--- check
if(!CAp::Assert(State.m_qprep.m_terminationtype==-3 || State.m_qprep.m_terminationtype==-4 || State.m_qprep.m_terminationtype==-5,__FUNCTION__+": unexpected completion code from QP solver"))
return(false);
if(State.m_qprep.m_terminationtype==-3)
{
iflag=-3;
label=87;
break;
}
bflag=false;
}
if(!bflag)
{
//--- Solution failed:
//--- try to increase lambda to make matrix positive definite and continue.
if(!IncreaseLambda(State.m_lambdav,State.m_nu))
{
iflag=-1;
label=87;
break;
}
label=86;
break;
}
//--- Step in State.XDir and it is bounded by StpMax.
//--- We should check stopping conditions on step size here.
//--- DeltaX, which is used for secant updates, is initialized here.
//--- This code is a bit tricky because sometimes XDir<>0, but
//--- it is so small that XDir+XBase==XBase (in finite precision
//--- arithmetics). So we set DeltaX to XBase, then
//--- add XDir, and then subtract XBase to get exact value of
//--- DeltaX.
//--- Step length is estimated using DeltaX.
//--- NOTE: stopping conditions are tested
//--- for fresh models only (ModelAge=0)
State.m_deltax=State.m_xdir;
State.m_deltaxready=true;
v=MathPow(State.m_deltax/State.m_s+0,2.0).Sum();
v=MathSqrt(v);
if(v>State.m_epsx)
{
label=88;
break;
}
if(State.m_modelage!=0)
{
label=90;
break;
}
//--- Step is too short, model is fresh and we can rely on it.
//--- Terminating.
State.m_repterminationtype=2;
if(!State.m_xrep)
{
label=92;
break;
}
State.m_x=State.m_xbase;
State.m_f=State.m_fbase;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=21;
label=-1;
break;
case 21:
State.m_xupdated=false;
case 92:
return(false);
case 90:
//--- Step is suspiciously short, but model is not fresh
//--- and we can't rely on it.
iflag=-2;
label=87;
break;
case 91:
case 88:
//--- Let's evaluate new step:
//--- a) if we have Fi vector, we evaluate it using rcomm, and
//--- then we manually calculate State.F as sum of squares of Fi[]
//--- b) if we have F value, we just evaluate it through rcomm interface
//--- We prefer (a) because we may need Fi vector for additional
//--- iterations
//--- check
if(!CAp::Assert(State.m_hasfi || State.m_hasf,__FUNCTION__+": internal error 2!"))
return(false);
State.m_x=State.m_xbase+State.m_xdir+0;
ClearRequestFields(State);
if(!State.m_hasfi)
{
label=94;
break;
}
State.m_needfi=true;
State.m_rstate.stage=22;
label=-1;
break;
case 22:
State.m_needfi=false;
v=CAblasF::RDotV2(m,State.m_fi);
State.m_f=v;
State.m_deltaf=State.m_fi-State.m_fibase+0;
State.m_deltafready=true;
label=95;
break;
case 94:
State.m_needf=true;
State.m_rstate.stage=23;
label=-1;
break;
case 23:
State.m_needf=false;
case 95:
State.m_repnfunc++;
if(!CMath::IsFinite(State.m_f))
{
//--- Integrity check failed, break!
State.m_repterminationtype=-8;
return(false);
}
if(State.m_f>=State.m_fbase)
{
//--- Increase lambda and continue
if(!IncreaseLambda(State.m_lambdav,State.m_nu))
{
iflag=-1;
label=87;
break;
}
label=86;
break;
}
//--- We've found our step!
iflag=0;
case 87:
if(State.m_userterminationneeded)
{
//--- User requested termination
State.m_x=State.m_xbase;
State.m_repterminationtype=8;
return(false);
}
State.m_nu=1;
//--- check
if(!CAp::Assert(iflag>=-3 && iflag<=0,__FUNCTION__+": internal integrity check failed!"))
return(false);
if(iflag==-3)
{
State.m_repterminationtype=-3;
return(false);
}
if(iflag==-2)
{
State.m_modelage=State.m_maxmodelage+1;
label=73;
break;
}
if(iflag==-1)
{
label=74;
break;
}
//--- Levenberg-Marquardt step is ready.
//--- Compare predicted vs. actual decrease and decide what to do with lambda.
//--- NOTE: we expect that State.DeltaX contains direction of step,
//--- State.F contains function value at new point.
//--- check
if(!CAp::Assert(State.m_deltaxready,__FUNCTION__+": deltaX is not ready"))
return(false);
t=0;
for(i=0; i<=n-1; i++)
{
v=CAblasF::RDotVR(n,State.m_deltax,State.m_quadraticmodel,i);
t+=State.m_deltax[i]*(State.m_gbase[i]+0.5*v);
}
State.m_predicteddecrease=-t;
State.m_actualdecrease=-(State.m_f-State.m_fbase);
if(State.m_predicteddecrease<=0.0)
{
label=74;
break;
}
v=State.m_actualdecrease/State.m_predicteddecrease;
if(v>=0.1)
{
label=96;
break;
}
if(IncreaseLambda(State.m_lambdav,State.m_nu))
{
label=98;
break;
}
//--- Lambda is too large, we have to break iterations.
State.m_repterminationtype=7;
if(!State.m_xrep)
{
label=100;
break;
}
State.m_x=State.m_xbase;
State.m_f=State.m_fbase;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=24;
label=-1;
break;
case 24:
State.m_xupdated=false;
case 100:
return(false);
case 98:
case 96:
if(v>0.5)
DecreaseLambda(State.m_lambdav,State.m_nu);
//--- Accept step, report it and
//--- test stopping conditions on iterations count and function decrease.
//--- NOTE: we expect that State.DeltaX contains direction of step,
//--- State.F contains function value at new point.
//--- NOTE2: we should update XBase ONLY. In the beginning of the next
//--- iteration we expect that State.FIBase is NOT updated and
//--- contains old value of a function vector.
State.m_xbase+=State.m_deltax;
if(!State.m_xrep)
{
label=102;
break;
}
State.m_x=State.m_xbase;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=25;
label=-1;
break;
case 25:
State.m_xupdated=false;
case 102:
State.m_repiterationscount++;
if(State.m_repiterationscount>=State.m_maxits && State.m_maxits>0)
State.m_repterminationtype=5;
if(State.m_repterminationtype<=0)
{
label=104;
break;
}
if(!State.m_xrep)
{
label=106;
break;
}
//--- Report: XBase contains new point, F contains function value at new point
State.m_x=State.m_xbase;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=26;
label=-1;
break;
case 26:
State.m_xupdated=false;
case 106:
return(false);
case 104:
State.m_modelage++;
label=73;
break;
case 74:
//--- Lambda is too large, we have to break iterations.
State.m_repterminationtype=7;
if(!State.m_xrep)
{
label=108;
break;
}
State.m_x=State.m_xbase;
State.m_f=State.m_fbase;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=27;
label=-1;
break;
case 27:
State.m_xupdated=false;
case 108:
case 37:
return(false);
}
//--- Saving State
State.m_rstate.ba[0]=bflag;
State.m_rstate.ia.Set(0,n);
State.m_rstate.ia.Set(1,m);
State.m_rstate.ia.Set(2,iflag);
State.m_rstate.ia.Set(3,i);
State.m_rstate.ia.Set(4,k);
State.m_rstate.ra.Set(0,v);
State.m_rstate.ra.Set(1,s);
State.m_rstate.ra.Set(2,t);
State.m_rstate.ra.Set(3,fnew);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary class for CMinComp |
//+------------------------------------------------------------------+
class CMinASAState
{
public:
//--- variables
int m_n;
double m_epsg;
double m_epsf;
double m_epsx;
int m_maxits;
bool m_xrep;
double m_stpmax;
int m_cgtype;
int m_k;
int m_nfev;
int m_mcstage;
int m_curalgo;
int m_acount;
double m_mu;
double m_finit;
double m_dginit;
double m_fold;
double m_stp;
double m_laststep;
double m_f;
bool m_needfg;
bool m_xupdated;
RCommState m_rstate;
int m_repiterationscount;
int m_repnfev;
int m_repterminationtype;
int m_debugrestartscount;
CLinMinState m_lstate;
double m_betahs;
double m_betady;
//--- arrays
double m_bndl[];
double m_bndu[];
double m_ak[];
double m_xk[];
double m_dk[];
double m_an[];
double m_xn[];
double m_dn[];
double m_d[];
double m_work[];
double m_yk[];
double m_gc[];
double m_x[];
double m_g[];
//--- constructor, destructor
CMinASAState(void);
~CMinASAState(void) {}
//--- copy
void Copy(CMinASAState &obj);
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinASAState::CMinASAState(void)
{
m_n=0;
m_epsg=0;
m_epsf=0;
m_epsx=0;
m_maxits=0;
m_xrep=false;
m_stpmax=0;
m_cgtype=0;
m_k=0;
m_nfev=0;
m_mcstage=0;
m_curalgo=0;
m_acount=0;
m_mu=0;
m_finit=0;
m_dginit=0;
m_fold=0;
m_stp=0;
m_laststep=0;
m_f=0;
m_needfg=false;
m_xupdated=false;
m_repiterationscount=0;
m_repnfev=0;
m_repterminationtype=0;
m_debugrestartscount=0;
m_betahs=0;
m_betady=0;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinASAState::Copy(CMinASAState &obj)
{
//--- copy variables
m_n=obj.m_n;
m_epsg=obj.m_epsg;
m_epsf=obj.m_epsf;
m_epsx=obj.m_epsx;
m_maxits=obj.m_maxits;
m_xrep=obj.m_xrep;
m_stpmax=obj.m_stpmax;
m_cgtype=obj.m_cgtype;
m_k=obj.m_k;
m_nfev=obj.m_nfev;
m_mcstage=obj.m_mcstage;
m_curalgo=obj.m_curalgo;
m_acount=obj.m_acount;
m_mu=obj.m_mu;
m_finit=obj.m_finit;
m_dginit=obj.m_dginit;
m_fold=obj.m_fold;
m_stp=obj.m_stp;
m_laststep=obj.m_laststep;
m_f=obj.m_f;
m_needfg=obj.m_needfg;
m_xupdated=obj.m_xupdated;
m_repiterationscount=obj.m_repiterationscount;
m_repnfev=obj.m_repnfev;
m_repterminationtype=obj.m_repterminationtype;
m_debugrestartscount=obj.m_debugrestartscount;
m_betahs=obj.m_betahs;
m_betady=obj.m_betady;
m_rstate.Copy(obj.m_rstate);
m_lstate.Copy(obj.m_lstate);
//--- copy arrays
ArrayCopy(m_bndl,obj.m_bndl);
ArrayCopy(m_bndu,obj.m_bndu);
ArrayCopy(m_ak,obj.m_ak);
ArrayCopy(m_xk,obj.m_xk);
ArrayCopy(m_dk,obj.m_dk);
ArrayCopy(m_an,obj.m_an);
ArrayCopy(m_xn,obj.m_xn);
ArrayCopy(m_dn,obj.m_dn);
ArrayCopy(m_d,obj.m_d);
ArrayCopy(m_work,obj.m_work);
ArrayCopy(m_yk,obj.m_yk);
ArrayCopy(m_gc,obj.m_gc);
ArrayCopy(m_x,obj.m_x);
ArrayCopy(m_g,obj.m_g);
}
//+------------------------------------------------------------------+
//| This class is a shell for class CMinASAState |
//+------------------------------------------------------------------+
class CMinASAStateShell
{
private:
CMinASAState m_innerobj;
public:
//--- constructors, destructor
CMinASAStateShell(void) {}
CMinASAStateShell(CMinASAState &obj) { m_innerobj.Copy(obj); }
~CMinASAStateShell(void) {}
//--- methods
bool GetNeedFG(void);
void SetNeedFG(const bool b);
bool GetXUpdated(void);
void SetXUpdated(const bool b);
double GetF(void);
void SetF(const double d);
CMinASAState *GetInnerObj(void);
};
//+------------------------------------------------------------------+
//| Returns the value of the variable needfg |
//+------------------------------------------------------------------+
bool CMinASAStateShell::GetNeedFG(void)
{
return(m_innerobj.m_needfg);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable needfg |
//+------------------------------------------------------------------+
void CMinASAStateShell::SetNeedFG(const bool b)
{
m_innerobj.m_needfg=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable xupdated |
//+------------------------------------------------------------------+
bool CMinASAStateShell::GetXUpdated(void)
{
return(m_innerobj.m_xupdated);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable xupdated |
//+------------------------------------------------------------------+
void CMinASAStateShell::SetXUpdated(const bool b)
{
m_innerobj.m_xupdated=b;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable f |
//+------------------------------------------------------------------+
double CMinASAStateShell::GetF(void)
{
return(m_innerobj.m_f);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable f |
//+------------------------------------------------------------------+
void CMinASAStateShell::SetF(const double d)
{
m_innerobj.m_f=d;
}
//+------------------------------------------------------------------+
//| Return object of class |
//+------------------------------------------------------------------+
CMinASAState *CMinASAStateShell::GetInnerObj(void)
{
return(GetPointer(m_innerobj));
}
//+------------------------------------------------------------------+
//| Auxiliary class for CMinComp |
//+------------------------------------------------------------------+
class CMinASAReport
{
public:
//--- variables
int m_iterationscount;
int m_nfev;
int m_terminationtype;
int m_activeconstraints;
//--- constructor, destructor
CMinASAReport(void) { ZeroMemory(this); }
~CMinASAReport(void) {}
//--- copy
void Copy(CMinASAReport &obj);
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinASAReport::Copy(CMinASAReport &obj)
{
//--- copy variables
m_iterationscount=obj.m_iterationscount;
m_nfev=obj.m_nfev;
m_terminationtype=obj.m_terminationtype;
m_activeconstraints=obj.m_activeconstraints;
}
//+------------------------------------------------------------------+
//| This class is a shell for class CMinASAReport |
//+------------------------------------------------------------------+
class CMinASAReportShell
{
private:
CMinASAReport m_innerobj;
public:
//--- constructors, destructor
CMinASAReportShell(void) {}
CMinASAReportShell(CMinASAReport &obj) { m_innerobj.Copy(obj); }
~CMinASAReportShell(void) {}
//--- methods
int GetIterationsCount(void);
void SetIterationsCount(const int i);
int GetNFev(void);
void SetNFev(const int i);
int GetTerminationType(void);
void SetTerminationType(const int i);
int GetActiveConstraints(void);
void SetActiveConstraints(const int i);
CMinASAReport *GetInnerObj(void);
};
//+------------------------------------------------------------------+
//| Returns the value of the variable iterationscount |
//+------------------------------------------------------------------+
int CMinASAReportShell::GetIterationsCount(void)
{
return(m_innerobj.m_iterationscount);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable iterationscount |
//+------------------------------------------------------------------+
void CMinASAReportShell::SetIterationsCount(const int i)
{
m_innerobj.m_iterationscount=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable nfev |
//+------------------------------------------------------------------+
int CMinASAReportShell::GetNFev(void)
{
return(m_innerobj.m_nfev);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable nfev |
//+------------------------------------------------------------------+
void CMinASAReportShell::SetNFev(const int i)
{
m_innerobj.m_nfev=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable terminationtype |
//+------------------------------------------------------------------+
int CMinASAReportShell::GetTerminationType(void)
{
return(m_innerobj.m_terminationtype);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable terminationtype |
//+------------------------------------------------------------------+
void CMinASAReportShell::SetTerminationType(const int i)
{
m_innerobj.m_terminationtype=i;
}
//+------------------------------------------------------------------+
//| Returns the value of the variable activeconstraints |
//+------------------------------------------------------------------+
int CMinASAReportShell::GetActiveConstraints(void)
{
return(m_innerobj.m_activeconstraints);
}
//+------------------------------------------------------------------+
//| Changing the value of the variable activeconstraints |
//+------------------------------------------------------------------+
void CMinASAReportShell::SetActiveConstraints(const int i)
{
m_innerobj.m_activeconstraints=i;
}
//+------------------------------------------------------------------+
//| Return object of class |
//+------------------------------------------------------------------+
CMinASAReport *CMinASAReportShell::GetInnerObj(void)
{
return(GetPointer(m_innerobj));
}
//+------------------------------------------------------------------+
//| Backward compatibility functions |
//+------------------------------------------------------------------+
class CMinComp
{
public:
//--- class constants
static const int m_n1;
static const int m_n2;
static const double m_stpmin;
static const double m_gtol;
static const double m_gpaftol;
static const double m_gpadecay;
static const double m_asarho;
//--- public methods
static void MinLBFGSSetDefaultPreconditioner(CMinLBFGSState &State);
static void MinLBFGSSetCholeskyPreconditioner(CMinLBFGSState &State,CMatrixDouble &p,const bool IsUpper);
static void MinBLEICSetBarrierWidth(CMinBLEICState &State,const double mu);
static void MinBLEICSetBarrierDecay(CMinBLEICState &State,const double mudecay);
static void MinASACreate(const int n,double &x[],double &bndl[],double &bndu[],CMinASAState &State);
static void MinASASetCond(CMinASAState &State,const double epsg,const double epsf,double epsx,const int m_maxits);
static void MinASASetXRep(CMinASAState &State,const bool needxrep);
static void MinASASetAlgorithm(CMinASAState &State,int algotype);
static void MinASASetStpMax(CMinASAState &State,const double stpmax);
static void MinASAResults(CMinASAState &State,double &x[],CMinASAReport &rep);
static void MinASAResultsBuf(CMinASAState &State,double &x[],CMinASAReport &rep);
static void MinASARestartFrom(CMinASAState &State,double &x[],double &bndl[],double &bndu[]);
static bool MinASAIteration(CMinASAState &State);
private:
//--- private methods
static double ASABoundedAntigradNorm(CMinASAState &State);
static double ASAGINorm(CMinASAState &State);
static double ASAD1Norm(CMinASAState &State);
static bool ASAUIsEmpty(CMinASAState &State);
static void ClearRequestFields(CMinASAState &State);
//--- auxiliary functions for MinASAIteration
static void Func_lbl_rcomm(CMinASAState &State,int n,int i,int mcinfo,int diffcnt,bool b,bool stepfound,double betak,double v,double vv);
static bool Func_lbl_15(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_17(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_19(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_21(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_24(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_26(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_27(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_29(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_31(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_35(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_39(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_43(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_49(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_51(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_52(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_53(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_55(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_59(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_63(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
static bool Func_lbl_65(CMinASAState &State,int &n,int &i,int &mcinfo,int &diffcnt,bool &b,bool &stepfound,double &betak,double &v,double &vv);
};
//+------------------------------------------------------------------+
//| Initialize constants |
//+------------------------------------------------------------------+
const int CMinComp::m_n1=2;
const int CMinComp::m_n2=2;
const double CMinComp::m_stpmin=1.0E-300;
const double CMinComp::m_gtol=0.3;
const double CMinComp::m_gpaftol=0.0001;
const double CMinComp::m_gpadecay=0.5;
const double CMinComp::m_asarho=0.5;
//+------------------------------------------------------------------+
//| Obsolete function, use MinLBFGSSetPrecDefault() instead. |
//+------------------------------------------------------------------+
void CMinComp::MinLBFGSSetDefaultPreconditioner(CMinLBFGSState &State)
{
CMinLBFGS::MinLBFGSSetPrecDefault(State);
}
//+------------------------------------------------------------------+
//| Obsolete function, use MinLBFGSSetCholeskyPreconditioner() |
//| instead. |
//+------------------------------------------------------------------+
void CMinComp::MinLBFGSSetCholeskyPreconditioner(CMinLBFGSState &State,
CMatrixDouble &p,
const bool IsUpper)
{
CMinLBFGS::MinLBFGSSetPrecCholesky(State,p,IsUpper);
}
//+------------------------------------------------------------------+
//| This is obsolete function which was used by previous version of |
//| the BLEIC optimizer. It does nothing in the current version of |
//| BLEIC. |
//+------------------------------------------------------------------+
void CMinComp::MinBLEICSetBarrierWidth(CMinBLEICState &State,
const double mu)
{
}
//+------------------------------------------------------------------+
//| This is obsolete function which was used by previous version of |
//| the BLEIC optimizer. It does nothing in the current version of |
//| BLEIC. |
//+------------------------------------------------------------------+
void CMinComp::MinBLEICSetBarrierDecay(CMinBLEICState &State,
const double mudecay)
{
}
//+------------------------------------------------------------------+
//| Obsolete optimization algorithm. |
//| Was replaced by MinBLEIC subpackage. |
//+------------------------------------------------------------------+
void CMinComp::MinASACreate(const int n,double &x[],double &bndl[],
double &bndu[],CMinASAState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N too small!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(bndl)>=n,__FUNCTION__+": Length(BndL)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(bndl,n),__FUNCTION__+": BndL contains infinite or NaN values!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(bndu)>=n,__FUNCTION__+": Length(BndU)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(bndu,n),__FUNCTION__+": BndU contains infinite or NaN values!"))
return;
for(int i=0; i<n; i++)
{
//--- check
if(!CAp::Assert(bndl[i]<=bndu[i],__FUNCTION__+": inconsistent bounds!"))
return;
//--- check
if(!CAp::Assert(bndl[i]<=x[i],__FUNCTION__+": infeasible X!"))
return;
//--- check
if(!CAp::Assert(x[i]<=bndu[i],__FUNCTION__+": infeasible X!"))
return;
}
//--- Initialize
State.m_n=n;
MinASASetCond(State,0,0,0,0);
MinASASetXRep(State,false);
MinASASetStpMax(State,0);
MinASASetAlgorithm(State,-1);
//--- allocation
ArrayResize(State.m_bndl,n);
ArrayResize(State.m_bndu,n);
ArrayResize(State.m_ak,n);
ArrayResize(State.m_xk,n);
ArrayResize(State.m_dk,n);
ArrayResize(State.m_an,n);
ArrayResize(State.m_xn,n);
ArrayResize(State.m_dn,n);
ArrayResize(State.m_x,n);
ArrayResize(State.m_d,n);
ArrayResize(State.m_g,n);
ArrayResize(State.m_gc,n);
ArrayResize(State.m_work,n);
ArrayResize(State.m_yk,n);
//--- function call
MinASARestartFrom(State,x,bndl,bndu);
}
//+------------------------------------------------------------------+
//| Obsolete optimization algorithm. |
//| Was replaced by MinBLEIC subpackage. |
//+------------------------------------------------------------------+
void CMinComp::MinASASetCond(CMinASAState &State,const double epsg,
const double epsf,double epsx,
const int m_maxits)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(epsg),__FUNCTION__+": EpsG is not finite number!"))
return;
//--- check
if(!CAp::Assert(epsg>=0.0,__FUNCTION__+": negative EpsG!"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(epsf),__FUNCTION__+": EpsF is not finite number!"))
return;
//--- check
if(!CAp::Assert(epsf>=0.0,__FUNCTION__+": negative EpsF!"))
return;
//--- check
if(!CAp::Assert(CMath::IsFinite(epsx),__FUNCTION__+": EpsX is not finite number!"))
return;
//--- check
if(!CAp::Assert(epsx>=0.0,__FUNCTION__+": negative EpsX!"))
return;
//--- check
if(!CAp::Assert(m_maxits>=0,__FUNCTION__+": negative MaxIts!"))
return;
//--- check
if(((epsg==0.0 && epsf==0.0) && epsx==0.0) && m_maxits==0)
epsx=1.0E-6;
//--- change values
State.m_epsg=epsg;
State.m_epsf=epsf;
State.m_epsx=epsx;
State.m_maxits=m_maxits;
}
//+------------------------------------------------------------------+
//| Obsolete optimization algorithm. |
//| Was replaced by MinBLEIC subpackage. |
//+------------------------------------------------------------------+
void CMinComp::MinASASetXRep(CMinASAState &State,const bool needxrep)
{
State.m_xrep=needxrep;
}
//+------------------------------------------------------------------+
//| Obsolete optimization algorithm. |
//| Was replaced by MinBLEIC subpackage. |
//+------------------------------------------------------------------+
void CMinComp::MinASASetAlgorithm(CMinASAState &State,int algotype)
{
//--- check
if(!CAp::Assert(algotype>=-1 && algotype<=1,__FUNCTION__+": incorrect AlgoType!"))
return;
//--- check
if(algotype==-1)
algotype=1;
//--- change value
State.m_cgtype=algotype;
}
//+------------------------------------------------------------------+
//| Obsolete optimization algorithm. |
//| Was replaced by MinBLEIC subpackage. |
//+------------------------------------------------------------------+
void CMinComp::MinASASetStpMax(CMinASAState &State,const double stpmax)
{
//--- check
if(!CAp::Assert(CMath::IsFinite(stpmax),__FUNCTION__+": StpMax is not finite!"))
return;
//--- check
if(!CAp::Assert(stpmax>=0.0,__FUNCTION__+": StpMax<0!"))
return;
//--- change value
State.m_stpmax=stpmax;
}
//+------------------------------------------------------------------+
//| Obsolete optimization algorithm. |
//| Was replaced by MinBLEIC subpackage. |
//+------------------------------------------------------------------+
void CMinComp::MinASAResults(CMinASAState &State,double &x[],CMinASAReport &rep)
{
//--- reset memory
ArrayResize(x,0);
//--- function call
MinASAResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| Obsolete optimization algorithm. |
//| Was replaced by MinBLEIC subpackage. |
//+------------------------------------------------------------------+
void CMinComp::MinASAResultsBuf(CMinASAState &State,double &x[],
CMinASAReport &rep)
{
//--- check
if(CAp::Len(x)<State.m_n)
ArrayResize(x,State.m_n);
//--- copy
for(int i_=0; i_<State.m_n; i_++)
x[i_]=State.m_x[i_];
//--- change values
rep.m_iterationscount=State.m_repiterationscount;
rep.m_nfev=State.m_repnfev;
rep.m_terminationtype=State.m_repterminationtype;
rep.m_activeconstraints=0;
for(int i=0; i<State.m_n; i++)
{
//--- check
if(State.m_ak[i]==0.0)
rep.m_activeconstraints=rep.m_activeconstraints+1;
}
}
//+------------------------------------------------------------------+
//| Obsolete optimization algorithm. |
//| Was replaced by MinBLEIC subpackage. |
//+------------------------------------------------------------------+
void CMinComp::MinASARestartFrom(CMinASAState &State,double &x[],double &bndl[],double &bndu[])
{
//--- create a variable
int i_=0;
//--- check
if(!CAp::Assert(CAp::Len(x)>=State.m_n,__FUNCTION__+": Length(X)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(x,State.m_n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(bndl)>=State.m_n,__FUNCTION__+": Length(BndL)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(bndl,State.m_n),__FUNCTION__+": BndL contains infinite or NaN values!"))
return;
//--- check
if(!CAp::Assert(CAp::Len(bndu)>=State.m_n,__FUNCTION__+": Length(BndU)<N!"))
return;
//--- check
if(!CAp::Assert(CApServ::IsFiniteVector(bndu,State.m_n),__FUNCTION__+": BndU contains infinite or NaN values!"))
return;
//--- copy
for(i_=0; i_<State.m_n; i_++)
State.m_x[i_]=x[i_];
for(i_=0; i_<State.m_n; i_++)
State.m_bndl[i_]=bndl[i_];
for(i_=0; i_<State.m_n; i_++)
State.m_bndu[i_]=bndu[i_];
State.m_laststep=0;
//--- allocation
State.m_rstate.ia.Resize(4);
ArrayResizeAL(State.m_rstate.ba,2);
State.m_rstate.ra.Resize(3);
State.m_rstate.stage=-1;
//--- function call
ClearRequestFields(State);
}
//+------------------------------------------------------------------+
//| Returns norm of bounded anti-gradient. |
//| Bounded antigradient is a vector obtained from anti-gradient by |
//| zeroing components which point outwards: |
//| result = norm(v) |
//| v[i]=0 if ((-g[i]<0)and(x[i]=bndl[i])) or |
//| ((-g[i]>0)and(x[i]=bndu[i])) |
//| v[i]=-g[i] otherwise |
//| This function may be used to check a stopping criterion. |
//+------------------------------------------------------------------+
double CMinComp::ASABoundedAntigradNorm(CMinASAState &State)
{
//--- create variables
double result=0;
double v=0;
for(int i=0; i<State.m_n; i++)
{
v=-State.m_g[i];
//--- check
if(State.m_x[i]==State.m_bndl[i] && -State.m_g[i]<0.0)
v=0;
//--- check
if(State.m_x[i]==State.m_bndu[i] && -State.m_g[i]>0.0)
v=0;
result=result+CMath::Sqr(v);
}
//--- return result
return(MathSqrt(result));
}
//+------------------------------------------------------------------+
//| Returns norm of GI(x). |
//| GI(x) is a gradient vector whose components associated with |
//| active constraints are zeroed. It differs from bounded |
//| anti-gradient because components of GI(x) are zeroed |
//| independently of sign(g[i]), and anti-gradient's components are |
//| zeroed with respect to both constraint and sign. |
//+------------------------------------------------------------------+
double CMinComp::ASAGINorm(CMinASAState &State)
{
double result=0;
for(int i=0; i<State.m_n; i++)
{
//--- check
if(State.m_x[i]!=State.m_bndl[i] && State.m_x[i]!=State.m_bndu[i])
result=result+CMath::Sqr(State.m_g[i]);
}
//--- return result
return(MathSqrt(result));
}
//+------------------------------------------------------------------+
//| Returns norm(D1(State.X)) |
//| For a meaning of D1 see 'NEW ACTIVE SET ALGORITHM FOR BOX |
//| CONSTRAINED OPTIMIZATION' by WILLIAM W. HAGER AND HONGCHAO ZHANG.|
//+------------------------------------------------------------------+
double CMinComp::ASAD1Norm(CMinASAState &State)
{
double result=0;
for(int i=0; i<State.m_n; i++)
result=result+CMath::Sqr(CApServ::BoundVal(State.m_x[i]-State.m_g[i],State.m_bndl[i],State.m_bndu[i])-State.m_x[i]);
//--- return result
return(MathSqrt(result));
}
//+------------------------------------------------------------------+
//| Returns True, if U set is empty. |
//| * State.X is used as point, |
//| * State.G - as gradient, |
//| * D is calculated within function (because State.D may have |
//| different meaning depending on current optimization algorithm) |
//| For a meaning of U see 'NEW ACTIVE SET ALGORITHM FOR BOX |
//| CONSTRAINED OPTIMIZATION' by WILLIAM W. HAGER AND HONGCHAO ZHANG.|
//+------------------------------------------------------------------+
bool CMinComp::ASAUIsEmpty(CMinASAState &State)
{
//--- create variables
double d=ASAD1Norm(State);
double d2=MathSqrt(d);
double d32=d*d2;
for(int i=0; i<State.m_n; i++)
{
//--- check
if(MathAbs(State.m_g[i])>=d2 && MathMin(State.m_x[i]-State.m_bndl[i],State.m_bndu[i]-State.m_x[i])>=d32)
return(false);
}
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Clears request fileds (to be sure that we don't forgot to clear |
//| something) |
//+------------------------------------------------------------------+
void CMinComp::ClearRequestFields(CMinASAState &State)
{
//--- change values
State.m_needfg=false;
State.m_xupdated=false;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
bool CMinComp::MinASAIteration(CMinASAState &State)
{
//--- create variables
int n=0;
int i=0;
double betak=0;
double v=0;
double vv=0;
int mcinfo=0;
bool b;
bool stepfound;
int diffcnt=0;
int i_=0;
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(State.m_rstate.stage>=0)
{
//--- initialization
n=State.m_rstate.ia[0];
i=State.m_rstate.ia[1];
mcinfo=State.m_rstate.ia[2];
diffcnt=State.m_rstate.ia[3];
b=State.m_rstate.ba[0];
stepfound=State.m_rstate.ba[1];
betak=State.m_rstate.ra[0];
v=State.m_rstate.ra[1];
vv=State.m_rstate.ra[2];
}
else
{
//--- initialization
n=-983;
i=-989;
mcinfo=-834;
diffcnt=900;
b=true;
stepfound=false;
betak=214;
v=-338;
vv=-686;
}
//--- check
if(State.m_rstate.stage==0)
{
//--- change value
State.m_needfg=false;
//--- check
if(!State.m_xrep)
return(Func_lbl_15(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- progress report
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=1;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//--- check
if(State.m_rstate.stage==1)
{
//--- change value
State.m_xupdated=false;
//--- function call, return result
return(Func_lbl_15(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//--- check
if(State.m_rstate.stage==2)
{
//--- change values
State.m_needfg=false;
State.m_repnfev=State.m_repnfev+1;
stepfound=State.m_f<=State.m_finit+m_gpaftol*State.m_dginit;
//--- function call, return result
return(Func_lbl_24(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//--- check
if(State.m_rstate.stage==3)
{
//--- change values
State.m_needfg=false;
State.m_repnfev=State.m_repnfev+1;
//--- check
if(State.m_stp<=m_stpmin)
{
for(i_=0; i_<n; i_++)
State.m_xn[i_]=State.m_x[i_];
//--- function call, return result
return(Func_lbl_26(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//--- check
if(State.m_f<=State.m_finit+State.m_stp*m_gpaftol*State.m_dginit)
{
//--- copy
for(i_=0; i_<n; i_++)
State.m_xn[i_]=State.m_x[i_];
//--- function call, return result
return(Func_lbl_26(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//--- change value
State.m_stp=State.m_stp*m_gpadecay;
//--- function call, return result
return(Func_lbl_27(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//--- check
if(State.m_rstate.stage==4)
{
//--- change value
State.m_xupdated=false;
//--- function call, return result
return(Func_lbl_29(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//--- check
if(State.m_rstate.stage==5)
{
//--- change value
State.m_xupdated=false;
//--- return result
return(false);
}
//--- check
if(State.m_rstate.stage==6)
{
//--- change value
State.m_xupdated=false;
//--- return result
return(false);
}
//--- check
if(State.m_rstate.stage==7)
{
//--- change value
State.m_xupdated=false;
//--- return result
return(false);
}
//--- check
if(State.m_rstate.stage==8)
{
//--- change value
State.m_xupdated=false;
//--- return result
return(false);
}
//--- check
if(State.m_rstate.stage==9)
{
//--- change value
State.m_needfg=false;
//--- postprocess data: zero components of G corresponding to
//--- the active constraints
for(i=0; i<n; i++)
{
//--- check
if(State.m_x[i]==State.m_bndl[i] || State.m_x[i]==State.m_bndu[i])
State.m_gc[i]=0;
else
State.m_gc[i]=State.m_g[i];
}
CLinMin::MCSrch(n,State.m_xn,State.m_f,State.m_gc,State.m_d,State.m_stp,State.m_stpmax,m_gtol,mcinfo,State.m_nfev,State.m_work,State.m_lstate,State.m_mcstage);
//--- function call, return result
return(Func_lbl_51(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//--- check
if(State.m_rstate.stage==10)
{
//--- change value
State.m_xupdated=false;
//--- function call, return result
return(Func_lbl_53(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//--- check
if(State.m_rstate.stage==11)
{
//--- change value
State.m_xupdated=false;
//--- return result
return(false);
}
//--- check
if(State.m_rstate.stage==12)
{
//--- change value
State.m_xupdated=false;
//--- return result
return(false);
}
//--- check
if(State.m_rstate.stage==13)
{
//--- change value
State.m_xupdated=false;
//--- return result
return(false);
}
//--- check
if(State.m_rstate.stage==14)
{
//--- change value
State.m_xupdated=false;
//--- return result
return(false);
}
//--- Routine body
//--- Prepare
n=State.m_n;
State.m_repterminationtype=0;
State.m_repiterationscount=0;
State.m_repnfev=0;
State.m_debugrestartscount=0;
State.m_cgtype=1;
//--- copy
for(i_=0; i_<n; i_++)
State.m_xk[i_]=State.m_x[i_];
for(i=0; i<n; i++)
{
//--- check
if(State.m_xk[i]==State.m_bndl[i] || State.m_xk[i]==State.m_bndu[i])
State.m_ak[i]=0;
else
State.m_ak[i]=1;
}
//--- change values
State.m_mu=0.1;
State.m_curalgo=0;
//--- Calculate F/G,initialize algorithm
ClearRequestFields(State);
//--- change values
State.m_needfg=true;
State.m_rstate.stage=0;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
void CMinComp::Func_lbl_rcomm(CMinASAState &State,int n,int i,
int mcinfo,int diffcnt,bool b,
bool stepfound,double betak,
double v,double vv)
{
//--- save
State.m_rstate.ia.Set(0,n);
State.m_rstate.ia.Set(1,i);
State.m_rstate.ia.Set(2,mcinfo);
State.m_rstate.ia.Set(3,diffcnt);
State.m_rstate.ba[0]=b;
State.m_rstate.ba[1]=stepfound;
State.m_rstate.ra.Set(0,betak);
State.m_rstate.ra.Set(1,v);
State.m_rstate.ra.Set(2,vv);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_15(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- check
if(ASABoundedAntigradNorm(State)<=State.m_epsg)
{
State.m_repterminationtype=4;
//--- return result
return(false);
}
State.m_repnfev=State.m_repnfev+1;
//--- Main cycle
//--- At the beginning of new iteration:
//--- * CurAlgo stores current algorithm selector
//--- * State.XK,State.F and State.G store current X/F/G
//--- * State.AK stores current set of active constraints
return(Func_lbl_17(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_17(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- GPA algorithm
if(State.m_curalgo!=0)
return(Func_lbl_19(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- change values
State.m_k=0;
State.m_acount=0;
//--- function call, return result
return(Func_lbl_21(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_19(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- CG algorithm
if(State.m_curalgo!=1)
return(Func_lbl_17(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- first,check that there are non-active constraints.
//--- move to GPA algorithm,if all constraints are active
b=true;
for(i=0; i<n; i++)
{
//--- check
if(State.m_ak[i]!=0.0)
{
b=false;
break;
}
}
//--- check
if(b)
{
State.m_curalgo=0;
//--- function call, return result
return(Func_lbl_17(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//--- CG iterations
State.m_fold=State.m_f;
for(int i_=0; i_<n; i_++)
State.m_xk[i_]=State.m_x[i_];
for(i=0; i<n; i++)
{
//--- change values
State.m_dk[i]=-(State.m_g[i]*State.m_ak[i]);
State.m_gc[i]=State.m_g[i]*State.m_ak[i];
}
//--- function call, return result
return(Func_lbl_49(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_21(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- Determine Dk=proj(xk - gk)-xk
for(i=0; i<n; i++)
State.m_d[i]=CApServ::BoundVal(State.m_xk[i]-State.m_g[i],State.m_bndl[i],State.m_bndu[i])-State.m_xk[i];
//--- Armijo line search.
//--- * exact search with alpha=1 is tried first,
//--- 'exact' means that we evaluate f() EXACTLY at
//--- bound(x-g,bndl,bndu),without intermediate floating
//--- point operations.
//--- * alpha<1 are tried if explicit search wasn't successful
//--- Result is placed into XN.
//--- Two types of search are needed because we can't
//--- just use second type with alpha=1 because in finite
//--- precision arithmetics (x1-x0)+x0 may differ from x1.
//--- So while x1 is correctly bounded (it lie EXACTLY on
//--- boundary,if it is active),(x1-x0)+x0 may be
//--- not bounded.
v=0.0;
for(int i_=0; i_<n; i_++)
v+=State.m_d[i_]*State.m_g[i_];
//--- change values
State.m_dginit=v;
State.m_finit=State.m_f;
//--- check
if(!(ASAD1Norm(State)<=State.m_stpmax || State.m_stpmax==0.0))
{
stepfound=false;
//--- function call, return result
return(Func_lbl_24(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//--- Try alpha=1 step first
for(i=0; i<n; i++)
State.m_x[i]=CApServ::BoundVal(State.m_xk[i]-State.m_g[i],State.m_bndl[i],State.m_bndu[i]);
//--- function call
ClearRequestFields(State);
//--- change values
State.m_needfg=true;
State.m_rstate.stage=2;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_24(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- check
if(!stepfound)
{
//--- alpha=1 is too large,try smaller values
State.m_stp=1;
//--- function call
CLinMin::LinMinNormalized(State.m_d,State.m_stp,n);
//--- change values
State.m_dginit=State.m_dginit/State.m_stp;
State.m_stp=m_gpadecay*State.m_stp;
//--- check
if(State.m_stpmax>0.0)
State.m_stp=MathMin(State.m_stp,State.m_stpmax);
//--- function call, return result
return(Func_lbl_27(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//--- we are at the boundary(ies)
for(int i_=0; i_<n; i_++)
State.m_xn[i_]=State.m_x[i_];
State.m_stp=1;
//--- function call, return result
return(Func_lbl_26(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_26(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
State.m_repiterationscount=State.m_repiterationscount+1;
//--- check
if(!State.m_xrep)
return(Func_lbl_29(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- progress report
ClearRequestFields(State);
//--- change values
State.m_xupdated=true;
State.m_rstate.stage=4;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_27(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
v=State.m_stp;
//--- copy
for(int i_=0; i_<n; i_++)
State.m_x[i_]=State.m_xk[i_];
for(int i_=0; i_<n; i_++)
State.m_x[i_]=State.m_x[i_]+v*State.m_d[i_];
ClearRequestFields(State);
//--- change values
State.m_needfg=true;
State.m_rstate.stage=3;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_29(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- Calculate new set of active constraints.
//--- Reset counter if active set was changed.
//--- Prepare for the new iteration
for(i=0; i<n; i++)
{
//--- check
if(State.m_xn[i]==State.m_bndl[i] || State.m_xn[i]==State.m_bndu[i])
State.m_an[i]=0;
else
State.m_an[i]=1;
}
for(i=0; i<n; i++)
{
//--- check
if(State.m_ak[i]!=State.m_an[i])
{
State.m_acount=-1;
break;
}
}
State.m_acount=State.m_acount+1;
//--- copy
for(int i_=0; i_<n; i_++)
State.m_xk[i_]=State.m_xn[i_];
for(int i_=0; i_<n; i_++)
State.m_ak[i_]=State.m_an[i_];
//--- Stopping conditions
if(!(State.m_repiterationscount>=State.m_maxits && State.m_maxits>0))
return(Func_lbl_31(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- Too many iterations
State.m_repterminationtype=5;
//--- check
if(!State.m_xrep)
return(false);
//--- function call
ClearRequestFields(State);
//--- change values
State.m_xupdated=true;
State.m_rstate.stage=5;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_31(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- check
if(ASABoundedAntigradNorm(State)>State.m_epsg)
return(Func_lbl_35(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- Gradient is small enough
State.m_repterminationtype=4;
//--- check
if(!State.m_xrep)
return(false);
//--- function call
ClearRequestFields(State);
//--- change values
State.m_xupdated=true;
State.m_rstate.stage=6;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_35(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- change value
v=0.0;
for(int i_=0; i_<n; i_++)
v+=State.m_d[i_]*State.m_d[i_];
//--- check
if(MathSqrt(v)*State.m_stp>State.m_epsx)
return(Func_lbl_39(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- Step size is too small,no further improvement is
//--- possible
State.m_repterminationtype=2;
//--- check
if(!State.m_xrep)
return(false);
//--- function call
ClearRequestFields(State);
//--- change values
State.m_xupdated=true;
State.m_rstate.stage=7;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_39(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- check
if(State.m_finit-State.m_f>State.m_epsf*MathMax(MathAbs(State.m_finit),MathMax(MathAbs(State.m_f),1.0)))
return(Func_lbl_43(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- F(k+1)-F(k) is small enough
State.m_repterminationtype=1;
//--- check
if(!State.m_xrep)
return(false);
ClearRequestFields(State);
//--- change values
State.m_xupdated=true;
State.m_rstate.stage=8;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_43(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- Decide - should we switch algorithm or not
if(ASAUIsEmpty(State))
{
//--- check
if(ASAGINorm(State)>=State.m_mu*ASAD1Norm(State))
{
State.m_curalgo=1;
//--- function call, return result
return(Func_lbl_19(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
else
State.m_mu=State.m_mu*m_asarho;
}
else
{
//--- check
if(State.m_acount==m_n1)
{
//--- check
if(ASAGINorm(State)>=State.m_mu*ASAD1Norm(State))
{
State.m_curalgo=1;
//--- function call, return result
return(Func_lbl_19(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
}
}
//--- Next iteration
State.m_k=State.m_k+1;
//--- function call, return result
return(Func_lbl_21(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_49(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- Store G[k] for later calculation of Y[k]
for(i=0; i<n; i++)
State.m_yk[i]=-State.m_gc[i];
//--- Make a CG step in direction given by DK[]:
//--- * calculate step. Step projection into feasible set
//--- is used. It has several benefits: a) step may be
//--- found with usual line search,b) multiple constraints
//--- may be activated with one step,c) activated constraints
//--- are detected in a natural way - just compare x[i] with
//--- bounds
//--- * update active set,set B to True,if there
//--- were changes in the set.
for(int i_=0; i_<n; i_++)
State.m_d[i_]=State.m_dk[i_];
for(int i_=0; i_<n; i_++)
State.m_xn[i_]=State.m_xk[i_];
//--- change values
State.m_mcstage=0;
State.m_stp=1;
//--- function call
CLinMin::LinMinNormalized(State.m_d,State.m_stp,n);
//--- check
if(State.m_laststep!=0.0)
State.m_stp=State.m_laststep;
//--- function call
CLinMin::MCSrch(n,State.m_xn,State.m_f,State.m_gc,State.m_d,State.m_stp,State.m_stpmax,m_gtol,mcinfo,State.m_nfev,State.m_work,State.m_lstate,State.m_mcstage);
//--- function call, return result
return(Func_lbl_51(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_51(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- check
if(State.m_mcstage==0)
return(Func_lbl_52(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- preprocess data: bound State.XN so it belongs to the
//--- feasible set and store it in the State.X
for(i=0; i<n; i++)
State.m_x[i]=CApServ::BoundVal(State.m_xn[i],State.m_bndl[i],State.m_bndu[i]);
//--- RComm
ClearRequestFields(State);
//--- change values
State.m_needfg=true;
State.m_rstate.stage=9;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_52(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
diffcnt=0;
for(i=0; i<n; i++)
{
//--- XN contains unprojected result,project it,
//--- save copy to X (will be used for progress reporting)
State.m_xn[i]=CApServ::BoundVal(State.m_xn[i],State.m_bndl[i],State.m_bndu[i]);
//--- update active set
if(State.m_xn[i]==State.m_bndl[i] || State.m_xn[i]==State.m_bndu[i])
State.m_an[i]=0;
else
State.m_an[i]=1;
//--- check
if(State.m_an[i]!=State.m_ak[i])
diffcnt=diffcnt+1;
State.m_ak[i]=State.m_an[i];
}
for(int i_=0; i_<n; i_++)
State.m_xk[i_]=State.m_xn[i_];
//--- change values
State.m_repnfev=State.m_repnfev+State.m_nfev;
State.m_repiterationscount=State.m_repiterationscount+1;
//--- check
if(!State.m_xrep)
return(Func_lbl_53(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- progress report
ClearRequestFields(State);
//--- change values
State.m_xupdated=true;
State.m_rstate.stage=10;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_53(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- Update Info about step length
v=0.0;
for(int i_=0; i_<n; i_++)
v+=State.m_d[i_]*State.m_d[i_];
State.m_laststep=MathSqrt(v)*State.m_stp;
//--- Check stopping conditions.
if(ASABoundedAntigradNorm(State)>State.m_epsg)
return(Func_lbl_55(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- Gradient is small enough
State.m_repterminationtype=4;
//--- check
if(!State.m_xrep)
return(false);
ClearRequestFields(State);
//--- change values
State.m_xupdated=true;
State.m_rstate.stage=11;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_55(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- check
if(!(State.m_repiterationscount>=State.m_maxits && State.m_maxits>0))
return(Func_lbl_59(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- Too many iterations
State.m_repterminationtype=5;
//--- check
if(!State.m_xrep)
return(false);
ClearRequestFields(State);
//--- change values
State.m_xupdated=true;
State.m_rstate.stage=12;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_59(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- check
if(!(ASAGINorm(State)>=State.m_mu*ASAD1Norm(State) && diffcnt==0))
return(Func_lbl_63(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- These conditions (EpsF/EpsX) are explicitly or implicitly
//--- related to the current step size and influenced
//--- by changes in the active constraints.
//--- For these reasons they are checked only when we don't
//--- want to 'unstick' at the end of the iteration and there
//--- were no changes in the active set.
//--- NOTE: consition |G|>=Mu*|D1| must be exactly opposite
//--- to the condition used to switch back to GPA. At least
//--- one inequality must be strict,otherwise infinite cycle
//--- may occur when |G|=Mu*|D1| (we DON'T test stopping
//--- conditions and we DON'T switch to GPA,so we cycle
//--- indefinitely).
if(State.m_fold-State.m_f>State.m_epsf*MathMax(MathAbs(State.m_fold),MathMax(MathAbs(State.m_f),1.0)))
return(Func_lbl_65(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- F(k+1)-F(k) is small enough
State.m_repterminationtype=1;
if(!State.m_xrep)
return(false);
//--- function call
ClearRequestFields(State);
//--- change values
State.m_xupdated=true;
State.m_rstate.stage=13;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_63(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- Check conditions for switching
if(ASAGINorm(State)<State.m_mu*ASAD1Norm(State))
{
State.m_curalgo=0;
//--- function call, return result
return(Func_lbl_17(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//--- check
if(diffcnt>0)
{
//--- check
if(ASAUIsEmpty(State) || diffcnt>=m_n2)
State.m_curalgo=1;
else
State.m_curalgo=0;
//--- function call, return result
return(Func_lbl_17(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//--- Calculate D(k+1)
//--- Line search may result in:
//--- * maximum feasible step being taken (already processed)
//--- * point satisfying Wolfe conditions
//--- * some kind of error (CG is restarted by assigning 0.0 to Beta)
if(mcinfo==1)
{
//--- Standard Wolfe conditions are satisfied:
//--- * calculate Y[K] and BetaK
for(int i_=0; i_<n; i_++)
State.m_yk[i_]=State.m_yk[i_]+State.m_gc[i_];
//--- change value
vv=0.0;
for(int i_=0; i_<n; i_++)
vv+=State.m_yk[i_]*State.m_dk[i_];
//--- change value
v=0.0;
for(int i_=0; i_<n; i_++)
v+=State.m_gc[i_]*State.m_gc[i_];
State.m_betady=v/vv;
//--- change value
v=0.0;
for(int i_=0; i_<n; i_++)
v+=State.m_gc[i_]*State.m_yk[i_];
State.m_betahs=v/vv;
//--- check
if(State.m_cgtype==0)
betak=State.m_betady;
//--- check
if(State.m_cgtype==1)
betak=MathMax(0,MathMin(State.m_betady,State.m_betahs));
}
else
{
//--- Something is wrong (may be function is too wild or too flat).
//--- We'll set BetaK=0,which will restart CG algorithm.
//--- We can stop later (during normal checks) if stopping conditions are met.
betak=0;
State.m_debugrestartscount=State.m_debugrestartscount+1;
}
//--- change values
for(int i_=0; i_<n; i_++)
State.m_dn[i_]=-State.m_gc[i_];
for(int i_=0; i_<n; i_++)
State.m_dn[i_]=State.m_dn[i_]+betak*State.m_dk[i_];
for(int i_=0; i_<n; i_++)
State.m_dk[i_]=State.m_dn[i_];
//--- update other information
State.m_fold=State.m_f;
State.m_k=State.m_k+1;
//--- function call, return result
return(Func_lbl_49(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
}
//+------------------------------------------------------------------+
//| Auxiliary function for MinASAIteration. Is a product to get rid |
//| of the operator unconditional jump goto. |
//+------------------------------------------------------------------+
bool CMinComp::Func_lbl_65(CMinASAState &State,int &n,int &i,
int &mcinfo,int &diffcnt,bool &b,
bool &stepfound,double &betak,
double &v,double &vv)
{
//--- check
if(State.m_laststep>State.m_epsx)
return(Func_lbl_63(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv));
//--- X(k+1)-X(k) is small enough
State.m_repterminationtype=2;
//--- check
if(!State.m_xrep)
return(false);
//--- function call
ClearRequestFields(State);
//--- change values
State.m_xupdated=true;
State.m_rstate.stage=14;
//--- Saving State
Func_lbl_rcomm(State,n,i,mcinfo,diffcnt,b,stepfound,betak,v,vv);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CLPQPServ
{
public:
static void ScaleShiftBCInplace(CRowDouble &s,CRowDouble &xorigin,CRowDouble &bndl,CRowDouble &bndu,int n);
static void ScaleShiftDenseBRLCInplace(CRowDouble &s,CRowDouble &xorigin,int n,CMatrixDouble &densea,CRowDouble &ab,CRowDouble &ar,int m);
static void ScaleShiftMixedBRLCInplace(CRowDouble &s,CRowDouble &xorigin,int n,CSparseMatrix &sparsea,int msparse,CMatrixDouble &densea,int mdense,CRowDouble &ab,CRowDouble &ar);
static void ScaleDenseQPInplace(CMatrixDouble &densea,bool IsUpper,int nmain,CRowDouble &denseb,int ntotal,CRowDouble &s);
static void ScaleSparseQPInplace(CRowDouble &s,int n,CSparseMatrix &sparsea,CRowDouble &denseb);
static void NormalizeDenseBRLCInplace(CMatrixDouble &densea,CRowDouble &ab,CRowDouble &ar,int n,int m,CRowDouble &rownorms,bool neednorms);
static void NormalizeMixedBRLCInplace(CSparseMatrix &sparsea,int msparse,CMatrixDouble &densea,int mdense,CRowDouble &ab,CRowDouble &ar,int n,bool limitedamplification,CRowDouble &rownorms,bool neednorms);
static double NormalizeDenseQPInplace(CMatrixDouble &densea,bool IsUpper,int nmain,CRowDouble &denseb,int ntotal);
static double NormalizeSparseQPInplace(CSparseMatrix &sparsea,bool IsUpper,CRowDouble &denseb,int n);
static void UnscaleUnshiftPointBC(CRowDouble &s,CRowDouble &xorigin,CRowDouble &rawbndl,CRowDouble &rawbndu,CRowDouble &sclsftbndl,CRowDouble &sclsftbndu,bool &HasBndL[],bool &HasBndU[],CRowDouble &x,int n);
};
//+------------------------------------------------------------------+
//| This function generates scaled (by S) and shifted (by XC) |
//| reformulation of the box constraints. |
//| INPUT PARAMETERS: |
//| S - scale vector, array[N]: |
//| * I-th element contains scale of I-th variable, |
//| * SC[I] > 0 |
//| XOrigin - origin term, array[N]. Can be zero. |
//| BndL - raw lower bounds, array[N] |
//| BndU - raw upper bounds, array[N] |
//| N - number of variables. |
//| OUTPUT PARAMETERS: |
//| BndL - replaced by scaled / shifted lower bounds, array[N]|
//| BndU - replaced by scaled / shifted upper bounds, array[N]|
//+------------------------------------------------------------------+
void CLPQPServ::ScaleShiftBCInplace(CRowDouble &s,
CRowDouble &xorigin,
CRowDouble &bndl,
CRowDouble &bndu,
int n)
{
//--- create variables
bool HasBndL=false;
bool HasBndU=false;
for(int i=0; i<n; i++)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(s[i]) && s[i]>0.0,__FUNCTION__+": S[i] is nonpositive"))
return;
if(!CAp::Assert(MathIsValidNumber(bndl[i]) || CInfOrNaN::NegativeInfinity()==bndl[i],__FUNCTION__+": BndL[i] is +INF or NAN"))
return;
if(!CAp::Assert(MathIsValidNumber(bndu[i]) || CInfOrNaN::PositiveInfinity()==bndu[i],__FUNCTION__+": BndU[i] is -INF or NAN"))
return;
//---
HasBndL=MathIsValidNumber(bndl[i]);
HasBndU=MathIsValidNumber(bndu[i]);
if(HasBndL && HasBndU && bndl[i]==bndu[i])
{
//--- Make sure that BndL[I]=BndU[I] bit-to-bit
//--- even with CRAZY optimizing compiler.
bndu.Set(i,(bndu[i]-xorigin[i])/s[i]);
bndl.Set(i,bndu[i]);
continue;
}
if(HasBndL)
{
bndl.Set(i,(bndl[i]-xorigin[i])/s[i]);
}
if(HasBndU)
{
bndu.Set(i,(bndu[i]-xorigin[i])/s[i]);
}
}
}
//+------------------------------------------------------------------+
//| This function generates scaled (by S) and shifted (by XC) |
//| reformulation of two-sided "lower-bound/range" constraints stored|
//| in dense format. |
//| INPUT PARAMETERS: |
//| S - scale vector, array[N]: |
//| * I-th element contains scale of I-th variable, |
//| * SC[I] > 0 |
//| XOrigin - origin term, array[N]. Can be zero. |
//| N - number of variables. |
//| DenseA - array[M, N], constraint matrix |
//| AB - lower bounds for constraints, always present and |
//| finite, array[M] |
//| AR - ranges for constraints, can be zero(equality |
//| constraint), positive(range constraint) or + INF |
//| (lower bound constraint), array[M] |
//| M - constraint count, M >= 0 |
//| OUTPUT PARAMETERS: |
//| DenseA - replaced by scaled / shifted constraints, |
//| array[M, N] |
//| AB - replaced by scaled / shifted lower bounds, array[M]|
//| AR - replaced by scaled / shifted ranges, array[M] |
//+------------------------------------------------------------------+
void CLPQPServ::ScaleShiftDenseBRLCInplace(CRowDouble &s,
CRowDouble &xorigin,
int n,
CMatrixDouble &densea,
CRowDouble &ab,
CRowDouble &ar,
int m)
{
//--- create variables
double v=0;
double vv=0;
for(int i=0; i<m; i++)
{
//--- Scale/shift constraint; shift its lower bound
//--- NOTE: range is NOT scaled or shifted
v=0.0;
for(int j=0; j<n; j++)
{
vv=densea.Get(i,j);
v+= vv*xorigin[j];
densea.Set(i,j,vv*s[j]);
}
ab.Add(i,-v);
}
}
//+------------------------------------------------------------------+
//| This function generates scaled (by S) and shifted (by XC) |
//| reformulation of two-sided "lower-bound/range" constraints stored|
//| in dense format. |
//| INPUT PARAMETERS: |
//| S - scale vector, array[N]: |
//| * I-th element contains scale of I-th variable, |
//| * SC[I] > 0 |
//| XOrigin - origin term, array[N]. Can be zero. |
//| N - number of variables. |
//| SparseA - sparse MSparse*N constraint matrix in CRS format; |
//| ignored if MSparse = 0. |
//| MSparse - dense constraint count, MSparse >= 0 |
//| DenseA - array[MDense, N], constraint matrix; ignored if |
//| MDense = 0. |
//| MDense - dense constraint count, MDense >= 0 |
//| AB - lower bounds for constraints, always present and |
//| finite, array[MSparse + MDense] |
//| AR - ranges for constraints, can be zero(equality |
//| constraint), positive(range constraint) or + INF |
//| (lower bound constraint), array[MSparse + MDense] |
//| OUTPUT PARAMETERS: |
//| DenseA - replaced by scaled / shifted constraints, |
//| array[MDense, N] |
//| SparseA - replaced by scaled / shifted constraints, |
//| array[MSparse, N] |
//| AB - replaced by scaled / shifted lower bounds, |
//| array[MDense + MSparse] |
//| AR - replaced by scaled / shifted ranges, |
//| array[MDense + MSparse] |
//+------------------------------------------------------------------+
void CLPQPServ::ScaleShiftMixedBRLCInplace(CRowDouble &s,
CRowDouble &xorigin,
int n,
CSparseMatrix &sparsea,
int msparse,
CMatrixDouble &densea,
int mdense,
CRowDouble &ab,
CRowDouble &ar)
{
//--- create variables
int i=0;
int j=0;
int k=0;
int k0=0;
int k1=0;
double v=0;
double vv=0;
//--- check
if(!CAp::Assert(msparse==0 || (sparsea.m_MatrixType==1 && sparsea.m_M==msparse && sparsea.m_N==n),__FUNCTION__+": non-CRS sparse constraint matrix!"))
return;
for(i=0; i<msparse; i++)
{
//--- Scale/shift constraint; shift its lower bound
//--- NOTE: range is NOT scaled or shifted
v=0.0;
k0=sparsea.m_RIdx[i];
k1=sparsea.m_RIdx[i+1];
for(k=k0; k<k1; k++)
{
j=sparsea.m_Idx[k];
vv=sparsea.m_Vals[k];
v+= vv*xorigin[j];
sparsea.m_Vals.Set(k,vv*s[j]);
}
ab.Add(i,-v);
}
for(i=0; i<mdense; i++)
{
//--- Scale/shift constraint; shift its lower bound
//--- NOTE: range is NOT scaled or shifted
v=0.0;
for(j=0; j<n; j++)
{
vv=densea.Get(i,j);
v+= vv*xorigin[j];
densea.Set(i,j,vv*s[j]);
}
ab.Add(msparse+i,- v);
}
}
//+------------------------------------------------------------------+
//| This function generates scaled (by S) reformulation of dense |
//| quadratic and linear terms in QP problem. |
//| INPUT PARAMETERS: |
//| N - number of variables. |
//| DenseA - array[NMain, NMain], quadratic term |
//| IsUpper - whether upper or lower triangle is present |
//| NMain - number of nonslack vars, 1 <= NMain <= NTotal |
//| DenseB - array[NTotal], linear term |
//| NTotal - total number of variables, NTotal >= 1 |
//| S - scale vector, array[NTotal]: |
//| * I-th element contains scale of I-th variable, |
//| * SC[I] > 0 |
//| OUTPUT PARAMETERS: |
//| DenseA - replaced by scaled term, array[N, N] |
//| DenseB - replaced by scaled term, array[N] |
//+------------------------------------------------------------------+
void CLPQPServ::ScaleDenseQPInplace(CMatrixDouble &densea,
bool IsUpper,
int nmain,
CRowDouble &denseb,
int ntotal,
CRowDouble &s)
{
//--- create variables
int i=0;
int j=0;
int j0=0;
int j1=0;
double si=0;
for(i=0; i<nmain; i++)
{
si=s[i];
if(IsUpper)
{
j0=i;
j1=nmain-1;
}
else
{
j0=0;
j1=i;
}
for(j=j0; j<=j1; j++)
densea.Mul(i,j,si*s[j]);
}
for(i=0; i<ntotal; i++)
denseb.Mul(i,s[i]);
}
//+------------------------------------------------------------------+
//| This function generates scaled (by S) reformulation of sparse |
//| quadratic and linear terms in QP problem. |
//| INPUT PARAMETERS: |
//| S - scale vector, array[N]: |
//| * I-th element contains scale of I-th variable, |
//| * SC[I] > 0 |
//| N - number of variables. |
//| SparseA - NxN CSparseMatrix in CRS format(any triangle can |
//| be present, we will scale everything) |
//| DenseB - array[N], linear term |
//| OUTPUT PARAMETERS: |
//| SparseA - replaced by scaled term |
//| DenseB - replaced by scaled term |
//+------------------------------------------------------------------+
void CLPQPServ::ScaleSparseQPInplace(CRowDouble &s,
int n,
CSparseMatrix &sparsea,
CRowDouble &denseb)
{
//--- create variables
int i=0;
int k0=0;
int k1=0;
int k=0;
double si=0;
//--- check
if(!CAp::Assert(sparsea.m_MatrixType==1 && sparsea.m_M==n && sparsea.m_N==n,__FUNCTION__+": SparseA in unexpected format"))
return;
for(i=0; i<n; i++)
{
si=s[i];
k0=sparsea.m_RIdx[i];
k1=sparsea.m_RIdx[i+1];
for(k=k0; k<k1; k++)
sparsea.m_Vals.Mul(k,si*s[sparsea.m_Idx[k]]);
denseb.Mul(i,si);
}
}
//+------------------------------------------------------------------+
//| This function normalizes two-sided "lower-bound/range" |
//| constraints stored in dense format in such a way that L2 norms |
//| of rows (right hand side NOT included) become equal to 1.0. |
//| Exactly zero rows are handled correctly. |
//| INPUT PARAMETERS: |
//| DenseA - array[M, N], constraint matrix |
//| AB - lower bounds for constraints, always present and |
//| finite, array[M] |
//| AR - ranges for constraints, can be zero (equality |
//| constraint), positive(range constraint) or + INF |
//| (lower bound constraint), array[M] |
//| N - number of variables, N >= 1. |
//| M - constraint count, M >= 0 |
//| NeedNorms- whether we need row norms or not |
//| OUTPUT PARAMETERS: |
//| DenseA - replaced by normalized constraints, array[M, N] |
//| AB - replaced by normalized lower bounds, array[M] |
//| AR - replaced by normalized ranges, array[M] |
//| RowNorms - if NeedNorms is true, leading M elements (resized |
//| if length is less than M) are filled by row norms |
//| before normalization was performed. |
//+------------------------------------------------------------------+
void CLPQPServ::NormalizeDenseBRLCInplace(CMatrixDouble &densea,
CRowDouble &ab,
CRowDouble &ar,
int n,
int m,
CRowDouble &rownorms,
bool neednorms)
{
//--- create variables
int i=0;
int j=0;
double v=0;
double vv=0;
if(neednorms)
rownorms.Resize(m);
for(i=0; i<m; i++)
{
vv=0.0;
for(j=0; j<n; j++)
{
v=densea.Get(i,j);
vv+=v*v;
}
vv=MathSqrt(vv);
if(neednorms)
rownorms.Set(i,vv);
if(vv>0.0)
{
vv=1/vv;
for(j=0; j<n; j++)
{
densea.Mul(i,j,vv);
}
ab.Mul(i,vv);
if(MathIsValidNumber(ar[i]))
ar.Mul(i,vv);
}
}
}
//+------------------------------------------------------------------+
//| This function normalizes two-sided "lower-bound/range" |
//| constraints stored in dense format in such a way that L2 norms |
//| of rows (right hand side NOT included) become equal to 1.0. |
//| Exactly zero rows are handled correctly. |
//| INPUT PARAMETERS: |
//| SparseA - sparse MSparse*N constraint matrix in CRS format; |
//| ignored if MSparse = 0. |
//| MSparse - dense constraint count, MSparse >= 0 |
//| DenseA - array[MDense, N], constraint matrix; ignored if |
//| MDense = 0. |
//| MDense - dense constraint count, MDense >= 0 |
//| AB - lower bounds for constraints, always present and |
//| finite, array[MSparse + MDense] |
//| AR - ranges for constraints, can be zero(equality |
//| constraint), positive(range constraint) or + INF |
//| (lower bound constraint), array[MSparse + MDense] |
//| N - number of variables, N >= 1. |
//| LimitedAmplification - whether row amplification is limited or|
//| not: |
//| * if False, rows with small norms(less than 1.0) |
//| are always normalized |
//| * if True, we do not increase individual row norms |
//| during normalization - only decrease. However, |
//| we may apply one amplification rount to entire |
//| constraint matrix, i.e. amplify all rows by same |
//| coefficient. As result, we do not overamplify |
//| any single row, but still make sure than entire |
//| problem is well scaled. If True, only large rows |
//| are normalized. |
//| NeedNorms- whether we need row norms or not |
//| OUTPUT PARAMETERS: |
//| DenseA - replaced by normalized constraints, array[M, N] |
//| AB - replaced by normalized lower bounds, array[M] |
//| AR - replaced by normalized ranges, array[M] |
//| RowNorms - if NeedNorms is true, leading M elements(resized |
//| if length is less than M) are filled by row norms |
//| before normalization was performed. |
//+------------------------------------------------------------------+
void CLPQPServ::NormalizeMixedBRLCInplace(CSparseMatrix &sparsea,
int msparse,
CMatrixDouble &densea,
int mdense,
CRowDouble &ab,
CRowDouble &ar,
int n,
bool limitedamplification,
CRowDouble &rownorms,
bool neednorms)
{
//--- create variables
int i=0;
int j=0;
int k=0;
int k0=0;
int k1=0;
double v=0;
double vv=0;
double maxnrm2=0;
//--- check
if(!CAp::Assert(msparse==0 || (sparsea.m_MatrixType==1 && sparsea.m_M==msparse && sparsea.m_N==n),__FUNCTION__+": non-CRS sparse constraint matrix!"))
return;
if(neednorms)
rownorms.Resize(mdense+msparse);
//--- First round of normalization - normalize row 2-norms subject to limited amplification status
maxnrm2=0;
for(i=0; i<msparse; i++)
{
vv=0.0;
k0=sparsea.m_RIdx[i];
k1=sparsea.m_RIdx[i+1];
for(k=k0; k<k1; k++)
{
v=sparsea.m_Vals[k];
vv+=v*v;
}
vv=MathSqrt(vv);
maxnrm2=MathMax(maxnrm2,vv);
if(limitedamplification)
vv=MathMax(vv,1.0);
if(neednorms)
rownorms.Set(i,vv);
if(vv>0.0)
{
vv=1/vv;
for(k=k0; k<k1; k++)
sparsea.m_Vals.Mul(k,vv);
ab.Mul(i,vv);
if(MathIsValidNumber(ar[i]))
ar.Mul(i,vv);
}
}
for(i=0; i<mdense; i++)
{
vv=0.0;
for(j=0; j<n; j++)
{
v=densea.Get(i,j);
vv+=v*v;
}
vv=MathSqrt(vv);
maxnrm2=MathMax(maxnrm2,vv);
if(limitedamplification)
vv=MathMax(vv,1.0);
if(neednorms)
rownorms.Set(msparse+i,vv);
if(vv>0.0)
{
vv=1/vv;
for(j=0; j<n; j++)
densea.Mul(i,j,vv);
ab.Mul(msparse+i,vv);
if(MathIsValidNumber(ar[msparse+i]))
ar.Mul(msparse+i,vv);
}
}
//--- If amplification was limited, perform second round of normalization
if(limitedamplification && maxnrm2<1.0 && maxnrm2>0.0)
{
if(neednorms)
CAblasF::RMulV(mdense+msparse,maxnrm2,rownorms);
vv=1/maxnrm2;
for(i=0; i<msparse; i++)
{
k0=sparsea.m_RIdx[i];
k1=sparsea.m_RIdx[i+1];
for(k=k0; k<k1; k++)
sparsea.m_Vals.Mul(k,vv);
ab.Mul(i,vv);
if(MathIsValidNumber(ar[i]))
ar.Mul(i,vv);
}
for(i=0; i<mdense; i++)
{
CAblasF::RMulR(n,vv,densea,i);
ab.Mul(msparse+i,vv);
if(MathIsValidNumber(ar[msparse+i]))
ar.Mul(msparse+i,vv);
}
}
}
//+------------------------------------------------------------------+
//| This function normalizes dense QP problem in such a way that |
//| maximum over its linear/quadratic coefficients max(max(A),max(B))|
//| becomes equal to 1.0. |
//| NOTE: completely zero A and B are handled correctly. |
//| INPUT PARAMETERS: |
//| DenseA - array[NMain, NMain], quadratic term |
//| IsUpper - whether upper or lower triangle is present |
//| NMain - number of nonslack vars, 1 <= NMain <= NTotal |
//| DenseB - array[NTotal], linear term |
//| NTotal - total number of variables. |
//| OUTPUT PARAMETERS: |
//| DenseA - replaced by normalized term |
//| DenseB - replaced by normalized term |
//| RESULT: max(max(A), max(B)) is returned |
//+------------------------------------------------------------------+
double CLPQPServ::NormalizeDenseQPInplace(CMatrixDouble &densea,
bool IsUpper,
int nmain,
CRowDouble &denseb,
int ntotal)
{
//--- return result
double result=0;
int i=0;
int j=0;
int j0=0;
int j1=0;
double mx=0;
double v=0;
mx=0;
for(i=0; i<nmain; i++)
{
if(IsUpper)
{
j0=i;
j1=nmain-1;
}
else
{
j0=0;
j1=i;
}
for(j=j0; j<=j1; j++)
mx=MathMax(mx,MathAbs(densea.Get(i,j)));
}
for(i=0; i<ntotal; i++)
mx=MathMax(mx,MathAbs(denseb[i]));
result=mx;
if(mx==0.0)
return(result);
v=1/mx;
for(i=0; i<nmain; i++)
{
if(IsUpper)
{
j0=i;
j1=nmain-1;
}
else
{
j0=0;
j1=i;
}
for(j=j0; j<=j1; j++)
densea.Mul(i,j,v);
}
for(i=0; i<ntotal; i++)
denseb.Mul(i,v);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function normalizes sparse QP problem in such a way that |
//| maximum over its linear/quadratic coefficients max(max(A),max(B))|
//| becomes equal to 1.0. |
//| NOTE: completely zero A and B are handled correctly. |
//| INPUT PARAMETERS: |
//| SparseA - Sparse NxN matrix, either upper or lower triangle, |
//| diagonal MUST be present |
//| IsUpper - which triangle is present(other one is ignored) |
//| DenseB - array[N], linear term |
//| N - number of variables. |
//| OUTPUT PARAMETERS: |
//| DenseA - replaced by normalized term, array[N, N] |
//| DenseB - replaced by normalized term, array[N] |
//| RESULT: max(max(A), max(B)) is returned |
//+------------------------------------------------------------------+
double CLPQPServ::NormalizeSparseQPInplace(CSparseMatrix &sparsea,
bool IsUpper,
CRowDouble &denseb,
int n)
{
//--- create variables
double result=0;
int i=0;
int k=0;
int k0=0;
int k1=0;
double mx=0;
double v=0;
//--- check
if(!CAp::Assert(sparsea.m_MatrixType==1 && sparsea.m_M==n && sparsea.m_N==n,__FUNCTION__+": SparseA in unexpected format"))
return(result);
mx=0;
for(i=0; i<n; i++)
{
//--- check
if(!CAp::Assert(sparsea.m_DIdx[i]+1==sparsea.m_UIdx[i],__FUNCTION__+": critical integrity check failed,sparse diagonal not found"))
return(result);
if(IsUpper)
{
k0=sparsea.m_DIdx[i];
k1=sparsea.m_RIdx[i+1]-1;
}
else
{
k0=sparsea.m_RIdx[i];
k1=sparsea.m_DIdx[i];
}
for(k=k0; k<=k1; k++)
mx=MathMax(mx,MathAbs(sparsea.m_Vals[k]));
mx=MathMax(mx,MathAbs(denseb[i]));
}
result=mx;
if(mx==0.0)
return(result);
v=1/mx;
for(i=0; i<n; i++)
{
k0=sparsea.m_RIdx[i];
k1=sparsea.m_RIdx[i+1]-1;
for(k=k0; k<=k1; k++)
sparsea.m_Vals.Mul(k,v);
denseb.Mul(i,v);
}
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function performs transformation of X from scaled / shifted |
//| coordinates to unscaled/unshifted ones, paying special attention |
//| to box constraints: |
//| * points which were exactly at the boundary before scaling will|
//| be mapped to corresponding boundary after scaling |
//| * in any case, unscaled box constraints will be satisfied |
//+------------------------------------------------------------------+
void CLPQPServ::UnscaleUnshiftPointBC(CRowDouble &s,
CRowDouble &xorigin,
CRowDouble &rawbndl,
CRowDouble &rawbndu,
CRowDouble &sclsftbndl,
CRowDouble &sclsftbndu,
bool &HasBndL[],
bool &HasBndU[],
CRowDouble &x,
int n)
{
for(int i=0; i<n; i++)
{
if(HasBndL[i] && x[i]<=sclsftbndl[i])
{
x.Set(i,rawbndl[i]);
continue;
}
if(HasBndU[i] && x[i]>=sclsftbndu[i])
{
x.Set(i,rawbndu[i]);
continue;
}
x.Set(i,x[i]*s[i]+xorigin[i]);
if(HasBndL[i] && x[i]<=rawbndl[i])
x.Set(i,rawbndl[i]);
if(HasBndU[i] && x[i]>=rawbndu[i])
x.Set(i,rawbndu[i]);
}
}
//+------------------------------------------------------------------+
//| This object stores temporaries of SQP subsolver. |
//+------------------------------------------------------------------+
struct CMinSQPSubSolver
{
int m_activesetsize;
int m_algokind;
bool m_hasal[];
bool m_hasau[];
bool m_HasBndL[];
bool m_HasBndU[];
CVIPMState m_ipmsolver;
CSparseMatrix m_sparsedummy;
CSparseMatrix m_sparseefflc;
CSparseMatrix m_sparserawlc;
CRowInt m_activeidx;
CRowDouble m_activerhs;
CRowDouble m_cural;
CRowDouble m_curau;
CRowDouble m_curb;
CRowDouble m_curbndl;
CRowDouble m_curbndu;
CRowDouble m_d0;
CRowDouble m_sk;
CRowDouble m_tmp0;
CRowDouble m_tmp1;
CRowDouble m_tmp2;
CRowDouble m_yk;
CMatrixDouble m_activea;
CMatrixDouble m_densedummy;
CMatrixDouble m_h;
//---
CMinSQPSubSolver(void);
~CMinSQPSubSolver(void) {}
//---
void Copy(const CMinSQPSubSolver &obj);
//--- overloading
void operator=(const CMinSQPSubSolver &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinSQPSubSolver::CMinSQPSubSolver(void)
{
m_activesetsize=0;
m_algokind=0;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinSQPSubSolver::Copy(const CMinSQPSubSolver &obj)
{
m_activesetsize=obj.m_activesetsize;
m_algokind=obj.m_algokind;
ArrayCopy(m_hasal,obj.m_hasal);
ArrayCopy(m_hasau,obj.m_hasau);
ArrayCopy(m_HasBndL,obj.m_HasBndL);
ArrayCopy(m_HasBndU,obj.m_HasBndU);
m_ipmsolver=obj.m_ipmsolver;
m_sparsedummy=obj.m_sparsedummy;
m_sparseefflc=obj.m_sparseefflc;
m_sparserawlc=obj.m_sparserawlc;
m_activeidx=obj.m_activeidx;
m_activerhs=obj.m_activerhs;
m_cural=obj.m_cural;
m_curau=obj.m_curau;
m_curb=obj.m_curb;
m_curbndl=obj.m_curbndl;
m_curbndu=obj.m_curbndu;
m_d0=obj.m_d0;
m_sk=obj.m_sk;
m_tmp0=obj.m_tmp0;
m_tmp1=obj.m_tmp1;
m_tmp2=obj.m_tmp2;
m_yk=obj.m_yk;
m_activea=obj.m_activea;
m_densedummy=obj.m_densedummy;
m_h=obj.m_h;
}
//+------------------------------------------------------------------+
//| This object stores temporaries for LagrangianFG() function |
//+------------------------------------------------------------------+
struct CMinSQPTmpLagrangian
{
CRowDouble m_sclagtmp0;
CRowDouble m_sclagtmp1;
CMinSQPTmpLagrangian(void) {}
~CMinSQPTmpLagrangian(void) {}
void Copy(const CMinSQPTmpLagrangian &obj)
{
m_sclagtmp0=obj.m_sclagtmp0;
m_sclagtmp1=obj.m_sclagtmp1;
}
//--- overloading
void operator=(const CMinSQPTmpLagrangian &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| This object stores temporaries for LagrangianFG() function |
//+------------------------------------------------------------------+
struct CMinSQPTmpMerit
{
public:
CRowDouble m_mftmp0;
CMinSQPTmpMerit(void) {}
~CMinSQPTmpMerit(void) {}
void Copy(const CMinSQPTmpMerit &obj) { m_mftmp0=obj.m_mftmp0; }
//--- overloading
void operator=(const CMinSQPTmpMerit &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| This object stores temporaries of Phase 1 (merit function |
//| optimization). |
//+------------------------------------------------------------------+
struct CMinSQPMeritPhaseState
{
int m_n;
int m_nec;
int m_nic;
int m_nlec;
int m_nlic;
int m_status;
bool m_increasebigc;
RCommState m_rmeritphasestate;
CRowDouble m_d;
CRowDouble m_dummylagmult;
CRowDouble m_dx;
CRowDouble m_lagmult;
CRowDouble m_penalties;
CRowDouble m_stepkfi;
CRowDouble m_stepkfic;
CRowDouble m_stepkfin;
CRowDouble m_stepklaggrad;
CRowDouble m_stepknlaggrad;
CRowDouble m_stepkx;
CRowDouble m_stepkxc;
CRowDouble m_stepkxn;
CMinSQPTmpMerit m_tmpmerit;
CMinSQPTmpLagrangian m_tmplagrangianfg;
CMatrixDouble m_stepkj;
CMatrixDouble m_stepkjc;
CMatrixDouble m_stepkjn;
CMinSQPMeritPhaseState(void);
~CMinSQPMeritPhaseState(void) {}
void Copy(const CMinSQPMeritPhaseState &obj);
//--- overloading
void operator=(const CMinSQPMeritPhaseState &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinSQPMeritPhaseState::CMinSQPMeritPhaseState(void)
{
m_n=0;
m_nec=0;
m_nic=0;
m_nlec=0;
m_nlic=0;
m_status=0;
m_increasebigc=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinSQPMeritPhaseState::Copy(const CMinSQPMeritPhaseState &obj)
{
m_n=obj.m_n;
m_nec=obj.m_nec;
m_nic=obj.m_nic;
m_nlec=obj.m_nlec;
m_nlic=obj.m_nlic;
m_status=obj.m_status;
m_increasebigc=obj.m_increasebigc;
m_rmeritphasestate=obj.m_rmeritphasestate;
m_d=obj.m_d;
m_dummylagmult=obj.m_dummylagmult;
m_dx=obj.m_dx;
m_lagmult=obj.m_lagmult;
m_penalties=obj.m_penalties;
m_stepkfi=obj.m_stepkfi;
m_stepkfic=obj.m_stepkfic;
m_stepkfin=obj.m_stepkfin;
m_stepklaggrad=obj.m_stepklaggrad;
m_stepknlaggrad=obj.m_stepknlaggrad;
m_stepkx=obj.m_stepkx;
m_stepkxc=obj.m_stepkxc;
m_stepkxn=obj.m_stepkxn;
m_tmpmerit=obj.m_tmpmerit;
m_tmplagrangianfg=obj.m_tmplagrangianfg;
m_stepkj=obj.m_stepkj;
m_stepkjc=obj.m_stepkjc;
m_stepkjn=obj.m_stepkjn;
}
//+------------------------------------------------------------------+
//| This object stores temporaries of SQP solver. |
//+------------------------------------------------------------------+
struct CMinSQPState
{
int m_fstagnationcnt;
int m_maxits;
int m_n;
int m_nec;
int m_nic;
int m_nlec;
int m_nlic;
int m_repbcidx;
int m_repiterationscount;
int m_replcidx;
int m_repnlcidx;
int m_repsimplexiterations1;
int m_repsimplexiterations2;
int m_repsimplexiterations3;
int m_repsimplexiterations;
int m_repterminationtype;
int m_trustradstagnationcnt;
double m_bigc;
double m_epsx;
double m_f;
double m_repbcerr;
double m_replcerr;
double m_repnlcerr;
double m_trustrad;
bool m_HasBndL[];
bool m_HasBndU[];
bool m_haslagmult;
bool m_needfij;
bool m_xupdated;
RCommState m_rstate;
CRowInt m_lcsrcidx;
CRowDouble m_backupfi;
CRowDouble m_backupx;
CRowDouble m_dummylagmult;
CRowDouble m_fi;
CRowDouble m_fscales;
CRowDouble m_meritlagmult;
CRowDouble m_s;
CRowDouble m_scaledbndl;
CRowDouble m_scaledbndu;
CRowDouble m_step0fi;
CRowDouble m_step0x;
CRowDouble m_stepkfi;
CRowDouble m_stepkx;
CRowDouble m_tracegamma;
CRowDouble m_x;
CMinSQPTmpMerit m_tmpmerit;
CMinSQPSubSolver m_subsolver;
CMinSQPMeritPhaseState m_meritstate;
CMatrixDouble m_abslagmemory;
CMatrixDouble m_j;
CMatrixDouble m_scaledcleic;
CMatrixDouble m_step0j;
CMatrixDouble m_stepkj;
//---
CMinSQPState(void);
~CMinSQPState(void) {}
//---
void Copy(const CMinSQPState &obj);
//--- overloading
void operator=(const CMinSQPState &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
CMinSQPState::CMinSQPState(void)
{
m_fstagnationcnt=0;
m_maxits=0;
m_n=0;
m_nec=0;
m_nic=0;
m_nlec=0;
m_nlic=0;
m_repbcidx=0;
m_repiterationscount=0;
m_replcidx=0;
m_repnlcidx=0;
m_repsimplexiterations1=0;
m_repsimplexiterations2=0;
m_repsimplexiterations3=0;
m_repsimplexiterations=0;
m_repterminationtype=0;
m_trustradstagnationcnt=0;
m_bigc=0;
m_epsx=0;
m_f=0;
m_repbcerr=0;
m_replcerr=0;
m_repnlcerr=0;
m_trustrad=0;
m_haslagmult=false;
m_needfij=false;
m_xupdated=false;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinSQPState::Copy(const CMinSQPState &obj)
{
m_fstagnationcnt=obj.m_fstagnationcnt;
m_maxits=obj.m_maxits;
m_n=obj.m_n;
m_nec=obj.m_nec;
m_nic=obj.m_nic;
m_nlec=obj.m_nlec;
m_nlic=obj.m_nlic;
m_repbcidx=obj.m_repbcidx;
m_repiterationscount=obj.m_repiterationscount;
m_replcidx=obj.m_replcidx;
m_repnlcidx=obj.m_repnlcidx;
m_repsimplexiterations1=obj.m_repsimplexiterations1;
m_repsimplexiterations2=obj.m_repsimplexiterations2;
m_repsimplexiterations3=obj.m_repsimplexiterations3;
m_repsimplexiterations=obj.m_repsimplexiterations;
m_repterminationtype=obj.m_repterminationtype;
m_trustradstagnationcnt=obj.m_trustradstagnationcnt;
m_bigc=obj.m_bigc;
m_epsx=obj.m_epsx;
m_f=obj.m_f;
m_repbcerr=obj.m_repbcerr;
m_replcerr=obj.m_replcerr;
m_repnlcerr=obj.m_repnlcerr;
m_trustrad=obj.m_trustrad;
ArrayCopy(m_HasBndL,obj.m_HasBndL);
ArrayCopy(m_HasBndU,obj.m_HasBndU);
m_haslagmult=obj.m_haslagmult;
m_needfij=obj.m_needfij;
m_xupdated=obj.m_xupdated;
m_rstate=obj.m_rstate;
m_lcsrcidx=obj.m_lcsrcidx;
m_backupfi=obj.m_backupfi;
m_backupx=obj.m_backupx;
m_dummylagmult=obj.m_dummylagmult;
m_fi=obj.m_fi;
m_fscales=obj.m_fscales;
m_meritlagmult=obj.m_meritlagmult;
m_s=obj.m_s;
m_scaledbndl=obj.m_scaledbndl;
m_scaledbndu=obj.m_scaledbndu;
m_step0fi=obj.m_step0fi;
m_step0x=obj.m_step0x;
m_stepkfi=obj.m_stepkfi;
m_stepkx=obj.m_stepkx;
m_tracegamma=obj.m_tracegamma;
m_x=obj.m_x;
m_tmpmerit=obj.m_tmpmerit;
m_subsolver=obj.m_subsolver;
m_meritstate=obj.m_meritstate;
m_abslagmemory=obj.m_abslagmemory;
m_j=obj.m_j;
m_scaledcleic=obj.m_scaledcleic;
m_step0j=obj.m_step0j;
m_stepkj=obj.m_stepkj;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CNLCSQP
{
public:
//--- constants
static const int m_fstagnationlimit;
static const int m_penaltymemlen;
static const int m_trustradstagnationlimit;
static const double m_augmentationfactor;
static const double m_inittrustrad;
static const double m_maxbigc;
static const double m_maxtrustraddecay;
static const double m_maxtrustradgrowth;
static const double m_meritfunctionbase;
static const double m_meritfunctiongain;
static const double m_sqpbigscale;
static const double m_sqpdeltadecrease;
static const double m_sqpdeltaincrease;
static const double m_sqpsmallscale;
static const double m_stagnationepsf;
static void MinSQPInitBuf(CRowDouble &bndl,CRowDouble &bndu,CRowDouble &s,CRowDouble &x0,int n,CMatrixDouble &cleic,CRowInt &lcsrcidx,int nec,int nic,int nlec,int nlic,double epsx,int m_maxits,CMinSQPState &State);
static bool MinSQPIteration(CMinSQPState &State,CSmoothnessMonitor &smonitor,bool userterminationneeded);
private:
static void InitQPSubSolver(CMinSQPState &sstate,CMinSQPSubSolver &subsolver);
static void QPSubSolverSetAlgoIPM(CMinSQPSubSolver &subsolver);
static bool QPSubProblemUpdateHessian(CMinSQPState &sstate,CMinSQPSubSolver &subsolver,CRowDouble &x0,CRowDouble &g0,CRowDouble &x1,CRowDouble &g1);
static void FASSolve(CMinSQPSubSolver &subsolver,CRowDouble &d0,CMatrixDouble &h,int nq,CRowDouble &b,int n,CRowDouble &bndl,CRowDouble &bndu,CSparseMatrix &a,int m,CRowDouble &al,CRowDouble &au,double trustrad,int &terminationtype,CRowDouble &d,CRowDouble &lagmult);
static bool QPSubproblemSolve(CMinSQPState &State,CMinSQPSubSolver &subsolver,CRowDouble &x,CRowDouble &fi,CMatrixDouble &jac,CRowDouble &d,CRowDouble &lagmult,int &terminationtype);
static void MeritPhaseInit(CMinSQPMeritPhaseState &meritstate,CRowDouble &curx,CRowDouble &curfi,CMatrixDouble &curj,int n,int nec,int nic,int nlec,int nlic,CMatrixDouble &abslagmemory,int memlen);
static bool MeritPhaseIteration(CMinSQPState &State,CMinSQPMeritPhaseState &meritstate,CSmoothnessMonitor &smonitor,bool userterminationneeded);
static void MeritPhaseResults(CMinSQPMeritPhaseState &meritstate,CRowDouble &curx,CRowDouble &curfi,CMatrixDouble &curj,CRowDouble &lagmult,bool &increasebigc,int &status);
static void SQPSendX(CMinSQPState &State,CRowDouble &xs);
static bool SQPRetrieveFIJ(CMinSQPState &State,CRowDouble &fis,CMatrixDouble &js);
static void SQPCopyState(CMinSQPState &State,CRowDouble &x0,CRowDouble &fi0,CMatrixDouble &j0,CRowDouble &x1,CRowDouble &fi1,CMatrixDouble &j1);
static void LagrangianFG(CMinSQPState &State,CRowDouble &x,double trustrad,CRowDouble &fi,CMatrixDouble &j,CRowDouble &lagmult,CMinSQPTmpLagrangian &tmp,double &f,CRowDouble &g);
static double MeritFunction(CMinSQPState &State,CRowDouble &x,CRowDouble &fi,CRowDouble &lagmult,CRowDouble &penalties,CMinSQPTmpMerit &tmp);
static double RawLagrangian(CMinSQPState &State,CRowDouble &x,CRowDouble &fi,CRowDouble &lagmult,CRowDouble &penalties,CMinSQPTmpMerit &tmp);
static void MeritFunctionAndRawLagrangian(CMinSQPState &State,CRowDouble &x,CRowDouble &fi,CRowDouble &lagmult,CRowDouble &penalties,CMinSQPTmpMerit &tmp,double &meritf,double &rawlag);
};
//+------------------------------------------------------------------+
//| Constants |
//+------------------------------------------------------------------+
const double CNLCSQP::m_sqpdeltadecrease=0.20;
const double CNLCSQP::m_sqpdeltaincrease=0.80;
const double CNLCSQP::m_maxtrustraddecay=0.1;
const double CNLCSQP::m_maxtrustradgrowth=1.333;
const double CNLCSQP::m_maxbigc=1.0E5;
const double CNLCSQP::m_meritfunctionbase=0.0;
const double CNLCSQP::m_meritfunctiongain=2.0;
const double CNLCSQP::m_augmentationfactor=10.0;
const double CNLCSQP::m_inittrustrad=0.1;
const double CNLCSQP::m_stagnationepsf=1.0E-12;
const int CNLCSQP::m_fstagnationlimit=20;
const int CNLCSQP::m_trustradstagnationlimit=10;
const double CNLCSQP::m_sqpbigscale=5.0;
const double CNLCSQP::m_sqpsmallscale=0.2;
const int CNLCSQP::m_penaltymemlen=5;
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CNLCSQP::MinSQPInitBuf(CRowDouble &bndl,CRowDouble &bndu,
CRowDouble &s,CRowDouble &x0,int n,
CMatrixDouble &cleic,CRowInt &lcsrcidx,
int nec,int nic,int nlec,int nlic,
double epsx,int m_maxits,CMinSQPState &State)
{
//--- create variables
int i=0;
int j=0;
double v=0;
double vv=0;
State.m_n=n;
State.m_nec=nec;
State.m_nic=nic;
State.m_nlec=nlec;
State.m_nlic=nlic;
//--- Prepare RCOMM State
State.m_rstate.ia.Resize(10);
ArrayResize(State.m_rstate.ba,4);
State.m_rstate.ra.Resize(7);
State.m_rstate.stage=-1;
State.m_needfij=false;
State.m_xupdated=false;
State.m_x.Resize(n);
State.m_fi.Resize(1+nlec+nlic);
State.m_j.Resize(1+nlec+nlic,n);
//--- Allocate memory.
ArrayResize(State.m_HasBndL,n);
ArrayResize(State.m_HasBndU,n);
State.m_s.Resize(n);
State.m_step0x.Resize(n);
State.m_stepkx.Resize(n);
State.m_backupx.Resize(n);
State.m_step0fi.Resize(1+nlec+nlic);
State.m_stepkfi.Resize(1+nlec+nlic);
State.m_backupfi.Resize(1+nlec+nlic);
State.m_step0j.Resize(1+nlec+nlic,n);
State.m_stepkj.Resize(1+nlec+nlic,n);
State.m_fscales.Resize(1+nlec+nlic);
State.m_tracegamma.Resize(1+nlec+nlic);
State.m_dummylagmult.Resize(nec+nic+nlec+nlic);
State.m_scaledbndl.Resize(n);
State.m_scaledbndu.Resize(n);
State.m_scaledcleic.Resize(nec+nic,n+1);
State.m_lcsrcidx.Resize(nec+nic);
State.m_meritlagmult.Resize(nec+nic+nlec+nlic);
State.m_abslagmemory=matrix<double>::Zeros(m_penaltymemlen,nec+nic+nlec+nlic);
//--- Prepare scaled problem
for(i=0; i<n; i++)
{
State.m_HasBndL[i]=MathIsValidNumber(bndl[i]);
State.m_HasBndU[i]=MathIsValidNumber(bndu[i]);
if(State.m_HasBndL[i])
State.m_scaledbndl.Set(i,bndl[i]/s[i]);
if(State.m_HasBndU[i])
State.m_scaledbndu.Set(i,bndu[i]/s[i]);
if(State.m_HasBndL[i] && State.m_HasBndU[i])
{
//--- check
if(!CAp::Assert(bndl[i]<=bndu[i],__FUNCTION__+": integrity check failed,box constraints are inconsistent"))
return;
}
State.m_step0x.Set(i,x0[i]/s[i]);
State.m_s.Set(i,s[i]);
}
//--- Permutation
State.m_lcsrcidx=lcsrcidx;
for(i=0; i<nec+nic; i++)
{
//--- Scale and normalize linear constraints
vv=0.0;
for(j=0; j<n; j++)
{
v=cleic.Get(i,j)*s[j];
State.m_scaledcleic.Set(i,j,v);
vv+=v*v;
}
vv=MathSqrt(vv);
State.m_scaledcleic.Set(i,n,cleic.Get(i,n));
if(vv>0.0)
State.m_scaledcleic.Row(i,State.m_scaledcleic[i]/vv);
}
//--- Initial enforcement of box constraints
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i])
State.m_step0x.Set(i,MathMax(State.m_step0x[i],State.m_scaledbndl[i]));
if(State.m_HasBndU[i])
State.m_step0x.Set(i,MathMin(State.m_step0x[i],State.m_scaledbndu[i]));
}
//--- Stopping criteria
State.m_epsx=epsx;
State.m_maxits=m_maxits;
//--- Report fields
State.m_repsimplexiterations=0;
State.m_repsimplexiterations1=0;
State.m_repsimplexiterations2=0;
State.m_repsimplexiterations3=0;
State.m_repterminationtype=0;
State.m_repbcerr=0;
State.m_repbcidx=-1;
State.m_replcerr=0;
State.m_replcidx=-1;
State.m_repnlcerr=0;
State.m_repnlcidx=-1;
State.m_repiterationscount=0;
//--- Integrity checks
if(!CAp::Assert(m_sqpdeltadecrease<m_sqpdeltaincrease,__FUNCTION__+": integrity check failed"))
return;
}
//+------------------------------------------------------------------+
//| This function performs actual processing for SQP algorithm. It |
//| expects that caller redirects its reverse communication requests |
//| NeedFiJ/XUpdated to external user who will provide analytic |
//| derivative (or handle reports about progress). |
//| In case external user does not have analytic derivative, it is |
//| responsibility of caller to intercept NeedFiJ request and replace|
//| it with appropriate numerical differentiation scheme. |
//| Results are stored: |
//| * point - in State.StepKX |
//| IMPORTANT: this function works with scaled problem formulation; |
//| it is responsibility of the caller to UnScale request |
//| and scale Jacobian. |
//| NOTE: SMonitor is expected to be correctly initialized smoothness|
//| monitor. |
//+------------------------------------------------------------------+
bool CNLCSQP::MinSQPIteration(CMinSQPState &State,
CSmoothnessMonitor &smonitor,
bool userterminationneeded)
{
//--- create variables
bool result=false;
int n=0;
int nslack=0;
int nec=0;
int nic=0;
int nlec=0;
int nlic=0;
int i=0;
int j=0;
double v=0;
double vv=0;
double mx=0;
int status=0;
double deltamax=0;
double multiplyby=0;
double setscaleto=0;
double prevtrustrad=0;
int subiterationidx=0;
bool trustradstagnated=false;
bool dotrace=false;
bool dodetailedtrace=false;
bool increasebigc=false;
int label=-1;
//--- Reverse communication preparations
//--- I know it looks ugly, but it works the same way
//--- anywhere from C++ to Python.
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(State.m_rstate.stage>=0)
{
n=State.m_rstate.ia[0];
nslack=State.m_rstate.ia[1];
nec=State.m_rstate.ia[2];
nic=State.m_rstate.ia[3];
nlec=State.m_rstate.ia[4];
nlic=State.m_rstate.ia[5];
i=State.m_rstate.ia[6];
j=State.m_rstate.ia[7];
status=State.m_rstate.ia[8];
subiterationidx=State.m_rstate.ia[9];
trustradstagnated=State.m_rstate.ba[0];
dotrace=State.m_rstate.ba[1];
dodetailedtrace=State.m_rstate.ba[2];
increasebigc=State.m_rstate.ba[3];
v=State.m_rstate.ra[0];
vv=State.m_rstate.ra[1];
mx=State.m_rstate.ra[2];
deltamax=State.m_rstate.ra[3];
multiplyby=State.m_rstate.ra[4];
setscaleto=State.m_rstate.ra[5];
prevtrustrad=State.m_rstate.ra[6];
}
else
{
n=359;
nslack=-58;
nec=-919;
nic=-909;
nlec=81;
nlic=255;
i=74;
j=-788;
status=809;
subiterationidx=205;
trustradstagnated=false;
dotrace=true;
dodetailedtrace=false;
increasebigc=true;
v=-541;
vv=-698;
mx=-900;
deltamax=-318;
multiplyby=-940;
setscaleto=1016;
prevtrustrad=-229;
}
switch(State.m_rstate.stage)
{
case 0:
State.m_needfij=false;
if(!SQPRetrieveFIJ(State,State.m_step0fi,State.m_step0j))
{
//--- Failed to retrieve function/Jaconian, infinities detected!
State.m_stepkx=State.m_step0x;
State.m_repterminationtype=-8;
return(false);
}
SQPCopyState(State,State.m_step0x,State.m_step0fi,State.m_step0j,State.m_stepkx,State.m_stepkfi,State.m_stepkj);
SQPSendX(State,State.m_stepkx);
State.m_f=State.m_stepkfi[0]*State.m_fscales[0];
State.m_xupdated=true;
State.m_rstate.stage=1;
break;
case 1:
State.m_xupdated=false;
COptServ::CheckLcViolation(State.m_scaledcleic,State.m_lcsrcidx,nec,nic,State.m_stepkx,n,State.m_replcerr,State.m_replcidx);
COptServ::UnScaleAndCheckNLcViolation(State.m_stepkfi,State.m_fscales,nlec,nlic,State.m_repnlcerr,State.m_repnlcidx);
//--- Trace output (if needed)
if(dotrace)
{
CAp::Trace("\n\n");
CAp::Trace("////////////////////////////////////////////////////////////////////////////////////////////////////\n");
CAp::Trace("//--- SQP SOLVER STARTED //\n");
CAp::Trace("////////////////////////////////////////////////////////////////////////////////////////////////////\n");
}
//--- Perform outer (NLC) iterations
State.m_bigc=500;
InitQPSubSolver(State,State.m_subsolver);
label=3;
break;
case 2:
label=5;
break;
default:
//--- Routine body
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
nlec=State.m_nlec;
nlic=State.m_nlic;
nslack=n+2*(nec+nlec)+(nic+nlic);
dotrace=CAp::IsTraceEnabled("SQP");
dodetailedtrace=dotrace && CAp::IsTraceEnabled("SQP.DETAILED");
//--- Prepare rcomm interface
State.m_needfij=false;
State.m_xupdated=false;
//--- Initialize algorithm data:
//---*Lagrangian and "Big C" estimates
//--- * trust region
//--- * initial function scales (vector of 1's)
//--- * current approximation of the Hessian matrix H (unit matrix)
//--- * initial linearized constraints
//--- * initial violation of linear/nonlinear constraints
State.m_fstagnationcnt=0;
State.m_trustradstagnationcnt=0;
State.m_trustrad=m_inittrustrad;
State.m_fscales.Fill(1);
State.m_tracegamma.Fill(0);
State.m_haslagmult=false;
//--- Avoid spurious warnings about possibly uninitialized vars
status=0;
//--- Evaluate function vector and Jacobian at Step0X, send first location report.
//--- Compute initial violation of constraints.
SQPSendX(State,State.m_step0x);
State.m_needfij=true;
State.m_rstate.stage=0;
break;
}
while(label>0)
{
switch(label)
{
case 3:
//--- Before beginning new outer iteration:
//--- * renormalize target function and/or constraints, if some of them have too large magnitudes
//--- * save initial point for the outer iteration
for(i=0; i<=nlec+nlic; i++)
{
//--- Determine (a) multiplicative coefficient applied to function value
//--- and Jacobian row, and (b) new value of the function scale.
mx=MathAbs(State.m_stepkj[i]+0).Max();
multiplyby=1.0;
setscaleto=State.m_fscales[i];
if(mx>=m_sqpbigscale)
{
multiplyby=1/mx;
setscaleto=State.m_fscales[i]*mx;
}
if(mx<=m_sqpsmallscale && State.m_fscales[i]>1.0)
{
if((State.m_fscales[i]*mx)>1.0)
{
multiplyby=1/mx;
setscaleto=State.m_fscales[i]*mx;
}
else
{
multiplyby=State.m_fscales[i];
setscaleto=1.0;
}
}
if(multiplyby!=1.0)
{
//--- Function #I needs renormalization:
//--- * update function vector element and Jacobian matrix row
//--- * update FScales[] and TraceGamma[] arrays
State.m_stepkfi.Mul(i,multiplyby);
State.m_stepkj.Row(i,State.m_stepkj[i]*multiplyby);
State.m_fscales.Set(i,setscaleto);
State.m_tracegamma.Mul(i,multiplyby);
}
}
//--- Trace output (if needed)
if(dotrace)
{
CAp::Trace(StringFormat("\n=== OUTER ITERATION %5d STARTED ==================================================================\n",State.m_repiterationscount));
if(dodetailedtrace)
{
CAp::Trace("> printing raw data (prior to applying variable and function scales)\n");
CAp::Trace("X (raw) = ");
CApServ::TraceVectorUnscaledUnshiftedAutopRec(State.m_step0x,n,State.m_s,true,State.m_s,false);
CAp::Trace("\n");
CAp::Trace("> printing scaled data (after applying variable and function scales)\n");
CAp::Trace("X (scaled) = ");
CApServ::TraceVectorAutopRec(State.m_step0x,0,n);
CAp::Trace("\n");
CAp::Trace("FScales = ");
CApServ::TraceVectorAutopRec(State.m_fscales,0,1+nlec+nlic);
CAp::Trace("\n");
CAp::Trace("GammaScl = ");
CApServ::TraceVectorAutopRec(State.m_tracegamma,0,1+nlec+nlic);
CAp::Trace("\n");
CAp::Trace("Fi (scaled) = ");
CApServ::TraceVectorAutopRec(State.m_stepkfi,0,1+nlec+nlic);
CAp::Trace("\n");
CAp::Trace("|Ji| (scaled) = ");
CApServ::TraceRowNrm1AutopRec(State.m_stepkj,0,1+nlec+nlic,0,n);
CAp::Trace("\n");
}
mx=0;
for(i=1; i<=nlec; i++)
mx=MathMax(mx,MathAbs(State.m_stepkfi[i]));
for(i=nlec+1; i<=nlec+nlic; i++)
mx=MathMax(mx,State.m_stepkfi[i]);
CAp::Trace(StringFormat("trustRad = %.3E\n",State.m_trustrad));
CAp::Trace(StringFormat("lin.violation = %.3E (scaled violation of linear constraints)\n",State.m_replcerr));
CAp::Trace(StringFormat("nlc.violation = %.3E (scaled violation of nonlinear constraints)\n",mx));
CAp::Trace(StringFormat("gamma0 = %.3E (Hessian 2-norm estimate for target)\n",State.m_tracegamma[0]));
j=(int)State.m_tracegamma.ArgMax();
CAp::Trace(StringFormat("gammaMax = %.3E (maximum over Hessian 2-norm estimates for target/constraints)\n",State.m_tracegamma[j]));
CAp::Trace(StringFormat("arg(gammaMax) = %d (function index; 0 for target,>0 for nonlinear constraints)\n",j));
}
//--- PHASE 2
//--- This phase is a primary part of the algorithm which is responsible for its
//--- convergence properties.
//--- It solves QP subproblem with possible activation and deactivation of constraints
//--- and then starts backtracking (step length is bounded by 1.0) merit function search
//--- (with second-order correction to deal with Maratos effect) on the direction produced
//--- by QP subproblem.
//--- This phase is everything we need to in order to have convergence; however,
//--- it has one performance-related issue: using "general" interior point QP solver
//--- results in slow solution times. Fast equality-constrained phase is essential for
//--- the quick convergence.
QPSubSolverSetAlgoIPM(State.m_subsolver);
SQPCopyState(State,State.m_stepkx,State.m_stepkfi,State.m_stepkj,State.m_step0x,State.m_step0fi,State.m_step0j);
MeritPhaseInit(State.m_meritstate,State.m_stepkx,State.m_stepkfi,State.m_stepkj,n,nec,nic,nlec,nlic,State.m_abslagmemory,m_penaltymemlen);
case 5:
if(MeritPhaseIteration(State,State.m_meritstate,smonitor,userterminationneeded))
{
State.m_rstate.stage=2;
label=-1;
break;
}
case 6:
MeritPhaseResults(State.m_meritstate,State.m_stepkx,State.m_stepkfi,State.m_stepkj,State.m_meritlagmult,increasebigc,status);
if(status==0)
{
label=4;
break;
}
//--- check
if(!CAp::Assert(status>0,__FUNCTION__+": integrity check failed"))
return(false);
State.m_haslagmult=true;
State.m_abslagmemory.DeleteRow(m_penaltymemlen);
State.m_abslagmemory.InsertRow(0);
State.m_abslagmemory.Row(0,State.m_meritlagmult.Abs()+0);
//--- Caller requested to update BigC - L1 penalty coefficient for linearized constraint violation
if(increasebigc)
State.m_bigc=MathMin(10*State.m_bigc,m_maxbigc);
//--- Update trust region.
//--- NOTE: when trust region radius remains fixed for a long time it may mean that we
//--- stagnated in eternal loop. In such cases we decrease it slightly in order
//--- to break possible loop. If such decrease was unnecessary, it may be easily
//--- fixed within few iterations.
deltamax=(MathAbs(State.m_step0x-State.m_stepkx+0)/State.m_trustrad).Max();
trustradstagnated=false;
State.m_trustradstagnationcnt++;
prevtrustrad=State.m_trustrad;
if(deltamax<=m_sqpdeltadecrease)
State.m_trustrad=State.m_trustrad*MathMax(deltamax/m_sqpdeltadecrease,m_maxtrustraddecay);
if(deltamax>=m_sqpdeltaincrease)
State.m_trustrad=State.m_trustrad*MathMin(deltamax/m_sqpdeltaincrease,m_maxtrustradgrowth);
if(State.m_trustrad<(0.99*prevtrustrad) || State.m_trustrad>(1.01*prevtrustrad))
State.m_trustradstagnationcnt=0;
if(State.m_trustradstagnationcnt>=m_trustradstagnationlimit)
{
State.m_trustrad=0.5*State.m_trustrad;
State.m_trustradstagnationcnt=0;
trustradstagnated=true;
}
//--- Trace
if(dotrace)
{
CAp::Trace("\n--- outer iteration ends ---------------------------------------------------------------------------\n");
CAp::Trace(StringFormat("deltaMax = %.3f (ratio of step length to trust radius)\n",deltamax));
CAp::Trace(StringFormat("newTrustRad = %.3E",State.m_trustrad));
if(!trustradstagnated)
{
if(State.m_trustrad>(double)(prevtrustrad))
CAp::Trace(",trust radius increased");
if(State.m_trustrad<(double)(prevtrustrad))
CAp::Trace(",trust radius decreased");
}
else
CAp::Trace(StringFormat(",trust radius forcibly decreased due to stagnation for %d iterations",m_trustradstagnationlimit));
CAp::Trace("\n");
if(increasebigc)
CAp::Trace(StringFormat("BigC = %.3E (short step was performed,but some constraints are still infeasible - increasing)\n",State.m_bigc));
}
//--- Advance outer iteration counter, test stopping criteria
State.m_repiterationscount++;
if(MathAbs(State.m_stepkfi[0]-State.m_step0fi[0])<=(m_stagnationepsf*MathAbs(State.m_step0fi[0])))
State.m_fstagnationcnt++;
else
State.m_fstagnationcnt=0;
if(State.m_trustrad<=State.m_epsx)
{
State.m_repterminationtype=2;
if(dotrace)
CAp::Trace(StringFormat("> stopping condition met: trust radius is smaller than %.3E\n",State.m_epsx));
label=4;
break;
}
if(State.m_maxits>0 && State.m_repiterationscount>=State.m_maxits)
{
State.m_repterminationtype=5;
if(dotrace)
CAp::Trace(StringFormat("> stopping condition met: %d iterations performed\n",State.m_repiterationscount));
label=4;
break;
}
if(State.m_fstagnationcnt>=m_fstagnationlimit)
{
State.m_repterminationtype=7;
if(dotrace)
CAp::Trace(StringFormat("> stopping criteria are too stringent: F stagnated for %d its,stopping\n",State.m_fstagnationcnt));
label=4;
break;
}
label=3;
break;
case 4:
COptServ::SmoothnessMonitorTraceStatus(smonitor,dotrace);
return(false);
break;
}
}
//--- Saving State
State.m_rstate.ba[0]=trustradstagnated;
State.m_rstate.ba[1]=dotrace;
State.m_rstate.ba[2]=dodetailedtrace;
State.m_rstate.ba[3]=increasebigc;
State.m_rstate.ia.Set(0,n);
State.m_rstate.ia.Set(1,nslack);
State.m_rstate.ia.Set(2,nec);
State.m_rstate.ia.Set(3,nic);
State.m_rstate.ia.Set(4,nlec);
State.m_rstate.ia.Set(5,nlic);
State.m_rstate.ia.Set(6,i);
State.m_rstate.ia.Set(7,j);
State.m_rstate.ia.Set(8,status);
State.m_rstate.ia.Set(9,subiterationidx);
State.m_rstate.ra.Set(0,v);
State.m_rstate.ra.Set(1,vv);
State.m_rstate.ra.Set(2,mx);
State.m_rstate.ra.Set(3,deltamax);
State.m_rstate.ra.Set(4,multiplyby);
State.m_rstate.ra.Set(5,setscaleto);
State.m_rstate.ra.Set(6,prevtrustrad);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| This function initializes SQP subproblem. |
//| Should be called once in the beginning of the optimization. |
//| INPUT PARAMETERS: |
//| SState - solver State |
//| Subsolver - SQP subproblem to initialize |
//| RETURN VALUE: |
//| True on success |
//| False on failure of the QP solver (unexpected... but possible |
//| due to numerical errors) |
//+------------------------------------------------------------------+
void CNLCSQP::InitQPSubSolver(CMinSQPState &sstate,
CMinSQPSubSolver &subsolver)
{
//--- create variables
int n=sstate.m_n;
int nec=sstate.m_nec;
int nic=sstate.m_nic;
int nlec=sstate.m_nlec;
int nlic=sstate.m_nlic;
int nslack=n+2*(nec+nlec)+(nic+nlic);
int lccnt=nec+nic+nlec+nlic;
int nnz=0;
int offs=0;
//--- Allocate temporaries
subsolver.m_cural.Resize(lccnt);
subsolver.m_curau.Resize(lccnt);
subsolver.m_curbndl.Resize(nslack);
subsolver.m_curbndu.Resize(nslack);
subsolver.m_curb.Resize(nslack);
subsolver.m_sk.Resize(n);
subsolver.m_yk.Resize(n);
//--- Initial State
subsolver.m_algokind=0;
subsolver.m_h=matrix<double>::Identity(n,n);
//--- Linear constraints do not change across subiterations, that's
//--- why we allocate storage for them at the start of the program.
//--- A full set of "raw" constraints is stored; later we will filter
//--- out inequality ones which are inactive anywhere in the current
//--- trust region.
//--- NOTE: because sparserawlc object stores only linear constraint
//--- (linearizations of nonlinear ones are not stored) we
//--- allocate only minimum necessary space.
nnz=sstate.m_scaledcleic.Compare(matrix<double>::Zeros(nec+nic,n));
subsolver.m_sparserawlc.m_RIdx.Resize(nec+nic+1);
subsolver.m_sparserawlc.m_Vals.Resize(nnz);
subsolver.m_sparserawlc.m_Idx.Resize(nnz);
subsolver.m_sparserawlc.m_DIdx.Resize(nec+nic);
subsolver.m_sparserawlc.m_UIdx.Resize(nec+nic);
offs=0;
subsolver.m_sparserawlc.m_RIdx.Set(0,0);
for(int i=0; i<nec+nic; i++)
{
for(int j=0; j<n; j++)
{
if(sstate.m_scaledcleic.Get(i,j)!=0.0)
{
//--- Primary part of the matrix
subsolver.m_sparserawlc.m_Vals.Set(offs,sstate.m_scaledcleic.Get(i,j));
subsolver.m_sparserawlc.m_Idx.Set(offs,j);
offs++;
}
}
subsolver.m_sparserawlc.m_RIdx.Set(i+1,offs);
}
subsolver.m_sparserawlc.m_MatrixType=1;
subsolver.m_sparserawlc.m_NInitialized=subsolver.m_sparserawlc.m_RIdx[nec+nic];
subsolver.m_sparserawlc.m_M=nec+nic;
subsolver.m_sparserawlc.m_N=n;
CSparse::SparseInitDUIdx(subsolver.m_sparserawlc);
}
//+------------------------------------------------------------------+
//| This function sets subsolver algorithm to interior point |
//| method(IPM) |
//+------------------------------------------------------------------+
void CNLCSQP::QPSubSolverSetAlgoIPM(CMinSQPSubSolver &subsolver)
{
subsolver.m_algokind=0;
}
//+------------------------------------------------------------------+
//| Updates Hessian estimate, uses regularized formula which prevents|
//| Hessian eigenvalues from decreasing below sqrt(Eps) and rejects |
//| updates larger than 1/sqrt(Eps) in magnitude. |
//| INPUT PARAMETERS: |
//| SState - solver State |
//| Subsolver - SQP subproblem to initialize |
//| X0, G0 - point #0 and gradient at #0, array[N] |
//| X1, G1 - point #1 and gradient at #1, array[N] |
//+------------------------------------------------------------------+
bool CNLCSQP::QPSubProblemUpdateHessian(CMinSQPState &sstate,
CMinSQPSubSolver &subsolver,
CRowDouble &x0,
CRowDouble &g0,
CRowDouble &x1,
CRowDouble &g1)
{
//--- create variables
bool result=false;
int i=0;
int n=0;
double shs=0;
double rawsy=0;
double sy=0;
double snrm2=0;
double ynrm2=0;
double v2=0;
double gk=0;
double sk=0;
double yk=0;
double mxs=0;
double mxy=0;
double mxhs=0;
double reg=0;
double big=0;
double growth=0;
double eigold=0;
double eignew=0;
double eigcorrection=0;
//--- Algorithm parameters
reg=100*MathSqrt(CMath::m_machineepsilon);
big=1/reg;
growth=100.0;
//--- Proceed
result=false;
n=sstate.m_n;
subsolver.m_tmp0.Resize(n);
rawsy=0;
sy=0;
snrm2=0;
ynrm2=0;
v2=0;
mxs=0;
mxy=0;
for(i=0; i<n; i++)
{
//--- Fetch components
sk=x1[i]-x0[i];
yk=g1[i]-g0[i];
gk=g0[i];
//--- Compute raw (S,Y) without regularization (to be used later
//--- during comparison with zero)
rawsy+=sk*yk;
//--- Convexify Y
yk+=reg*sk;
//--- Compute various coefficients using regularized values
sy+=sk*yk;
snrm2+=sk*sk;
ynrm2+=yk*yk;
v2+=gk*gk;
mxs=MathMax(mxs,MathAbs(sk));
mxy=MathMax(mxy,MathAbs(yk));
subsolver.m_sk.Set(i,sk);
subsolver.m_yk.Set(i,yk);
}
shs=CAblas::RMatrixSyvMVect(n,subsolver.m_h,0,0,true,subsolver.m_sk,0,subsolver.m_tmp0);
CAblas::RMatrixGemVect(n,n,1.0,subsolver.m_h,0,0,0,subsolver.m_sk,0,0.0,subsolver.m_tmp0,0);
mxhs=(subsolver.m_tmp0.Abs()+0).Max();
//--- Skip updates if (Sk,Yk)<=0 or Sk*H*Sk<=0
//--- NOTE: we use 0.5*(SY+RawSY) in place of (Sk,Yk) which allows us to have slight
//--- nonconvexity due to numerical noise.
if((0.5*(sy+rawsy))<=0.0 || shs<=0.0 || snrm2<=0.0)
return(result);
//--- check
if(!CAp::Assert(sy>0.0,__FUNCTION__+": integrity check failed"))
return(false);
//--- Skip updates with too short steps
//--- NOTE: may prevent us from updating Hessian near the solution
if(mxs<=CApServ::Coalesce(sstate.m_epsx,MathSqrt(CMath::m_machineepsilon)))
return(result);
//--- Too large Hessian updates sometimes may come from noisy or nonsmooth problems.
//--- Skip updates with max(Yk)^2/(Yk,Sk)>=BIG or max(H*Sk)^2/(Sk*H*Sk)>=BIG
if((CMath::Sqr(mxy)/sy)>=big)
return(result);
if((CMath::Sqr(mxhs)/shs)>=big)
return(result);
//--- Compare eigenvalues of H: old one removed by update, and new one.
//--- We require that new eigenvalue is not much larger/smaller than the old one.
//--- In order to enforce this condition we compute correction coefficient and
//--- multiply one of the rank-1 updates by this coefficient.
eigold=shs/snrm2;
eignew=ynrm2/sy;
eigcorrection=1.0;
if(eignew>(eigold*growth))
eigcorrection=1/(eignew/(eigold*growth));
if(eignew<(eigold/growth))
eigcorrection=1/(eignew/(eigold/growth));
//--- Update Hessian
CAblas::RMatrixGer(n,n,subsolver.m_h,0,0,-(1/shs),subsolver.m_tmp0,0,subsolver.m_tmp0,0);
CAblas::RMatrixGer(n,n,subsolver.m_h,0,0,eigcorrection*(1/sy),subsolver.m_yk,0,subsolver.m_yk,0);
//--- return result
return(true);
}
//+------------------------------------------------------------------+
//| This function solves QP subproblem given by initial point X, |
//| function vector Fi and Jacobian Jac, and returns estimates of |
//| Lagrangian multipliers and search direction D[]. |
//+------------------------------------------------------------------+
void CNLCSQP::FASSolve(CMinSQPSubSolver &subsolver,
CRowDouble &d0,
CMatrixDouble &h,
int nq,
CRowDouble &b,
int n,
CRowDouble &bndl,
CRowDouble &bndu,
CSparseMatrix &a,
int m,
CRowDouble &al,
CRowDouble &au,
double trustrad,
int &terminationtype,
CRowDouble &d,
CRowDouble &lagmult)
{
//--- create variables
int i=0;
terminationtype=1;
//--- Initial point, integrity check for constraints
ArrayResize(subsolver.m_HasBndL,n);
ArrayResize(subsolver.m_HasBndU,n);
for(i=0; i<n; i++)
{
subsolver.m_HasBndL[i]=MathIsValidNumber(bndl[i]);
subsolver.m_HasBndU[i]=MathIsValidNumber(bndu[i]);
//--- check
if(!CAp::Assert(!subsolver.m_HasBndL[i] || bndl[i]<=d0[i],__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(!subsolver.m_HasBndU[i] || bndu[i]>=(double)(d0[i]),__FUNCTION__+": integrity check failed"))
return;
d.Set(i,d0[i]);
}
ArrayResize(subsolver.m_hasal,m);
ArrayResize(subsolver.m_hasau,m);
for(i=0; i<m; i++)
{
subsolver.m_hasal[i]=MathIsValidNumber(al[i]);
subsolver.m_hasau[i]=MathIsValidNumber(au[i]);
if(subsolver.m_hasal[i] && subsolver.m_hasau[i])
{
//--- check
if(!CAp::Assert(al[i]<=au[i],__FUNCTION__+": integrity check failed"))
return;
}
}
subsolver.m_activea.Resize(n,n);
subsolver.m_activerhs.Resize(n);
subsolver.m_activeidx.Resize(n);
subsolver.m_activesetsize=0;
//--- Activate equality constraints (at most N)
for(i=0; i<m; i++)
{
if(subsolver.m_hasal[i] && subsolver.m_hasau[i] && al[i]==au[i])
{
//--- Stop if full set of constraints is activated
if(subsolver.m_activesetsize>=n)
break;
}
}
subsolver.m_tmp0=vector<double>::Full(n,trustrad);
subsolver.m_tmp1=vector<double>::Zeros(n);
CVIPMSolver::VIPMInitDenseWithSlacks(subsolver.m_ipmsolver,subsolver.m_tmp0,subsolver.m_tmp1,nq,n);
CVIPMSolver::VIPMSetQuadraticLinear(subsolver.m_ipmsolver,h,subsolver.m_sparsedummy,0,true,b);
CVIPMSolver::VIPMSetConstraints(subsolver.m_ipmsolver,bndl,bndu,a,m,subsolver.m_densedummy,0,al,au);
CVIPMSolver::VIPMOptimize(subsolver.m_ipmsolver,false,subsolver.m_tmp0,subsolver.m_tmp1,subsolver.m_tmp2,terminationtype);
if(terminationtype<=0)
return;
d=subsolver.m_tmp0;
lagmult=subsolver.m_tmp2;
}
//+------------------------------------------------------------------+
//| This function solves QP subproblem given by initial point X, |
//| function vector Fi and Jacobian Jac, and returns estimates of |
//| Lagrangian multipliers and search direction D[]. |
//+------------------------------------------------------------------+
bool CNLCSQP::QPSubproblemSolve(CMinSQPState &State,
CMinSQPSubSolver &subsolver,
CRowDouble &x,
CRowDouble &fi,
CMatrixDouble &jac,
CRowDouble &d,
CRowDouble &lagmult,
int &terminationtype)
{
//--- create variables
bool result=false;
int n=State.m_n;
int nec=State.m_nec;
int nic=State.m_nic;
int nlec=State.m_nlec;
int nlic=State.m_nlic;
int nslack=n+2*(nec+nlec)+(nic+nlic);
int lccnt=nec+nic+nlec+nlic;
//--- Locations of slack variables
int offsslackec=n;
int offsslacknlec=n+2*nec;
int offsslackic=n+2*nec+2*nlec;
int offsslacknlic=n+2*(nec+nlec)+nic;
int i=0;
int j=0;
int k=0;
double v=0;
double vv=0;
double vright=0;
double vmax=0;
int offs=0;
int nnz=0;
int j0=0;
int j1=0;
terminationtype=0;
//--- Prepare temporary structures
subsolver.m_cural.Resize(lccnt);
subsolver.m_curau.Resize(lccnt);
subsolver.m_d0=vector<double>::Zeros(nslack);
//--- Prepare default solution: all zeros
result=true;
terminationtype=0;
d.Fill(0.0);
lagmult.Fill(0);
//--- Linear term B
//--- NOTE: elements [N,NSlack) are equal to bigC + perturbation to improve numeric properties of QP problem
subsolver.m_curb=jac[0]+0;
subsolver.m_curb.Resize(n);
v=subsolver.m_curb.Dot(subsolver.m_curb);
v=CApServ::Coalesce(MathSqrt(v),1.0);
subsolver.m_curb.Resize(nslack);
for(i=n; i<nslack; i++)
subsolver.m_curb.Set(i,(State.m_bigc+1.0/(1+i))*v);
//--- Trust radius constraints for primary variables
for(i=0; i<n; i++)
{
subsolver.m_curbndl.Set(i,-State.m_trustrad);
subsolver.m_curbndu.Set(i,State.m_trustrad);
if(State.m_HasBndL[i])
subsolver.m_curbndl.Set(i,MathMax(subsolver.m_curbndl[i],State.m_scaledbndl[i]-x[i]));
if(State.m_HasBndU[i])
subsolver.m_curbndu.Set(i,MathMin(subsolver.m_curbndu[i],State.m_scaledbndu[i]-x[i]));
}
//--- Prepare storage for "effective" constraining matrix
nnz=subsolver.m_sparserawlc.m_RIdx[nec+nic];
for(i=0; i<nlec+nlic; i++)
{
for(j=0; j<n; j++)
if(jac.Get(1+i,j)!=0.0)
nnz++;
}
nnz=nnz+2*nec+nic;
nnz=nnz+2*nlec+nlic;
subsolver.m_sparseefflc.m_RIdx.Resize(lccnt+1);
subsolver.m_sparseefflc.m_Vals.Resize(nnz);
subsolver.m_sparseefflc.m_Idx.Resize(nnz);
subsolver.m_sparseefflc.m_DIdx.Resize(lccnt);
subsolver.m_sparseefflc.m_UIdx.Resize(lccnt);
subsolver.m_sparseefflc.m_M=0;
subsolver.m_sparseefflc.m_N=nslack;
subsolver.m_sparseefflc.m_MatrixType=1;
//--- Append linear equality/inequality constraints
//--- Scan sparsified linear constraints stored in sparserawlc[], skip ones
//--- which are inactive anywhere in the trust region.
subsolver.m_tmp0=x;
subsolver.m_tmp0.Resize(nslack);
for(i=n; i<nslack; i++)
subsolver.m_tmp0.Set(i,0);
for(i=0; i<nec+nic; i++)
{
//--- Calculate:
//--- * VRight - product of X[] (extended with zeros up to NSlack elements)
//--- and AR[i] - Ith row of sparserawlc matrix.
//--- * VMax - maximum value of X*ARi computed over trust region
vright=0;
vmax=0;
j0=subsolver.m_sparserawlc.m_RIdx[i];
j1=subsolver.m_sparserawlc.m_RIdx[i+1];
for(k=j0; k<j1; k++)
{
j=subsolver.m_sparserawlc.m_Idx[k];
v=subsolver.m_tmp0[j];
vv=subsolver.m_sparserawlc.m_Vals[k];
vright+=vv*v;
if(vv>=0)
vmax+=vv*(v+subsolver.m_curbndu[j]);
else
vmax+=vv*(v+subsolver.m_curbndl[j]);
}
//--- If constraint is an inequality one and guaranteed to be inactive
//--- within trust region, it is skipped (row itself is retained but
//--- filled by zeros).
if(i>=nec && vmax<=State.m_scaledcleic.Get(i,n))
{
offs=subsolver.m_sparseefflc.m_RIdx[i];
subsolver.m_sparseefflc.m_Vals.Set(offs,-1);
subsolver.m_sparseefflc.m_Idx.Set(offs,offsslackic+(i-nec));
subsolver.m_sparseefflc.m_RIdx.Set(i+1,offs+1);
subsolver.m_cural.Set(i,0.0);
subsolver.m_curau.Set(i,0.0);
subsolver.m_curbndl.Set(offsslackic+(i-nec),0);
subsolver.m_curbndu.Set(offsslackic+(i-nec),0);
continue;
}
//--- Start working on row I
offs=subsolver.m_sparseefflc.m_RIdx[i];
//--- Copy constraint from sparserawlc[] to sparseefflc[]
j0=subsolver.m_sparserawlc.m_RIdx[i];
j1=subsolver.m_sparserawlc.m_RIdx[i+1];
for(k=j0; k<j1; k++)
{
subsolver.m_sparseefflc.m_Idx.Set(offs,subsolver.m_sparserawlc.m_Idx[k]);
subsolver.m_sparseefflc.m_Vals.Set(offs,subsolver.m_sparserawlc.m_Vals[k]);
offs++;
}
//--- Set up slack variables
if(i<nec)
{
subsolver.m_sparseefflc.m_Vals.Set(offs,-1);
subsolver.m_sparseefflc.m_Vals.Set(offs+1,1);
subsolver.m_sparseefflc.m_Idx.Set(offs,offsslackec+2*i);
subsolver.m_sparseefflc.m_Idx.Set(offs+1,offsslackec+2*i+1);
offs+=2;
}
else
{
//--- Slack variables for inequality constraints
subsolver.m_sparseefflc.m_Vals.Set(offs,-1);
subsolver.m_sparseefflc.m_Idx.Set(offs,offsslackic+(i-nec));
offs++;
}
//--- Finalize row
subsolver.m_sparseefflc.m_RIdx.Set(i+1,offs);
//--- Set up bounds and slack part of D0.
//--- NOTE: bounds for equality and inequality constraints are
//--- handled differently
v=vright-State.m_scaledcleic.Get(i,n);
if(i<nec)
{
subsolver.m_cural.Set(i,-v);
subsolver.m_curau.Set(i,-v);
subsolver.m_curbndl.Set(offsslackec+2*i,0);
subsolver.m_curbndl.Set(offsslackec+2*i+1,0);
subsolver.m_curbndu.Set(offsslackec+2*i,MathAbs(v));
subsolver.m_curbndu.Set(offsslackec+2*i+1,MathAbs(v));
if(v>=0.0)
{
subsolver.m_d0.Set(offsslackec+2*i+0,MathAbs(v));
subsolver.m_d0.Set(offsslackec+2*i+1,0);
}
else
{
subsolver.m_d0.Set(offsslackec+2*i+0,0);
subsolver.m_d0.Set(offsslackec+2*i+1,MathAbs(v));
}
}
else
{
subsolver.m_cural.Set(i,AL_NEGINF);
subsolver.m_curau.Set(i,-v);
subsolver.m_curbndl.Set(offsslackic+(i-nec),0);
subsolver.m_curbndu.Set(offsslackic+(i-nec),MathMax(v,0));
subsolver.m_d0.Set(offsslackic+(i-nec),MathMax(v,0));
}
}
subsolver.m_sparseefflc.m_M+=(nec+nic);
//--- Append nonlinear equality/inequality constraints
for(i=0; i<nlec+nlic; i++)
{
//--- Calculate scale coefficient
vv=0;
for(j=0; j<n; j++)
{
v=jac.Get(1+i,j);
vv+=v*v;
}
vv=1/CApServ::Coalesce(MathSqrt(vv),1);
//--- Copy scaled row
offs=subsolver.m_sparseefflc.m_RIdx[subsolver.m_sparseefflc.m_M+i];
for(j=0; j<n; j++)
{
if(jac.Get(1+i,j)!=0.0)
{
subsolver.m_sparseefflc.m_Vals.Set(offs,vv*jac.Get(1+i,j));
subsolver.m_sparseefflc.m_Idx.Set(offs,j);
offs++;
}
}
if(i<nlec)
{
//--- Add slack terms for equality constraints
subsolver.m_sparseefflc.m_Vals.Set(offs,-1);
subsolver.m_sparseefflc.m_Vals.Set(offs+1,1);
subsolver.m_sparseefflc.m_Idx.Set(offs,offsslacknlec+2*i);
subsolver.m_sparseefflc.m_Idx.Set(offs+1,offsslacknlec+2*i+1);
offs+=2;
}
else
{
//--- Add slack terms for inequality constraints
subsolver.m_sparseefflc.m_Vals.Set(offs,-1);
subsolver.m_sparseefflc.m_Idx.Set(offs,offsslacknlic+(i-nlec));
offs++;
}
subsolver.m_sparseefflc.m_RIdx.Set(subsolver.m_sparseefflc.m_M+i+1,offs);
//--- Set box constraints on slack variables and bounds on linear equality/inequality constraints
v=vv*fi[1+i];
if(i<nlec)
{
//--- Equality constraint
subsolver.m_cural.Set(subsolver.m_sparseefflc.m_M+i,-v);
subsolver.m_curau.Set(subsolver.m_sparseefflc.m_M+i,-v);
subsolver.m_curbndl.Set(offsslacknlec+2*i,0);
subsolver.m_curbndl.Set(offsslacknlec+2*i+1,0);
subsolver.m_curbndu.Set(offsslacknlec+2*i,MathAbs(v));
subsolver.m_curbndu.Set(offsslacknlec+2*i+1,MathAbs(v));
if(v>=0.0)
{
subsolver.m_d0.Set(offsslacknlec+2*i,MathAbs(v));
subsolver.m_d0.Set(offsslacknlec+2*i+1,0);
}
else
{
subsolver.m_d0.Set(offsslacknlec+2*i,0);
subsolver.m_d0.Set(offsslacknlec+2*i+1,MathAbs(v));
}
}
else
{
//--- Inequality constraint
subsolver.m_cural.Set(subsolver.m_sparseefflc.m_M+i,AL_NEGINF);
subsolver.m_curau.Set(subsolver.m_sparseefflc.m_M+i,-v);
subsolver.m_curbndl.Set(offsslacknlic+(i-nlec),0);
subsolver.m_curbndu.Set(offsslacknlic+(i-nlec),MathMax(v,0));
subsolver.m_d0.Set(offsslacknlic+(i-nlec),MathMax(v,0));
}
}
subsolver.m_sparseefflc.m_M+=(nlec+nlic);
//--- Finalize sparse matrix structure
//--- check
if(!CAp::Assert(subsolver.m_sparseefflc.m_RIdx[subsolver.m_sparseefflc.m_M]<=subsolver.m_sparseefflc.m_Idx.Size(),__FUNCTION__+": critical integrity check failed"))
return(false);
if(!CAp::Assert(subsolver.m_sparseefflc.m_RIdx[subsolver.m_sparseefflc.m_M]<=subsolver.m_sparseefflc.m_Vals.Size(),__FUNCTION__+": critical integrity check failed"))
return(false);
subsolver.m_sparseefflc.m_NInitialized=subsolver.m_sparseefflc.m_RIdx[subsolver.m_sparseefflc.m_M];
CSparse::SparseInitDUIdx(subsolver.m_sparseefflc);
//--- Solve quadratic program
switch(subsolver.m_algokind)
{
case 0:
//--- Use dense IPM.
//--- We always treat its result as a valid solution, even for TerminationType<=0.
//--- In case anything is wrong with solution vector, we will detect it during line
//--- search phase (merit function does not increase).
//--- NOTE: because we cleaned up constraints that are DEFINITELY inactive within
//--- trust region, we do not have to worry about StopOnExcessiveBounds option.
subsolver.m_tmp0=vector<double>::Full(nslack,State.m_trustrad);
subsolver.m_tmp1=vector<double>::Zeros(nslack);
CVIPMSolver::VIPMInitDenseWithSlacks(subsolver.m_ipmsolver,subsolver.m_tmp0,subsolver.m_tmp1,n,nslack);
CVIPMSolver::VIPMSetQuadraticLinear(subsolver.m_ipmsolver,subsolver.m_h,subsolver.m_sparsedummy,0,true,subsolver.m_curb);
CVIPMSolver::VIPMSetConstraints(subsolver.m_ipmsolver,subsolver.m_curbndl,subsolver.m_curbndu,subsolver.m_sparseefflc,subsolver.m_sparseefflc.m_M,subsolver.m_densedummy,0,subsolver.m_cural,subsolver.m_curau);
CVIPMSolver::VIPMOptimize(subsolver.m_ipmsolver,false,subsolver.m_tmp0,subsolver.m_tmp1,subsolver.m_tmp2,terminationtype);
d=subsolver.m_tmp0;
lagmult=subsolver.m_tmp2;
break;
case 1:
//--- Use fast active set
FASSolve(subsolver,subsolver.m_d0,subsolver.m_h,n,subsolver.m_curb,nslack,subsolver.m_curbndl,subsolver.m_curbndu,subsolver.m_sparseefflc,subsolver.m_sparseefflc.m_M,subsolver.m_cural,subsolver.m_curau,State.m_trustrad,terminationtype,d,lagmult);
if(terminationtype<=0)
{
//--- QP solver failed due to numerical errors; exit
result=false;
}
break;
default:
//--- Unexpected
CAp::Assert(false,__FUNCTION__+": unexpected subsolver type");
result=false;
break;
}
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function initializes MeritPhase temporaries. It should be |
//| called before beginning of each new iteration. You may call it |
//| multiple times for the same instance of MeritPhase temporaries. |
//| INPUT PARAMETERS: |
//| MeritState - instance to be initialized. |
//| N - problem dimensionality |
//| NEC, NIC - linear equality / inequality constraint count |
//| NLEC, NLIC - nonlinear equality / inequality constraint count|
//| AbsLagMemory - array[MemLen, NEC + NIC + NLEC + NLIC], stores |
//| absolute values of Lagrange multipliers for the |
//| last MemLen iterations |
//| MemLen - memory length |
//| OUTPUT PARAMETERS: |
//| MeritState - instance being initialized |
//+------------------------------------------------------------------+
void CNLCSQP::MeritPhaseInit(CMinSQPMeritPhaseState &meritstate,
CRowDouble &curx,
CRowDouble &curfi,
CMatrixDouble &curj,
int n,
int nec,
int nic,
int nlec,
int nlic,
CMatrixDouble &abslagmemory,
int memlen)
{
int nslack=n+2*(nec+nlec)+(nic+nlic);
meritstate.m_n=n;
meritstate.m_nec=nec;
meritstate.m_nic=nic;
meritstate.m_nlec=nlec;
meritstate.m_nlic=nlic;
meritstate.m_penalties=vector<double>::Zeros(nec+nic+nlec+nlic);
for(int i=0; i<memlen; i++)
CAblasF::RMergeMaxRV(nec+nic+nlec+nlic,abslagmemory,i,meritstate.m_penalties);
meritstate.m_stepkx=curx;
meritstate.m_stepkfi=curfi;
meritstate.m_stepkj=curj;
meritstate.m_stepkx.Resize(n);
meritstate.m_stepkfi.Resize(1+nlec+nlic);
meritstate.m_stepkj.Resize(1+nlec+nlic,n);
meritstate.m_d.Resize(nslack);
meritstate.m_dx.Resize(nslack);
meritstate.m_stepkxc.Resize(n);
meritstate.m_stepkxn.Resize(n);
meritstate.m_stepkfic.Resize(1+nlec+nlic);
meritstate.m_stepkfin.Resize(1+nlec+nlic);
meritstate.m_stepkj.Resize(1+nlec+nlic,n);
meritstate.m_stepkjc.Resize(1+nlec+nlic,n);
meritstate.m_stepkjn.Resize(1+nlec+nlic,n);
meritstate.m_stepklaggrad.Resize(n);
meritstate.m_stepknlaggrad.Resize(n);
meritstate.m_lagmult.Resize(nec+nic+nlec+nlic);
meritstate.m_dummylagmult.Resize(nec+nic+nlec+nlic);
meritstate.m_rmeritphasestate.ia.Resize(8);
ArrayResize(meritstate.m_rmeritphasestate.ba,4);
meritstate.m_rmeritphasestate.ra.Resize(12);
meritstate.m_rmeritphasestate.stage=-1;
}
//+------------------------------------------------------------------+
//| This function tries to perform either phase #1 or phase #3 step. |
//| Former corresponds to linear model step (without conjugacy |
//| constraints) with correction for nonlinearity ("second order |
//| correction"). Such correction helps to overcome Maratos effect |
//| (a tendency of L1 penalized merit functions to reject nonzero |
//| steps). |
//| Latter is a step using linear model with no second order |
//| correction. |
//| INPUT PARAMETERS: |
//| State - SQP solver State |
//| SMonitor - smoothness monitor |
//| UserTerminationNeeded - True if user requested termination |
//| OUTPUT PARAMETERS: |
//| State - RepTerminationType is set to current termination |
//| code (if Status = 0). |
//| Status - when reverse communication is done, Status is set |
//| to: |
//| * positive value, if we can proceed to the next |
//| stage of the outer iteration |
//| * zero, if algorithm is terminated |
//| (RepTerminationType is set to appropriate value) |
//+------------------------------------------------------------------+
bool CNLCSQP::MeritPhaseIteration(CMinSQPState &State,
CMinSQPMeritPhaseState &meritstate,
CSmoothnessMonitor &smonitor,
bool userterminationneeded)
{
//--- create variables
bool result=false;
int n=0;
int nslack=0;
int nec=0;
int nic=0;
int nlec=0;
int nlic=0;
int i=0;
int j=0;
double v=0;
double vv=0;
double mx=0;
double f0=0;
double f1=0;
double nu=0;
double localstp=0;
double tol=0;
double stepklagval=0;
double stepknlagval=0;
bool hessianupdateperformed=false;
bool dotrace=false;
bool doprobing=false;
bool dotracexd=false;
double stp=0;
double expandedrad=0;
int label=-1;
//--- Reverse communication preparations
//--- I know it looks ugly, but it works the same way
//--- anywhere from C++ to Python.
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(meritstate.m_rmeritphasestate.stage>=0)
{
n=meritstate.m_rmeritphasestate.ia[0];
nslack=meritstate.m_rmeritphasestate.ia[1];
nec=meritstate.m_rmeritphasestate.ia[2];
nic=meritstate.m_rmeritphasestate.ia[3];
nlec=meritstate.m_rmeritphasestate.ia[4];
nlic=meritstate.m_rmeritphasestate.ia[5];
i=meritstate.m_rmeritphasestate.ia[6];
j=meritstate.m_rmeritphasestate.ia[7];
hessianupdateperformed=meritstate.m_rmeritphasestate.ba[0];
dotrace=meritstate.m_rmeritphasestate.ba[1];
doprobing=meritstate.m_rmeritphasestate.ba[2];
dotracexd=meritstate.m_rmeritphasestate.ba[3];
v=meritstate.m_rmeritphasestate.ra[0];
vv=meritstate.m_rmeritphasestate.ra[1];
mx=meritstate.m_rmeritphasestate.ra[2];
f0=meritstate.m_rmeritphasestate.ra[3];
f1=meritstate.m_rmeritphasestate.ra[4];
nu=meritstate.m_rmeritphasestate.ra[5];
localstp=meritstate.m_rmeritphasestate.ra[6];
tol=meritstate.m_rmeritphasestate.ra[7];
stepklagval=meritstate.m_rmeritphasestate.ra[8];
stepknlagval=meritstate.m_rmeritphasestate.ra[9];
stp=meritstate.m_rmeritphasestate.ra[10];
expandedrad=meritstate.m_rmeritphasestate.ra[11];
}
else
{
n=-536;
nslack=487;
nec=-115;
nic=886;
nlec=346;
nlic=-722;
i=-413;
j=-461;
hessianupdateperformed=true;
dotrace=true;
doprobing=false;
dotracexd=false;
v=306;
vv=-1011;
mx=951;
f0=-463;
f1=88;
nu=-861;
localstp=-678;
tol=-731;
stepklagval=-675;
stepknlagval=-763;
stp=-233;
expandedrad=-936;
}
switch(meritstate.m_rmeritphasestate.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
case 3:
label=3;
break;
default:
//--- Routine body
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
nlec=State.m_nlec;
nlic=State.m_nlic;
nslack=n+2*(nec+nlec)+(nic+nlic);
dotrace=CAp::IsTraceEnabled("SQP");
dotracexd=dotrace && CAp::IsTraceEnabled("SQP.DETAILED");
doprobing=CAp::IsTraceEnabled("SQP.PROBING");
//--- check
if(!CAp::Assert(meritstate.m_lagmult.Size()>=nec+nic+nlec+nlic,__FUNCTION__+": integrity check failed"))
return(false);
//--- Report iteration beginning
if(dotrace)
CAp::Trace("\n--- quadratic step ---------------------------------------------------------------------------------\n");
//--- Default decision is to continue algorithm
meritstate.m_status=1;
meritstate.m_increasebigc=false;
stp=0;
//--- Determine step direction using initial quadratic model.
//--- Update penalties vector with current Lagrange multipliers.
if(!QPSubproblemSolve(State,State.m_subsolver,meritstate.m_stepkx,meritstate.m_stepkfi,meritstate.m_stepkj,meritstate.m_d,meritstate.m_lagmult,j))
{
if(dotrace)
CAp::Trace(StringFormat("> [WARNING] QP subproblem failed with TerminationType=%d\n",j));
result=false;
return(result);
}
if(dotrace)
CAp::Trace(StringFormat("> QP subproblem solved with TerminationType=%d\n",j));
for(i=0; i<=nec+nic+nlec+nlic-1; i++)
meritstate.m_penalties.Set(i,MathMax(meritstate.m_penalties[i],MathAbs(meritstate.m_lagmult[i])));
//--- Perform merit function line search.
//--- First, we try unit step. If it does not decrease merit function,
//--- a second-order correction is tried (helps to combat Maratos effect).
localstp=1.0;
f0=MeritFunction(State,meritstate.m_stepkx,meritstate.m_stepkfi,meritstate.m_lagmult,meritstate.m_penalties,meritstate.m_tmpmerit);
for(i=0; i<n; i++)
meritstate.m_stepkxn.Set(i,meritstate.m_stepkx[i]+meritstate.m_d[i]*localstp);
SQPSendX(State,meritstate.m_stepkxn);
State.m_needfij=true;
meritstate.m_rmeritphasestate.stage=0;
label=-1;
break;
}
//--- Main loop
while(label>=0)
{
switch(label)
{
case 0:
State.m_needfij=false;
if(!SQPRetrieveFIJ(State,meritstate.m_stepkfin,meritstate.m_stepkjn))
{
//--- Failed to retrieve func/Jac, infinities detected
State.m_repterminationtype=-8;
meritstate.m_status=0;
if(dotrace)
CAp::Trace("[ERROR] infinities in target/constraints are detected\n");
return(false);
}
f1=MeritFunction(State,meritstate.m_stepkxn,meritstate.m_stepkfin,meritstate.m_lagmult,meritstate.m_penalties,meritstate.m_tmpmerit);
if(f1<f0)
{
label=4;
break;
}
//--- Full step increases merit function. Let's compute second order
//--- correction to the constraint model and recompute trial step D:
//--- * use original model of the target
//--- * extrapolate model of nonlinear constraints at StepKX+D back to origin
if(dotrace)
CAp::Trace("> preparing second-order correction\n");
meritstate.m_stepkfic.Set(0,meritstate.m_stepkfi[0]);
meritstate.m_stepkjc=meritstate.m_stepkj;
for(i=1; i<=nlec+nlic; i++)
{
v=CAblasF::RDotVR(n,meritstate.m_d,meritstate.m_stepkj,i);
meritstate.m_stepkjc.Row(i,meritstate.m_stepkj,i);
meritstate.m_stepkfic.Set(i,meritstate.m_stepkfin[i]-v);
}
if(!QPSubproblemSolve(State,State.m_subsolver,meritstate.m_stepkx,meritstate.m_stepkfic,meritstate.m_stepkjc,meritstate.m_dx,meritstate.m_dummylagmult,j))
{
if(dotrace)
CAp::Trace("> [WARNING] second-order QP subproblem failed\n");
return(false);
}
if(dotrace)
CAp::Trace(StringFormat("> second-order QP subproblem solved with TerminationType=%d\n",j));
meritstate.m_d=meritstate.m_dx;
//--- Perform line search, we again try full step (maybe it will work after SOC)
localstp=1.0;
nu=0.5;
f1=f0;
COptServ::SmoothnessMonitorStartLineSearch(smonitor,meritstate.m_stepkx,meritstate.m_stepkfi,meritstate.m_stepkj);
case 6:
for(i=0; i<n; i++)
meritstate.m_stepkxn.Set(i,meritstate.m_stepkx[i]+meritstate.m_d[i]*localstp);
SQPSendX(State,meritstate.m_stepkxn);
State.m_needfij=true;
meritstate.m_rmeritphasestate.stage=1;
label=-1;
break;
case 1:
State.m_needfij=false;
if(!SQPRetrieveFIJ(State,meritstate.m_stepkfin,meritstate.m_stepkjn))
{
//--- Failed to retrieve func/Jac, infinities detected
State.m_repterminationtype=-8;
meritstate.m_status=0;
if(dotrace)
CAp::Trace("[ERROR] infinities in target/constraints are detected\n");
return(false);
}
COptServ::SmoothnessMonitorEnqueuePoint(smonitor,meritstate.m_d,localstp,meritstate.m_stepkxn,meritstate.m_stepkfin,meritstate.m_stepkjn);
f1=MeritFunction(State,meritstate.m_stepkxn,meritstate.m_stepkfin,meritstate.m_lagmult,meritstate.m_penalties,meritstate.m_tmpmerit);
if(f1<f0)
{
//--- Step is found!
label=7;
break;
}
if(localstp<0.001)
{
//--- Step is shorter than 0.001 times current search direction,
//--- it means that no good step can be found.
localstp=0;
SQPCopyState(State,meritstate.m_stepkx,meritstate.m_stepkfi,meritstate.m_stepkj,meritstate.m_stepkxn,meritstate.m_stepkfin,meritstate.m_stepkjn);
label=7;
break;
}
localstp=nu*localstp;
nu=MathMax(0.1,0.5*nu);
label=6;
break;
case 7:
COptServ::SmoothnessMonitorFinalizeLineSearch(smonitor);
case 4:
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i])
meritstate.m_stepkxn.Set(i,MathMax(meritstate.m_stepkxn[i],State.m_scaledbndl[i]));
if(State.m_HasBndU[i])
meritstate.m_stepkxn.Set(i,MathMin(meritstate.m_stepkxn[i],State.m_scaledbndu[i]));
}
if(userterminationneeded)
{
//--- User requested termination, break before we move to new point
State.m_repterminationtype=8;
meritstate.m_status=0;
if(dotrace)
CAp::Trace("> user requested termination\n");
return(false);
}
LagrangianFG(State,meritstate.m_stepkx,State.m_trustrad,meritstate.m_stepkfi,meritstate.m_stepkj,meritstate.m_lagmult,meritstate.m_tmplagrangianfg,stepklagval,meritstate.m_stepklaggrad);
LagrangianFG(State,meritstate.m_stepkxn,State.m_trustrad,meritstate.m_stepkfin,meritstate.m_stepkjn,meritstate.m_lagmult,meritstate.m_tmplagrangianfg,stepknlagval,meritstate.m_stepknlaggrad);
//--- Decide whether we want to request increase BigC (a constraint enforcing multiplier for L1 penalized
//--- QP subproblem) or not.
//--- An increase is NOT needed if at least one of the following holds:
//--- * present value of BigC is already nearly maximum
//--- * a long step was performed
//--- * any single constraint can be made feasible within box with radius slightly larger max|D|
//--- Thus, BigC is requested to be increased if a short step was made, but there are some
//--- constraints that are infeasible within max|D|-sized box
if(CAblasF::RMaxAbsV(n,meritstate.m_d)<(0.9*State.m_trustrad) && State.m_bigc<(0.9*m_maxbigc))
{
expandedrad=1.1*CAblasF::RMaxAbsV(n,meritstate.m_d);
tol=MathMax(MathSqrt(CMath::m_machineepsilon)*State.m_trustrad,1000*CMath::m_machineepsilon);
for(i=0; i<nec+nic; i++)
{
v=0;
vv=0;
for(j=0; j<n; j++)
{
v+=State.m_scaledcleic.Get(i,j)*State.m_stepkx[j];
vv+=MathAbs(State.m_scaledcleic.Get(i,j)*expandedrad);
}
v-=State.m_scaledcleic.Get(i,n);
if(i>=nec)
v=MathMax(v,0.0);
meritstate.m_increasebigc=meritstate.m_increasebigc || MathAbs(v)>(vv+tol);
}
for(i=1; i<=nlec+nlic; i++)
{
v=State.m_stepkfi[i];
vv=0;
for(j=0; j<n; j++)
{
vv+=MathAbs(State.m_stepkj.Get(i,j)*expandedrad);
}
if(i>=nlec+1)
v=MathMax(v,0.0);
meritstate.m_increasebigc=meritstate.m_increasebigc || MathAbs(v)>(vv+tol);
}
}
//--- Trace
if(!dotrace)
{
label=8;
break;
}
//--- Perform agressive probing of the search direction - additional function evaluations
//--- which help us to determine possible discontinuity and nonsmoothness of the problem
if(!doprobing)
{
label=10;
break;
}
COptServ::SmoothnessMonitorStartProbing(smonitor,1.0,2,State.m_trustrad);
COptServ::SmoothnessMonitorStartLineSearch(smonitor,meritstate.m_stepkx,meritstate.m_stepkfi,meritstate.m_stepkj);
case 12:
if(!COptServ::SmoothnessMonitorProbe(smonitor))
{
label=13;
break;
}
for(j=0; j<n; j++)
{
meritstate.m_stepkxc.Set(j,meritstate.m_stepkx[j]+meritstate.m_d[j]*smonitor.m_probingstp);
if(State.m_HasBndL[j])
meritstate.m_stepkxc.Set(j,MathMax(meritstate.m_stepkxc[j],State.m_scaledbndl[j]));
if(State.m_HasBndU[j])
meritstate.m_stepkxc.Set(j,MathMin(meritstate.m_stepkxc[j],State.m_scaledbndu[j]));
}
SQPSendX(State,meritstate.m_stepkxc);
State.m_needfij=true;
meritstate.m_rmeritphasestate.stage=2;
label=-1;
break;
case 2:
State.m_needfij=false;
if(!SQPRetrieveFIJ(State,meritstate.m_stepkfic,meritstate.m_stepkjc))
{
label=13;
break;
}
smonitor.m_probingf.Set(0,RawLagrangian(State,meritstate.m_stepkxc,meritstate.m_stepkfic,meritstate.m_lagmult,meritstate.m_penalties,meritstate.m_tmpmerit));
smonitor.m_probingf.Set(1,meritstate.m_stepkfic[0]);
COptServ::SmoothnessMonitorEnqueuePoint(smonitor,meritstate.m_d,smonitor.m_probingstp,meritstate.m_stepkxc,meritstate.m_stepkfic,meritstate.m_stepkjc);
label=12;
break;
case 13:
COptServ::SmoothnessMonitorFinalizeLineSearch(smonitor);
CAp::Trace("*** ------------------------------------------------------------\n");
CAp::Trace("*** | probing search direction suggested by QP subproblem |\n");
CAp::Trace("*** ------------------------------------------------------------\n");
CAp::Trace("*** | Step | Lagrangian (unaugmentd)| Target function |\n");
CAp::Trace("*** |along D| must be smooth | must be smooth |\n");
CAp::Trace("*** | | function | slope | function | slope |\n");
COptServ::SmoothnessMonitorTraceProbingResults(smonitor);
case 10:
//--- Update debug curvature information - TraceGamma[]
v=0;
mx=0;
for(j=0; j<n; j++)
{
vv=meritstate.m_stepkxn[j]-meritstate.m_stepkx[j];
mx=MathMax(mx,MathAbs(vv));
v+=vv*vv;
}
if(v>0.0)
{
//--- Step is long enough, update curvature information (used for debugging)
for(i=0; i<=nlec+nlic; i++)
{
vv=0;
for(j=0; j<n; j++)
vv+=(meritstate.m_stepkjn.Get(i,j)-meritstate.m_stepkj.Get(i,j))*(meritstate.m_stepkxn[j]-meritstate.m_stepkx[j]);
State.m_tracegamma.Set(i,MathMax(State.m_tracegamma[i],MathAbs(vv/(v+100*n*CMath::m_machineepsilon*CMath::m_machineepsilon))));
}
}
//--- Output other information
mx=(meritstate.m_d.Abs()/State.m_trustrad).Max();
if(localstp>0.0)
CAp::Trace("> nonzero linear step was performed\n");
else
CAp::Trace("> zero linear step was performed\n");
CAp::Trace(StringFormat("max(|Di|)/TrustRad = %.6f\n",mx));
CAp::Trace(StringFormat("stp = %.6f\n",localstp));
if(dotracexd)
{
CAp::Trace("X0 (scaled) = ");
CApServ::TraceVectorAutopRec(meritstate.m_stepkx,0,n);
CAp::Trace("\n");
CAp::Trace("D (scaled) = ");
CApServ::TraceVectorAutopRec(meritstate.m_d,0,n);
CAp::Trace("\n");
CAp::Trace("X1 (scaled) = ");
CApServ::TraceVectorAutopRec(meritstate.m_stepkxn,0,n);
CAp::Trace("\n");
}
CAp::Trace(StringFormat("meritF: %14.6E -> %14.6E (delta=%11.3E)\n",f0,f1,f1 - f0));
CAp::Trace(StringFormat("scaled-targetF: %14.6E -> %14.6E (delta=%11.3E)\n",meritstate.m_stepkfi[0],meritstate.m_stepkfin[0],meritstate.m_stepkfin[0] - meritstate.m_stepkfi[0]));
CAp::Trace("> evaluating possible Hessian update\n");
v=(meritstate.m_stepkxn-meritstate.m_stepkx+0).Dot(meritstate.m_stepknlaggrad-meritstate.m_stepklaggrad+0);
CAp::Trace(StringFormat("(Sk,Yk) = %.3E\n",v));
v=MathPow(meritstate.m_stepkxn-meritstate.m_stepkx+0,2.0).Sum();
CAp::Trace(StringFormat("(Sk,Sk) = %.3E\n",v));
v=MathPow(meritstate.m_stepknlaggrad-meritstate.m_stepklaggrad+0,2.0).Sum();
CAp::Trace(StringFormat("(Yk,Yk) = %.3E\n",v));
v=0;
for(i=0; i<n; i++)
{
v+=CMath::Sqr(meritstate.m_stepkxn[i]-meritstate.m_stepkx[i])
*State.m_subsolver.m_h.Get(i,i);
for(j=i+1; j<n; j++)
v+=2*(meritstate.m_stepkxn[i]-meritstate.m_stepkx[i])*State.m_subsolver.m_h.Get(i,j)*(meritstate.m_stepkxn[j]-meritstate.m_stepkx[j]);
}
CAp::Trace(StringFormat("Sk*Bk*Sk = %.3E\n",v));
v=State.m_subsolver.m_h.Get(0,0);
for(i=1; i<n; i++)
v=MathMin(v,State.m_subsolver.m_h.Get(i,i));
CAp::Trace(StringFormat("mindiag(Bk) = %.3E\n",v));
v=State.m_subsolver.m_h.Get(0,0);
for(i=1; i<n; i++)
v=MathMax(v,State.m_subsolver.m_h.Get(i,i));
CAp::Trace(StringFormat("maxdiag(Bk) = %.3E\n",v));
case 8:
//--- Perform Hessian update
hessianupdateperformed=false;
if(localstp>0.0)
hessianupdateperformed=QPSubProblemUpdateHessian(State,State.m_subsolver,meritstate.m_stepkx,meritstate.m_stepklaggrad,meritstate.m_stepkxn,meritstate.m_stepknlaggrad);
if(dotrace)
{
if(hessianupdateperformed)
{
CAp::Trace("> Hessian updated\n");
v=State.m_subsolver.m_h.Get(0,0);
for(i=0; i<n; i++)
v=MathMin(v,State.m_subsolver.m_h.Get(i,i));
CAp::Trace(StringFormat("mindiag(Bk) = %.3E\n",v));
v=State.m_subsolver.m_h.Get(0,0);
for(i=0; i<n; i++)
v=MathMax(v,State.m_subsolver.m_h.Get(i,i));
CAp::Trace(StringFormat("maxdiag(Bk) = %.3E\n",v));
}
else
CAp::Trace("> skipping Hessian update\n");
}
//--- Move to new point
stp=localstp;
SQPCopyState(State,meritstate.m_stepkxn,meritstate.m_stepkfin,meritstate.m_stepkjn,meritstate.m_stepkx,meritstate.m_stepkfi,meritstate.m_stepkj);
if(localstp<=0.0)
{
label=14;
break;
}
//--- Report one more inner iteration
SQPSendX(State,meritstate.m_stepkx);
State.m_f=meritstate.m_stepkfi[0]*State.m_fscales[0];
State.m_xupdated=true;
meritstate.m_rmeritphasestate.stage=3;
label=-1;
break;
case 3:
State.m_xupdated=false;
//--- Update constraint violations
COptServ::CheckLcViolation(State.m_scaledcleic,State.m_lcsrcidx,nec,nic,meritstate.m_stepkx,n,State.m_replcerr,State.m_replcidx);
COptServ::UnScaleAndCheckNLcViolation(meritstate.m_stepkfi,State.m_fscales,nlec,nlic,State.m_repnlcerr,State.m_repnlcidx);
case 14:
return(false);
}
}
//--- Saving State
result=true;
meritstate.m_rmeritphasestate.ba[0]=hessianupdateperformed;
meritstate.m_rmeritphasestate.ba[1]=dotrace;
meritstate.m_rmeritphasestate.ba[2]=doprobing;
meritstate.m_rmeritphasestate.ba[3]=dotracexd;
meritstate.m_rmeritphasestate.ia.Set(0,n);
meritstate.m_rmeritphasestate.ia.Set(1,nslack);
meritstate.m_rmeritphasestate.ia.Set(2,nec);
meritstate.m_rmeritphasestate.ia.Set(3,nic);
meritstate.m_rmeritphasestate.ia.Set(4,nlec);
meritstate.m_rmeritphasestate.ia.Set(5,nlic);
meritstate.m_rmeritphasestate.ia.Set(6,i);
meritstate.m_rmeritphasestate.ia.Set(7,j);
meritstate.m_rmeritphasestate.ra.Set(0,v);
meritstate.m_rmeritphasestate.ra.Set(1,vv);
meritstate.m_rmeritphasestate.ra.Set(2,mx);
meritstate.m_rmeritphasestate.ra.Set(3,f0);
meritstate.m_rmeritphasestate.ra.Set(4,f1);
meritstate.m_rmeritphasestate.ra.Set(5,nu);
meritstate.m_rmeritphasestate.ra.Set(6,localstp);
meritstate.m_rmeritphasestate.ra.Set(7,tol);
meritstate.m_rmeritphasestate.ra.Set(8,stepklagval);
meritstate.m_rmeritphasestate.ra.Set(9,stepknlagval);
meritstate.m_rmeritphasestate.ra.Set(10,stp);
meritstate.m_rmeritphasestate.ra.Set(11,expandedrad);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function initializes MeritPhase temporaries. It should be |
//| called before beginning of each new iteration. You may call it |
//| multiple times for the same instance of MeritPhase temporaries. |
//| INPUT PARAMETERS: |
//| MeritState - instance to be initialized. |
//| N - problem dimensionality |
//| NEC, NIC - linear equality / inequality constraint count |
//| NLEC, NLIC - nonlinear equality / inequality constraint count|
//| OUTPUT PARAMETERS: |
//| IncreaseBigC - whether increasing BigC is suggested (we |
//| detected infeasible constraints that are NOT |
//| improved) or not. |
//| MeritState - instance being initialized |
//+------------------------------------------------------------------+
void CNLCSQP::MeritPhaseResults(CMinSQPMeritPhaseState &meritstate,
CRowDouble &curx,
CRowDouble &curfi,
CMatrixDouble &curj,
CRowDouble &lagmult,
bool &increasebigc,
int &status)
{
//--- copy State
increasebigc=meritstate.m_increasebigc;
status=meritstate.m_status;
curx=meritstate.m_stepkx;
curfi=meritstate.m_stepkfi;
curj=meritstate.m_stepkj;
lagmult=meritstate.m_lagmult;
}
//+------------------------------------------------------------------+
//| Copies X to State.X |
//+------------------------------------------------------------------+
void CNLCSQP::SQPSendX(CMinSQPState &State,
CRowDouble &xs)
{
int n=State.m_n;
for(int i=0; i<n; i++)
{
if(State.m_HasBndL[i] && xs[i]<=State.m_scaledbndl[i])
{
State.m_x.Set(i,State.m_scaledbndl[i]);
continue;
}
if(State.m_HasBndU[i] && xs[i]>=State.m_scaledbndu[i])
{
State.m_x.Set(i,State.m_scaledbndu[i]);
continue;
}
State.m_x.Set(i,xs[i]);
}
}
//+------------------------------------------------------------------+
//| Retrieves F-vector and scaled Jacobian, copies them to FiS and JS|
//| Returns: |
//| True on success, |
//| False on failure(when F or J are not finite numbers). |
//+------------------------------------------------------------------+
bool CNLCSQP::SQPRetrieveFIJ(CMinSQPState &State,
CRowDouble &fis,
CMatrixDouble &js)
{
//--- create variables
bool result=false;
int n=State.m_n;
int nlec=State.m_nlec;
int nlic=State.m_nlic;
double v=0;
double vv=0;
//--- main loop
v=0;
for(int i=0; i<=nlec+nlic; i++)
{
vv=1/State.m_fscales[i];
fis.Set(i,vv*State.m_fi[i]);
v=0.1*v+fis[i];
for(int j=0; j<n; j++)
{
js.Set(i,j,vv*State.m_j.Get(i,j));
v=0.1*v+js.Get(i,j);
}
}
result=MathIsValidNumber(v);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Copies State (X point, Fi vector, J jacobian) to preallocated |
//| storage. |
//+------------------------------------------------------------------+
void CNLCSQP::SQPCopyState(CMinSQPState &State,
CRowDouble &x0,
CRowDouble &fi0,
CMatrixDouble &j0,
CRowDouble &x1,
CRowDouble &fi1,
CMatrixDouble &j1)
{
x1=x0;
fi1=fi0;
j1=j0;
}
//+------------------------------------------------------------------+
//| This function calculates Lagrangian of the problem (in scaled |
//| variables): its value and gradient. |
//| Additionally it also estimates violation of linear constraints at|
//| the point as well as index of the most violated constraint |
//+------------------------------------------------------------------+
void CNLCSQP::LagrangianFG(CMinSQPState &State,
CRowDouble &x,
double trustrad,
CRowDouble &fi,
CMatrixDouble &j,
CRowDouble &lagmult,
CMinSQPTmpLagrangian &tmp,
double &f,
CRowDouble &g)
{
//--- create variables
int n=State.m_n;
int nec=State.m_nec;
int nic=State.m_nic;
int nlec=State.m_nlec;
int nlic=State.m_nlic;
int i=0;
double v=0;
double vlag=0;
double vact=0;
double vd=0;
bool usesparsegemv=false;
//--- Target function
f=fi[0];
g=j[0]+0;
//--- Lagrangian terms for linear constraints, constraint violations
if(nec+nic>0)
{
usesparsegemv=State.m_subsolver.m_sparserawlc.m_RIdx[nec+nic]<CApServ::SparseLevel2Density()*n*(nec+nic);
tmp.m_sclagtmp0.Resize(MathMax(nec+nic,n));
tmp.m_sclagtmp1.Resize(MathMax(nec+nic,n));
if(usesparsegemv)
CSparse::SparseMV(State.m_subsolver.m_sparserawlc,x,tmp.m_sclagtmp0);
else
CAblas::RMatrixGemVect(nec+nic,n,1.0,State.m_scaledcleic,0,0,0,x,0,0.0,tmp.m_sclagtmp0,0);
for(i=0; i<nec+nic; i++)
{
//--- Prepare
v=tmp.m_sclagtmp0[i]-State.m_scaledcleic.Get(i,n);
vlag=lagmult[i];
tmp.m_sclagtmp1.Set(i,0);
//--- Primary Lagrangian term
vact=v;
vd=1;
f+=vlag*vact;
tmp.m_sclagtmp1.Add(i,vlag*vd);
//--- Quadratic augmentation term
if(i<nec || v>0)
vact=v;
else
vact=0;
f+=0.5*m_augmentationfactor*vact*vact;
tmp.m_sclagtmp1.Add(i,m_augmentationfactor*vact);
}
if(usesparsegemv)
{
CSparse::SparseMTV(State.m_subsolver.m_sparserawlc,tmp.m_sclagtmp1,tmp.m_sclagtmp0);
g+=tmp.m_sclagtmp0;
}
else
CAblas::RMatrixGemVect(n,nec+nic,1.0,State.m_scaledcleic,0,0,1,tmp.m_sclagtmp1,0,1.0,g,0);
}
//--- Lagrangian terms for nonlinear constraints
tmp.m_sclagtmp1=vector<double>::Zeros(nlec+nlic);
for(i=0; i<nlec+nlic; i++)
{
v=fi[1+i];
vlag=lagmult[nec+nic+i];
//--- Lagrangian term
vact=v;
vd=1;
f+=vlag*vact;
tmp.m_sclagtmp1.Add(i,vlag*vd);
//--- Augmentation term
if(i<nlec || v>0)
vact=v;
else
vact=0;
f+=0.5*m_augmentationfactor*vact*vact;
tmp.m_sclagtmp1.Add(i,m_augmentationfactor*vact);
}
CAblas::RMatrixGemVect(n,nlec+nlic,1.0,j,1,0,1,tmp.m_sclagtmp1,0,1.0,g,0);
}
//+------------------------------------------------------------------+
//| This function calculates L1 - penalized merit function |
//+------------------------------------------------------------------+
double CNLCSQP::MeritFunction(CMinSQPState &State,
CRowDouble &x,
CRowDouble &fi,
CRowDouble &lagmult,
CRowDouble &penalties,
CMinSQPTmpMerit &tmp)
{
//--- create variables
double tmp0=0;
double tmp1=0;
//--- function call
MeritFunctionAndRawLagrangian(State,x,fi,lagmult,penalties,tmp,tmp0,tmp1);
//--- return result
return(tmp0);
}
//+------------------------------------------------------------------+
//| This function calculates raw (unaugmented and smooth) Lagrangian |
//+------------------------------------------------------------------+
double CNLCSQP::RawLagrangian(CMinSQPState &State,
CRowDouble &x,
CRowDouble &fi,
CRowDouble &lagmult,
CRowDouble &penalties,
CMinSQPTmpMerit &tmp)
{
//--- create variables
double tmp0=0;
double tmp1=0;
//--- function call
MeritFunctionAndRawLagrangian(State,x,fi,lagmult,penalties,tmp,tmp0,tmp1);
//--- return result
return(tmp1);
}
//+------------------------------------------------------------------+
//| This function calculates L1-penalized merit function and raw |
//| (smooth and un-augmented) Lagrangian |
//+------------------------------------------------------------------+
void CNLCSQP::MeritFunctionAndRawLagrangian(CMinSQPState &State,
CRowDouble &x,
CRowDouble &fi,
CRowDouble &lagmult,
CRowDouble &penalties,
CMinSQPTmpMerit &tmp,
double &meritf,
double &rawlag)
{
//--- create variables
int n=State.m_n;
int nec=State.m_nec;
int nic=State.m_nic;
int nlec=State.m_nlec;
int nlic=State.m_nlic;
int i=0;
double v=0;
//--- Merit function and Lagrangian: primary term
meritf=fi[0];
rawlag=fi[0];
//--- Merit function: augmentation and penalty for linear constraints
tmp.m_mftmp0.Resize(nec+nic);
CAblas::RMatrixGemVect(nec+nic,n,1.0,State.m_scaledcleic,0,0,0,x,0,0.0,tmp.m_mftmp0,0);
for(i=0; i<nec+nic; i++)
{
v=tmp.m_mftmp0[i]-State.m_scaledcleic.Get(i,n);
if(i<nec)
{
//--- Merit function: augmentation term + L1 penalty term
meritf+=0.5*m_augmentationfactor*v*v;
meritf+=m_meritfunctionbase*MathAbs(v)+m_meritfunctiongain*MathAbs(1+penalties[i])*MathAbs(v);
//--- Raw Lagrangian
rawlag+=lagmult[i]*v;
}
else
{
//--- Merit function: augmentation term + L1 penalty term
meritf+=0.5*m_augmentationfactor*CMath::Sqr(MathMax(v,0));
meritf+=m_meritfunctionbase*MathMax(v,0)+m_meritfunctiongain*MathAbs(1+penalties[i])*MathMax(v,0);
//--- Raw Lagrangian
rawlag+=lagmult[i]*v;
}
}
//--- Merit function: augmentation and penalty for nonlinear constraints
for(i=0; i<nlec+nlic; i++)
{
v=fi[1+i];
if(i<nlec)
{
//--- Merit function: augmentation term + L1 penalty term
meritf+=0.5*m_augmentationfactor*v*v;
meritf+=m_meritfunctionbase*MathAbs(v)+m_meritfunctiongain*MathAbs(1+penalties[nec+nic+i])*MathAbs(v);
//--- Raw Lagrangian
rawlag+=lagmult[nec+nic+i]*v;
}
else
{
//--- Merit function: augmentation term + L1 penalty term
meritf+=0.5*m_augmentationfactor*CMath::Sqr(MathMax(v,0));
meritf+=m_meritfunctionbase*MathMax(v,0)+m_meritfunctiongain*MathAbs(1+penalties[nec+nic+i])*MathMax(v,0);
//--- Raw Lagrangian
rawlag+=lagmult[nec+nic+i]*v;
}
}
}
//+------------------------------------------------------------------+
//| This object stores transformation used to convert solution |
//| (primal and dual) to original variables. It is also used |
//| to store temporaries. |
//+------------------------------------------------------------------+
struct CPresolveInfo
{
int m_newm;
int m_newn;
int m_oldm;
int m_oldn;
double m_costscale;
CSparseMatrix m_sparsea;
CRowDouble m_al;
CRowDouble m_au;
CRowDouble m_bndl;
CRowDouble m_bndu;
CRowDouble m_c;
CRowDouble m_colscales;
CRowDouble m_rawbndl;
CRowDouble m_rawbndu;
CRowDouble m_rowscales;
//--- constructor / destructor
CPresolveInfo(void);
~CPresolveInfo(void) {}
//---
void Copy(const CPresolveInfo &obj);
//--- overloading
void operator=(const CPresolveInfo &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CPresolveInfo::CPresolveInfo(void)
{
m_newm=0;
m_newn=0;
m_oldm=0;
m_oldn=0;
m_costscale=0;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CPresolveInfo::Copy(const CPresolveInfo &obj)
{
m_newm=obj.m_newm;
m_newn=obj.m_newn;
m_oldm=obj.m_oldm;
m_oldn=obj.m_oldn;
m_costscale=obj.m_costscale;
m_sparsea=obj.m_sparsea;
m_al=obj.m_al;
m_au=obj.m_au;
m_bndl=obj.m_bndl;
m_bndu=obj.m_bndu;
m_c=obj.m_c;
m_colscales=obj.m_colscales;
m_rawbndl=obj.m_rawbndl;
m_rawbndu=obj.m_rawbndu;
m_rowscales=obj.m_rowscales;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CLPQPPresolve
{
public:
static void PresolveNoneScaleUser(CRowDouble &s,CRowDouble &c,CRowDouble &bndl,CRowDouble &bndu,int n,CSparseMatrix &sparsea,CRowDouble &al,CRowDouble &au,int k,CPresolveInfo &Info);
static void PresolveBwd(CPresolveInfo &Info,CRowDouble &x,CRowInt &stats,CRowDouble &lagbc,CRowDouble &laglc);
};
//+------------------------------------------------------------------+
//| No presolve, just user-supplied scaling + constraint and cost |
//| vector normalization. |
//| INPUT PARAMETERS: |
//| S - array[N], user - supplied scale vector, S[I] > 0 |
//| C - array[N], costs |
//| BndL - array[N], lower bounds (may contain - INF) |
//| BndU - array[N], upper bounds (may contain + INF) |
//| N - variable count, N > 0 |
//| SparseA - matrix[K, N], sparse constraints |
//| AL - array[K], lower constraint bounds (may contain |
//| -INF) |
//| AU - array[K], upper constraint bounds (may contain |
//| +INF) |
//| K - constraint count, K >= 0 |
//| Info - presolve Info structure; temporaries allocated |
//| during previous calls may be reused by this |
//| function. |
//| OUTPUT PARAMETERS: |
//| Info - contains transformed C, BndL, bndU, SparseA, AL, |
//| AU and information necessary to perform backward |
//| transformation. |
//| Following fields can be acessed: |
//| * Info.NewN > 0 for transformed problem size |
//| * Info.NewM >= 0 for transformed constraint |
//| count |
//| * always: Info.C, Info.BndL, Info.BndU - |
//| array[NewN] |
//| * for Info.NewM > 0: Info.SparseA, Info.AL, |
//| Info.AU |
//| NOTE: this routine does not reallocate arrays if NNew <= NOld |
//| and/or KNew <= KOld. |
//+------------------------------------------------------------------+
void CLPQPPresolve::PresolveNoneScaleUser(CRowDouble &s,
CRowDouble &c,
CRowDouble &bndl,
CRowDouble &bndu,
int n,
CSparseMatrix &sparsea,
CRowDouble &al,
CRowDouble &au,
int k,
CPresolveInfo &Info)
{
//--- create variables
int i=0;
int j=0;
int j0=0;
int j1=0;
double v=0;
double avgln=0;
//--- Integrity checks
if(!CAp::Assert(bndl.Size()>=n,__FUNCTION__+": Length(BndL)<N"))
return;
if(!CAp::Assert(bndu.Size()>=n,__FUNCTION__+": Length(BndU)<N"))
return;
if(!CAp::Assert(s.Size()>=n,__FUNCTION__+": Length(S)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(s,n),__FUNCTION__+": S contains infinite or NaN elements"))
return;
if(!CAp::Assert(c.Size()>=n,__FUNCTION__+": Length(C)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(c,n),__FUNCTION__+": C contains infinite or NaN elements"))
return;
if(!CAp::Assert(k>=0,__FUNCTION__+": K<0"))
return;
if(!CAp::Assert(k==0 || CSparse::SparseIsCRS(sparsea),__FUNCTION__+": A is not CRS"))
return;
if(!CAp::Assert(k==0 || sparsea.m_M==k,__FUNCTION__+": rows(A)<>K"))
return;
if(!CAp::Assert(k==0 || sparsea.m_N==n,__FUNCTION__+": cols(A)<>N"))
return;
//--- Save original problem formulation
Info.m_newn=n;
Info.m_oldn=n;
Info.m_newm=k;
Info.m_oldm=k;
for(i=0; i<n; i++)
{
//--- check
if(!CAp::Assert(s[i]>0,__FUNCTION__+": S<=0"))
return;
if(!CAp::Assert(MathIsValidNumber(bndl[i]) || IsNegInf(bndl[i]),__FUNCTION__+": BndL contains NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(bndu[i]) || IsPosInf(bndu[i]),__FUNCTION__+": BndU contains NAN or -INF"))
return;
}
Info.m_colscales=s;
Info.m_rawbndl=bndl;
Info.m_rawbndu=bndu;
Info.m_rawbndl.Resize(n);
Info.m_rawbndu.Resize(n);
Info.m_colscales.Resize(n);
//--- Scale cost and box constraints
Info.m_c=c*s+0;
Info.m_bndl=bndl/s+0;
Info.m_bndu=bndu/s+0;
Info.m_c.Resize(n);
Info.m_bndl.Resize(n);
Info.m_bndu.Resize(n);
avgln=0;
for(i=0; i<n; i++)
avgln+=MathLog(1+MathAbs(Info.m_c[i]));
Info.m_costscale=MathExp(avgln/n);
Info.m_c/=Info.m_costscale;
//--- Quick exit if no linear constraints is present
if(k==0)
return;
//--- Scale constraint matrix
Info.m_al=al;
Info.m_au=au;
CSparse::SparseCopyBuf(sparsea,Info.m_sparsea);
Info.m_rowscales=vector<double>::Zeros(k);
for(i=0; i<k; i++)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(Info.m_al[i]) || IsNegInf(Info.m_al[i]),__FUNCTION__+": AL contains NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(Info.m_au[i]) || IsPosInf(Info.m_au[i]),__FUNCTION__+": AU contains NAN or -INF"))
return;
j0=Info.m_sparsea.m_RIdx[i];
j1=Info.m_sparsea.m_RIdx[i+1];
for(j=j0; j<j1; j++)
{
v=s[Info.m_sparsea.m_Idx[j]]*Info.m_sparsea.m_Vals[j];
Info.m_sparsea.m_Vals.Set(j,v);
Info.m_rowscales.Set(i,MathMax(Info.m_rowscales[i],v));
}
Info.m_rowscales.Set(i,MathMax(Info.m_rowscales[i],1.0));
v=1/Info.m_rowscales[i];
for(j=j0; j<j1; j++)
Info.m_sparsea.m_Vals.Mul(j,v);
Info.m_al.Mul(i,v);
Info.m_au.Mul(i,v);
}
}
//+------------------------------------------------------------------+
//| Backward transformation which extracts original solution from |
//| that of the converted problem. |
//| Below NNew/KNew correspond to transformed problem size (as |
//| returned by the presolve routine) and NOld/KOld correspond to |
//| original problem size (as specified by caller). We expect that |
//| caller knows these sizes, so this routine does not report them. |
//| INPUT PARAMETERS: |
//| Info - presolve Info structure |
//| X - array[NNew], transformed solution(primal variables)|
//| Stats - array[NNew + MNew], transformed constraint status |
//| (negative - at lower bound, positive - at upper |
//| bound, zero - inactive). |
//| LagBC - array[NNew], transformed Lagrange multipliers |
//| LagLC - array[KNew], transformed Lagrange multipliers |
//| OUTPUT PARAMETERS: |
//| X - array[NOld], original solution(primal variables) |
//| Stats - array[NOld + MOld], original constraint status |
//| LagBC - array[NOld], Lagrange multipliers |
//| LagLC - array[KOld], Lagrange multipliers |
//| NOTE: this routine does not reallocate arrays if NOld <= NNew |
//| and / or KOld <= KNew. |
//+------------------------------------------------------------------+
void CLPQPPresolve::PresolveBwd(CPresolveInfo &Info,
CRowDouble &x,
CRowInt &stats,
CRowDouble &lagbc,
CRowDouble &laglc)
{
//--- create variables
int n=Info.m_oldn;
int m=Info.m_oldm;
int i=0;
//--- check
if(!CAp::Assert(Info.m_oldn==Info.m_newn,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(Info.m_oldm==Info.m_newm,__FUNCTION__+": integrity check failed"))
return;
for(i=0; i<n; i++)
{
if(stats[i]<0)
{
x.Set(i,Info.m_rawbndl[i]);
continue;
}
if(stats[i]>0)
{
x.Set(i,Info.m_rawbndu[i]);
continue;
}
x.Mul(i,Info.m_colscales[i]);
if(MathIsValidNumber(Info.m_rawbndl[i]))
x.Set(i,MathMax(x[i],Info.m_rawbndl[i]));
if(MathIsValidNumber(Info.m_rawbndu[i]))
x.Set(i,MathMin(x[i],Info.m_rawbndu[i]));
}
for(i=0; i<n; i++)
lagbc.Mul(i,Info.m_costscale/Info.m_colscales[i]) ;
for(i=0; i<m; i++)
laglc.Mul(i,Info.m_costscale/Info.m_rowscales[i]) ;
}
//+------------------------------------------------------------------+
//| This object stores Settings for dual simplex solver |
//+------------------------------------------------------------------+
struct CDualSimplexSettings
{
int m_maxtrfage;
int m_pricing;
int m_ratiotest;
int m_shifting;
int m_trftype;
double m_dtolabs;
double m_perturbmag;
double m_pivottol;
double m_xtolabs;
double m_xtolrelabs;
//--- constructor / destructor
CDualSimplexSettings(void) { ZeroMemory(this); }
~CDualSimplexSettings(void) {}
//---
void Copy(const CDualSimplexSettings &obj);
//--- overloading
void operator=(const CDualSimplexSettings &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CDualSimplexSettings::Copy(const CDualSimplexSettings &obj)
{
m_maxtrfage=obj.m_maxtrfage;
m_pricing=obj.m_pricing;
m_ratiotest=obj.m_ratiotest;
m_shifting=obj.m_shifting;
m_trftype=obj.m_trftype;
m_dtolabs=obj.m_dtolabs;
m_perturbmag=obj.m_perturbmag;
m_pivottol=obj.m_pivottol;
m_xtolabs=obj.m_xtolabs;
m_xtolrelabs=obj.m_xtolrelabs;
}
//+------------------------------------------------------------------+
//| A vector that is stored in dual dense / sparse format. |
//+------------------------------------------------------------------+
struct CDSSVector
{
int m_k;
int m_n;
CRowInt m_idx;
CRowDouble m_dense;
CRowDouble m_vals;
//--- constructor / destructor
CDSSVector(void) { m_k=0; m_n=0; }
~CDSSVector(void) {}
//---
void Copy(const CDSSVector &obj);
//--- overloading
void operator=(const CDSSVector &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CDSSVector::Copy(const CDSSVector &obj)
{
m_k=obj.m_k;
m_n=obj.m_n;
m_idx=obj.m_idx;
m_dense=obj.m_dense;
m_vals=obj.m_vals;
}
//+------------------------------------------------------------------+
//| This object stores basis and its triangular factorization for |
//| DualSimplexState |
//+------------------------------------------------------------------+
struct CDualSimplexBasis
{
int m_m;
int m_ns;
int m_statfact;
int m_statupdt;
int m_trfage;
int m_trftype;
double m_eminu;
double m_statoffdiag;
bool m_dsevalid;
bool m_isbasic[];
bool m_isvalidtrf;
CSparseMatrix m_sparsel;
CSparseMatrix m_sparselu1;
CSparseMatrix m_sparselu2;
CSparseMatrix m_sparseludbg;
CSparseMatrix m_sparseu;
CSparseMatrix m_sparseut;
CSLUV2Buffer m_lubuf2;
CRowInt m_colpermbwd;
CRowInt m_densep2;
CRowInt m_densep2c;
CRowInt m_dk;
CRowInt m_idx;
CRowInt m_nidx;
CRowInt m_nrs;
CRowInt m_rk;
CRowInt m_rowpermbwd;
CRowInt m_tcinvidx;
CRowInt m_tmpi;
CRowInt m_utmpi;
CRowDouble m_densemu;
CRowDouble m_densepfieta;
CRowDouble m_dseweights;
CRowDouble m_utmp0;
CRowDouble m_wtmp0;
CRowDouble m_wtmp1;
CRowDouble m_wtmp2;
CMatrixDouble m_denselu2;
CMatrixDouble m_denselu;
//--- constructor / destructor
CDualSimplexBasis(void);
~CDualSimplexBasis(void) {}
//---
void Copy(const CDualSimplexBasis &obj);
//--- overloading
void operator=(const CDualSimplexBasis &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CDualSimplexBasis::CDualSimplexBasis(void)
{
m_m=0;
m_ns=0;
m_statfact=0;
m_statupdt=0;
m_trfage=0;
m_trftype=0;
m_eminu=0;
m_statoffdiag=0;
m_dsevalid=false;
m_isvalidtrf=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CDualSimplexBasis::Copy(const CDualSimplexBasis &obj)
{
m_m=obj.m_m;
m_ns=obj.m_ns;
m_statfact=obj.m_statfact;
m_statupdt=obj.m_statupdt;
m_trfage=obj.m_trfage;
m_trftype=obj.m_trftype;
m_eminu=obj.m_eminu;
m_statoffdiag=obj.m_statoffdiag;
m_dsevalid=obj.m_dsevalid;
ArrayCopy(m_isbasic,obj.m_isbasic);
m_isvalidtrf=obj.m_isvalidtrf;
m_sparsel=obj.m_sparsel;
m_sparselu1=obj.m_sparselu1;
m_sparselu2=obj.m_sparselu2;
m_sparseludbg=obj.m_sparseludbg;
m_sparseu=obj.m_sparseu;
m_sparseut=obj.m_sparseut;
m_lubuf2=obj.m_lubuf2;
m_colpermbwd=obj.m_colpermbwd;
m_densep2=obj.m_densep2;
m_densep2c=obj.m_densep2c;
m_dk=obj.m_dk;
m_idx=obj.m_idx;
m_nidx=obj.m_nidx;
m_nrs=obj.m_nrs;
m_rk=obj.m_rk;
m_rowpermbwd=obj.m_rowpermbwd;
m_tcinvidx=obj.m_tcinvidx;
m_tmpi=obj.m_tmpi;
m_utmpi=obj.m_utmpi;
m_densemu=obj.m_densemu;
m_densepfieta=obj.m_densepfieta;
m_dseweights=obj.m_dseweights;
m_utmp0=obj.m_utmp0;
m_wtmp0=obj.m_wtmp0;
m_wtmp1=obj.m_wtmp1;
m_wtmp2=obj.m_wtmp2;
m_denselu2=obj.m_denselu2;
m_denselu=obj.m_denselu;
}
//+------------------------------------------------------------------+
//| This object stores subproblem for DualSimplexState object |
//+------------------------------------------------------------------+
struct CDualSimplexSubproblem
{
int m_m;
int m_ns;
int m_state;
CRowInt m_bndt;
CRowInt m_bndtb;
CRowDouble m_bndl;
CRowDouble m_bndlb;
CRowDouble m_bndtollb;
CRowDouble m_bndtolub;
CRowDouble m_bndu;
CRowDouble m_bndub;
CRowDouble m_d;
CRowDouble m_effc;
CRowDouble m_rawc;
CRowDouble m_xa;
CRowDouble m_xb;
//--- constructor / destructor
CDualSimplexSubproblem(void) { m_m=0; m_ns=0; m_state=0; }
~CDualSimplexSubproblem(void) {}
//---
void Copy(const CDualSimplexSubproblem &obj);
//--- overloading
void operator=(const CDualSimplexSubproblem &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CDualSimplexSubproblem::Copy(const CDualSimplexSubproblem &obj)
{
m_m=obj.m_m;
m_ns=obj.m_ns;
m_state=obj.m_state;
m_bndt=obj.m_bndt;
m_bndtb=obj.m_bndtb;
m_bndl=obj.m_bndl;
m_bndlb=obj.m_bndlb;
m_bndtollb=obj.m_bndtollb;
m_bndtolub=obj.m_bndtolub;
m_bndu=obj.m_bndu;
m_bndub=obj.m_bndub;
m_d=obj.m_d;
m_effc=obj.m_effc;
m_rawc=obj.m_rawc;
m_xa=obj.m_xa;
m_xb=obj.m_xb;
}
//+------------------------------------------------------------------+
//| This object stores State of the DSS solver. |
//| MUST be initialized with DSSInit(). |
//+------------------------------------------------------------------+
struct CDualSimplexState
{
int m_m;
int m_ns;
int m_possibleflipscnt;
int m_repdualbtrantime;
int m_repdualftrantime;
int m_repdualpivotrowtime;
int m_repdualpricingtime;
int m_repdualratiotesttime;
int m_repdualupdatesteptime;
int m_repfilldensemucnt;
int m_repfillpivotrowcnt;
int m_repfillrhorcnt;
int m_repiterationscount1;
int m_repiterationscount2;
int m_repiterationscount3;
int m_repiterationscount;
int m_repphase1time;
int m_repphase2time;
int m_repphase3time;
int m_repterminationtype;
double m_repfilldensemu;
double m_repfillpivotrow;
double m_repfillrhor;
bool m_dodetailedtrace;
bool m_dotimers;
bool m_dotrace;
CSparseMatrix m_a;
CSparseMatrix m_at;
CRowInt m_eligiblealphar;
CRowInt m_harrisset;
CRowInt m_possibleflips;
CRowInt m_repstats;
CRowInt m_ustmpi;
CRowDouble m_alphaq;
CRowDouble m_alphaqim;
CRowDouble m_btrantmp0;
CRowDouble m_btrantmp1;
CRowDouble m_btrantmp2;
CRowDouble m_dfctmp0;
CRowDouble m_dfctmp1;
CRowDouble m_dfctmp2;
CRowDouble m_ftrantmp0;
CRowDouble m_ftrantmp1;
CRowDouble m_rawbndl;
CRowDouble m_rawbndu;
CRowDouble m_replagbc;
CRowDouble m_replaglc;
CRowDouble m_repx;
CRowDouble m_rowscales;
CRowDouble m_tau;
CRowDouble m_tmp0;
CRowDouble m_tmp1;
CRowDouble m_tmp2;
CDualSimplexSubproblem m_phase1;
CDualSimplexSubproblem m_phase3;
CDualSimplexSubproblem m_primary;
CDualSimplexBasis m_basis;
CDSSVector m_alphar;
CDSSVector m_rhor;
CApBuff m_xydsbuf;
//--- constructor / destructor
CDualSimplexState(void);
~CDualSimplexState(void) {}
//---
void Copy(const CDualSimplexState &obj);
//--- overloading
void operator=(const CDualSimplexState &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CDualSimplexState::CDualSimplexState(void)
{
m_m=0;
m_ns=0;
m_possibleflipscnt=0;
m_repdualbtrantime=0;
m_repdualftrantime=0;
m_repdualpivotrowtime=0;
m_repdualpricingtime=0;
m_repdualratiotesttime=0;
m_repdualupdatesteptime=0;
m_repfilldensemucnt=0;
m_repfillpivotrowcnt=0;
m_repfillrhorcnt=0;
m_repiterationscount1=0;
m_repiterationscount2=0;
m_repiterationscount3=0;
m_repiterationscount=0;
m_repphase1time=0;
m_repphase2time=0;
m_repphase3time=0;
m_repterminationtype=0;
m_repfilldensemu=0;
m_repfillpivotrow=0;
m_repfillrhor=0;
m_dodetailedtrace=false;
m_dotimers=false;
m_dotrace=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CDualSimplexState::Copy(const CDualSimplexState &obj)
{
m_m=obj.m_m;
m_ns=obj.m_ns;
m_possibleflipscnt=obj.m_possibleflipscnt;
m_repdualbtrantime=obj.m_repdualbtrantime;
m_repdualftrantime=obj.m_repdualftrantime;
m_repdualpivotrowtime=obj.m_repdualpivotrowtime;
m_repdualpricingtime=obj.m_repdualpricingtime;
m_repdualratiotesttime=obj.m_repdualratiotesttime;
m_repdualupdatesteptime=obj.m_repdualupdatesteptime;
m_repfilldensemucnt=obj.m_repfilldensemucnt;
m_repfillpivotrowcnt=obj.m_repfillpivotrowcnt;
m_repfillrhorcnt=obj.m_repfillrhorcnt;
m_repiterationscount1=obj.m_repiterationscount1;
m_repiterationscount2=obj.m_repiterationscount2;
m_repiterationscount3=obj.m_repiterationscount3;
m_repiterationscount=obj.m_repiterationscount;
m_repphase1time=obj.m_repphase1time;
m_repphase2time=obj.m_repphase2time;
m_repphase3time=obj.m_repphase3time;
m_repterminationtype=obj.m_repterminationtype;
m_repfilldensemu=obj.m_repfilldensemu;
m_repfillpivotrow=obj.m_repfillpivotrow;
m_repfillrhor=obj.m_repfillrhor;
m_dodetailedtrace=obj.m_dodetailedtrace;
m_dotimers=obj.m_dotimers;
m_dotrace=obj.m_dotrace;
m_a=obj.m_a;
m_at=obj.m_at;
m_eligiblealphar=obj.m_eligiblealphar;
m_harrisset=obj.m_harrisset;
m_possibleflips=obj.m_possibleflips;
m_repstats=obj.m_repstats;
m_ustmpi=obj.m_ustmpi;
m_alphaq=obj.m_alphaq;
m_alphaqim=obj.m_alphaqim;
m_btrantmp0=obj.m_btrantmp0;
m_btrantmp1=obj.m_btrantmp1;
m_btrantmp2=obj.m_btrantmp2;
m_dfctmp0=obj.m_dfctmp0;
m_dfctmp1=obj.m_dfctmp1;
m_dfctmp2=obj.m_dfctmp2;
m_ftrantmp0=obj.m_ftrantmp0;
m_ftrantmp1=obj.m_ftrantmp1;
m_rawbndl=obj.m_rawbndl;
m_rawbndu=obj.m_rawbndu;
m_replagbc=obj.m_replagbc;
m_replaglc=obj.m_replaglc;
m_repx=obj.m_repx;
m_rowscales=obj.m_rowscales;
m_tau=obj.m_tau;
m_tmp0=obj.m_tmp0;
m_tmp1=obj.m_tmp1;
m_tmp2=obj.m_tmp2;
m_phase1=obj.m_phase1;
m_phase3=obj.m_phase3;
m_primary=obj.m_primary;
m_basis=obj.m_basis;
m_alphar=obj.m_alphar;
m_rhor=obj.m_rhor;
m_xydsbuf=obj.m_xydsbuf;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CRevisedDualSimplex
{
public:
//--- constants
static const int m_maxforcedrestarts;
static const int m_safetrfage;
static const int m_defaultmaxtrfage;
static const double m_minbeta;
static const double m_maxudecay;
static const double m_shiftlen;
static const double m_alphatrigger;
static const double m_alphatrigger2;
static const int m_ssinvalid;
static const int m_ssvalidxn;
static const int m_ssvalid;
static const int m_ccfixed;
static const int m_cclower;
static const int m_ccupper;
static const int m_ccrange;
static const int m_ccfree;
static const int m_ccinfeasible;
static void DSSSettingsInit(CDualSimplexSettings &Settings);
static void DSSInit(int n,CDualSimplexState &s);
static void DSSSetProblem(CDualSimplexState &State,CRowDouble &c,CRowDouble &bndl,CRowDouble &bndu,CMatrixDouble &densea,CSparseMatrix &sparsea,int akind,CRowDouble &al,CRowDouble &au,int k,CDualSimplexBasis &proposedbasis,int basisinittype,CDualSimplexSettings &Settings);
static void DSSExportBasis(CDualSimplexState &State,CDualSimplexBasis &basis);
static void DSSOptimize(CDualSimplexState &State,CDualSimplexSettings &Settings);
private:
static void SubproblemInit(int n,CDualSimplexSubproblem &s);
static void SubproblemInitPhase1(CDualSimplexSubproblem &s0,CDualSimplexBasis &basis,CDualSimplexSubproblem &s1);
static void SubproblemInitPhase3(CDualSimplexSubproblem &s0,CDualSimplexSubproblem &s1);
static void SubproblemInferInitialXN(CDualSimplexState &State,CDualSimplexSubproblem &s);
static void SubproblemHandleXNUpdate(CDualSimplexState &State,CDualSimplexSubproblem &s);
static double InitialDualFeasibilityCorrection(CDualSimplexState &State,CDualSimplexSubproblem &s,CDualSimplexSettings &Settings);
static void Shifting(CDualSimplexState &State,CDualSimplexSubproblem &s,CDSSVector &alphar,double delta,int q,double alpharpiv,double &thetad,CDualSimplexSettings &Settings);
static void PricingStep(CDualSimplexState &State,CDualSimplexSubproblem &s,bool phase1pricing,int &p,int &r,double &delta,CDualSimplexSettings &Settings);
static void BTranStep(CDualSimplexState &State,CDualSimplexSubproblem &s,int r,CDSSVector &rhor,CDualSimplexSettings &Settings);
static void PivotRowStep(CDualSimplexState &State,CDualSimplexSubproblem &s,CDSSVector &rhor,CDSSVector &alphar,CDualSimplexSettings &Settings);
static void FTranStep(CDualSimplexState &State,CDualSimplexSubproblem &s,CDSSVector &rhor,int q,CRowDouble &alphaq,CRowDouble &alphaqim,CRowDouble &tau,CDualSimplexSettings &Settings);
static void RatioTest(CDualSimplexState &State,CDualSimplexSubproblem &s,CDSSVector &alphar,double delta,int p,int &q,double &alpharpiv,double &thetad,CRowInt &possibleflips,int &possibleflipscnt,CDualSimplexSettings &Settings);
static void UpdateStep(CDualSimplexState &State,CDualSimplexSubproblem &s,int p,int q,int r,double delta,double alphapiv,double thetap,double thetad,CRowDouble &alphaq,CRowDouble &alphaqim,CDSSVector &alphar,CRowDouble &tau,CRowInt &possiblealpharflips,int possiblealpharflipscnt,CDualSimplexSettings &Settings);
static bool RefactorizationRequired(CDualSimplexState &State,CDualSimplexSubproblem &s,int q,double alpharpiv,int r,double alphaqpiv);
static void CacheBoundInfo(CDualSimplexSubproblem &s,int i,int k,CDualSimplexSettings &Settings);
static void SolveSubproblemDual(CDualSimplexState &State,CDualSimplexSubproblem &s,bool IsPhase1,CDualSimplexSettings &Settings,int &Info);
static void SolveSubproblemPrimal(CDualSimplexState &State,CDualSimplexSubproblem &s,CDualSimplexSettings &Settings,int &Info);
static void InvokePhase1(CDualSimplexState &State,CDualSimplexSettings &Settings);
static void DSSOptimizeWrk(CDualSimplexState &State,CDualSimplexSettings &Settings);
static void SolveBoxOnly(CDualSimplexState &State);
static void SetZeroXYStats(CDualSimplexState &State);
static void BasisInit(int ns,int m,CDualSimplexBasis &s);
static void BasisClearStats(CDualSimplexBasis &s);
static bool BasisTryResize(CDualSimplexBasis &s,int newm,CSparseMatrix &at,CDualSimplexSettings &Settings);
static double BasisMinimumDiagonalElement(CDualSimplexBasis &s);
static void BasisExportTo(CDualSimplexBasis &s0,CDualSimplexBasis &s1);
static bool BasisTryImportFrom(CDualSimplexBasis &s0,CDualSimplexBasis &s1,CSparseMatrix &at,CDualSimplexSettings &Settings);
static void BasisFreshTrf(CDualSimplexBasis &s,CSparseMatrix &at,CDualSimplexSettings &Settings);
static double BasisFreshTRFUnsafe(CDualSimplexBasis &s,CSparseMatrix &at,CDualSimplexSettings &Settings);
static void BasisRequestWeights(CDualSimplexBasis &s,CDualSimplexSettings &Settings);
static void BasisUpdateTrf(CDualSimplexBasis &s,CSparseMatrix &at,int p,int q,CRowDouble &alphaq,CRowDouble &alphaqim,int r,CRowDouble &tau,CDualSimplexSettings &Settings);
static void BasisSolve(CDualSimplexBasis &s,CRowDouble &r,CRowDouble &x,CRowDouble &tmpx);
static void BasisSolveX(CDualSimplexBasis &s,CRowDouble &r,CRowDouble &x,CRowDouble &xim,bool needintermediate,CRowDouble &tx);
static void BasisSolveT(CDualSimplexBasis &s,CRowDouble &r,CRowDouble &x,CRowDouble &tx);
static void ComputeAnXn(CDualSimplexState &State,CDualSimplexSubproblem &subproblem,CRowDouble &x,CRowDouble &y);
static void ComputeAnTV(CDualSimplexState &State,CRowDouble &y,CRowDouble &r);
static bool HasBndL(CDualSimplexSubproblem &subproblem,int i);
static bool HasBndU(CDualSimplexSubproblem &subproblem,int i);
static bool IsFree(CDualSimplexSubproblem &subproblem,int i);
static void DowngradeState(CDualSimplexSubproblem &subproblem,int s);
static double DualFeasibilityError(CDualSimplexState &State,CDualSimplexSubproblem &s);
static bool IsDualFeasible(CDualSimplexState &State,CDualSimplexSubproblem &s,CDualSimplexSettings &Settings);
static void PivotToBWD(CRowInt &p,int m,CRowInt &bwd);
static void InverseCyclicPermutation(CRowInt &bwd,int m,int d,CRowInt &tmpi);
static void OffloadBasicComponents(CDualSimplexSubproblem &s,CDualSimplexBasis &basis,CDualSimplexSettings &Settings);
static void RecombineBasicNonBasicX(CDualSimplexSubproblem &s,CDualSimplexBasis &basis);
static void SetXYDStats(CDualSimplexState &State,CDualSimplexSubproblem &s,CDualSimplexBasis &basis,CApBuff &buffers,CRowDouble &x,CRowDouble &lagbc,CRowDouble &laglc,CRowInt &stats);
static void DVAlloc(CDSSVector &x,int n);
static void DVInit(CDSSVector &x,int n);
static void DVDenseToSparse(CDSSVector &x);
static void DVSparseToDense(CDSSVector &x);
static double SparsitYOf(CRowDouble &x,int n);
static void UpdateAvgCounter(double v,double &acc,int &cnt);
};
//+------------------------------------------------------------------+
//| Constants |
//+------------------------------------------------------------------+
const int CRevisedDualSimplex::m_maxforcedrestarts=1;
const int CRevisedDualSimplex::m_safetrfage=5;
const int CRevisedDualSimplex::m_defaultmaxtrfage=100;
const double CRevisedDualSimplex::m_minbeta=1.0E-4;
const double CRevisedDualSimplex::m_maxudecay=0.001;
const double CRevisedDualSimplex::m_shiftlen=1.0E-12;
const double CRevisedDualSimplex::m_alphatrigger=1.0E8*CMath::m_machineepsilon;
const double CRevisedDualSimplex::m_alphatrigger2=0.001;
const int CRevisedDualSimplex::m_ssinvalid=0;
const int CRevisedDualSimplex::m_ssvalidxn=1;
const int CRevisedDualSimplex::m_ssvalid=2;
const int CRevisedDualSimplex::m_ccfixed=0;
const int CRevisedDualSimplex::m_cclower=1;
const int CRevisedDualSimplex::m_ccupper=2;
const int CRevisedDualSimplex::m_ccrange=3;
const int CRevisedDualSimplex::m_ccfree=4;
const int CRevisedDualSimplex::m_ccinfeasible=5;
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::DSSSettingsInit(CDualSimplexSettings &Settings)
{
Settings.m_xtolabs=1.0E-6;
Settings.m_dtolabs=1.0E-6;
Settings.m_xtolrelabs=0.01;
Settings.m_pivottol=10*MathSqrt(CMath::m_machineepsilon);
Settings.m_perturbmag=10*Settings.m_pivottol;
Settings.m_maxtrfage=m_defaultmaxtrfage;
Settings.m_trftype=3;
Settings.m_ratiotest=1;
Settings.m_pricing=1;
Settings.m_shifting=2;
}
//+------------------------------------------------------------------+
//| This function initializes DSS structure. Previously allocated |
//| memory is reused as much as possible. |
//| Default State of the problem is zero cost vector, all variables |
//| are fixed at zero. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::DSSInit(int n,CDualSimplexState &s)
{
//--- check
if(!CAp::Assert(n>0,__FUNCTION__+": N<=0"))
return;
s.m_ns=n;
s.m_m=0;
s.m_rawbndl=vector<double>::Zeros(n);
s.m_rawbndu=vector<double>::Zeros(n);
SubproblemInit(n,s.m_primary);
BasisInit(n,0,s.m_basis);
s.m_repx=vector<double>::Zeros(n);
s.m_replagbc.Resize(n);
s.m_repstats.Resize(n);
s.m_repstats.Fill(1);
s.m_dotrace=false;
s.m_dodetailedtrace=false;
s.m_dotimers=false;
}
//+------------------------------------------------------------------+
//| This function specifies LP problem |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinLPCreate() |
//| call. |
//| BndL - lower bounds, array[N]. |
//| BndU - upper bounds, array[N]. |
//| DenseA - dense array[K, N], dense linear constraints (not |
//| supported in present version) |
//| SparseA - sparse linear constraints, sparsematrix[K, N] in |
//| CRS format |
//| AKind - type of A: 0 for dense, 1 for sparse |
//| AL, AU - lower and upper bounds, array[K] |
//| K - number of equality/inequality constraints, K >= 0. |
//| ProposedBasis - basis to import from (if BasisType = 2) |
//| BasisInitType - what to do with basis: |
//| * 0 - set new basis to all-logicals |
//| * 1 - try to reuse previous basis as much as |
//| possible |
//| * 2 - try to import basis from ProposedBasis |
//| Settings - algorithm Settings |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::DSSSetProblem(CDualSimplexState &State,
CRowDouble &c,
CRowDouble &bndl,
CRowDouble &bndu,
CMatrixDouble &densea,
CSparseMatrix &sparsea,
int akind,
CRowDouble &al,
CRowDouble &au,
int k,
CDualSimplexBasis &proposedbasis,
int basisinittype,
CDualSimplexSettings &Settings)
{
//--- create variables
int ns=State.m_primary.m_ns;
int oldm=State.m_primary.m_m;
int i=0;
int j=0;
int jj=0;
int offs=0;
int j0=0;
int j1=0;
bool processed=false;
bool basisinitialized=false;
double v=0;
//--- Integrity checks
if(!CAp::Assert(bndl.Size()>=ns,__FUNCTION__+": Length(BndL)<N"))
return;
if(!CAp::Assert(bndu.Size()>=ns,__FUNCTION__+": Length(BndU)<N"))
return;
if(!CAp::Assert(c.Size()>=ns,__FUNCTION__+": Length(C)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(c,ns),__FUNCTION__+": C contains infinite or NaN elements"))
return;
if(!CAp::Assert(akind==0 || akind==1,__FUNCTION__+": incorrect AKind"))
return;
if(!CAp::Assert(basisinittype==0 || basisinittype==1 || basisinittype==2,__FUNCTION__+": incorrect BasisInitType"))
return;
if(!CAp::Assert(k>=0,__FUNCTION__+": K<0"))
return;
if(k>0 && akind==1)
{
if(!CAp::Assert(sparsea.m_M==k,__FUNCTION__+": rows(A)<>K"))
return;
if(!CAp::Assert(sparsea.m_N==ns,__FUNCTION__+": cols(A)<>N"))
return;
}
//--- Downgrade State
DowngradeState(State.m_primary,m_ssinvalid);
//--- Reallocate storage
State.m_primary.m_bndl.Resize(ns+k);
State.m_primary.m_bndu.Resize(ns+k);
State.m_primary.m_bndt.Resize(ns+k);
State.m_primary.m_effc=vector<double>::Zeros(ns+k);
State.m_primary.m_rawc=vector<double>::Zeros(ns+k);
State.m_primary.m_xa.Resize(ns+k);
State.m_primary.m_d.Resize(ns+k);
State.m_primary.m_xb.Resize(k);
State.m_primary.m_bndlb.Resize(k);
State.m_primary.m_bndub.Resize(k);
State.m_primary.m_bndtb.Resize(k);
State.m_primary.m_bndtollb.Resize(k);
State.m_primary.m_bndtolub.Resize(k);
//--- Save original problem formulation
State.m_ns=ns;
State.m_m=k;
State.m_rawbndl=bndl;
State.m_rawbndu=bndu;
//--- Setup cost, scale and box constraints
for(i=0; i<ns; i++)
{
State.m_primary.m_rawc.Set(i,c[i]);
State.m_primary.m_effc.Set(i,c[i]);
}
for(i=0; i<ns; i++)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(bndl[i]) || IsNegInf(bndl[i]),__FUNCTION__+": BndL contains NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(bndu[i]) || IsPosInf(bndu[i]),__FUNCTION__+": BndU contains NAN or -INF"))
return;
State.m_primary.m_bndl.Set(i,bndl[i]);
State.m_primary.m_bndu.Set(i,bndu[i]);
//--- Set bound type
if(MathIsValidNumber(bndl[i]) && MathIsValidNumber(bndu[i]))
{
if(bndl[i]>bndu[i])
State.m_primary.m_bndt.Set(i,m_ccinfeasible);
if(bndl[i]<bndu[i])
State.m_primary.m_bndt.Set(i,m_ccrange);
if(bndl[i]==bndu[i])
State.m_primary.m_bndt.Set(i,m_ccfixed);
continue;
}
if(MathIsValidNumber(bndl[i]) && !MathIsValidNumber(bndu[i]))
{
State.m_primary.m_bndt.Set(i,m_cclower);
continue;
}
if(!MathIsValidNumber(bndl[i]) && MathIsValidNumber(bndu[i]))
{
State.m_primary.m_bndt.Set(i,m_ccupper);
continue;
}
//--- check
if(!CAp::Assert(IsNegInf(bndl[i]) && IsPosInf(bndu[i]),__FUNCTION__+": integrity check failed"))
return;
State.m_primary.m_bndt.Set(i,m_ccfree);
}
//--- Quick exit if no linear constraints is present
if(k==0)
{
State.m_primary.m_m=0;
BasisInit(State.m_primary.m_ns,State.m_primary.m_m,State.m_basis);
return;
}
//--- Extend A with structural terms and transpose it:
//--- * allocate place for A^T extended with logical part.
//--- * copy with transposition
//--- * perform integrity check for array sizes
//--- * manually append new items
//--- * update DIdx/UIdx
processed=false;
State.m_primary.m_m=k;
if(akind==0)
{
CAp::Assert(false,__FUNCTION__+": does not support dense inputs yet");
return;
}
if(akind==1)
{
//--- Transpose constraints matrix, apply column and row scaling.
//--- Extend it with identity submatrix.
//--- NOTE: in order to improve stability of LU factorization we
//--- normalize rows using 2-norm, not INF-norm. Having rows
//--- normalized with 2-norm makes every element less than
//--- 1.0 in magnitude, which allows us later to move logical
//--- columns to the beginning of LU factors without loosing
//--- stability.
State.m_at.m_Vals.Resize(sparsea.m_RIdx[k]+k);
State.m_at.m_Idx.Resize(sparsea.m_RIdx[k]+k);
State.m_at.m_RIdx.Resize(ns+k+1);
State.m_at.m_DIdx.Resize(ns+k);
State.m_at.m_UIdx.Resize(ns+k);
CSparse::SparseCopyTransposeCRSBuf(sparsea,State.m_at);
State.m_rowscales=vector<double>::Zeros(k);
for(i=0; i<ns; i++)
{
j0=State.m_at.m_RIdx[i];
j1=State.m_at.m_RIdx[i+1];
for(j=j0; j<j1; j++)
{
v=State.m_at.m_Vals[j];
jj=State.m_at.m_Idx[j];
State.m_rowscales.Set(jj,State.m_rowscales[jj]+v*v);
}
}
for(i=0; i<k; i++)
State.m_rowscales.Set(i,CApServ::Coalesce(MathSqrt(State.m_rowscales[i]),1));
State.m_tmp0=State.m_rowscales.Pow(-1.0)+0;
State.m_tmp0.Resize(k);
for(i=0; i<ns; i++)
{
j0=State.m_at.m_RIdx[i];
j1=State.m_at.m_RIdx[i+1];
for(j=j0; j<j1; j++)
State.m_at.m_Vals.Mul(j,State.m_tmp0[State.m_at.m_Idx[j]]);
}
//--- check
if(!CAp::Assert(State.m_at.m_Vals.Size()>=sparsea.m_RIdx[k]+k,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(State.m_at.m_Idx.Size()>=sparsea.m_RIdx[k]+k,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(State.m_at.m_RIdx.Size()>=ns+k+1,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(State.m_at.m_DIdx.Size()>=ns+k,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(State.m_at.m_UIdx.Size()>=ns+k,__FUNCTION__+": integrity check failed"))
return;
offs=State.m_at.m_RIdx[ns];
for(i=0; i<k; i++)
{
State.m_at.m_Vals.Set(offs+i,-1.0);
State.m_at.m_Idx.Set(offs+i,i);
State.m_at.m_RIdx.Set(ns+i+1,State.m_at.m_RIdx[ns+i]+1);
State.m_at.m_NInitialized++;
}
State.m_at.m_M+=k;
CSparse::SparseInitDUIdx(State.m_at);
CSparse::SparseCopyTransposeCRSBuf(State.m_at,State.m_a);
processed=true;
}
//--- check
if(!CAp::Assert(processed,__FUNCTION__+": integrity check failed (akind)"))
return;
//--- Copy AL, AU to BndL/BndT
for(i=0; i<k; i++)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(al[i]) || IsNegInf(al[i]),__FUNCTION__+": AL contains NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(au[i]) || IsPosInf(au[i]),__FUNCTION__+": AU contains NAN or -INF"))
return;
State.m_primary.m_bndl.Set(ns+i,al[i]/State.m_rowscales[i]);
State.m_primary.m_bndu.Set(ns+i,au[i]/State.m_rowscales[i]);
//--- Set bound type
if(MathIsValidNumber(al[i]) && MathIsValidNumber(au[i]))
{
if(al[i]>au[i])
State.m_primary.m_bndt.Set(ns+i,m_ccinfeasible);
if(al[i]<au[i])
State.m_primary.m_bndt.Set(ns+i,m_ccrange);
if(al[i]==au[i])
State.m_primary.m_bndt.Set(ns+i,m_ccfixed);
continue;
}
if(MathIsValidNumber(al[i]) && !MathIsValidNumber(au[i]))
{
State.m_primary.m_bndt.Set(ns+i,m_cclower);
continue;
}
if(!MathIsValidNumber(al[i]) && MathIsValidNumber(au[i]))
{
State.m_primary.m_bndt.Set(ns+i,m_ccupper);
continue;
}
//--- check
if(!CAp::Assert(IsNegInf(al[i]) && IsPosInf(au[i]),__FUNCTION__+": integrity check faoled"))
return;
State.m_primary.m_bndt.Set(ns+i,m_ccfree);
}
//--- Depending on BasisInitType either start from all-logical basis
//--- or try to reuse already existing basis.
//--- NOTE: current version does not support basis shrinkage, only
//--- growing basis can be reused.
basisinitialized=false;
if(basisinittype==2)
{
//--- Import basis from one proposed by caller
//--- check
if(!CAp::Assert(proposedbasis.m_ns==State.m_primary.m_ns,__FUNCTION__+": unable to import basis,sizes do not match"))
return;
if(!CAp::Assert(proposedbasis.m_m==State.m_primary.m_m,__FUNCTION__+": unable to import basis,sizes do not match"))
return;
basisinitialized=BasisTryImportFrom(State.m_basis,proposedbasis,State.m_at,Settings);
}
if(basisinittype==1 && State.m_primary.m_m>=oldm)
{
//--- New rows were added, try to reuse previous basis
for(i=oldm; i<State.m_primary.m_m; i++)
{
State.m_primary.m_rawc.Set(ns+i,0.0);
State.m_primary.m_effc.Set(ns+i,0.0);
State.m_primary.m_xa.Set(ns+i,0.0);
State.m_primary.m_d.Set(ns+i,0.0);
}
basisinitialized=BasisTryResize(State.m_basis,State.m_primary.m_m,State.m_at,Settings);
}
if(!basisinitialized)
{
//--- Straightforward code for all-logicals basis
for(i=0; i<k; i++)
{
State.m_primary.m_rawc.Set(ns+i,0.0);
State.m_primary.m_effc.Set(ns+i,0.0);
State.m_primary.m_xa.Set(ns+i,0.0);
State.m_primary.m_d.Set(ns+i,0.0);
}
BasisInit(State.m_primary.m_ns,State.m_primary.m_m,State.m_basis);
BasisFreshTrf(State.m_basis,State.m_at,Settings);
}
State.m_replaglc.Resize(State.m_primary.m_m);
State.m_repstats.Resize(State.m_primary.m_ns+State.m_primary.m_m);
}
//+------------------------------------------------------------------+
//| This function exports basis from the primary (phase II) |
//| subproblem. |
//| INPUT PARAMETERS: |
//| State - structure |
//| OUTPUT PARAMETERS: |
//| Basis - current basis exported (no factorization, only set |
//| of basis / nonbasic variables) |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::DSSExportBasis(CDualSimplexState &State,
CDualSimplexBasis &basis)
{
BasisExportTo(State.m_basis,basis);
}
//+------------------------------------------------------------------+
//| This function solves LP problem with dual simplex solver. |
//| INPUT PARAMETERS: |
//| State - State |
//| Solution results can be found in fields of State which are |
//| explicitly declared as accessible by external code. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::DSSOptimize(CDualSimplexState &State,
CDualSimplexSettings &Settings)
{
//--- create variables
int nx=State.m_primary.m_ns+State.m_primary.m_m;
int i=0;
double v=0;
int cnt1=0;
int cnt2=0;
int cntfx=0;
int cntfr=0;
int cntif=0;
int ttotal=0;
//--- Trace Settings
State.m_dotrace=CAp::IsTraceEnabled("DSS");
State.m_dodetailedtrace=State.m_dotrace && CAp::IsTraceEnabled("DSS.DETAILED");
State.m_dotimers=CAp::IsTraceEnabled("TIMERS.DSS");
//--- Init report fields
State.m_repiterationscount=0;
State.m_repiterationscount1=0;
State.m_repiterationscount2=0;
State.m_repiterationscount3=0;
State.m_repterminationtype=1;
State.m_repphase1time=0;
State.m_repphase2time=0;
State.m_repphase3time=0;
State.m_repdualpricingtime=0;
State.m_repdualbtrantime=0;
State.m_repdualpivotrowtime=0;
State.m_repdualratiotesttime=0;
State.m_repdualftrantime=0;
State.m_repdualupdatesteptime=0;
State.m_repfillpivotrow=0;
State.m_repfillpivotrowcnt=0;
State.m_repfillrhor=0;
State.m_repfillrhorcnt=0;
State.m_repfilldensemu=0;
State.m_repfilldensemucnt=0;
BasisClearStats(State.m_basis);
//--- Setup timer (if needed)
if(State.m_dotimers)
ttotal=(int)(int)(GetTickCount()/10000);
//--- Trace output (if needed)
if(State.m_dotrace || State.m_dotimers)
{
CAp::Trace("\n\n");
CAp::Trace("////////////////////////////////////////////////////////////////////////////////////////////////////\n");
CAp::Trace("//--- DUAL SIMPLEX SOLVER STARTED //\n");
CAp::Trace("////////////////////////////////////////////////////////////////////////////////////////////////////\n");
CAp::Trace("> problem size:\n");
CAp::Trace(StringFormat("N = %12d (variables)\n",State.m_primary.m_ns));
CAp::Trace(StringFormat("M = %12d (constraints)\n",State.m_primary.m_m));
}
if(State.m_dotrace)
{
CAp::Trace("> variable stats:\n");
cnt1=0;
cnt2=0;
cntfx=0;
cntfr=0;
cntif=0;
for(i=0; i<State.m_primary.m_ns; i++)
{
if(State.m_primary.m_bndt[i]==m_cclower || State.m_primary.m_bndt[i]==m_ccupper)
cnt1++;
if(State.m_primary.m_bndt[i]==m_ccrange)
cnt2++;
if(State.m_primary.m_bndt[i]==m_ccfixed)
cntfx++;
if(State.m_primary.m_bndt[i]==m_ccfree)
cntfr++;
if(State.m_primary.m_bndt[i]==m_ccinfeasible)
cntif++;
}
CAp::Trace(StringFormat("UBnd/LBnd = %12d\n",cnt1));
CAp::Trace(StringFormat("Range = %12d\n",cnt2));
CAp::Trace(StringFormat("Fixed = %12d\n",cntfx));
CAp::Trace(StringFormat("Free = %12d\n",cntfr));
CAp::Trace(StringFormat("Infeas = %12d\n",cntif));
CAp::Trace("> constraint stats:\n");
cnt1=0;
cnt2=0;
cntfx=0;
cntfr=0;
cntif=0;
for(i=State.m_primary.m_ns-1; i<nx; i++)
{
if(State.m_primary.m_bndt[i]==m_cclower || State.m_primary.m_bndt[i]==m_ccupper)
cnt1++;
if(State.m_primary.m_bndt[i]==m_ccrange)
cnt2++;
if(State.m_primary.m_bndt[i]==m_ccfixed)
cntfx++;
if(State.m_primary.m_bndt[i]==m_ccfree)
cntfr++;
if(State.m_primary.m_bndt[i]==m_ccinfeasible)
cntif++;
}
CAp::Trace(StringFormat("ubnd/lbnd = %12d\n",cnt1));
CAp::Trace(StringFormat("range = %12d\n",cnt2));
CAp::Trace(StringFormat("fixed = %12d\n",cntfx));
CAp::Trace(StringFormat("free = %12d\n",cntfr));
CAp::Trace(StringFormat("infeas = %12d\n",cntif));
v=0;
for(i=0; i<State.m_primary.m_ns; i++)
if(MathIsValidNumber(State.m_primary.m_bndl[i]))
v=MathMax(v,MathAbs(State.m_primary.m_bndl[i]));
CAp::Trace(StringFormat("|BndL| = %.3E\n",v));
v=0;
for(i=0; i<State.m_primary.m_ns; i++)
if(MathIsValidNumber(State.m_primary.m_bndu[i]))
v=MathMax(v,MathAbs(State.m_primary.m_bndu[i]));
CAp::Trace(StringFormat("|BndU| = %.3E\n",v));
v=0;
for(i=State.m_primary.m_ns; i<nx; i++)
if(MathIsValidNumber(State.m_primary.m_bndl[i]))
v=MathMax(v,MathAbs(State.m_primary.m_bndl[i]));
CAp::Trace(StringFormat("|AL| = %.3E\n",v));
v=0;
for(i=State.m_primary.m_ns; i<nx; i++)
if(MathIsValidNumber(State.m_primary.m_bndu[i]))
v=MathMax(v,MathAbs(State.m_primary.m_bndu[i]));
CAp::Trace(StringFormat("|AU| = %.3E\n",v));
}
//--- Call actual workhorse function
DSSOptimizeWrk(State,Settings);
//--- Print reports
if(State.m_dotrace)
{
CAp::Trace("\n");
CAp::Trace("****************************************************************************************************\n");
CAp::Trace("* PRINTING ITERATION STATISTICS *\n");
CAp::Trace("****************************************************************************************************\n");
CAp::Trace("> iteration counts:\n");
CAp::Trace(StringFormat("Phase 1 = %12d\n",State.m_repiterationscount1));
CAp::Trace(StringFormat("Phase 2 = %12d\n",State.m_repiterationscount2));
CAp::Trace(StringFormat("Phase 3 = %12d\n",State.m_repiterationscount3));
CAp::Trace("> factorization statistics:\n");
CAp::Trace(StringFormat("FactCnt = %12d (LU factorizations)\n",State.m_basis.m_statfact));
CAp::Trace(StringFormat("UpdtCnt = %12d (LU updates)\n",State.m_basis.m_statupdt));
CAp::Trace(StringFormat("RefactPeriod= %12.1f (average refactorization interval)\n",(State.m_basis.m_statfact + State.m_basis.m_statupdt) / CApServ::Coalesce(State.m_basis.m_statfact,1)));
CAp::Trace(StringFormat("LU-NZR = %12.1f (average LU nonzeros per row)\n",State.m_basis.m_statoffdiag / (CApServ::Coalesce(State.m_m,1)*CApServ::Coalesce(State.m_basis.m_statfact + State.m_basis.m_statupdt,1))));
CAp::Trace("> sparsity counters (average fill factors):\n");
if(State.m_dodetailedtrace)
{
CAp::Trace(StringFormat("RhoR = %12.4f (BTran result)\n",State.m_repfillrhor / CApServ::Coalesce(State.m_repfillrhorcnt,1)));
CAp::Trace(StringFormat("AlphaR = %12.4f (pivot row)\n",State.m_repfillpivotrow / CApServ::Coalesce(State.m_repfillpivotrowcnt,1)));
if(State.m_basis.m_trftype==3)
CAp::Trace(StringFormat("Mu = %12.4f (Forest-Tomlin factor)\n",State.m_repfilldensemu / CApServ::Coalesce(State.m_repfilldensemucnt,1)));
}
else
CAp::Trace("...skipped,need DUALSIMPLEX.DETAILED trace tag\n");
}
if(State.m_dotimers)
{
ttotal=(int)(int)(GetTickCount()/10000)-ttotal;
CAp::Trace("\n");
CAp::Trace("****************************************************************************************************\n");
CAp::Trace("* PRINTING DUAL SIMPLEX TIMERS *\n");
CAp::Trace("****************************************************************************************************\n");
CAp::Trace("> total time:\n");
CAp::Trace(StringFormat("Time = %12d ms\n",ttotal));
CAp::Trace("> time by phase:\n");
CAp::Trace(StringFormat("Phase 1 = %12d ms\n",State.m_repphase1time));
CAp::Trace(StringFormat("Phase 2 = %12d ms\n",State.m_repphase2time));
CAp::Trace(StringFormat("Phase 3 = %12d ms\n",State.m_repphase3time));
CAp::Trace("> time by step (dual phases 1 and 2):\n");
CAp::Trace(StringFormat("Pricing = %12d ms\n",State.m_repdualpricingtime));
CAp::Trace(StringFormat("BTran = %12d ms\n",State.m_repdualbtrantime));
CAp::Trace(StringFormat("PivotRow = %12d ms\n",State.m_repdualpivotrowtime));
CAp::Trace(StringFormat(__FUNCTION__+" = %12d ms\n",State.m_repdualratiotesttime));
CAp::Trace(StringFormat(__FUNCTION__+" = %12d ms\n",State.m_repdualftrantime));
CAp::Trace(StringFormat("Update = %12d ms\n",State.m_repdualupdatesteptime));
}
}
//+------------------------------------------------------------------+
//| This function initializes subproblem structure. Previously |
//| allocated memory is reused as much as possible. |
//| Default State of the problem is zero cost vector, all variables |
//| are fixed at zero, linear constraint matrix is zero. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::SubproblemInit(int n,CDualSimplexSubproblem &s)
{
//--- check
if(!CAp::Assert(n>0,__FUNCTION__+": N<=0"))
return;
s.m_ns=n;
s.m_m=0;
s.m_state=m_ssinvalid;
s.m_xb.Resize(0);
s.m_xa=vector<double>::Zeros(n);
s.m_d=vector<double>::Zeros(n);
s.m_rawc=vector<double>::Zeros(n);
s.m_effc=vector<double>::Zeros(n);
s.m_bndl=vector<double>::Zeros(n);
s.m_bndu=vector<double>::Zeros(n);
s.m_bndt.Resize(n);
s.m_bndt.Fill(m_ccfixed);
}
//+------------------------------------------------------------------+
//| This function initializes phase #1 subproblem which minimizes sum|
//| of dual infeasibilities. It is required that total count of |
//| non-boxed non-fixed variables is at least M. |
//| It splits out basic components of XA[] to XB[] |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::SubproblemInitPhase1(CDualSimplexSubproblem &s0,
CDualSimplexBasis &basis,
CDualSimplexSubproblem &s1)
{
//--- copy
s1=s0;
for(int i=0; i<s1.m_ns+s1.m_m; i++)
{
if(s1.m_bndt[i]==m_cclower)
{
s1.m_bndt.Set(i,m_ccrange);
s1.m_bndl.Set(i,0);
s1.m_bndu.Set(i,1);
s1.m_xa.Set(i,0);
continue;
}
if(s1.m_bndt[i]==m_ccupper)
{
s1.m_bndt.Set(i,m_ccrange);
s1.m_bndl.Set(i,-1);
s1.m_bndu.Set(i,0);
s1.m_xa.Set(i,0);
continue;
}
if(s1.m_bndt[i]==m_ccfree)
{
s1.m_bndt.Set(i,m_ccrange);
s1.m_bndl.Set(i,-1);
s1.m_bndu.Set(i,1);
if(s1.m_effc[i]>=0.0)
s1.m_xa.Set(i,-1);
else
s1.m_xa.Set(i,1);
continue;
}
s1.m_bndt.Set(i,m_ccfixed);
s1.m_bndl.Set(i,0);
s1.m_bndu.Set(i,0);
s1.m_xa.Set(i,0);
}
s1.m_state=m_ssvalidxn;
}
//+------------------------------------------------------------------+
//| This function initializes phase #3 subproblem which applies |
//| primal simplex method to the result of the phase #2. |
//| It also performs modification of the subproblem in order to |
//| ensure that initial point is primal feasible. |
//| NOTE: this function expects that all components (basic and |
//| nonbasic ones) are stored in XA[] |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::SubproblemInitPhase3(CDualSimplexSubproblem &s0,
CDualSimplexSubproblem &s1)
{
s1=s0;
s1.m_state=m_ssvalidxn;
}
//+------------------------------------------------------------------+
//| This function infers nonbasic variables of X using sign of |
//| effective C[]. |
//| Only non - basic components of XN are changed; everything else |
//| is NOT updated. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::SubproblemInferInitialXN(CDualSimplexState &State,
CDualSimplexSubproblem &s)
{
//--- create variables
int i=0;
int ii=0;
int bndt=0;
for(ii=0; ii<s.m_ns; ii++)
{
i=State.m_basis.m_nidx[ii];
bndt=s.m_bndt[i];
if(bndt==m_ccfixed || bndt==m_ccrange)
{
if(s.m_effc[i]>=0)
s.m_xa.Set(i,s.m_bndl[i]);
else
s.m_xa.Set(i,s.m_bndu[i]);
continue;
}
if(bndt==m_cclower)
{
s.m_xa.Set(i,s.m_bndl[i]);
continue;
}
if(bndt==m_ccupper)
{
s.m_xa.Set(i,s.m_bndu[i]);
continue;
}
if(bndt==m_ccfree)
{
s.m_xa.Set(i,0.0);
continue;
}
CAp::Assert(false,__FUNCTION__+": integrity check failed (infeasible constraint)");
return;
}
s.m_state=m_ssvalidxn;
}
//+------------------------------------------------------------------+
//| This function infers basic variables of X using values of |
//| non-basic vars and updates reduced cost vector D and target |
//| function Z. Sets State age to zero. |
//| D[] is allocated during computations. |
//| Temporary vectors Tmp0 and Tmp1 are used(reallocated as needed). |
//| NOTE: this function expects that both nonbasic and basic |
//| components are stored in XA[]. XB[] array is not referenced|
//+------------------------------------------------------------------+
void CRevisedDualSimplex::SubproblemHandleXNUpdate(CDualSimplexState &State,
CDualSimplexSubproblem &s)
{
//--- create variables
int nn=s.m_ns;
int m=s.m_m;
int i=0;
int j=0;
//--- check
if(!CAp::Assert(s.m_state>=m_ssvalidxn,__FUNCTION__+": integrity check failed (XN is not valid)"))
return;
//--- Compute nonbasic components
ComputeAnXn(State,s,s.m_xa,State.m_tmp0);
BasisSolve(State.m_basis,State.m_tmp0,State.m_tmp1,State.m_tmp2);
for(i=0; i<m; i++)
s.m_xa.Set(State.m_basis.m_idx[i],-State.m_tmp1[i]);
//--- Compute D
for(i=0; i<m; i++)
State.m_tmp0.Set(i,s.m_effc[State.m_basis.m_idx[i]]);
BasisSolveT(State.m_basis,State.m_tmp0,State.m_tmp1,State.m_tmp2);
ComputeAnTV(State,State.m_tmp1,s.m_d);
for(i=0; i<nn; i++)
{
j=State.m_basis.m_nidx[i];
s.m_d.Set(j,s.m_effc[j]-s.m_d[j]);
}
//--- Update State validity/age
s.m_state=m_ssvalid;
}
//+------------------------------------------------------------------+
//| This function performs initial dual feasibility correction on the|
//| subproblem. It assumes that problem State is at least ssValidXN. |
//| After call to this function the problem State is set to ssValid. |
//| This function returns dual feasibility error after dual |
//| feasibility correction. |
//| NOTE: this function expects that both nonbasic and basic |
//| components are stored in XA[]. XB[] array is not referenced|
//+------------------------------------------------------------------+
double CRevisedDualSimplex::InitialDualFeasibilityCorrection(CDualSimplexState &State,
CDualSimplexSubproblem &s,
CDualSimplexSettings &Settings)
{
//--- create variables
int nn=s.m_ns;
int m=s.m_m;
double result=0;
CRowDouble dummy;
int ii=0;
int i=0;
int j=0;
bool flipped=false;
double v=0;
double dj=0;
double xj=0;
int bndt=0;
//--- check
if(!CAp::Assert(s.m_state>=m_ssvalidxn,__FUNCTION__+": XN is invalid"))
return(result);
//--- Prepare
State.m_dfctmp0.Resize(m);
State.m_dfctmp1.Resize(m);
//--- Recompute D[] using fresh factorization
BasisFreshTrf(State.m_basis,State.m_at,Settings);
for(i=0; i<m; i++)
State.m_dfctmp0.Set(i,s.m_effc[State.m_basis.m_idx[i]]);
BasisSolveT(State.m_basis,State.m_dfctmp0,State.m_dfctmp1,State.m_dfctmp2);
ComputeAnTV(State,State.m_dfctmp1,s.m_d);
for(i=0; i<nn; i++)
{
j=State.m_basis.m_nidx[i];
s.m_d.Set(j,s.m_effc[j]-s.m_d[j]);
}
//--- Perform flips for dual-infeasible boxed variables
result=0;
flipped=false;
for(ii=0; ii<nn; ii++)
{
j=State.m_basis.m_nidx[ii];
bndt=s.m_bndt[j];
//--- Boxed variables, perform DFC
if(bndt==m_ccrange)
{
dj=s.m_d[j];
xj=s.m_xa[j];
if(xj==s.m_bndl[j] && dj<0)
{
s.m_xa.Set(j,s.m_bndu[j]);
flipped=true;
continue;
}
if(xj==s.m_bndu[j] && dj>0)
{
s.m_xa.Set(j,s.m_bndl[j]);
flipped=true;
continue;
}
continue;
}
//--- Non-boxed variables, compute dual feasibility error
if(bndt==m_ccfixed)
continue;
if(bndt==m_cclower)
{
v=-s.m_d[j];
if(v>result)
result=v;
continue;
}
if(bndt==m_ccupper)
{
v=s.m_d[j];
if(v>result)
result=v;
continue;
}
if(bndt==m_ccfree)
{
result=MathMax(result,MathAbs(s.m_d[j]));
continue;
}
}
//--- Recompute basic components of X[]
if(flipped || s.m_state<m_ssvalid)
{
ComputeAnXn(State,s,s.m_xa,State.m_dfctmp0);
BasisSolve(State.m_basis,State.m_dfctmp0,State.m_dfctmp1,State.m_dfctmp2);
for(i=0; i<m; i++)
s.m_xa.Set(State.m_basis.m_idx[i],-State.m_dfctmp1[i]);
}
//--- Update State validity/age
s.m_state=m_ssvalid;
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function performs Shifting using current algorithm as |
//| specified by Settings.m_shifting. |
//| It accepts following parameters: |
//| * AlphaR - pivot row |
//| * Delta - delta from pricing step |
//| * Q - variable selected by ratio test |
//| * AlphaRPiv - pivot element, Q-th element of AlphaR (because |
//| alphaR is stored in compressed format, we can't |
//| extract it easily) |
//| * ThetaD - dual step length |
//| If no shifts are necessary, it silently returns. If shifts are |
//| necessary, it modifies ThetaD, S.D, S.EffC according to Shifting |
//| algorithm. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::Shifting(CDualSimplexState &State,
CDualSimplexSubproblem &s,
CDSSVector &alphar,
double delta,
int q,
double alpharpiv,
double &thetad,
CDualSimplexSettings &Settings)
{
//--- create variables
int dir=0;
double sft=0;
int ii=0;
int j=0;
int bndt=0;
if(q<0)
return;
switch(Settings.m_shifting)
{
//--- No shifts
case 0:
break;
//--- EXPAND with ThetaD=0
case 1:
dir=(int)MathSign(delta);
if((thetad*dir)>=0.0)
break;
s.m_effc.Add(q,- s.m_d[q]);
s.m_d.Set(q,0);
thetad=0;
break;
//--- EXPAND with ThetaD=ShiftLen
case 2:
dir=(int)MathSign(delta);
if((thetad*dir)>0.0)
break;
//--- Ensure that non-zero step is performed
thetad=dir*m_shiftlen;
//--- Shift Q-th coefficient
sft=thetad*(dir*alpharpiv)-s.m_d[q];
s.m_effc.Add(q,sft);
s.m_d.Add(q,sft);
//--- Shift other coefficients
for(ii=0; ii<alphar.m_k; ii++)
{
j=alphar.m_idx[ii];
bndt=s.m_bndt[j];
if((j==q || bndt==m_ccfixed) || bndt==m_ccfree)
continue;
sft=thetad*(dir*alphar.m_vals[ii])-s.m_d[j];
//--- Handle variables at lower bound
if(bndt==m_cclower || (bndt==m_ccrange && s.m_xa[j]==s.m_bndl[j]))
{
sft-=Settings.m_dtolabs;
if(sft>0)
{
s.m_effc.Add(j,sft);
s.m_d.Add(j,sft);
}
continue;
}
if(bndt==m_ccupper || (bndt==m_ccrange && s.m_xa[j]==s.m_bndu[j]))
{
sft+=Settings.m_dtolabs;
if(sft<0)
{
s.m_effc.Add(j,sft);
s.m_d.Add(j,sft);
}
continue;
}
}
break;
//--- Done
default:
CAp::Assert(false,__FUNCTION__+": unexpected Shifting type");
break;
}
}
//+------------------------------------------------------------------+
//| This function performs pricing step |
//| Additional parameters: |
//| * Phase1Pricing - if True, then special Phase #1 restriction|
//| is applied to leaving variables: only |
//| those are eligible which will move to zero|
//| bound after basis change. |
//| This trick allows to accelerate and stabilize phase #1. See |
//| Robert Fourer, 'Notes on the dual simplex method', draft report, |
//| 1994, for more Info. |
//| Returns: |
//| * leaving variable index P |
//| * its index R in the basis, in [0, M) range |
//| * Delta - difference between variable value and corresponding |
//| bound |
//| NOTE: this function expects that basic components are stored in |
//| XB[]; corresponding entries of XA[] are ignored. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::PricingStep(CDualSimplexState &State,
CDualSimplexSubproblem &s,
bool phase1pricing,
int &p,
int &r,
double &delta,
CDualSimplexSettings &Settings)
{
//--- create variables
int m=s.m_m;
int i=0;
int bi=0;
double v=0;
double vtarget=0;
double xbi=0;
double bndl=0;
double bndu=0;
double vdiff=0;
double vtest=0;
double invw=0;
int bndt=0;
bool hasboth=false;
bool hasl=false;
bool hasu=false;
int t0=0;
p=0;
r=0;
delta=0;
//--- Integrity checks
if(!CAp::Assert(s.m_state==m_ssvalid,__FUNCTION__+": invalid X"))
return;
if(!CAp::Assert(m>0,__FUNCTION__+": M<=0"))
return;
//--- Timers
t0=0;
if(State.m_dotimers)
t0=(int)(GetTickCount()/10000);
//--- Pricing
if(Settings.m_pricing==0)
{
//--- "Most infeasible" pricing
p=-1;
r=-1;
delta=0;
vtarget=0;
for(i=0; i<m; i++)
{
bndt=s.m_bndtb[i];
hasboth=bndt==3 || bndt==0;
hasl=hasboth || bndt==1;
hasu=hasboth || bndt==2;
xbi=s.m_xb[i];
if(hasl)
{
bndl=s.m_bndlb[i];
vdiff=xbi-bndl;
v=-vdiff;
if(v>s.m_bndtollb[i] && v>vtarget)
{
//--- Special phase 1 pricing: do not choose variables which move to non-zero bound
if(phase1pricing && !(bndl==0.0))
continue;
//--- Proceed as usual
p=State.m_basis.m_idx[i];
r=i;
delta=vdiff;
vtarget=v;
continue;
}
}
if(hasu)
{
bndu=s.m_bndub[i];
vdiff=xbi-bndu;
v=vdiff;
if(v>s.m_bndtolub[i] && v>vtarget)
{
//--- Special phase 1 pricing: do not choose variables which move to non-zero bound
if(phase1pricing && !(bndu==0.0))
continue;
//--- Proceed as usual
p=State.m_basis.m_idx[i];
r=i;
delta=vdiff;
vtarget=v;
continue;
}
}
}
//--- Trace/profile
if(State.m_dotrace)
{
CAp::Trace("> pricing: most infeasible variable removed\n");
CAp::Trace(StringFormat("P = %12d (R=%d)\n",p,r));
CAp::Trace(StringFormat("Delta = %12.3E\n",delta));
}
if(State.m_dotimers)
State.m_repdualpricingtime+=(int)(GetTickCount()/10000)-t0;
//--- Done
return;
}
if(Settings.m_pricing==-1 || Settings.m_pricing==1)
{
//--- Dual steepest edge pricing
BasisRequestWeights(State.m_basis,Settings);
p=-1;
r=-1;
delta=0;
vtarget=0;
for(i=0; i<m; i++)
{
bi=State.m_basis.m_idx[i];
bndt=s.m_bndtb[i];
hasboth=bndt==3 || bndt==0;
hasl=hasboth || bndt==1;
hasu=hasboth || bndt==2;
xbi=s.m_xb[i];
invw=1/State.m_basis.m_dseweights[i];
if(hasl)
{
bndl=s.m_bndlb[i];
vdiff=xbi-bndl;
vtest=vdiff*vdiff*invw;
if(vdiff<-s.m_bndtollb[i] && (p<0 || vtest>vtarget))
{
//--- Special phase 1 pricing: do not choose variables which move to non-zero bound
if(phase1pricing && !(bndl==0.0))
continue;
//--- Proceed as usual
p=bi;
r=i;
delta=vdiff;
vtarget=vtest;
continue;
}
}
if(hasu)
{
bndu=s.m_bndub[i];
vdiff=xbi-bndu;
vtest=vdiff*vdiff*invw;
if(vdiff>s.m_bndtolub[i] && (p<0 || vtest>vtarget))
{
//--- Special phase 1 pricing: do not choose variables which move to non-zero bound
if(phase1pricing && !(bndu==0.0))
continue;
//--- Proceed as usual
p=bi;
r=i;
delta=vdiff;
vtarget=vtest;
continue;
}
}
}
//--- Trace/profile
if(State.m_dotrace)
{
CAp::Trace("> dual steepest edge pricing: leaving variable found\n");
CAp::Trace(StringFormat("P = %12d (variable index)\n",p));
CAp::Trace(StringFormat("R = %12d (variable index in basis)\n",r));
CAp::Trace(StringFormat("Delta = %12.3E (primal infeasibility removed)\n",delta));
}
if(State.m_dotimers)
State.m_repdualpricingtime+=(int)(GetTickCount()/10000)-t0;
//--- Done
return;
}
CAp::Assert(false,__FUNCTION__+": unknown pricing type");
}
//+------------------------------------------------------------------+
//| This function performs BTran step |
//| Accepts: |
//| * R, index of the leaving variable in the basis, in [0, M) |
//| range |
//| Returns: |
//| * RhoR, array[M], BTran result |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::BTranStep(CDualSimplexState &State,
CDualSimplexSubproblem &s,
int r,
CDSSVector &rhor,
CDualSimplexSettings &Settings)
{
//--- create variables
int m=s.m_m;
int t0=0;
//--- Integrity checks
if(!CAp::Assert(m>0,__FUNCTION__+": M<=0"))
return;
//--- Timers
if(State.m_dotimers)
t0=(int)(GetTickCount()/10000);
//--- BTran
State.m_btrantmp0=vector<double>::Zeros(m);
State.m_btrantmp1.Resize(m);
State.m_btrantmp2.Resize(m);
State.m_btrantmp0.Set(r,1);
DVAlloc(rhor,m);
BasisSolveT(State.m_basis,State.m_btrantmp0,rhor.m_dense,State.m_btrantmp1);
DVDenseToSparse(rhor);
//--- Timers
if(State.m_dotimers)
State.m_repdualbtrantime+=((int)(GetTickCount()/10000)-t0);
}
//+------------------------------------------------------------------+
//| This function performs PivotRow step |
//| Accepts: |
//| * RhoR, BTRan result |
//| Returns: |
//| * AlphaR, array[N + M], pivot row |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::PivotRowStep(CDualSimplexState &State,
CDualSimplexSubproblem &s,
CDSSVector &rhor,
CDSSVector &alphar,
CDualSimplexSettings &Settings)
{
//--- create variables
int m=s.m_m;
int ns=s.m_ns;
int nx=s.m_ns+s.m_m;
int i=0;
int j=0;
int k=0;
int jj=0;
int j0=0;
int j1=0;
int alphark=0;
double v=0;
int t0=0;
double avgcolwise=0;
double avgrowwise=0;
//--- Integrity checks
if(!CAp::Assert(m>0,__FUNCTION__+": M<=0"))
return;
//--- Timers
if(State.m_dotimers)
t0=(int)(GetTickCount()/10000);
//--- Determine operation counts for columnwise and rowwise approaches
avgrowwise=rhor.m_k*((double)State.m_at.m_RIdx[nx]/(double)m);
avgcolwise=ns*((double)State.m_at.m_RIdx[nx]/(double)nx);
//--- Pivot row
if(avgrowwise<avgcolwise)
{
//--- Use rowwise algorithm
DVInit(alphar,nx);
for(i=0; i<rhor.m_k; i++)
{
k=rhor.m_idx[i];
v=rhor.m_vals[i];
j0=State.m_a.m_RIdx[k];
j1=State.m_a.m_RIdx[k+1];
for(j=j0; j<j1; j++)
{
jj=State.m_a.m_Idx[j];
alphar.m_dense.Add(jj,v*State.m_a.m_Vals[j]);
}
}
alphark=0;
for(i=0; i<nx; i++)
{
if(!State.m_basis.m_isbasic[i])
{
//--- Fetch nonbasic nonzeros to sparse part
v=alphar.m_dense[i];
if(v!=0.0)
{
alphar.m_idx.Set(alphark,i);
alphar.m_vals.Set(alphark,v);
alphark++;
}
}
else
{
//--- Enforce condition that basic elements of AlphaR are exactly zero
alphar.m_dense.Set(i,0);
}
}
alphar.m_k=alphark;
}
else
{
//--- Use colwise algorithm
DVAlloc(alphar,nx);
alphark=0;
for(i=0; i<ns; i++)
{
k=State.m_basis.m_nidx[i];
j0=State.m_at.m_RIdx[k];
j1=State.m_at.m_RIdx[k+1];
v=0;
for(j=j0; j<j1; j++)
{
v+=State.m_at.m_Vals[j]*rhor.m_dense[State.m_at.m_Idx[j]];
}
if(v!=0.0)
{
alphar.m_idx.Set(alphark,k);
alphar.m_vals.Set(alphark,v);
alphark++;
}
}
alphar.m_k=alphark;
DVSparseToDense(alphar);
}
//--- Timers and tracing
if(State.m_dodetailedtrace)
{
UpdateAvgCounter(rhor.m_k/CApServ::Coalesce(rhor.m_n,1),State.m_repfillrhor,State.m_repfillrhorcnt);
UpdateAvgCounter(alphar.m_k/CApServ::Coalesce(alphar.m_n,1),State.m_repfillpivotrow,State.m_repfillpivotrowcnt);
}
if(State.m_dotimers)
State.m_repdualpivotrowtime+=((int)(GetTickCount()/10000)-t0);
}
//+------------------------------------------------------------------+
//| This function performs FTran step |
//| Accepts: |
//| * RhoR, array[M] |
//| * Q, index of the entering variable, in [0, NX) range |
//| Returns: |
//| * AlphaQ, array[M], FTran result |
//| * AlphaQim, array[M], intermediate FTran result used by Forest-|
//| Tomlin update |
//| * Tau, array[M], used to compute DSE temporaries |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::FTranStep(CDualSimplexState &State,
CDualSimplexSubproblem &s,
CDSSVector &rhor,
int q,
CRowDouble &alphaq,
CRowDouble &alphaqim,
CRowDouble &tau,
CDualSimplexSettings &Settings)
{
//--- create variables
int m=s.m_m;
int j=0;
int j0=0;
int j1=0;
int t0=0;
//--- Integrity checks
if(!CAp::Assert(m>0,__FUNCTION__+": M<=0"))
return;
//--- Timers
if(State.m_dotimers)
t0=(int)(GetTickCount()/10000);
//--- FTran
State.m_ftrantmp0=vector<double>::Zeros(m);
j0=State.m_at.m_RIdx[q];
j1=State.m_at.m_RIdx[q+1];
for(j=j0; j<j1; j++)
State.m_ftrantmp0.Set(State.m_at.m_Idx[j],State.m_at.m_Vals[j]);
BasisSolveX(State.m_basis,State.m_ftrantmp0,alphaq,alphaqim,true,State.m_ftrantmp1);
//--- check
if(!CAp::Assert(Settings.m_pricing==-1 || Settings.m_pricing==0 || Settings.m_pricing==1,__FUNCTION__+": unexpected Settings.Pricing"))
return;
if(Settings.m_pricing==1)
BasisSolve(State.m_basis,rhor.m_dense,tau,State.m_ftrantmp1);
//--- Timers
if(State.m_dotimers)
State.m_repdualftrantime+=((int)(GetTickCount()/10000)-t0);
}
//+------------------------------------------------------------------+
//| This function performs ratio test, either simple one or BFRT. |
//| It accepts following parameters: |
//| * AlphaR - pivot row |
//| * Delta - delta from pricing step |
//| * P - index of leaving variable from pricing step |
//| It returns following results: |
//| * Q - non-negative value for success, negative for |
//| primal infeasible problem |
//| * AlphaRPiv - AlphaR[Q] (due to AlphaR being stored in sparse |
//| format this value is difficult to extract by |
//| index Q). |
//| * ThetaD - dual step length |
//| * PossibleFlips[PossibleFlipsCnt] - for possible flip indexes |
//| (for BFRT this set coincides with actual flips, |
//| but stabilizing BFRT is a bit more complex - |
//| some variables in PossibleFlips[] may need |
//| flipping and some not) |
//| Internally it uses following fields of State for temporaries: |
//| * EligibleAlphaR |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::RatioTest(CDualSimplexState &State,
CDualSimplexSubproblem &s,
CDSSVector &alphar,
double delta,
int p,
int &q,
double &alpharpiv,
double &thetad,
CRowInt &possibleflips,
int &possibleflipscnt,
CDualSimplexSettings &Settings)
{
//--- create variables
int nx=s.m_ns+s.m_m;
int j=0;
int nj=0;
int dir=0;
double vx=0;
double vp=0;
int ej=0;
double alpharej=0;
double vtarget=0;
double vtest=0;
int eligiblecnt=0;
int originaleligiblecnt=0;
int bndt=0;
double alphawaver=0;
double adelta=0;
int idx=0;
double vtheta=0;
int t0=0;
q=0;
alpharpiv=0;
thetad=0;
//--- check
if(!CAp::Assert(delta!=0.0,__FUNCTION__+": zero delta"))
return;
if(!CAp::Assert(s.m_state==m_ssvalid,__FUNCTION__+": invalid X"))
return;
//--- Timers
if(State.m_dotimers)
t0=(int)(GetTickCount()/10000);
//--- Clear output
q=-1;
alpharpiv=0;
thetad=0;
possibleflipscnt=0;
//--- Prepare temporaries
//--- Scaled tolerances are used to test AlphaWaveR for positivity/negativity,
//--- scale of I-th tolerance is calculated as ratio of ColScale[I] and ColScale[P].
dir=(int)MathSign(delta);
possibleflips.Resize(nx);
//--- Prepare set of eligible variables
//--- NOTE: free variables are immediately chosen at this stage
State.m_eligiblealphar.Resize(alphar.m_k);
eligiblecnt=0;
for(j=0; j<alphar.m_k; j++)
{
nj=alphar.m_idx[j];
bndt=s.m_bndt[nj];
//--- Handle fixed and free variables: fixed ones are not eligible,
//--- free non-basic variables are always and immediately eligible
if(bndt==m_ccfixed)
continue;
if(bndt==m_ccfree)
{
q=nj;
thetad=0;
alpharpiv=alphar.m_vals[j];
if(State.m_dotrace)
{
CAp::Trace("> ratio test: quick exit,found free nonbasic variable\n");
CAp::Trace(StringFormat("Q = %12d (variable selected)\n",q));
CAp::Trace(StringFormat("ThetaD = %12.3E (dual step length)\n",thetad));
}
if(State.m_dotimers)
State.m_repdualratiotesttime+=((int)(GetTickCount()/10000)-t0);
return;
}
//--- Handle lower/upper/range constraints
vx=s.m_xa[nj];
vp=Settings.m_pivottol;
alphawaver=dir*alphar.m_vals[j];
if(bndt==m_cclower || (bndt==m_ccrange && vx==s.m_bndl[nj]))
{
if(alphawaver>vp)
{
State.m_eligiblealphar.Set(eligiblecnt,j);
eligiblecnt++;
continue;
}
}
if(bndt==m_ccupper || (bndt==m_ccrange && vx==s.m_bndu[nj]))
{
if(alphawaver<-vp)
{
State.m_eligiblealphar.Set(eligiblecnt,j);
eligiblecnt++;
continue;
}
}
}
originaleligiblecnt=eligiblecnt;
//--- Simple ratio test.
if(Settings.m_ratiotest==0)
{
//--- Ratio test
vtarget=0;
for(j=0; j<eligiblecnt; j++)
{
ej=State.m_eligiblealphar[j];
nj=alphar.m_idx[ej];
alpharej=alphar.m_vals[ej];
//--- More general case
alphawaver=dir*alpharej;
vtest=s.m_d[nj]/alphawaver;
if(q<0 || vtest<vtarget)
{
q=nj;
alpharpiv=alpharej;
vtarget=vtest;
thetad=s.m_d[nj]/alpharej;
}
}
Shifting(State,s,alphar,delta,q,alpharpiv,thetad,Settings);
//--- Trace
if(State.m_dotrace)
{
CAp::Trace("> dual ratio test:\n");
CAp::Trace(StringFormat("|E| = %12d (eligible set size)\n",originaleligiblecnt));
CAp::Trace(StringFormat("Q = %12d (variable selected)\n",q));
CAp::Trace(StringFormat("ThetaD = %12.3E (dual step length)\n",thetad));
}
if(State.m_dotimers)
State.m_repdualratiotesttime+=((int)(GetTickCount()/10000)-t0);
//--- Done
return;
}
//--- Bounds flipping ratio test
if(Settings.m_ratiotest==1)
{
adelta=MathAbs(delta);
//--- Quick exit
if(eligiblecnt==0)
{
if(State.m_dotrace)
CAp::Trace("> ratio test: quick exit,no eligible variables\n");
return;
}
//--- BFRT
while(eligiblecnt>0)
{
//--- Find Q satisfying BFRT criteria
idx=-1;
q=-1;
alpharpiv=0;
vtarget=0;
for(j=0; j<eligiblecnt; j++)
{
ej=State.m_eligiblealphar[j];
nj=alphar.m_idx[ej];
alpharej=alphar.m_vals[ej];
vtheta=s.m_d[nj]/alpharej;
vtest=dir*vtheta;
if(q<0 || vtest<vtarget)
{
q=nj;
alpharpiv=alpharej;
vtarget=vtest;
thetad=vtheta;
idx=j;
}
}
//--- check
if(!CAp::Assert(q>=0,__FUNCTION__+": integrity check failed (BFRT)"))
return;
//--- BFRT mini-iterations will be terminated upon discovery
//--- of non-boxed variable or upon exhausting of eligible set.
if(s.m_bndt[q]!=m_ccrange)
break;
if(eligiblecnt==1)
break;
//--- Update and test ADelta. Break BFRT mini-iterations once
//--- we get negative slope.
adelta-=(s.m_bndu[q]-s.m_bndl[q])*MathAbs(alpharpiv);
if(adelta<=0.0)
break;
//--- Update eligible set, record flip
possibleflips.Set(possibleflipscnt,State.m_eligiblealphar[idx]);
possibleflipscnt++;
State.m_eligiblealphar.Set(idx,State.m_eligiblealphar[eligiblecnt-1]);
eligiblecnt--;
}
//--- check
if(!CAp::Assert(q>=0,__FUNCTION__+": unexpected failure"))
return;
thetad=s.m_d[q]/alpharpiv;
Shifting(State,s,alphar,delta,q,alpharpiv,thetad,Settings);
//--- Trace
if(State.m_dotrace)
{
CAp::Trace("> dual bounds flipping ratio test:\n");
CAp::Trace(StringFormat("|E| = %12d (eligible set size)\n",originaleligiblecnt));
CAp::Trace(StringFormat("Q = %12d (variable selected)\n",q));
CAp::Trace(StringFormat("ThetaD = %12.3E (dual step length)\n",thetad));
CAp::Trace(StringFormat("Flips = %12d (possible bound flips)\n",State.m_possibleflipscnt));
}
if(State.m_dotimers)
State.m_repdualratiotesttime+=((int)(GetTickCount()/10000)-t0);
//--- Done
return;
}
//--- Unknown test type
CAp::Assert(false,__FUNCTION__+": integrity check failed,unknown test type");
}
//+------------------------------------------------------------------+
//| This function performs update of XB, XN, D and Z during final |
//| step of revised dual simplex method. |
//| It also updates basis cache of the subproblem (s.bcache field). |
//| Depending on Settings.m_ratiotest, following operations are |
//| performed: |
//| * Settings.m_ratiotest = 0 -> simple update is performed |
//| * Settings.m_ratiotest = 1 -> bounds flipping ratio test update|
//| is performed |
//| * Settings.m_ratiotest = 2 -> stabilizing bounds flipping ratio|
//| test update is performed |
//| It accepts following parameters: |
//| * P - index of leaving variable from pricing step |
//| * Q - index of entering variable. |
//| * R - index of leaving variable in AlphaQ |
//| * Delta - delta from pricing step |
//| * AlphaPiv - pivot element (in absence of numerical rounding it|
//| is AlphaR[Q] = AlphaQ[R]) |
//| * ThetaP - primal step length |
//| * ThetaD - dual step length |
//| * AlphaQ - pivot column |
//| * AlphaQim - intermediate result from Ftran for AlphaQ, used |
//| for Forest-Tomlin update, not referenced when other|
//| update scheme is set |
//| * AlphaR - pivot row |
//| * Tau - tau-vector for DSE pricing (ignored if simple |
//| pricing is used) |
//| * PossibleAlphaRFlips, PossibleAlphaRFlipsCnt - outputs of the |
//| RatioTest() information about possible variable |
//| flips - indexes of AlphaR positions which are |
//| considered for flipping due to BFRT (however, we |
//| have to check residual costs before actually |
//| flipping variables - it is possible that some |
//| variables in this set actually do not need flipping|
//| It performs following operations: |
//| * basis update |
//| * update of XB / BndTB / BndLB / BndUB[] and XA[] (basic and |
//| nonbasic components), D |
//| * update of pricing weights |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::UpdateStep(CDualSimplexState &State,
CDualSimplexSubproblem &s,
int p,
int q,
int r,
double delta,
double alphapiv,
double thetap,
double thetad,
CRowDouble &alphaq,
CRowDouble &alphaqim,
CDSSVector &alphar,
CRowDouble &tau,
CRowInt &possiblealpharflips,
int possiblealpharflipscnt,
CDualSimplexSettings &Settings)
{
//--- create variables
int nx=s.m_ns+s.m_m;
int m=s.m_m;
int ii=0;
int j=0;
int k=0;
int aj=0;
int k0=0;
int k1=0;
double bndl=0;
double bndu=0;
bool flipped=false;
double flip=0;
double dj=0;
int dir=0;
int idx=0;
int actualflipscnt=0;
int t0=0;
int alpharlen=0;
//--- Integrity checks
if(!CAp::Assert(Settings.m_ratiotest==0 || Settings.m_ratiotest==1 || Settings.m_ratiotest==2,__FUNCTION__+": invalid X"))
return;
if(!CAp::Assert(s.m_state==m_ssvalid,__FUNCTION__+": invalid X"))
return;
if(!CAp::Assert(p>=0 && q>=0,__FUNCTION__+": invalid P/Q"))
return;
if(!CAp::Assert(delta!=0.0,__FUNCTION__+": Delta=0"))
return;
if(!CAp::Assert(alphapiv!=0.0,__FUNCTION__+": AlphaPiv=0"))
return;
//--- Timers
if(State.m_dotimers)
t0=(int)(GetTickCount()/10000);
//--- Prepare
dir=(int)MathSign(delta);
alpharlen=alphar.m_k;
flip=0;
State.m_tmp0=vector<double>::Zeros(m);
State.m_ustmpi.Resize(nx);
actualflipscnt=0;
//--- Evaluate and update non-basic elements of D
for(ii=0; ii<alpharlen; ii++)
{
j=alphar.m_idx[ii];
s.m_d.Add(j,- thetad*alphar.m_vals[ii]);
}
for(ii=0; ii<possiblealpharflipscnt; ii++)
{
aj=possiblealpharflips[ii];
j=alphar.m_idx[aj];
dj=s.m_d[j];
bndl=s.m_bndl[j];
bndu=s.m_bndu[j];
flipped=false;
if(s.m_xa[j]==bndl && dj<0)
{
flip=bndu-bndl;
flipped=true;
}
else
{
if(s.m_xa[j]==bndu && dj>0)
{
flip=bndl-bndu;
flipped=true;
}
}
if(flipped)
{
delta-=dir*(bndu-bndl)*MathAbs(alphar.m_vals[aj]);
State.m_ustmpi.Set(actualflipscnt,j);
actualflipscnt++;
k0=State.m_at.m_RIdx[j];
k1=State.m_at.m_RIdx[j+1];
for(k=k0; k<k1; k++)
{
idx=State.m_at.m_Idx[k];
State.m_tmp0.Add(idx,flip*State.m_at.m_Vals[k]);
}
}
}
s.m_d.Set(p,-thetad);
s.m_d.Set(q,0.0);
//--- Apply BFRT update (aka long dual step) or simple ratio update
if(actualflipscnt>0)
{
thetap=delta/alphapiv;
k0=State.m_at.m_RIdx[q];
k1=State.m_at.m_RIdx[q+1]-1;
for(k=k0; k<=k1; k++)
{
idx=State.m_at.m_Idx[k];
State.m_tmp0.Add(idx,thetap*State.m_at.m_Vals[k]);
}
BasisSolve(State.m_basis,State.m_tmp0,State.m_tmp1,State.m_tmp2);
for(j=0; j<m; j++)
s.m_xb.Add(j,-State.m_tmp1[j]);
for(ii=0; ii<=actualflipscnt-1; ii++)
{
j=State.m_ustmpi[ii];
if(s.m_xa[j]==s.m_bndl[j])
s.m_xa.Set(j,s.m_bndu[j]);
else
s.m_xa.Set(j,s.m_bndl[j]);
}
s.m_xb.Set(r,s.m_xa[q]+thetap);
if(dir<0)
s.m_xa.Set(p,s.m_bndl[p]);
else
s.m_xa.Set(p,s.m_bndu[p]);
}
else
{
for(j=0; j<m; j++)
s.m_xb.Add(j,-thetap*alphaq[j]);
s.m_xb.Set(r,s.m_xa[q]+thetap);
if(dir<0)
s.m_xa.Set(p,s.m_bndl[p]);
else
s.m_xa.Set(p,s.m_bndu[p]);
}
//--- Update basis
BasisUpdateTrf(State.m_basis,State.m_at,p,q,alphaq,alphaqim,r,tau,Settings);
//--- Update cached variables
CacheBoundInfo(s,r,q,Settings);
//--- Tracing and timers
if(State.m_dodetailedtrace)
{
if(State.m_basis.m_trftype==3)
UpdateAvgCounter(SparsitYOf(State.m_basis.m_densemu,State.m_basis.m_trfage*m),State.m_repfilldensemu,State.m_repfilldensemucnt);
}
if(State.m_dotimers)
{
State.m_repdualupdatesteptime+=((int)(GetTickCount()/10000)-t0);
}
}
//+------------------------------------------------------------------+
//| This function performs several checks for accumulation of errors |
//| during factorization update. It returns True if refactorization |
//| is advised. |
//+------------------------------------------------------------------+
bool CRevisedDualSimplex::RefactorizationRequired(CDualSimplexState &State,
CDualSimplexSubproblem &s,
int q,
double alpharpiv,
int r,
double alphaqpiv)
{
//--- create variables
bool result=false;
int m=s.m_m;
double mx=0;
double v=0;
//--- Quick exit
if(State.m_basis.m_trfage<=m_safetrfage)
return(result);
//--- Compare Q-th entry of the pivot row AlphaR with R-th entry of the AlphaQ;
//--- ideally, both should match exactly. The difference is a rough estimate
//--- of the magnitude of the numerical errors.
mx=(State.m_alphaq.Abs()+0).Max();
result=result || MathAbs(alphaqpiv-alpharpiv)>(m_alphatrigger*(1.0+mx));
result=result || MathAbs(alphaqpiv-alpharpiv)>(m_alphatrigger2*MathAbs(alpharpiv));
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function caches information for I-th column of the basis, |
//| which is assumed to store variable K: |
//| * lower bound in S.BndLB[I] = S.BndL[K] |
//| * upper bound in S.BndUB[I] = S.BndU[K] |
//| * bound type in S.BndTB[I] = S.BndT[K] |
//| * lower bound primal error tolerance in S.BndTolLB[I] |
//| (nonnegative) |
//| * upper bound primal error tolerance in S.BndTolLB[I] |
//| (nonnegative). |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::CacheBoundInfo(CDualSimplexSubproblem &s,
int i,
int k,
CDualSimplexSettings &Settings)
{
s.m_bndlb.Set(i,s.m_bndl[k]);
s.m_bndub.Set(i,s.m_bndu[k]);
s.m_bndtb.Set(i,s.m_bndt[k]);
s.m_bndtollb.Set(i,Settings.m_xtolabs+Settings.m_xtolrelabs*Settings.m_xtolabs*MathAbs(s.m_bndlb[i]));
s.m_bndtolub.Set(i,Settings.m_xtolabs+Settings.m_xtolrelabs*Settings.m_xtolabs*MathAbs(s.m_bndub[i]));
}
//+------------------------------------------------------------------+
//| This function performs actual solution of dual simplex subproblem|
//| (either primary one or phase 1 one). |
//| A problem with following properties is expected: |
//| * M > 0 |
//| * feasible box constraints |
//| * dual feasible initial basis |
//| * actual initial point XC and target value Z |
//| * actual reduced cost vector D |
//| * pricing weights being set to 1.0 or copied from previous |
//| problem |
//| Returns: |
//| * Info = +1 for success, -3 for infeasible |
//| * IterationsCount is increased by amount of iterations |
//| performed |
//| NOTE: this function internally uses separate storage of basic and|
//| nonbasic components; however, all inputs and outputs use |
//| single array S.XA[] to store both basic and nonbasic |
//| variables. It transparently splits variables on input and |
//| recombines them on output. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::SolveSubproblemDual(CDualSimplexState &State,
CDualSimplexSubproblem &s,
bool IsPhase1,
CDualSimplexSettings &Settings,
int &Info)
{
//--- create variables
int nx=s.m_ns+s.m_m;
int m=s.m_m;
int i=0;
int p=0;
int r=0;
int q=0;
double alpharpiv=0;
double alphaqpiv=0;
double thetad=0;
double thetap=0;
double delta=0;
int forcedrestarts=0;
Info=0;
//--- Integrity checks
if(!CAp::Assert(s.m_state==m_ssvalid,__FUNCTION__+": X is not valid"))
return;
if(!CAp::Assert(m>0,__FUNCTION__+": M<=0"))
return;
for(i=0; i<nx; i++)
if(!CAp::Assert(s.m_bndt[i]!=m_ccinfeasible,__FUNCTION__+": infeasible box constraints"))
return;
if(!CAp::Assert(IsDualFeasible(State,s,Settings),__FUNCTION__+": dual infeasible initial basis"))
return;
//--- Actual processing
OffloadBasicComponents(s,State.m_basis,Settings);
Info=0;
State.m_tmp0.Resize(m);
while(true)
{
//--- Iteration report
if(State.m_dotrace)
{
i=State.m_repiterationscount2;
if(IsPhase1)
i=State.m_repiterationscount1;
CAp::Trace(StringFormat("=== ITERATION %5d STARTED ========================================================================\n",i));
}
//--- Pricing
PricingStep(State,s,IsPhase1,p,r,delta,Settings);
if(delta==0.0)
{
//--- Solved! Feasible and bounded!
if(State.m_dotrace)
CAp::Trace("> pricing: feasible point found\n");
RecombineBasicNonBasicX(s,State.m_basis);
Info=1;
return;
}
//--- BTran
BTranStep(State,s,r,State.m_rhor,Settings);
//--- Pivot row
PivotRowStep(State,s,State.m_rhor,State.m_alphar,Settings);
//--- Ratio test
RatioTest(State,s,State.m_alphar,delta,p,q,alpharpiv,thetad,State.m_possibleflips,State.m_possibleflipscnt,Settings);
if(q<0)
{
//--- Do we have fresh factorization and State? If not,
//--- refresh them prior to declaring that we have no solution.
if(State.m_basis.m_trfage>0 && forcedrestarts<m_maxforcedrestarts)
{
if(State.m_dotrace)
CAp::Trace(StringFormat("> ratio test: failed,basis is old (age=%d),forcing restart (%d of %d)\n",State.m_basis.m_trfage,forcedrestarts,m_maxforcedrestarts - 1));
BasisFreshTrf(State.m_basis,State.m_at,Settings);
SubproblemHandleXNUpdate(State,s);
OffloadBasicComponents(s,State.m_basis,Settings);
forcedrestarts++;
continue;
}
//--- Dual unbounded, primal infeasible
if(State.m_dotrace)
CAp::Trace("> ratio test: failed,results are accepted\n");
RecombineBasicNonBasicX(s,State.m_basis);
Info=-3;
return;
}
thetap=delta/alpharpiv;
//--- FTran, including additional FTran for DSE weights (if needed)
//--- NOTE: AlphaQim is filled by intermediate FTran result which is useful
//--- for Forest-Tomlin update scheme. If not Forest-Tomlin update is
//--- used, then it is not set.
FTranStep(State,s,State.m_rhor,q,State.m_alphaq,State.m_alphaqim,State.m_tau,Settings);
alphaqpiv=State.m_alphaq[r];
//--- Check numerical accuracy, trigger refactorization if needed
if(RefactorizationRequired(State,s,q,alpharpiv,r,alphaqpiv))
{
if(State.m_dotrace)
CAp::Trace("> refactorization test: numerical errors are too large,forcing refactorization and restart\n");
BasisFreshTrf(State.m_basis,State.m_at,Settings);
SubproblemHandleXNUpdate(State,s);
OffloadBasicComponents(s,State.m_basis,Settings);
continue;
}
//--- Basis change and update
UpdateStep(State,s,p,q,r,delta,alpharpiv,thetap,thetad,State.m_alphaq,State.m_alphaqim,State.m_alphar,State.m_tau,State.m_possibleflips,State.m_possibleflipscnt,Settings);
State.m_repiterationscount++;
if(IsPhase1)
State.m_repiterationscount1++;
else
State.m_repiterationscount2++;
}
}
//+------------------------------------------------------------------+
//| This function solves simplex subproblem using primal simplex |
//| method. |
//| A problem with following properties is expected: |
//| * M > 0 |
//| * feasible box constraints |
//| * primal feasible initial basis |
//| * actual initial point XC and target value Z |
//| * actual reduced cost vector D |
//| * pricing weights being set to 1.0 or copied from previous |
//| problem |
//| Returns: |
//| * Info = +1 for success, -3 for infeasible |
//| * IterationsCount is increased by amount of iterations |
//| performed |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::SolveSubproblemPrimal(CDualSimplexState &State,
CDualSimplexSubproblem &s,
CDualSimplexSettings &Settings,
int &Info)
{
//--- create variables
int nn=s.m_ns;
int nx=s.m_ns+s.m_m;
int m=s.m_m;
int i=0;
int j=0;
double v=0;
double vmax=0;
int bi=0;
double dj=0;
int bndt=0;
int q=0;
int p=0;
int r=0;
int dir=0;
double lim=0;
bool haslim=false;
double thetap=0;
double xbnd=0;
double flip=0;
int canddir=0;
double candlim=0;
double candflip=0;
int j0=0;
int j1=0;
double alphawave=0;
double vp=0;
double vb=0;
double vx=0;
double vtest=0;
double vv=0;
Info=0;
//--- Integrity checks
if(!CAp::Assert(s.m_state==m_ssvalid,__FUNCTION__+": X is not valid"))
return;
if(!CAp::Assert(m>0,__FUNCTION__+": M<=0"))
return;
for(i=0; i<nx; i++)
if(!CAp::Assert(s.m_bndt[i]!=m_ccinfeasible,__FUNCTION__+": infeasible box constraints"))
return;
//--- Actual processing
Info=1;
State.m_tmp0.Resize(m);
while(true)
{
//--- Iteration report
if(State.m_dotrace)
{
i=State.m_repiterationscount3;
CAp::Trace(StringFormat("=== ITERATION %5d STARTED ========================================================================\n",i));
}
//--- Primal simplex pricing step: we implement the very basic version
//--- of the pricing step because it is expected that primal simplex method
//--- is used just to apply quick correction after removal of the perturbation.
q=-1;
vmax=0;
dir=0;
lim=CMath::m_maxrealnumber;
haslim=false;
flip=0;
canddir=0;
for(i=0; i<nn; i++)
{
j=State.m_basis.m_nidx[i];
dj=s.m_d[j];
bndt=s.m_bndt[j];
if(bndt==m_ccfixed)
continue;
if(bndt==m_ccrange)
{
v=0;
candlim=s.m_bndu[j]-s.m_bndl[j];
candflip=0;
if(s.m_xa[j]==s.m_bndl[j])
{
v=-dj;
canddir=1;
candflip=s.m_bndu[j];
}
if(s.m_xa[j]==s.m_bndu[j])
{
v=dj;
canddir=-1;
candflip=s.m_bndl[j];
}
if(v>vmax)
{
vmax=v;
dir=canddir;
lim=candlim;
haslim=true;
flip=candflip;
q=j;
}
continue;
}
v=0;
canddir=0;
if(bndt==m_cclower)
{
v=-dj;
canddir=1;
}
if(bndt==m_ccupper)
{
v=dj;
canddir=-1;
}
if(bndt==m_ccfree)
{
v=MathAbs(dj);
canddir=-(int)MathSign(dj);
}
if(v>vmax)
{
vmax=v;
dir=canddir;
lim=CMath::m_maxrealnumber;
haslim=false;
q=j;
}
continue;
}
if(vmax<=Settings.m_dtolabs)
{
//--- Solved: primal and dual feasible!
if(State.m_dotrace)
CAp::Trace("> primal pricing: feasible point found\n");
return;
}
//--- check
if(!CAp::Assert(q>=0,__FUNCTION__+": integrity check failed"))
return;
if(State.m_dotrace)
{
CAp::Trace("> primal pricing: found entering variable\n");
CAp::Trace(StringFormat("Q = %12d (variable selected)\n",q));
CAp::Trace(StringFormat("|D| = %12.3E (dual infeasibility)\n",vmax));
}
//--- FTran and textbook ratio test (again, we expect primal phase to terminate quickly)
//--- NOTE: AlphaQim is filled by intermediate FTran result which is useful
//--- for Forest-Tomlin update scheme. If not Forest-Tomlin update is
//--- used, then it is not set.
State.m_tmp0.Fill(0);
j0=State.m_at.m_RIdx[q];
j1=State.m_at.m_RIdx[q+1];
for(j=j0; j<j1; j++)
State.m_tmp0.Set(State.m_at.m_Idx[j],State.m_at.m_Vals[j]);
BasisSolveX(State.m_basis,State.m_tmp0,State.m_alphaq,State.m_alphaqim,true,State.m_tmp2);
vp=Settings.m_pivottol;
p=-1;
r=-1;
thetap=0;
xbnd=0;
for(i=0; i<m; i++)
{
bi=State.m_basis.m_idx[i];
alphawave=-(dir*State.m_alphaq[i]);
vx=s.m_xa[bi];
if(alphawave<-vp && HasBndL(s,bi))
{
vb=s.m_bndl[bi];
if(vx<=vb)
{
//--- X[Bi] is already out of bounds due to rounding errors, perform Shifting
vb=vx-m_shiftlen;
s.m_bndl.Set(bi,vx);
}
vtest=(vb-vx)/alphawave;
if(p<0 || vtest<thetap)
{
p=bi;
r=i;
thetap=vtest;
xbnd=vb;
}
}
if(alphawave>vp && HasBndU(s,bi))
{
vb=s.m_bndu[bi];
if(vx>=vb)
{
//--- X[Bi] is already out of bounds due to rounding errors, perform Shifting
vb=vx+m_shiftlen;
s.m_bndu.Set(bi,vb);
}
vtest=(vb-vx)/alphawave;
if(p<0 || vtest<thetap)
{
p=bi;
r=i;
thetap=vtest;
xbnd=vb;
}
}
}
if(p<0 && !haslim)
{
//--- Primal unbounded
Info=-4;
if(State.m_dotrace)
CAp::Trace("> primal ratio test: dual infeasible,primal unbounded\n");
return;
}
if(State.m_dotrace)
{
CAp::Trace("> primal ratio test: found leaving variable\n");
CAp::Trace(StringFormat("P = %12d (variable index)\n",p));
CAp::Trace(StringFormat("R = %12d (variable index in basis)\n",r));
CAp::Trace(StringFormat("ThetaP = %12.3E (primal step length)\n",thetap));
}
//--- Update step
if(p>=0 && (!haslim || thetap<lim))
{
//--- One of the basic variables hit the boundary and become non-basic.
//--- Perform update:
//--- * update basic elements of X[] (X[p] is explicitly set to the
//--- boundary value) and X[q]
//--- * update target value Z
//--- * update factorization
//--- * update D[]
State.m_tmp0.Resize(m);
for(i=0; i<m; i++)
{
bi=State.m_basis.m_idx[i];
vv=thetap*(dir*State.m_alphaq[i]);
s.m_xa.Add(bi,-vv);
}
s.m_xa.Set(p,xbnd);
s.m_xa.Add(q,dir*thetap);
State.m_tmp0.Fill(0);
BasisUpdateTrf(State.m_basis,State.m_at,p,q,State.m_alphaq,State.m_alphaqim,r,State.m_tmp0,Settings);
for(i=0; i<m; i++)
State.m_tmp0.Set(i,s.m_effc[State.m_basis.m_idx[i]]);
BasisSolveT(State.m_basis,State.m_tmp0,State.m_tmp1,State.m_tmp2);
ComputeAnTV(State,State.m_tmp1,s.m_d);
for(i=0; i<nn; i++)
{
j=State.m_basis.m_nidx[i];
s.m_d.Set(j,s.m_effc[j]-s.m_d[j]);
}
}
else
{
//--- Basis does not change because Qth variable flips from one bound
//--- to another one long before we encounter the boundary
s.m_xa.Set(q,flip);
for(i=0; i<m; i++)
{
bi=State.m_basis.m_idx[i];
vv=lim*(dir*State.m_alphaq[i]);
s.m_xa.Add(bi,-vv);
}
}
State.m_repiterationscount++;
State.m_repiterationscount3++;
}
}
//+------------------------------------------------------------------+
//| This function estimates feasibility properties of the current |
//| basis and invokes phase 1 if necessary. |
//| A problem with following properties is expected: |
//| * M > 0 |
//| * feasible box constraints |
//| * some initial basis (can be dual infeasible) with actual |
//| factorization |
//| * actual initial point XC and target value Z |
//| * actual reduced cost vector D |
//| It returns: |
//| * +1 if dual feasible basis was found |
//| * -4 if problem is dual infeasible |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::InvokePhase1(CDualSimplexState &State,
CDualSimplexSettings &Settings)
{
//--- create variables
int m=State.m_primary.m_m;
double dualerr=0;
State.m_repterminationtype=0;
//--- Integrity checks
if(!CAp::Assert(State.m_primary.m_state==m_ssvalid,__FUNCTION__+": invalid primary X"))
return;
if(!CAp::Assert(m>0,__FUNCTION__+": M<=0"))
return;
//--- Is it dual feasible from the very beginning (or maybe after initial DFC)?
if(State.m_dotrace)
CAp::Trace("> performing initial dual feasibility correction...\n");
dualerr=InitialDualFeasibilityCorrection(State,State.m_primary,Settings);
if(State.m_dotrace)
CAp::Trace(StringFormat("> initial dual feasibility correction done\ndualErr = %.3E\n",dualerr));
if(dualerr<=Settings.m_dtolabs)
{
if(State.m_dotrace)
CAp::Trace("> solution is dual feasible,phase 1 is done\n");
State.m_repterminationtype=1;
return;
}
if(State.m_dotrace)
{
CAp::Trace("> solution is not dual feasible,proceeding to full-scale phase 1\n");
CAp::Trace("\n");
CAp::Trace("****************************************************************************************************\n");
CAp::Trace("* PHASE 1 OF DUAL SIMPLEX SOLVER *\n");
CAp::Trace("****************************************************************************************************\n");
}
//--- Solve phase #1 subproblem
SubproblemInitPhase1(State.m_primary,State.m_basis,State.m_phase1);
if(State.m_dotrace)
CAp::Trace("> performing phase 1 dual feasibility correction...\n");
dualerr=InitialDualFeasibilityCorrection(State,State.m_phase1,Settings);
if(State.m_dotrace)
CAp::Trace(StringFormat("> phase 1 dual feasibility correction done\ndualErr = %.3E\n",dualerr));
SolveSubproblemDual(State,State.m_phase1,true,Settings,State.m_repterminationtype);
//--- check
if(!CAp::Assert(State.m_repterminationtype>0,__FUNCTION__+": unexpected failure of phase #1"))
return;
State.m_repterminationtype=1;
//--- Setup initial basis for phase #2 using solution of phase #1
if(State.m_dotrace)
CAp::Trace("> setting up phase 2 initial solution\n");
SubproblemInferInitialXN(State,State.m_primary);
dualerr=InitialDualFeasibilityCorrection(State,State.m_primary,Settings);
if(dualerr>Settings.m_dtolabs)
{
if(State.m_dotrace)
CAp::Trace("> initial dual feasibility correction failed! terminating...\n");
State.m_repterminationtype=-4;
return;
}
State.m_repterminationtype=1;
}
//+------------------------------------------------------------------+
//| This function performs actual solution. |
//| INPUT PARAMETERS: |
//| State - State |
//| Solution results can be found in fields of State which are |
//| explicitly declared as accessible by external code. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::DSSOptimizeWrk(CDualSimplexState &State,
CDualSimplexSettings &Settings)
{
//--- create variables
int nx=State.m_primary.m_ns+State.m_primary.m_m;
int m=State.m_primary.m_m;
int i=0;
int j=0;
double v=0;
CHighQualityRandState rs;
int t0=0;
//--- Handle case when M=0; after this block we assume that M>0.
if(m==0)
{
//--- Trace
if(State.m_dotrace)
CAp::Trace("> box-only LP problem,quick solution\n");
//--- Solve
SolveBoxOnly(State);
return;
}
//--- Most basic check for correctness of box and/or linear constraints
for(j=0; j<nx; j++)
{
if(State.m_primary.m_bndt[j]==m_ccinfeasible)
{
//--- Set error flag and generate some point to return
if(State.m_dotrace)
CAp::Trace("[WARNING] infeasible box constraint (or range constraint with AL>AU) found,terminating\n");
State.m_repterminationtype=-3;
SetZeroXYStats(State);
return;
}
}
//--- Initialization:
//--- * initial perturbed C[]
CHighQualityRand::HQRndSeed(7456,2355,rs);
for(i=0; i<nx; i++)
{
if(!IsFree(State.m_primary,i))
{
//--- apply perturbation
v=Settings.m_perturbmag*(1+MathAbs(State.m_primary.m_rawc[i]))*(1+CHighQualityRand::HQRndUniformR(rs));
if(!HasBndL(State.m_primary,i))
v=-v;
State.m_primary.m_effc.Set(i,State.m_primary.m_rawc[i]+v);
}
}
//--- Solve phase 1 subproblem, then perturbed subproblem
BasisFreshTrf(State.m_basis,State.m_at,Settings);
if(State.m_primary.m_state==m_ssinvalid)
SubproblemInferInitialXN(State,State.m_primary);
if(State.m_primary.m_state==m_ssvalidxn)
SubproblemHandleXNUpdate(State,State.m_primary);
//--- check
if(!CAp::Assert(State.m_primary.m_state==m_ssvalid,__FUNCTION__+": integrity check failed (init)"))
return;
if(State.m_dotimers)
t0=(int)(GetTickCount()/10000);
InvokePhase1(State,Settings);
if(State.m_dotimers)
State.m_repphase1time=(int)(GetTickCount()/10000)-t0;
if(State.m_repterminationtype<=0)
{
//--- Primal unbounded, dual infeasible
if(!CAp::Assert(State.m_repterminationtype==-4,__FUNCTION__+": integrity check for InvokePhase1() result failed"))
return;
if(State.m_dotrace)
CAp::Trace("> the problem is dual infeasible,primal unbounded\n> done\n");
SetXYDStats(State,State.m_primary,State.m_basis,State.m_xydsbuf,State.m_repx,State.m_replagbc,State.m_replaglc,State.m_repstats);
return;
}
if(State.m_dotrace)
{
CAp::Trace("\n");
CAp::Trace("****************************************************************************************************\n");
CAp::Trace("* PHASE 2 OF DUAL SIMPLEX SOLVER *\n");
CAp::Trace("****************************************************************************************************\n");
}
if(State.m_dotimers)
t0=(int)(GetTickCount()/10000);
SolveSubproblemDual(State,State.m_primary,false,Settings,State.m_repterminationtype);
if(State.m_dotimers)
State.m_repphase2time=(int)(GetTickCount()/10000)-t0;
if(State.m_repterminationtype<=0)
{
//--- Primal infeasible
if(!CAp::Assert(State.m_repterminationtype==-3,__FUNCTION__+": integrity check for SolveSubproblemDual() result failed"))
return;
if(State.m_dotrace)
CAp::Trace("> the problem is primal infeasible\n> done\n");
SetXYDStats(State,State.m_primary,State.m_basis,State.m_xydsbuf,State.m_repx,State.m_replagbc,State.m_replaglc,State.m_repstats);
return;
}
//--- Remove perturbation from the cost vector,
//--- then use primal simplex to enforce dual feasibility
//--- after removal of the perturbation (if necessary).
if(State.m_dotrace)
{
CAp::Trace("\n");
CAp::Trace("****************************************************************************************************\n");
CAp::Trace("* PHASE 3 OF DUAL SIMPLEX SOLVER (perturbation removed from cost vector) *\n");
CAp::Trace("****************************************************************************************************\n");
}
if(State.m_dotimers)
t0=(int)(GetTickCount()/10000);
SubproblemInitPhase3(State.m_primary,State.m_phase3);
State.m_phase3.m_effc=State.m_primary.m_rawc;
//--- check
if(!CAp::Assert(State.m_phase3.m_state>=m_ssvalidxn,__FUNCTION__+": integrity check failed (remove perturbation)"))
return;
SubproblemHandleXNUpdate(State,State.m_phase3);
SolveSubproblemPrimal(State,State.m_phase3,Settings,State.m_repterminationtype);
if(State.m_dotimers)
State.m_repphase3time=(int)(GetTickCount()/10000)-t0;
if(State.m_repterminationtype<=0)
{
//--- Dual infeasible, primal unbounded
//--- check
if(!CAp::Assert(State.m_repterminationtype==-4,__FUNCTION__+": integrity check for SolveSubproblemPrimal() result failed"))
return;
if(State.m_dotrace)
CAp::Trace("> the problem is primal unbounded\n> done\n");
SetXYDStats(State,State.m_phase3,State.m_basis,State.m_xydsbuf,State.m_repx,State.m_replagbc,State.m_replaglc,State.m_repstats);
return;
}
State.m_primary.m_xa=State.m_phase3.m_xa;
for(i=0; i<nx; i++)
{
if(HasBndL(State.m_primary,i))
State.m_primary.m_xa.Set(i,MathMax(State.m_primary.m_xa[i],State.m_primary.m_bndl[i]));
if(HasBndU(State.m_primary,i))
State.m_primary.m_xa.Set(i,MathMin(State.m_primary.m_xa[i],State.m_primary.m_bndu[i]));
}
//--- Primal and dual feasible, problem solved
State.m_repterminationtype=1;
SetXYDStats(State,State.m_primary,State.m_basis,State.m_xydsbuf,State.m_repx,State.m_replagbc,State.m_replaglc,State.m_repstats);
}
//+------------------------------------------------------------------+
//| Box - constrained solver; sets State.RepX, State.RepStats and |
//| State.RepTerminationType, does not change other fields. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::SolveBoxOnly(CDualSimplexState &State)
{
//--- create variables
int i=0;
int ns=State.m_primary.m_ns;
//--- check
if(!CAp::Assert(State.m_primary.m_m==0,__FUNCTION__+": integrity check failed"))
return;
State.m_replagbc=vector<double>::Zeros(ns);
for(i=0; i<ns; i++)
{
//--- Handle infeasible variable
if(State.m_primary.m_bndt[i]==m_ccinfeasible)
{
State.m_repterminationtype=-3;
State.m_repx.Set(i,0.5*(State.m_primary.m_bndl[i]+State.m_primary.m_bndu[i]));
State.m_repstats.Set(i,0);
continue;
}
//--- Handle fixed variable
if(State.m_primary.m_bndt[i]==m_ccfixed)
{
State.m_repx.Set(i,State.m_primary.m_bndl[i]);
State.m_repstats.Set(i,-1);
State.m_replagbc.Set(i,-State.m_primary.m_rawc[i]);
continue;
}
//--- Handle non-zero cost component
if((double)(State.m_primary.m_rawc[i])>0.0)
{
if(State.m_primary.m_bndt[i]!=m_ccrange && State.m_primary.m_bndt[i]!=m_cclower)
{
if(State.m_repterminationtype>0)
State.m_repterminationtype=-4;
if(State.m_primary.m_bndt[i]==m_ccupper)
{
State.m_repx.Set(i,State.m_primary.m_bndu[i]);
State.m_repstats.Set(i,1);
}
else
{
State.m_repx.Set(i,0);
State.m_repstats.Set(i,0);
}
State.m_replagbc.Set(i,0);
}
else
{
State.m_repx.Set(i,State.m_primary.m_bndl[i]);
State.m_repstats.Set(i,-1);
State.m_replagbc.Set(i,-State.m_primary.m_rawc[i]);
}
continue;
}
if(State.m_primary.m_rawc[i]<0.0)
{
if(State.m_primary.m_bndt[i]!=m_ccrange && State.m_primary.m_bndt[i]!=m_ccupper)
{
if(State.m_repterminationtype>0)
State.m_repterminationtype=-4;
if(State.m_primary.m_bndt[i]==m_cclower)
{
State.m_repx.Set(i,State.m_primary.m_bndl[i]);
State.m_repstats.Set(i,-1);
}
else
{
State.m_repx.Set(i,0);
State.m_repstats.Set(i,0);
}
State.m_replagbc.Set(i,0);
}
else
{
State.m_repx.Set(i,State.m_primary.m_bndu[i]);
State.m_repstats.Set(i,1);
State.m_replagbc.Set(i,-State.m_primary.m_rawc[i]);
}
continue;
}
//--- Handle non-free variable with zero cost component
if(State.m_primary.m_bndt[i]==m_ccupper || State.m_primary.m_bndt[i]==m_ccrange)
{
State.m_repx.Set(i,State.m_primary.m_bndu[i]);
State.m_repstats.Set(i,1);
State.m_replagbc.Set(i,0);
continue;
}
if(State.m_primary.m_bndt[i]==m_cclower)
{
State.m_repx.Set(i,State.m_primary.m_bndl[i]);
State.m_repstats.Set(i,-1);
State.m_replagbc.Set(i,0);
continue;
}
//--- Free variable, zero cost component
//--- check
if(!CAp::Assert(State.m_primary.m_bndt[i]==m_ccfree,__FUNCTION__+": integrity check failed"))
return;
State.m_repx.Set(i,0);
State.m_repstats.Set(i,0);
State.m_replagbc.Set(i,0);
}
}
//+------------------------------------------------------------------+
//| Zero - fill RepX, RepLagBC, RepLagLC, RepStats. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::SetZeroXYStats(CDualSimplexState &State)
{
State.m_repx=vector<double>::Zeros(State.m_primary.m_ns);
State.m_replagbc=vector<double>::Zeros(State.m_primary.m_ns);
State.m_replaglc=vector<double>::Zeros(State.m_primary.m_m);
State.m_repstats.Resize(State.m_primary.m_ns+State.m_primary.m_m);
State.m_repstats.Fill(0);
}
//+------------------------------------------------------------------+
//| This function initializes basis structure; no triangular |
//| factorization is prepared yet. Previously allocated memory is |
//| reused. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::BasisInit(int ns,int m,CDualSimplexBasis &s)
{
s.m_ns=ns;
s.m_m=m;
s.m_idx.Resize(m);
s.m_nidx.Resize(ns);
ArrayResize(s.m_isbasic,ns+m);
ArrayFill(s.m_isbasic,0,ns,false);
ArrayFill(s.m_isbasic,ns,m,true);
for(int i=0; i<ns; i++)
s.m_nidx.Set(i,i);
for(int i=0; i<m; i++)
s.m_idx.Set(i,ns+i);
s.m_trftype=3;
s.m_trfage=0;
s.m_isvalidtrf=false;
s.m_dseweights=vector<double>::Ones(m);
s.m_dsevalid=false;
BasisClearStats(s);
}
//+------------------------------------------------------------------+
//| This function clears internal performance counters of the basis |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::BasisClearStats(CDualSimplexBasis &s)
{
s.m_statfact=0;
s.m_statupdt=0;
s.m_statoffdiag=0;
}
//+------------------------------------------------------------------+
//| This function resizes basis. It is assumed that constraint matrix|
//| is completely overwritten by new one, but both matrices are |
//| similar enough so we can reuse previous basis. |
//| Dual steepest edge weights are invalidated by this function. |
//| This function: |
//| * tries to resize basis |
//| * if possible, returns True and valid basis with valid |
//| factorization |
//| * if resize is impossible (or abandoned due to stability |
//| reasons), it returns False and basis object is left in the |
//| invalid State(you have to reinitialize it by all-logicals |
//| basis) |
//| Following types of resize are supported: |
//| * new basis size is larger than previous one => logical |
//| elements are added to the new basis |
//| * basis sizes match => no operation is performed |
//| * new basis size is zero => basis is set to zero |
//| This function: |
//| * requires valid triangular factorization at S on entry |
//| * replaces it by another, valid factorization |
//| * checks that new factorization deviates from the previous one |
//| not too much by comparing magnitudes of min[abs(u_ii)] in |
//| both factorization (sharp decrease results in attempt to |
//| resize being abandoned |
//| IMPORTANT: if smooth resize is not possible, this function throws|
//| an exception! It is responsibility of the caller to |
//| check that smooth resize is possible |
//+------------------------------------------------------------------+
bool CRevisedDualSimplex::BasisTryResize(CDualSimplexBasis &s,
int newm,
CSparseMatrix &at,
CDualSimplexSettings &Settings)
{
//--- create variables
bool result=false;
int ns=s.m_ns;
int oldm=s.m_m;
double oldminu=0;
double newminu=0;
//--- Quick exit strategies
if(newm==0)
{
BasisInit(ns,0,s);
return(true);
}
//--- Same size or larger
if(newm>=oldm)
{
//--- check
if(!CAp::Assert(s.m_isvalidtrf || oldm==0,__FUNCTION__+": needs valid TRF in S"))
return(false);
//--- Save information about matrix conditioning
oldminu=BasisMinimumDiagonalElement(s);
//--- Growth if needed
s.m_m=newm;
s.m_idx.Resize(newm);
ArrayResize(s.m_isbasic,ns+newm);
for(int i=oldm; i<newm; i++)
{
s.m_idx.Set(i,ns+i);
s.m_isbasic[ns+i]=true;
}
//--- DSE weights are invalid and filled by 1.0
s.m_dseweights=vector<double>::Ones(newm);
s.m_dsevalid=false;
//--- Invalidate TRF.
//--- Try to refactorize.
s.m_isvalidtrf=false;
newminu=BasisFreshTRFUnsafe(s,at,Settings);
result=newminu>=(m_maxudecay*oldminu);
//--- return result
return(result);
}
//--- unexpected branch
CAp::Assert(false,__FUNCTION__+": unexpected branch");
return(false);
}
//+------------------------------------------------------------------+
//| This function returns minimum diagonal element of S. Result = 1 |
//| is returned for M = 0. |
//+------------------------------------------------------------------+
double CRevisedDualSimplex::BasisMinimumDiagonalElement(CDualSimplexBasis &s)
{
//--- create variables
int m=s.m_m;
double v=0;
double vv=0;
//--- Quick exit
if(m==0)
return(1);
//--- check
if(!CAp::Assert(s.m_trftype==0 || s.m_trftype==1 || s.m_trftype==2 || s.m_trftype==3,__FUNCTION__+": unexpected TRF type"))
return(0);
if(!CAp::Assert(s.m_isvalidtrf,__FUNCTION__+": TRF is invalid"))
return(0);
v=CMath::m_maxrealnumber;
for(int i=0; i<m; i++)
{
vv=0;
if(s.m_trftype==0 || s.m_trftype==1)
vv=s.m_denselu.Get(i,i);
if(s.m_trftype==2 || s.m_trftype==3)
vv=CSparse::SparseGetDiagonal(s.m_sparseu,i);
if(vv<0)
vv=-vv;
if(vv<v)
v=vv;
}
//--- return result
return(v);
}
//+------------------------------------------------------------------+
//| This function exports division of variables into basic / nonbasic|
//| ones; only basic / nonbasic sets are exported - triangular |
//| factorization is NOT exported; however, valid triangular |
//| factorization IS required in order to perform exporting. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::BasisExportTo(CDualSimplexBasis &s0,
CDualSimplexBasis &s1)
{
//--- copy
s1.m_ns=s0.m_ns;
s1.m_m=s0.m_m;
s1.m_idx=s0.m_idx;
s1.m_nidx=s0.m_nidx;
ArrayCopy(s1.m_isbasic,s0.m_isbasic);
s1.m_isvalidtrf=false;
s1.m_trftype=-1;
s1.m_dsevalid=false;
if(s0.m_m>0)
{
//--- check
if(!CAp::Assert(s0.m_isvalidtrf,"BasisExport: valid factorization is required for source basis"))
return;
s1.m_eminu=BasisMinimumDiagonalElement(s0);
}
else
s1.m_eminu=1;
}
//+------------------------------------------------------------------+
//| This function imports from S1 to S0 a division of variables into |
//| basic / nonbasic ones; only basic / nonbasic sets are imported. |
//| Triangular factorization is not imported; however, this function |
//| checks that new factorization deviates from the previous one not |
//| too much by comparing magnitudes of min[abs(u_ii)] in both |
//| factorization (basis being imported stores statistics about U). |
//| Sharp decrease of diagonal elements means that we have too |
//| unstable situation which results in import being abandoned. In |
//| this case False is returned, and the basis S0 is left in the |
//| indeterminate invalid State(you have to reinitialize it by |
//| all-logicals). |
//| IMPORTANT: if metrics of S0 and S1 do not match, an exception |
//| will be generated. |
//+------------------------------------------------------------------+
bool CRevisedDualSimplex::BasisTryImportFrom(CDualSimplexBasis &s0,
CDualSimplexBasis &s1,
CSparseMatrix &at,
CDualSimplexSettings &Settings)
{
//--- create variables
bool result=false;
double newminu=0;
//--- check
if(!CAp::Assert(s0.m_ns==s1.m_ns,__FUNCTION__+": structural variable counts do not match"))
return(false);
BasisClearStats(s0);
s0.m_m=s1.m_m;
s0.m_idx=s1.m_idx;
s0.m_nidx=s1.m_nidx;
ArrayCopy(s0.m_isbasic,s1.m_isbasic);
s0.m_isvalidtrf=false;
s0.m_dseweights=vector<double>::Ones(s1.m_m);
s0.m_dsevalid=false;
newminu=BasisFreshTRFUnsafe(s0,at,Settings);
result=newminu>=(m_maxudecay*s1.m_eminu);
if(!result)
{
s0.m_isvalidtrf=false;
s0.m_trftype=-1;
}
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function computes fresh triangular factorization. |
//| If TRF of age 0 (fresh) is already present, no new factorization |
//| is calculated. If factorization has exactly zero element along |
//| diagonal, this function generates exception. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::BasisFreshTrf(CDualSimplexBasis &s,
CSparseMatrix &at,
CDualSimplexSettings &Settings)
{
double v=BasisFreshTRFUnsafe(s,at,Settings);
CAp::Assert(v>0.0,__FUNCTION__+": degeneracy of B is detected");
}
//+------------------------------------------------------------------+
//| This function computes fresh triangular factorization. |
//| If TRF of age 0 (fresh) is already present, no new factorization |
//| is calculated. |
//| It returns min[abs(u[i, i])] which can be used to determine |
//| whether factorization is degenerate or not (it will|
//| factorize anything, the question is whether it is |
//| possible to use factorization) |
//+------------------------------------------------------------------+
double CRevisedDualSimplex::BasisFreshTRFUnsafe(CDualSimplexBasis &s,
CSparseMatrix &at,
CDualSimplexSettings &Settings)
{
//--- create variables
int m=s.m_m;
int ns=s.m_ns;
double result=0;
int i=0;
int j=0;
int k=0;
int j0=0;
int j1=0;
int k1=0;
int nzl=0;
int nzu=0;
int nlogical=0;
int nstructural=0;
int offs=0;
int offs1=0;
int offs2=0;
//--- Compare TRF type with one required by Settings, invalidation and refresh otherwise
if(s.m_trftype!=Settings.m_trftype)
{
s.m_trftype=Settings.m_trftype;
s.m_isvalidtrf=false;
result=BasisFreshTRFUnsafe(s,at,Settings);
//--- return result
return(result);
}
//--- Is it valid and fresh?
if(s.m_isvalidtrf && s.m_trfage==0)
{
result=BasisMinimumDiagonalElement(s);
//--- return result
return(result);
}
//--- Dense TRF
if(s.m_trftype==0 || s.m_trftype==1)
{
s.m_colpermbwd.Resize(m);
for(i=0; i<m; i++)
s.m_colpermbwd.Set(i,i);
s.m_denselu=matrix<double>::Zeros(m,m);
for(i=0; i<m; i++)
{
j0=at.m_RIdx[s.m_idx[i]];
j1=at.m_RIdx[s.m_idx[i]+1];
for(j=j0; j<j1; j++)
s.m_denselu.Set(i,at.m_Idx[j],at.m_Vals[j]);
}
CTrFac::RMatrixLU(s.m_denselu,m,m,s.m_tmpi);
PivotToBWD(s.m_tmpi,m,s.m_rowpermbwd);
s.m_isvalidtrf=true;
s.m_trfage=0;
s.m_statfact++;
s.m_statoffdiag+=CMath::Sqr(m-1);
result=BasisMinimumDiagonalElement(s);
//--- return result
return(result);
}
//--- Sparse TRF (with either PFI or Forest-Tomlin)
if(s.m_trftype==2 || s.m_trftype==3)
{
//--- Determine permutation which moves logical variables
//--- to the beginning.
//--- NOTE: this reordering results in stable factorization
//--- because we prenormalized constraints with 2-norm,
//--- all elements in the logical columns are less than
//--- 1.0 in magnitude.
//--- After this block is done we have following arrays:
//--- * tCInvIdx[j], which is an inverse of ColPermBwf[]
s.m_tcinvidx.Resize(m);
s.m_rowpermbwd.Resize(m);
s.m_colpermbwd.Resize(m);
for(i=0; i<m; i++)
{
s.m_tcinvidx.Set(i,i);
s.m_rowpermbwd.Set(i,i);
s.m_colpermbwd.Set(i,i);
}
nlogical=0;
for(i=0; i<m; i++)
{
if(s.m_idx[i]>=ns)
{
s.m_rowpermbwd.Swap(nlogical,i);
j1=s.m_tcinvidx[s.m_idx[i]-ns];
s.m_colpermbwd.Swap(j1,nlogical);
s.m_tcinvidx.Set(s.m_colpermbwd[nlogical],nlogical);
s.m_tcinvidx.Set(s.m_colpermbwd[j1],j1);
nlogical++;
}
}
CTSort::SortMiddleI(s.m_colpermbwd,nlogical,m-nlogical);
for(i=0; i<m; i++)
s.m_tcinvidx.Set(s.m_colpermbwd[i],i);
nstructural=m-nlogical;
//--- Prepare SparseLU1 to receive factored out logical part of the matrix
//--- and SparseLU2 to receive structural part of the matrix.
s.m_sparselu1.m_RIdx.Resize(nstructural+1);
s.m_sparselu1.m_DIdx.Resize(nstructural);
s.m_sparselu1.m_UIdx.Resize(nstructural);
s.m_sparselu1.m_MatrixType=1;
s.m_sparselu1.m_M=nstructural;
s.m_sparselu1.m_N=nlogical;
s.m_sparselu1.m_RIdx.Set(0,0);
s.m_sparselu2.m_RIdx.Resize(nstructural+1);
s.m_sparselu2.m_DIdx.Resize(nstructural);
s.m_sparselu2.m_UIdx.Resize(nstructural);
s.m_sparselu2.m_MatrixType=1;
s.m_sparselu2.m_M=nstructural;
s.m_sparselu2.m_N=nstructural;
s.m_sparselu2.m_RIdx.Set(0,0);
//--- Reorder array, perform LU factorization
for(k=0; k<nstructural; k++)
{
//--- Make sure SparseLU1 and SparseLU2 have enough place.
offs1=s.m_sparselu1.m_RIdx[k];
offs2=s.m_sparselu2.m_RIdx[k];
s.m_sparselu1.m_Idx.Resize(offs1+m);
s.m_sparselu1.m_Vals.Resize(offs1+m);
s.m_sparselu2.m_Idx.Resize(offs2+m);
s.m_sparselu2.m_Vals.Resize(offs2+m);
//--- Extract K-th row of the SparseLU1/2 (I-th row of the original matrix)
i=s.m_rowpermbwd[k+nlogical];
j0=at.m_RIdx[s.m_idx[i]];
j1=at.m_RIdx[s.m_idx[i]+1];
for(j=j0; j<j1; j++)
{
k1=s.m_tcinvidx[at.m_Idx[j]];
if(k1<nlogical)
{
//--- Append element to SparseLU1
s.m_sparselu1.m_Idx.Set(offs1,k1);
s.m_sparselu1.m_Vals.Set(offs1,at.m_Vals[j]);
offs1++;
}
else
{
//--- Append element to SparseLU2
s.m_sparselu2.m_Idx.Set(offs2,k1-nlogical);
s.m_sparselu2.m_Vals.Set(offs2,at.m_Vals[j]);
offs2++;
}
}
//--- Elements added to the last row of LU1 can be unordered,
//--- so it needs resorting.
//--- LU2 does NOT need resorting because trailing NStructural
//--- elements of permutation were post-sorted to produce
//--- already sorted results.
CTSort::TagSortMiddleIR(s.m_sparselu1.m_Idx,s.m_sparselu1.m_Vals,s.m_sparselu1.m_RIdx[k],offs1-s.m_sparselu1.m_RIdx[k]);
s.m_sparselu1.m_RIdx.Set(k+1,offs1);
s.m_sparselu2.m_RIdx.Set(k+1,offs2);
}
s.m_sparselu1.m_NInitialized=s.m_sparselu1.m_RIdx[nstructural];
s.m_sparselu2.m_NInitialized=s.m_sparselu2.m_RIdx[nstructural];
CSparse::SparseInitDUIdx(s.m_sparselu1);
CSparse::SparseInitDUIdx(s.m_sparselu2);
if(nstructural>0)
{
CSpTrf::SpTrfLU(s.m_sparselu2,2,s.m_densep2,s.m_densep2c,s.m_lubuf2);
for(i=0; i<nstructural; i++)
{
s.m_rowpermbwd.Swap(i+nlogical,s.m_densep2[i]+nlogical);
s.m_colpermbwd.Swap(i+nlogical,s.m_densep2c[i]+nlogical);
}
//--- Process L factor:
//--- 1. count number of non-zeros in the L factor,
//--- 2. fill NLogical*NLogical leading block
//--- 3. NStructural*M bottom block
nzl=nlogical;
for(i=0; i<nstructural; i++)
{
k=s.m_lubuf2.m_RowPermRawIdx[i];
nzl+=(s.m_sparselu1.m_RIdx[k+1]-s.m_sparselu1.m_RIdx[k]);
nzl+=1+(s.m_sparselu2.m_DIdx[i]-s.m_sparselu2.m_RIdx[i]);
}
s.m_sparsel.m_Vals.Resize(nzl);
s.m_sparsel.m_Idx.Resize(nzl);
s.m_sparsel.m_RIdx.Resize(m+1);
s.m_sparsel.m_DIdx.Resize(m);
s.m_sparsel.m_UIdx.Resize(m);
s.m_sparsel.m_MatrixType=1;
s.m_sparsel.m_M=m;
s.m_sparsel.m_N=m;
s.m_sparsel.m_NInitialized=nzl;
s.m_sparsel.m_RIdx.Set(0,0);
for(i=0; i<nlogical; i++)
{
s.m_sparsel.m_Idx.Set(i,i);
s.m_sparsel.m_Vals.Set(i,1.0);
s.m_sparsel.m_RIdx.Set(i+1,i+1);
}
for(i=0; i<nstructural; i++)
{
offs=s.m_sparsel.m_RIdx[nlogical+i];
k=s.m_lubuf2.m_RowPermRawIdx[i];
j0=s.m_sparselu1.m_RIdx[k];
j1=s.m_sparselu1.m_RIdx[k+1];
for(j=j0; j<j1; j++)
{
s.m_sparsel.m_Idx.Set(offs,s.m_sparselu1.m_Idx[j]);
s.m_sparsel.m_Vals.Set(offs,-s.m_sparselu1.m_Vals[j]);
offs++;
}
j0=s.m_sparselu2.m_RIdx[i];
j1=s.m_sparselu2.m_DIdx[i];
for(j=j0; j<j1; j++)
{
s.m_sparsel.m_Idx.Set(offs,nlogical+s.m_sparselu2.m_Idx[j]);
s.m_sparsel.m_Vals.Set(offs,s.m_sparselu2.m_Vals[j]);
offs++;
}
s.m_sparsel.m_Idx.Set(offs,nlogical+i);
s.m_sparsel.m_Vals.Set(offs,1.0);
offs++;
s.m_sparsel.m_RIdx.Set(nlogical+i+1,offs);
}
//--- check
if(!CAp::Assert(s.m_sparsel.m_NInitialized==s.m_sparsel.m_RIdx[m],__FUNCTION__+": integrity check failed"))
return(0);
CSparse::SparseInitDUIdx(s.m_sparsel);
//--- Process U factor:
//--- 1. count number of non-zeros in the U factor,
//--- 2. fill NLogical*NLogical leading block
//--- 3. NStructural*NStructural bottom block
nzu=nlogical;
for(i=0; i<=nstructural-1; i++)
nzu+=1+(s.m_sparselu2.m_RIdx[i+1]-s.m_sparselu2.m_UIdx[i]);
s.m_sparseu.m_Vals=vector<double>::Full(nzu,-1.0);
s.m_sparseu.m_Idx.Resize(nzu);
s.m_sparseu.m_RIdx.Resize(m+1);
s.m_sparseu.m_DIdx.Resize(m);
s.m_sparseu.m_UIdx.Resize(m);
s.m_sparseu.m_MatrixType=1;
s.m_sparseu.m_M=m;
s.m_sparseu.m_N=m;
s.m_sparseu.m_NInitialized=nzu;
s.m_sparseu.m_RIdx.Set(0,0);
for(i=0; i<nlogical; i++)
{
s.m_sparseu.m_Idx.Set(i,i);
s.m_sparseu.m_RIdx.Set(i+1,i+1);
}
for(i=0; i<nstructural; i++)
{
offs=s.m_sparseu.m_RIdx[nlogical+i];
s.m_sparseu.m_Idx.Set(offs,nlogical+i);
j=s.m_sparselu2.m_DIdx[i];
if(j<s.m_sparselu2.m_UIdx[i])
{
//--- check
if(!CAp::Assert(s.m_sparselu2.m_Idx[j]==i,__FUNCTION__+": integrity check failed"))
return(0);
s.m_sparseu.m_Vals.Set(offs,s.m_sparselu2.m_Vals[j]);
}
else
{
s.m_sparseu.m_Vals.Set(offs,0);
}
offs++;
j0=s.m_sparselu2.m_UIdx[i];
j1=s.m_sparselu2.m_RIdx[i+1];
for(j=j0; j<j1; j++)
{
s.m_sparseu.m_Idx.Set(offs,nlogical+s.m_sparselu2.m_Idx[j]);
s.m_sparseu.m_Vals.Set(offs,s.m_sparselu2.m_Vals[j]);
offs++;
}
s.m_sparseu.m_RIdx.Set(nlogical+i+1,offs);
}
//--- check
if(!CAp::Assert(s.m_sparseu.m_NInitialized==s.m_sparseu.m_RIdx[m],__FUNCTION__+": integrity check failed"))
return(0);
CSparse::SparseInitDUIdx(s.m_sparseu);
}
else
{
s.m_nrs.Resize(m);
for(i=0; i<m; i++)
s.m_nrs.Set(i,1);
CSparse::SparseCreateCRSBuf(m,m,s.m_nrs,s.m_sparsel);
for(i=0; i<nlogical; i++)
CSparse::SparseSet(s.m_sparsel,i,i,1.0);
CSparse::SparseCreateCRSBuf(m,m,s.m_nrs,s.m_sparseu);
for(i=0; i<nlogical; i++)
CSparse::SparseSet(s.m_sparseu,i,i,-1.0);
}
CSparse::SparseCopyTransposeCRSBuf(s.m_sparseu,s.m_sparseut);
s.m_isvalidtrf=true;
s.m_trfage=0;
s.m_statfact=s.m_statfact+1;
s.m_statoffdiag+=(s.m_sparsel.m_RIdx[m]-m)+(s.m_sparseu.m_RIdx[m]-m);
result=BasisMinimumDiagonalElement(s);
//--- return result
return(result);
}
//---unexpected TRF type
CAp::Assert(false,__FUNCTION__+": unexpected TRF type");
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function fills S.DSEWeights by actual weights according to |
//| current Settings and sets validity flag. |
//| Basis object MUST store valid triangular factorization, otherwise|
//| this function throws an exception. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::BasisRequestWeights(CDualSimplexBasis &s,
CDualSimplexSettings &Settings)
{
//--- create variables
int m=s.m_m;
int ns=s.m_ns;
int i=0;
int j=0;
double v=0;
double vv=0;
//--- check
if(!CAp::Assert(Settings.m_pricing==-1 || Settings.m_pricing==0 || Settings.m_pricing==1,__FUNCTION__+": unknown pricing type"))
return;
if(!CAp::Assert(s.m_isvalidtrf,__FUNCTION__+": factorization is not computed prior to calling this function"))
return;
//--- If weights are valid, return immediately
if(s.m_dsevalid)
return;
//--- Compute weights from scratch
if(Settings.m_pricing==-1 || Settings.m_pricing==1)
{
for(i=0; i<m; i++)
{
if(s.m_idx[i]<ns)
{
//--- Structural variable, DSE weight is computed by definition
s.m_wtmp0=vector<double>::Zeros(m);
s.m_wtmp1.Resize(m);
s.m_wtmp0.Set(i,1);
BasisSolveT(s,s.m_wtmp0,s.m_wtmp1,s.m_wtmp2);
v=s.m_wtmp1.Dot(s.m_wtmp1);
s.m_dseweights.Set(i,v);
}
else
{
//--- Logical variable, weight can be set to 1.0
s.m_dseweights.Set(i,1.0);
}
}
s.m_dsevalid=true;
return;
}
//--- Compute weights from scratch
if(Settings.m_pricing==0)
{
s.m_dseweights=vector<double>::Ones(m);
s.m_dsevalid=true;
}
else
CAp::Assert(false,__FUNCTION__+": unexpected pricing type");
}
//+------------------------------------------------------------------+
//| This function updates triangular factorization by adding Q to |
//| basis and removing P from basis. It also updates index tables |
//| IsBasic[], BasicIdx[], Basis.NIdx[]. |
//| AlphaQim contains intermediate result from Ftran for AlphaQ, it |
//| is used by Forest - Tomlin update scheme. If other update is |
//| used, it is not referenced at all. |
//| X[], D[], Z are NOT recomputed. |
//| Tau is used if Settings.Pricing = 1, ignored otherwise. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::BasisUpdateTrf(CDualSimplexBasis &s,
CSparseMatrix &at,
int p,
int q,
CRowDouble &alphaq,
CRowDouble &alphaqim,
int r,
CRowDouble &tau,
CDualSimplexSettings &Settings)
{
//--- create variables
int m=s.m_m;
int nn=s.m_ns;
int i=0;
int j=0;
bool processed=false;
double invaq=0;
int dstoffs=0;
int srcoffs=0;
int srcidx=0;
double srcval=0;
double vcorner=0;
int idxd=0;
double v=0;
//--- Update index tables
s.m_isbasic[p]=false;
s.m_isbasic[q]=true;
for(i=0; i<m; i++)
{
if(s.m_idx[i]==p)
{
s.m_idx.Set(i,q);
break;
}
}
for(i=0; i<nn; i++)
{
if(s.m_nidx[i]==q)
{
s.m_nidx.Set(i,p);
break;
}
}
//--- Update dense factorization
if(s.m_trftype!=Settings.m_trftype || s.m_trftype==0 || !s.m_isvalidtrf || s.m_trfage>=Settings.m_maxtrfage)
{
//--- Complete refresh is needed for factorization
s.m_isvalidtrf=false;
BasisFreshTrf(s,at,Settings);
}
else
{
processed=false;
if(s.m_trftype==0 || s.m_trftype==1 || s.m_trftype==2)
{
//--- Dense/sparse factorizations with dense PFI
//--- check
if(!CAp::Assert(alphaq[r]!=0.0,__FUNCTION__+": integrity check failed,AlphaQ[R]=0"))
return;
s.m_densepfieta.Resize((s.m_trfage+1)*m);
s.m_rk.Resize(s.m_trfage+1);
s.m_rk.Set(s.m_trfage,r);
invaq=1.0/alphaq[r];
for(i=0; i<m; i++)
{
if(i!=r)
s.m_densepfieta.Set(s.m_trfage*m+i,-(alphaq[i]*invaq));
else
s.m_densepfieta.Set(s.m_trfage*m+i,invaq);
}
s.m_trfage++;
s.m_statupdt++;
s.m_statoffdiag+=CMath::Sqr(m-1);
processed=true;
}
if(s.m_trftype==3)
{
//--- Sparse factorization with Forest-Tomlin update
//--- check
if(!CAp::Assert(alphaq[r]!=0.0,__FUNCTION__+": integrity check failed,AlphaQ[R]=0"))
return;
s.m_densemu.Resize((s.m_trfage+1)*m);
s.m_rk.Resize(s.m_trfage+1);
s.m_dk.Resize(s.m_trfage+1);
s.m_utmp0=vector<double>::Zeros(m);
//--- Determine D - index of row being overwritten by Forest-Tomlin update
idxd=-1;
for(i=0; i<m; i++)
{
if(s.m_rowpermbwd[i]==r)
{
idxd=i;
break;
}
}
//--- check
if(!CAp::Assert(idxd>=0,__FUNCTION__+": unexpected integrity check failure"))
return;
s.m_rk.Set(s.m_trfage,r);
s.m_dk.Set(s.m_trfage,idxd);
//--- Modify L with permutation which moves D-th row/column to the end:
//--- * rows 0...D-1 are left intact
//--- * rows D+1...M-1 are moved one position up, with columns 0..D-1
//--- retained as is, and columns D+1...M-1 being moved one position left.
//--- * last row is filled by permutation/modification of AlphaQim
//--- Determine FT update coefficients in the process.
s.m_sparsel.m_Idx.Resize(s.m_sparsel.m_RIdx[m]+m);
s.m_sparsel.m_Vals.Resize(s.m_sparsel.m_RIdx[m]+m);
for(i=idxd+1; i<m; i++)
{
j=s.m_sparsel.m_RIdx[i+1]-1;
if(s.m_sparsel.m_Idx[j]!=i || s.m_sparsel.m_Vals[j]!=1)
{
//--- check
if(!CAp::Assert(false,__FUNCTION__+": integrity check failed for sparse L"))
return;
}
dstoffs=s.m_sparsel.m_RIdx[i-1];
srcoffs=s.m_sparsel.m_RIdx[i];
//--- Read first element in the row (it has at least one - unit diagonal)
srcidx=s.m_sparsel.m_Idx[srcoffs];
srcval=s.m_sparsel.m_Vals[srcoffs];
//--- Read/write columns 0...D-1
while(srcidx<idxd)
{
s.m_sparsel.m_Idx.Set(dstoffs,srcidx);
s.m_sparsel.m_Vals.Set(dstoffs,srcval);
dstoffs++;
srcoffs++;
srcidx=s.m_sparsel.m_Idx[srcoffs];
srcval=s.m_sparsel.m_Vals[srcoffs];
}
//--- If we have non-zero element in column D, use it as
//--- right-hand side of intermediate linear system which
//--- is used to determine coefficients of update matrix.
if(srcidx==idxd)
{
s.m_utmp0.Set(i-1,srcval);
srcoffs++;
srcidx=s.m_sparsel.m_Idx[srcoffs];
srcval=s.m_sparsel.m_Vals[srcoffs];
}
//--- Process columns D+1...I-1
v=s.m_utmp0[i-1];
while(srcidx<i)
{
s.m_sparsel.m_Idx.Set(dstoffs,srcidx-1);
s.m_sparsel.m_Vals.Set(dstoffs,srcval);
v-=srcval*s.m_utmp0[srcidx-1];
dstoffs++;
srcoffs++;
srcidx=s.m_sparsel.m_Idx[srcoffs];
srcval=s.m_sparsel.m_Vals[srcoffs];
}
s.m_utmp0.Set(i-1,v);
//--- Write out unit diagonal, finalize row
s.m_sparsel.m_Idx.Set(dstoffs,i-1);
s.m_sparsel.m_Vals.Set(dstoffs,1);
dstoffs++;
s.m_sparsel.m_RIdx.Set(i,dstoffs);
}
s.m_utmp0.Set(m-1,1);
dstoffs=s.m_sparsel.m_RIdx[m-1];
for(j=0; j<=idxd-1; j++)
{
v=alphaqim[j];
if(v!=0)
{
s.m_sparsel.m_Idx.Set(dstoffs,j);
s.m_sparsel.m_Vals.Set(dstoffs,v);
dstoffs++;
}
}
vcorner=alphaqim[idxd];
for(j=idxd+1; j<m; j++)
{
v=alphaqim[j];
if(v!=0)
{
s.m_sparsel.m_Idx.Set(dstoffs,j-1);
s.m_sparsel.m_Vals.Set(dstoffs,v);
dstoffs++;
vcorner-=v*s.m_utmp0[j-1];
}
}
s.m_sparsel.m_Idx.Set(dstoffs,m-1);
s.m_sparsel.m_Vals.Set(dstoffs,1);
dstoffs++;
s.m_sparsel.m_RIdx.Set(m,dstoffs);
s.m_sparsel.m_NInitialized=s.m_sparsel.m_RIdx[m];
for(i=0; i<m; i++)
{
j=s.m_sparsel.m_RIdx[i+1];
s.m_sparsel.m_DIdx.Set(i,j-1);
s.m_sparsel.m_UIdx.Set(i,j);
}
//--- check
if(!CAp::Assert(vcorner!=0,__FUNCTION__+": corner element is zero,degeneracy detected"))
return;
v=1/vcorner;
for(i=0; i<m-1; i++)
s.m_densemu.Set(s.m_trfage*m+i,-(s.m_utmp0[i]*v));
s.m_densemu.Set(s.m_trfage*m+m-1,v);
//--- Multiply row permutation matrix by cyclic permutation applied to L
InverseCyclicPermutation(s.m_rowpermbwd,m,idxd,s.m_utmpi);
//--- Done
s.m_trfage++;
s.m_statupdt++;
s.m_statoffdiag+=(s.m_sparsel.m_RIdx[m]-m)+(s.m_sparseu.m_RIdx[m]-m);
processed=true;
}
//--- check
if(!CAp::Assert(processed,__FUNCTION__+": unexpected TRF type"))
return;
}
//--- Update pricing weights
//--- check
if(!CAp::Assert(Settings.m_pricing==-1 || Settings.m_pricing==0 || Settings.m_pricing==1,__FUNCTION__+": unexpected Settings.Pricing"))
return;
processed=false;
switch(Settings.m_pricing)
{
case -1:
//--- Weights are recomputed from scratch at every step.
//--- VERY, VERY time consuming, used only for debug purposes.
s.m_dsevalid=false;
BasisRequestWeights(s,Settings);
processed=true;
break;
case 0:
//--- Weights are filled by 1.0
if(!s.m_dsevalid)
{
s.m_dseweights.Fill(1.0);
s.m_dsevalid=true;
}
processed=true;
break;
case 1:
//--- Weights are computed using DSE update formula.
if(s.m_dsevalid)
{
//--- Compute using update formula
for(i=0; i<m; i++)
if(i!=r)
{
s.m_dseweights.Add(i,-2*(alphaq[i]/alphaq[r])*tau[i]+s.m_dseweights[r]*CMath::Sqr(alphaq[i]/alphaq[r]));
s.m_dseweights.Set(i,MathMax(s.m_dseweights[i],m_minbeta));
}
s.m_dseweights.Mul(r,1.0/CMath::Sqr(alphaq[r]));
}
else
{
//--- No prior values, compute from scratch (usually it is done only once)
BasisRequestWeights(s,Settings);
}
processed=true;
break;
}
CAp::Assert(processed,__FUNCTION__+": unexpected pricing type");
}
//+------------------------------------------------------------------+
//| This function computes solution to B*x = r. |
//| Output array is reallocated if needed. Temporary array TmpX[] is |
//| used and reallocated if necessary. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::BasisSolve(CDualSimplexBasis &s,
CRowDouble &r,
CRowDouble &x,
CRowDouble &tmpx)
{
BasisSolveX(s,r,x,x,false,tmpx);
}
//+------------------------------------------------------------------+
//| This function computes solution to B*x = r. It also additionally |
//| outputs intermediate result of multiplication by |
//| inv(DS) * inv(U) * inv(colPerm), a value essential for |
//| Forest - Tomlin update. |
//| Output arrays are reallocated if needed. Temporary array TX[] can|
//| be used / reallocated. |
//| If NeedIntermediate is False or Forest - Tomlin updates are not |
//| used, then Xim[] is not referenced at all. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::BasisSolveX(CDualSimplexBasis &s,
CRowDouble &r,
CRowDouble &x,
CRowDouble &xim,
bool needintermediate,
CRowDouble &tx)
{
//--- create variables
int m=s.m_m;
int i=0;
int d=0;
int k=0;
double v=0;
double vd=0;
double vv=0;
bool processed=false;
//--- check
if(!CAp::Assert(s.m_isvalidtrf,__FUNCTION__+": integrity check failed"))
return;
tx.Resize(m);
//--- Dense/sparse factorizations with dense PFI
//--- NOTE: although we solve B*x=r, internally we store factorization of B^T
if((s.m_trftype==0 || s.m_trftype==1) || s.m_trftype==2)
{
//--- check
if(!CAp::Assert(s.m_trfage==0 || s.m_trftype!=0,__FUNCTION__+": integrity check failed TrfAge vs TrfType"))
return;
x.Resize(m);
for(i=0; i<m; i++)
x.Set(i,r[s.m_colpermbwd[i]]);
if(s.m_trftype==0 || s.m_trftype==1)
{
//--- Dense TRF
CAblas::RMatrixTrsVect(m,s.m_denselu,0,0,true,false,1,x,0);
CAblas::RMatrixTrsVect(m,s.m_denselu,0,0,false,true,1,x,0);
}
else
{
//--- Sparse TRF
CSparse::SparseTRSV(s.m_sparseu,true,false,1,x);
CSparse::SparseTRSV(s.m_sparsel,false,false,1,x);
}
for(i=0; i<m; i++)
tx.Set(s.m_rowpermbwd[i],x[i]);
x=tx;
for(k=0; k<s.m_trfage; k++)
{
v=x[s.m_rk[k]];
for(i=0; i<m; i++)
x.Add(i,s.m_densepfieta[k*m+i]*v);
x.Add(s.m_rk[k],-v);
}
processed=true;
}
//--- Sparse factorization with Forest-Tomlin update
//--- NOTE: although we solve B*x=r, internally we store factorization of B^T
if(s.m_trftype==3)
{
x.Resize(m);
for(i=0; i<m; i++)
x.Set(i,r[s.m_colpermbwd[i]]);
CSparse::SparseTRSV(s.m_sparseu,true,false,1,x);
for(k=0; k<s.m_trfage; k++)
{
//--- The code below is an amalgamation of two parts:
//--- cyclic permutation
//--- V:=X[D];
//--- for I:=D to M-2 do
//--- X[I]:=X[I+1];
//--- X[M-1]:=V;
//--- and triangular factor
//--- V:=0;
//--- for I:=D to M-1 do
//--- V:=V+X[I]*S.DenseMu[K*M+I];
//--- X[M-1]:=V;
d=s.m_dk[k];
vv=0;
vd=x[d];
for(i=d; i<m-1; i++)
{
v=x[i+1];
x.Set(i,v);
vv+=v*s.m_densemu[k*m+i];
}
x.Set(m-1,vv+vd*s.m_densemu[k*m+m-1]);
}
if(needintermediate)
xim=x;
CSparse::SparseTRSV(s.m_sparsel,false,false,1,x);
for(i=0; i<m; i++)
tx.Set(s.m_rowpermbwd[i],x[i]);
x=tx;
processed=true;
}
//--- Integrity check
if(!CAp::Assert(processed,__FUNCTION__+": unsupported TRF type"))
return;
v=x.Sum();
CAp::Assert(MathIsValidNumber(v),__FUNCTION__+": integrity check failed (degeneracy in B?)");
}
//+------------------------------------------------------------------+
//| This function computes solution to(B^T)*x = r. |
//| Output array is reallocated if needed. TX[] temporary is |
//| reallocated if needed |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::BasisSolveT(CDualSimplexBasis &s,
CRowDouble &r,
CRowDouble &x,
CRowDouble &tx)
{
//--- create variables
int m=s.m_m;
int i=0;
int d=0;
int k=0;
double v=0;
double vm=0;
bool processed=false;
//--- check
if(!CAp::Assert(s.m_isvalidtrf,__FUNCTION__+": integrity check failed"))
return;
tx.Resize(m);
//--- Dense factorizations
if(s.m_trftype==0 || s.m_trftype==1 || s.m_trftype==2)
{
//--- check
if(!CAp::Assert(s.m_trfage==0 || s.m_trftype!=0,__FUNCTION__+": integrity check failed TrfAge vs TrfType"))
return;
x=r;
x.Resize(m);
for(k=s.m_trfage-1; k>=0; k--)
{
v=0;
for(i=0; i<m; i++)
v+=s.m_densepfieta[k*m+i]*x[i];
x.Set(s.m_rk[k],v);
}
for(i=0; i<m; i++)
tx.Set(i,x[s.m_rowpermbwd[i]]);
x=tx;
if(s.m_trftype==0 || s.m_trftype==1)
{
//--- Dense TRF
CAblas::RMatrixTrsVect(m,s.m_denselu,0,0,false,true,0,x,0);
CAblas::RMatrixTrsVect(m,s.m_denselu,0,0,true,false,0,x,0);
}
else
{
//--- Sparse TRF
CSparse::SparseTRSV(s.m_sparsel,false,false,0,x);
CSparse::SparseTRSV(s.m_sparseu,true,false,0,x);
}
for(i=0; i<m; i++)
tx.Set(s.m_colpermbwd[i],x[i]);
x=tx;
processed=true;
}
//--- Sparse factorization with Forest-Tomlin update
if(s.m_trftype==3)
{
x=r;
x.Resize(m);
for(i=0; i<m; i++)
tx.Set(i,x[s.m_rowpermbwd[i]]);
x=tx;
CSparse::SparseTRSV(s.m_sparsel,false,false,0,x);
for(k=s.m_trfage-1; k>=0; k--)
{
//--- The code below is an amalgamation of two parts:
//--- triangular factor
//--- V:=X[M-1];
//--- for I:=D to M-2 do
//--- X[I]:=X[I]+S.DenseMu[K*M+I]*V;
//--- X[M-1]:=S.DenseMu[K*M+(M-1)]*V;
//--- inverse of cyclic permutation
//--- V:=X[M-1];
//--- for I:=M-1 downto D+1 do
//--- X[I]:=X[I-1];
//--- X[D]:=V;
d=s.m_dk[k];
vm=x[m-1];
v=s.m_densemu[k*m+(m-1)]*vm;
if(vm!=0)
{
//--- X[M-1] is non-zero, apply update
for(i=m-2; i>=d; i--)
x.Set(i+1,x[i]+s.m_densemu[k*m+i]*vm);
}
else
{
//--- X[M-1] is zero, just cyclic permutation
for(i=m-2; i>=d; i--)
x.Set(i+1,x[i]);
}
x.Set(d,v);
}
CSparse::SparseTRSV(s.m_sparseut,false,false,1,x);
for(i=0; i<m; i++)
tx.Set(s.m_colpermbwd[i],x[i]);
x=tx;
processed=true;
}
//--- Integrity check
if(!CAp::Assert(processed,__FUNCTION__+": unsupported TRF type"))
return;
v=x.Sum();
CAp::Assert(MathIsValidNumber(v),__FUNCTION__+": integrity check failed (degeneracy in B?)");
}
//+------------------------------------------------------------------+
//| This function computes product AN*XN, where AN is a non-basic |
//| subset of columns of A, and XN is a non - basic subset of columns|
//| of X. |
//| Output array is reallocated if its size is too small. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::ComputeAnXn(CDualSimplexState &State,
CDualSimplexSubproblem &subproblem,
CRowDouble &x,
CRowDouble &y)
{
//--- create variables
int nx=subproblem.m_ns+subproblem.m_m;
int m=subproblem.m_m;
int nn=nx-m;
int i=0;
int j=0;
int k=0;
int j0=0;
int j1=0;
double v=0;
//--- Integrity check
if(!CAp::Assert(subproblem.m_state>=m_ssvalidxn,"ComputeANXN: XN is invalid"))
return;
//--- Compute
y=vector<double>::Zeros(m);
for(i=0; i<nn; i++)
{
j0=State.m_at.m_RIdx[State.m_basis.m_nidx[i]];
j1=State.m_at.m_RIdx[State.m_basis.m_nidx[i]+1];
v=x[State.m_basis.m_nidx[i]];
for(j=j0; j<j1; j++)
{
k=State.m_at.m_Idx[j];
y.Add(k,v*State.m_at.m_Vals[j]);
}
}
}
//+------------------------------------------------------------------+
//| This function computes product(AN^T)*y, where AN is a non-basic |
//| subset of columns of A, and y is some vector. |
//| Output array is set to full NX - sized length, with basic |
//| components of the output being set to zeros. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::ComputeAnTV(CDualSimplexState &State,
CRowDouble &y,
CRowDouble &r)
{
//--- create variables
int nx=State.m_ns+State.m_m;
int m=State.m_m;
int nn=nx-m;
int i=0;
int j=0;
int j0=0;
int j1=0;
double v=0;
//--- Allocate output, set to zero
r=vector<double>::Zeros(nx);
for(i=0; i<nn; i++)
{
j0=State.m_at.m_RIdx[State.m_basis.m_nidx[i]];
j1=State.m_at.m_RIdx[State.m_basis.m_nidx[i]+1];
v=0;
for(j=j0; j<j1; j++)
v+=State.m_at.m_Vals[j]*y[State.m_at.m_Idx[j]];
r.Set(State.m_basis.m_nidx[i],v);
}
}
//+------------------------------------------------------------------+
//| Returns True if I-th lower bound is present |
//+------------------------------------------------------------------+
bool CRevisedDualSimplex::HasBndL(CDualSimplexSubproblem &subproblem,
int i)
{
int k=subproblem.m_bndt[i];
//--- check
if(k==0 || k==1 || k==3)
return(true);
if(k==2 || k==4)
return(false);
CAp::Assert(false,__FUNCTION__+": integrity check failed");
return(false);
}
//+------------------------------------------------------------------+
//| Returns True if I-th upper bound is present |
//+------------------------------------------------------------------+
bool CRevisedDualSimplex::HasBndU(CDualSimplexSubproblem &subproblem,
int i)
{
int k=subproblem.m_bndt[i];
//--- check
if(k==0 || k==2 || k==3)
return(true);
if(k==1 || k==4)
return(false);
CAp::Assert(false,__FUNCTION__+": integrity check failed");
return(false);
}
//+------------------------------------------------------------------+
//| Returns True if I-th variable if free |
//+------------------------------------------------------------------+
bool CRevisedDualSimplex::IsFree(CDualSimplexSubproblem &subproblem,
int i)
{
int k=subproblem.m_bndt[i];
//--- check
if(k==0 || k==1 || k==2 || k==3)
return(false);
if(k==4)
return(true);
CAp::Assert(false,__FUNCTION__+": integrity check failed");
return(false);
}
//+------------------------------------------------------------------+
//| Downgrades problem State to the specified one(if status is lower |
//| than one specified by user, nothing is changed) |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::DowngradeState(CDualSimplexSubproblem &subproblem,
int s)
{
subproblem.m_state=MathMin(subproblem.m_state,s);
}
//+------------------------------------------------------------------+
//| Returns maximum dual infeasibility(only non - basic variables are|
//| checked, we assume that basic variables are good enough). |
//+------------------------------------------------------------------+
double CRevisedDualSimplex::DualFeasibilityError(CDualSimplexState &State,
CDualSimplexSubproblem &s)
{
//--- create variables
int nn=s.m_ns;
double result=0;
int i=0;
int j=0;
int bndt=0;
//--- check
if(!CAp::Assert(s.m_state==m_ssvalid,__FUNCTION__+": invalid X"))
return(0);
for(i=0; i<nn; i++)
{
j=State.m_basis.m_nidx[i];
bndt=s.m_bndt[j];
if(bndt==m_ccfixed)
continue;
if(bndt==m_ccrange)
{
if(s.m_xa[j]==s.m_bndl[j])
{
result=MathMax(result,-s.m_d[j]);
continue;
}
if(s.m_xa[j]==s.m_bndu[j])
{
result=MathMax(result,s.m_d[j]);
continue;
}
CAp::Assert(false,__FUNCTION__+": integrity check failed");
return(0);
}
if(bndt==m_cclower)
{
//--- check
if(!CAp::Assert(s.m_xa[j]==s.m_bndl[j],__FUNCTION__+": integrity check failed"))
return(0);
result=MathMax(result,-s.m_d[j]);
continue;
}
if(bndt==m_ccupper)
{
//--- check
if(!CAp::Assert(s.m_xa[j]==s.m_bndu[j],__FUNCTION__+": integrity check failed"))
return(0);
result=MathMax(result,s.m_d[j]);
continue;
}
if(bndt==m_ccfree)
{
result=MathMax(result,MathAbs(s.m_d[j]));
continue;
}
CAp::Assert(false,__FUNCTION__+": integrity check failed (infeasible constraint)");
return(0);
}
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Returns True for dual feasible basis(some minor dual feasibility |
//| error is allowed), False otherwise |
//+------------------------------------------------------------------+
bool CRevisedDualSimplex::IsDualFeasible(CDualSimplexState &State,
CDualSimplexSubproblem &s,
CDualSimplexSettings &Settings)
{
return (DualFeasibilityError(State,s)<=Settings.m_dtolabs);
}
//+------------------------------------------------------------------+
//| Transforms sequence of pivot permutations P0*P1*...*Pm to |
//| forward / backward permutation representation. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::PivotToBWD(CRowInt &p,
int m,
CRowInt &bwd)
{
//--- create variables
int i=0;
int k=0;
int t=0;
bwd.Resize(m);
for(i=0; i<m; i++)
bwd.Set(i,i);
for(i=0; i<m; i++)
{
k=p[i];
if(k!=i)
bwd.Swap(i,k);
}
}
//+------------------------------------------------------------------+
//| Applies inverse cyclic permutation of [D, M - 1) (element D is |
//| moved to the end, the rest of elements is shifted one position |
//| backward) to the already existing permutation. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::InverseCyclicPermutation(CRowInt &bwd,
int m,
int d,
CRowInt &tmpi)
{
//--- create variables
int i=0;
int k=0;
//--- update Bwd[]
k=bwd[d];
for(i=d; i<m-1; i++)
bwd.Set(i,bwd[i+1]);
bwd.Set(m-1,k);
}
//+------------------------------------------------------------------+
//| Offloads basic components of X[], BndT[], BndL[], BndU[] to |
//| XB / BndTB / BndLB / BndUB. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::OffloadBasicComponents(CDualSimplexSubproblem &s,
CDualSimplexBasis &basis,
CDualSimplexSettings &Settings)
{
int m=basis.m_m;
for(int i=0; i<m; i++)
{
s.m_xb.Set(i,s.m_xa[basis.m_idx[i]]);
CacheBoundInfo(s,i,basis.m_idx[i],Settings);
}
}
//+------------------------------------------------------------------+
//| Recombines basic and non - basic components in X[] |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::RecombineBasicNonBasicX(CDualSimplexSubproblem &s,
CDualSimplexBasis &basis)
{
int m=basis.m_m;
//--- copy
for(int i=0; i<m; i++)
s.m_xa.Set(basis.m_idx[i],s.m_xb[i]);
}
//+------------------------------------------------------------------+
//| Computes Stats array |
//| INPUT PARAMETERS: |
//| S - problem, contains current solution at S.XA |
//| Basis - basis |
//| X - possibly preallocated output buffer |
//| LagBC - possibly preallocated output buffer |
//| LagLC - possibly preallocated output buffer |
//| Stats - possibly preallocated output buffer |
//| Buffers - temporary buffers |
//| OUTPUT PARAMETERS: |
//| X - array[NS], solution |
//| LagBC - array[NS], Lagrange multipliers for box constraints|
//| LagLC - array[M], Lagrange multipliers for linear |
//| constraints |
//| Stats - array[NS + M], primary / slack variable stats: |
//| * -1 = variable at lower bound |
//| * +1 = variable at upper bound |
//| * 0 = basic or free (possibly nonbasic) |
//| variable fixed variables may be set |
//| to + 1 or - 1 |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::SetXYDStats(CDualSimplexState &State,
CDualSimplexSubproblem &s,
CDualSimplexBasis &basis,
CApBuff &buffers,
CRowDouble &x,
CRowDouble &lagbc,
CRowDouble &laglc,
CRowInt &stats)
{
//--- create variables
int m=s.m_m;
int ns=s.m_ns;
int nx=s.m_ns+s.m_m;
int i=0;
int j=0;
//--- Prepare
x.Resize(ns);
laglc.Resize(m);
stats.Resize(nx);
lagbc=vector<double>::Zeros(ns);
//--- Compute Y (in Buffers.RA1) and D (in Buffers.RA3)
buffers.m_ra0.Resize(m);
buffers.m_ra1.Resize(m);
buffers.m_ra3.Resize(nx);
for(i=0; i<m; i++)
buffers.m_ra0.Set(i,s.m_rawc[basis.m_idx[i]]);
BasisSolveT(basis,buffers.m_ra0,buffers.m_ra1,buffers.m_ra2);
ComputeAnTV(State,buffers.m_ra1,buffers.m_ra3);
for(i=0; i<ns; i++)
{
j=State.m_basis.m_nidx[i];
buffers.m_ra3.Set(j,State.m_primary.m_rawc[j]-buffers.m_ra3[j]);
if(j<ns)
lagbc.Set(j,-buffers.m_ra3[j]);
}
for(i=0; i<m; i++)
buffers.m_ra3.Set(State.m_basis.m_idx[i],0);
//--- Compute X, Y, Stats
x=s.m_xa;
for(i=0; i<ns; i++)
{
if(MathIsValidNumber(State.m_rawbndl[i]))
x.Set(i,MathMax(x[i],State.m_rawbndl[i]));
if(MathIsValidNumber(State.m_rawbndu[i]))
x.Set(i,MathMin(x[i],State.m_rawbndu[i]));
}
for(i=0; i<ns; i++)
{
if(basis.m_isbasic[i])
{
lagbc.Set(i,0);
continue;
}
if(s.m_bndt[i]==m_ccfixed)
continue;
if(HasBndL(s,i) && s.m_xa[i]==s.m_bndl[i])
{
lagbc.Set(i,MathMin(lagbc[i],0.0));
continue;
}
if(HasBndU(s,i) && s.m_xa[i]==s.m_bndu[i])
{
lagbc.Set(i,MathMax(lagbc[i],0.0));
continue;
}
//--- check
if(!CAp::Assert(!HasBndL(s,i) && !HasBndU(s,i),__FUNCTION__+": integrity check failed (zetta5)"))
return;
lagbc.Set(i,0);
}
for(i=0; i<m; i++)
laglc.Set(i,-(buffers.m_ra1[i]/State.m_rowscales[i]));
for(i=0; i<nx; i++)
{
if(basis.m_isbasic[i])
{
stats.Set(i,0);
continue;
}
if(HasBndL(s,i) && s.m_xa[i]==s.m_bndl[i])
{
stats.Set(i,-1);
continue;
}
if(HasBndU(s,i) && s.m_xa[i]==s.m_bndu[i])
{
stats.Set(i,1);
continue;
}
//--- check
if(!CAp::Assert(!HasBndL(s,i) && !HasBndU(s,i),__FUNCTION__+": integrity check failed (zetta5)"))
return;
stats.Set(i,0);
}
}
//+------------------------------------------------------------------+
//| Initializes vector, sets all internal arrays to length N (so that|
//| we may store any vector without reallocation). Previously |
//| allocated memory is reused as much as possible. |
//| No zero - filling is performed, X.K is undefined. Only X.N is set|
//| INPUT PARAMETERS: |
//| X - temporary buffers |
//| OUTPUT PARAMETERS: |
//| X - preallocated vector, X.N = N, contents undefined|
//+------------------------------------------------------------------+
void CRevisedDualSimplex::DVAlloc(CDSSVector &x,int n)
{
x.m_idx.Resize(n);
x.m_vals.Resize(n);
x.m_dense.Resize(n);
x.m_n=n;
}
//+------------------------------------------------------------------+
//| Initializes vector, sets all internal arrays to length N and |
//| zero - fills them. Previously allocated memory is reused as much |
//| as possible. |
//| INPUT PARAMETERS: |
//| X - temporary buffers |
//| OUTPUT PARAMETERS: |
//| X - preallocated vector: |
//| * X.N = N |
//| * X.K = 0 |
//| * X.Dense is zero - filled. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::DVInit(CDSSVector &x,int n)
{
x.m_idx.Resize(n);
x.m_vals.Resize(n);
x.m_dense=vector<double>::Zeros(n);
x.m_n=n;
x.m_k=0;
}
//+------------------------------------------------------------------+
//| Copies dense part to sparse one. |
//| INPUT PARAMETERS: |
//| X - allocated vector; dense part must be valid |
//| OUTPUT PARAMETERS: |
//| X - both dense and sparse parts are valid. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::DVDenseToSparse(CDSSVector &x)
{
//--- create variables
int n=x.m_n;
int k=0;
double v=0;
x.m_idx.Resize(n);
x.m_vals.Resize(n);
for(int i=0; i<n; i++)
{
v=x.m_dense[i];
if(v!=0.0)
{
x.m_idx.Set(k,i);
x.m_vals.Set(k,v);
k++;
}
}
x.m_k=k;
}
//+------------------------------------------------------------------+
//| Copies sparse part to dense one. |
//| INPUT PARAMETERS: |
//| X - allocated vector; sparse part must be valid |
//| OUTPUT PARAMETERS: |
//| X - both dense and sparse parts are valid. |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::DVSparseToDense(CDSSVector &x)
{
//--- create variables
int n=x.m_n;
int k=x.m_k;
x.m_dense=vector<double>::Zeros(n);
for(int i=0; i<k; i++)
x.m_dense.Set(x.m_idx[i],x.m_vals[i]);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
double CRevisedDualSimplex::SparsitYOf(CRowDouble &x,int n)
{
//--- create variables
double result=0;
int i=0;
int k=0;
double mx=1.0;
//--- Quick exit
if(n<=1)
return(0);
for(i=0; i<n; i++)
mx=MathMax(mx,MathAbs(x[i]));
mx=1.0E5*CMath::m_machineepsilon*mx;
k=0;
for(i=0; i<n; i++)
if(MathAbs(x[i])>mx)
k++;
result=(double)k/(double)n;
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CRevisedDualSimplex::UpdateAvgCounter(double v,
double &acc,
int &cnt)
{
acc=acc+v;
cnt=cnt+1;
}
//+------------------------------------------------------------------+
//| This object stores linear solver State. |
//| You should use functions provided by MinLP subpackage to work |
//| with this object |
//+------------------------------------------------------------------+
struct CMinLPState
{
int m_algokind;
int m_m;
int m_n;
int m_repiterationscount;
int m_repm;
int m_repn;
int m_repterminationtype;
double m_dsseps;
double m_ipmeps;
double m_ipmlambda;
double m_repdualerror;
double m_repf;
double m_repprimalerror;
double m_repslackerror;
CVIPMState m_ipm;
CSparseMatrix m_a;
CSparseMatrix m_ipmquadratic;
CRowInt m_adddtmpi;
CRowInt m_cs;
CRowDouble m_adddtmpr;
CRowDouble m_al;
CRowDouble m_au;
CRowDouble m_bndl;
CRowDouble m_bndu;
CRowDouble m_c;
CRowDouble m_lagbc;
CRowDouble m_laglc;
CRowDouble m_s;
CRowDouble m_tmpax;
CRowDouble m_tmpg;
CRowDouble m_units;
CRowDouble m_xs;
CRowDouble m_zeroorigin;
CPresolveInfo m_presolver;
CDualSimplexState m_dss;
//--- constructor / destructor
CMinLPState(void);
~CMinLPState(void) {}
//---
void Copy(const CMinLPState &obj);
//--- overloading
void operator=(const CMinLPState &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinLPState::CMinLPState(void)
{
m_algokind=0;
m_m=0;
m_n=0;
m_repiterationscount=0;
m_repm=0;
m_repn=0;
m_repterminationtype=0;
m_dsseps=0;
m_ipmeps=0;
m_ipmlambda=0;
m_repdualerror=0;
m_repf=0;
m_repprimalerror=0;
m_repslackerror=0;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinLPState::Copy(const CMinLPState &obj)
{
m_algokind=obj.m_algokind;
m_m=obj.m_m;
m_n=obj.m_n;
m_repiterationscount=obj.m_repiterationscount;
m_repm=obj.m_repm;
m_repn=obj.m_repn;
m_repterminationtype=obj.m_repterminationtype;
m_dsseps=obj.m_dsseps;
m_ipmeps=obj.m_ipmeps;
m_ipmlambda=obj.m_ipmlambda;
m_repdualerror=obj.m_repdualerror;
m_repf=obj.m_repf;
m_repprimalerror=obj.m_repprimalerror;
m_repslackerror=obj.m_repslackerror;
m_ipm=obj.m_ipm;
m_a=obj.m_a;
m_ipmquadratic=obj.m_ipmquadratic;
m_adddtmpi=obj.m_adddtmpi;
m_cs=obj.m_cs;
m_adddtmpr=obj.m_adddtmpr;
m_al=obj.m_al;
m_au=obj.m_au;
m_bndl=obj.m_bndl;
m_bndu=obj.m_bndu;
m_c=obj.m_c;
m_lagbc=obj.m_lagbc;
m_laglc=obj.m_laglc;
m_s=obj.m_s;
m_tmpax=obj.m_tmpax;
m_tmpg=obj.m_tmpg;
m_units=obj.m_units;
m_xs=obj.m_xs;
m_zeroorigin=obj.m_zeroorigin;
m_presolver=obj.m_presolver;
m_dss=obj.m_dss;
}
//+------------------------------------------------------------------+
//| This structure stores optimization report: |
//| * f target function value |
//| * lagbc Lagrange coefficients for box constraints |
//| * laglc Lagrange coefficients for linear constraints |
//| * y dual variables |
//| * stats array[N + M], statuses of box(N) and linear(M) |
//| constraints. This array is filled only by DSS |
//| algorithm because IPM always stops at INTERIOR |
//| point: |
//| * stats[i] > 0 => constraint at upper bound |
//| (also used for free |
//| non-basic variables set |
//| to zero) |
//| * stats[i] < 0 => constraint at lower bound |
//| * stats[i] = 0 => constraint is inactive, |
//| basic variable |
//| * primalerror primal feasibility error |
//| * dualerror dual feasibility error |
//| * slackerror complementary slackness error |
//| * iterationscount iteration count |
//| * terminationtype completion code(see below) |
//| COMPLETION CODES: |
//| * -4 LP problem is primal unbounded(dual infeasible) |
//| * -3 LP problem is primal infeasible(dual unbounded) |
//| * 1..4 successful completion |
//| * 5 MaxIts steps was taken |
//| * 7 stopping conditions are too stringent, further |
//| improvement is impossible, X contains best point|
//| found so far. |
//| LAGRANGE COEFFICIENTS: |
//| Positive Lagrange coefficient means that constraint is at its |
//| upper bound. Negative coefficient means that constraint is at |
//| its lower bound. It is expected that at solution the dual |
//| feasibility condition holds: |
//| C+SUM(Ei*LagBC[i], i=0..m_n-1)+SUM(Ai*LagLC[i], i=0..m_m-1) ~ 0 |
//| where: |
//| * C is a cost vector(linear term) |
//| * Ei is a vector with 1.0 at position I and 0 in other |
//| positions |
//| * Ai is an I-th row of linear constraint matrix |
//+------------------------------------------------------------------+
struct CMinLPReport
{
int m_iterationscount;
int m_terminationtype;
double m_dualerror;
double m_f;
double m_primalerror;
double m_slackerror;
CRowInt m_stats;
CRowDouble m_lagbc;
CRowDouble m_laglc;
CRowDouble m_y;
//--- constructor / destructor
CMinLPReport(void);
~CMinLPReport(void) {}
void Copy(const CMinLPReport &obj);
//--- overloading
void operator=(const CMinLPReport &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinLPReport::CMinLPReport(void)
{
m_iterationscount=0;
m_terminationtype=0;
m_dualerror=0;
m_f=0;
m_primalerror=0;
m_slackerror=0;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinLPReport::Copy(const CMinLPReport &obj)
{
m_iterationscount=obj.m_iterationscount;
m_terminationtype=obj.m_terminationtype;
m_dualerror=obj.m_dualerror;
m_f=obj.m_f;
m_primalerror=obj.m_primalerror;
m_slackerror=obj.m_slackerror;
m_stats=obj.m_stats;
m_lagbc=obj.m_lagbc;
m_laglc=obj.m_laglc;
m_y=obj.m_y;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CMinLP
{
public:
//--- constants
static const int m_alllogicalsbasis;
static void MinLPCreate(int n,CMinLPState &State);
static void MinLPSetAlgoDSS(CMinLPState &State,double eps);
static void MinLPSetAlgoIPM(CMinLPState &State,double eps);
static void MinLPSetCost(CMinLPState &State,CRowDouble &c);
static void MinLPSetScale(CMinLPState &State,CRowDouble &s);
static void MinLPSetBC(CMinLPState &State,CRowDouble &bndl,CRowDouble &bndu);
static void MinLPSetBCAll(CMinLPState &State,double bndl,double bndu);
static void MinLPSetBCi(CMinLPState &State,int i,double bndl,double bndu);
static void MinLPSetLC(CMinLPState &State,CMatrixDouble &a,CRowInt &ct,int k);
static void MinLPSetLC2Dense(CMinLPState &State,CMatrixDouble &a,CRowDouble &al,CRowDouble &au,int k);
static void MinLPSetLC2(CMinLPState &State,CSparseMatrix &a,CRowDouble &al,CRowDouble &au,int k);
static void MinLPAddLC2Dense(CMinLPState &State,CRowDouble &a,double al,double au);
static void MinLPAddLC2(CMinLPState &State,CRowInt &idxa,CRowDouble &vala,int nnz,double al,double au);
static void MinLPOptimize(CMinLPState &State);
static void MinLPResults(CMinLPState &State,CRowDouble &x,CMinLPReport &rep);
static void MinLPResultsBuf(CMinLPState &State,CRowDouble &x,CMinLPReport &rep);
private:
static void ClearReportFields(CMinLPState &State);
};
//+------------------------------------------------------------------+
//| Constants |
//+------------------------------------------------------------------+
const int CMinLP::m_alllogicalsbasis=0;
//+------------------------------------------------------------------+
//| LINEAR PROGRAMMING |
//| The subroutine creates LP solver. After initial creation it |
//| contains default optimization problem with zero cost vector and |
//| all variables being fixed to zero values and no constraints. |
//| In order to actually solve something you should: |
//| * set cost vector with MinLPSetCost() |
//| * set variable bounds with MinLPSetBC() or MinLPSetBCAll() |
//| * specify constraint matrix with one of the following |
//| functions: |
//| [*] MinLPSetLC() for dense one-sided constraints |
//| [*] MinLPSetLC2Dense() for dense two-sided constraints |
//| [*] MinLPSetLC2() for sparse two-sided constraints |
//| [*] MinLPAddLC2Dense() to add one dense row to constraint |
//| matrix |
//| [*] MinLPAddLC2() to add one row to constraint matrix |
//| (compressed format) |
//| * call MinLPOptimize() to run the solver and MinLPResults() to |
//| get the solution vector and additional information. |
//| By default, LP solver uses best algorithm available. As of ALGLIB|
//| 3.17, sparse interior point (barrier) solver is used. Future |
//| releases of ALGLIB may introduce other solvers. |
//| User may choose specific LP algorithm by calling: |
//| * MinLPSetAlgoDSS() for revised dual simplex method with DSE |
//| pricing and bounds flipping ratio test (aka long dual step). |
//| Large - scale sparse LU solverwith Forest - Tomlin update is |
//| used internally as linear algebra driver. |
//| * MinLPSetAlgoIPM() for sparse interior point method |
//| INPUT PARAMETERS: |
//| N - problem size |
//| OUTPUT PARAMETERS: |
//| State - optimizer in the default State |
//+------------------------------------------------------------------+
void CMinLP::MinLPCreate(int n,CMinLPState &State)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
//--- Initialize
State.m_n=n;
State.m_m=0;
MinLPSetAlgoIPM(State,0.0);
State.m_ipmlambda=0;
State.m_c=vector<double>::Zeros(n);
State.m_s=vector<double>::Ones(n);
State.m_bndl=vector<double>::Zeros(n);
State.m_bndu=vector<double>::Zeros(n);
State.m_xs=vector<double>::Ones(n);
ClearReportFields(State);
}
//+------------------------------------------------------------------+
//| This function sets LP algorithm to revised dual simplex method. |
//| ALGLIB implementation of dual simplex method supports advanced |
//| performance and stability improvements like DSE pricing, bounds |
//| flipping ratio test (aka long dual step), Forest - Tomlin update,|
//| Shifting. |
//| INPUT PARAMETERS: |
//| State - optimizer |
//| Eps - stopping condition, Eps >= 0: |
//| * should be small number about 1E-6 or 1E-7. |
//| * zero value means that solver automatically |
//| selects good value (can be different in |
//| different ALGLIB versions) |
//| * default value is zero |
//| Algorithm stops when relative error is less than Eps. |
//| ===== TRACING DSS SOLVER ======================================= |
//| DSS solver supports advanced tracing capabilities. You can trace |
//| algorithm output by specifying following trace symbols |
//| (case-insensitive) by means of trace_file() call: |
//| * 'DSS' - for basic trace of algorithm steps and decisions|
//| Only short scalars (function values and deltas) |
//| are printed. N-dimensional quantities like |
//| search directions are NOT printed. |
//| * 'DSS.DETAILED' - for output of points being visited and |
//| search directions. |
//| This symbol also implicitly defines 'DSS'. You can control output|
//| format by additionally specifying: |
//| * nothing to output in 6-digit exponential format |
//| * 'PREC.E15' to output in 15-digit exponential format |
//| * 'PREC.F6' to output in 6-digit fixed - point format |
//| By default trace is disabled and adds no overhead to the |
//| optimization process. However, specifying any of the symbols |
//| adds some formatting and output - related overhead. |
//| You may specify multiple symbols by separating them with commas: |
//| > |
//| >CAlglib::Trace_File("DSS,PREC.F6", "path/to/trace.log") |
//+------------------------------------------------------------------+
void CMinLP::MinLPSetAlgoDSS(CMinLPState &State,
double eps)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(eps),__FUNCTION__+": Eps is not finite number"))
return;
if(!CAp::Assert(eps>=0.0,__FUNCTION__+": Eps<0"))
return;
State.m_algokind=1;
if(eps==0.0)
eps=1.0E-6;
State.m_dsseps=eps;
}
//+------------------------------------------------------------------+
//| This function sets LP algorithm to sparse interior point method. |
//| ALGORITHM INFORMATION: |
//| * this algorithm is our implementation of interior point |
//| method as formulated by R.J.Vanderbei, with minor |
//| modifications to the algorithm (damped Newton directions are |
//| extensively used) |
//| * like all interior point methods, this algorithm tends to |
//| converge in roughly same number of iterations (between 15 |
//| and 50) independently from the problem dimensionality |
//| INPUT PARAMETERS: |
//| State - optimizer |
//| Eps - stopping condition, Eps >= 0: |
//| * should be small number about 1E-7 or 1E-8. |
//| * zero value means that solver automatically |
//| selects good value (can be different in different|
//| ALGLIB versions) |
//| * default value is zero |
//| Algorithm stops when primal error AND dual error AND |
//| duality gap are less than Eps. |
//| ===== TRACING IPM SOLVER ======================================= |
//| IPM solver supports advanced tracing capabilities. You can trace |
//| algorithm output by specifying following trace symbols |
//| (case-insensitive) by means of trace_file() call: |
//| * 'IPM' - for basic trace of algorithm steps and |
//| decisions. Only short scalars (function |
//| values and deltas) are printed. N-dimensional|
//| quantities like search directions are NOT |
//| printed. |
//| * 'IPM.DETAILED' - for output of points being visited and |
//| search directions |
//| This symbol also implicitly defines 'IPM'. You can output format |
//| by additionally specifying: |
//| * nothing to output in 6-digit exponential format |
//| * 'PREC.E15' to output in 15 - digit exponential format |
//| * 'PREC.F6' to output in 6-digit fixed-point format |
//| By default trace is disabled and adds no overhead to the |
//| optimization process. However, specifying any of the symbols |
//| adds some formatting and output - related overhead. |
//| You may specify multiple symbols by separating them with commas: |
//| > |
//| >CAlglib::Trace_File("IPM,PREC.F6", "path/to/trace.log") |
//+------------------------------------------------------------------+
void CMinLP::MinLPSetAlgoIPM(CMinLPState &State,double eps)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(eps),__FUNCTION__+": Eps is not finite number"))
return;
if(!CAp::Assert(eps>=0.0,__FUNCTION__+": Eps<0"))
return;
State.m_algokind=2;
State.m_ipmeps=eps;
State.m_ipmlambda=0.0;
}
//+------------------------------------------------------------------+
//| This function sets cost term for LP solver. |
//| By default, cost term is zero. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| C - cost term, array[N]. |
//+------------------------------------------------------------------+
void CMinLP::MinLPSetCost(CMinLPState &State,CRowDouble &c)
{
int n=State.m_n;
//--- check
if(!CAp::Assert(c.Size()>=n,__FUNCTION__+": Length(C)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(c,n),__FUNCTION__+": C contains infinite or NaN elements"))
return;
State.m_c=c;
State.m_c.Resize(n);
}
//+------------------------------------------------------------------+
//| This function sets scaling coefficients. |
//| ALGLIB optimizers use scaling matrices to test stopping |
//| conditions and as preconditioner. |
//| Scale of the I-th variable is a translation invariant measure of:|
//| a) "how large" the variable is |
//| b) how large the step should be to make significant changes in |
//| the function |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| S - array[N], non-zero scaling coefficients S[i] may |
//| be negative, sign doesn't matter. |
//+------------------------------------------------------------------+
void CMinLP::MinLPSetScale(CMinLPState &State,CRowDouble &s)
{
if(!CAp::Assert(CAp::Len(s)>=State.m_n,__FUNCTION__+": Length(S)<N"))
return;
for(int i=0; i<State.m_n; i++)
{
if(!CAp::Assert(MathIsValidNumber(s[i]),__FUNCTION__+": S contains infinite or NAN elements"))
return;
if(!CAp::Assert(s[i]!=0.0,__FUNCTION__+": S contains zero elements"))
return;
}
State.m_s=s.Abs()+0;
}
//+------------------------------------------------------------------+
//| This function sets box constraints for LP solver (all variables |
//| at once, different constraints for different variables). |
//| The default State of constraints is to have all variables fixed |
//| at zero. You have to overwrite it by your own constraint vector. |
//| Constraint status is preserved until constraints are explicitly |
//| overwritten with another MinLPSetBC() call, overwritten with |
//| MinLPSetBCAll(), or partially overwritten with minlmsetbci() call|
//| Following types of constraints are supported: |
//| DESCRIPTION CONSTRAINT HOW TO SPECIFY |
//| fixed variable x[i] = Bnd[i] BndL[i] = BndU[i]|
//| lower bound BndL[i] <= x[i] BndU[i] = +INF |
//| upper bound x[i] <= BndU[i] BndL[i] = -INF |
//| range BndL[i] <= x[i] <= BndU[i] ... |
//| free variable - BndL[I] = -INF, BndU[I] + INF|
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| BndL - lower bounds, array[N]. |
//| BndU - upper bounds, array[N]. |
//| NOTE: infinite values can be specified by means of AL_POSINF and |
//| AL_NEGINF |
//| NOTE: you may replace infinities by very small/very large values,|
//| but it is not recommended because large numbers may |
//| introduce large numerical errors in the algorithm. |
//| NOTE: if constraints for all variables are same you may use |
//| MinLPSetBCAll() which allows to specify constraints without|
//| using arrays. |
//| NOTE: BndL > BndU will result in LP problem being recognized as |
//| infeasible. |
//+------------------------------------------------------------------+
void CMinLP::MinLPSetBC(CMinLPState &State,CRowDouble &bndl,CRowDouble &bndu)
{
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(bndl)>=n,__FUNCTION__+": Length(BndL)<N"))
return;
if(!CAp::Assert(CAp::Len(bndu)>=n,__FUNCTION__+": Length(BndU)<N"))
return;
for(int i=0; i<n; i++)
{
if(!CAp::Assert(MathIsValidNumber(bndl[i]) || IsNegInf(bndl[i]),__FUNCTION__+": BndL contains NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(bndu[i]) || IsPosInf(bndu[i]),__FUNCTION__+": BndU contains NAN or -INF"))
return;
}
State.m_bndl=bndl;
State.m_bndu=bndu;
}
//+------------------------------------------------------------------+
//| This function sets box constraints for LP solver(all variables at|
//| once, same constraints for all variables) |
//| The default State of constraints is to have all variables fixed |
//| at zero. You have to overwrite it by your own constraint vector. |
//| Constraint status is preserved until constraints are explicitly |
//| overwritten with another MinLPSetBC() call or partially |
//| overwritten with MinLPSetBCAll(). |
//| Following types of constraints are supported: |
//| DESCRIPTION CONSTRAINT HOW TO SPECIFY |
//| fixed variable x[i] = Bnd[i] BndL[i] = BndU[i]|
//| lower bound BndL[i] <= x[i] BndU[i] = +INF |
//| upper bound x[i] <= BndU[i] BndL[i] = -INF |
//| range BndL[i] <= x[i] <= BndU[i] ... |
//| free variable - BndL[I] = -INF, BndU[I] + INF|
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| BndL - lower bound, same for all variables |
//| BndU - upper bound, same for all variables |
//| NOTE: infinite values can be specified by means of AL_POSINF and |
//| AL_NEGINF |
//| NOTE: you may replace infinities by very small/very large values,|
//| but it is not recommended because large numbers may |
//| introduce large numerical errors in the algorithm. |
//| NOTE: MinLPSetBC() can be used to specify different constraints |
//| for different variables. |
//| NOTE: BndL > BndU will result in LP problem being recognized as |
//| infeasible. |
//+------------------------------------------------------------------+
void CMinLP::MinLPSetBCAll(CMinLPState &State,double bndl,double bndu)
{
int n=State.m_n;
//--- check
if(!CAp::Assert(MathIsValidNumber(bndl) || IsNegInf(bndl),__FUNCTION__+": BndL is NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(bndu) || IsPosInf(bndu),__FUNCTION__+": BndU is NAN or -INF"))
return;
State.m_bndl.Fill(bndl);
State.m_bndu.Fill(bndu);
}
//+------------------------------------------------------------------+
//| This function sets box constraints for I-th variable (other |
//| variables are not modified). |
//| The default State of constraints is to have all variables fixed |
//| at zero. You have to overwrite it by your own constraint vector. |
//| Following types of constraints are supported: |
//| DESCRIPTION CONSTRAINT HOW TO SPECIFY |
//| fixed variable x[i] = Bnd[i] BndL[i] = BndU[i]|
//| lower bound BndL[i] <= x[i] BndU[i] = +INF |
//| upper bound x[i] <= BndU[i] BndL[i] = -INF |
//| range BndL[i] <= x[i] <= BndU[i] ... |
//| free variable - BndL[I] = -INF, BndU[I] + INF|
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| I - variable index, in [0, N) |
//| BndL - lower bound for I-th variable |
//| BndU - upper bound for I-th variable |
//| NOTE: infinite values can be specified by means of AL_POSINF and |
//| AL_NEGINF |
//| NOTE: you may replace infinities by very small/very large values,|
//| but it is not recommended because large numbers may |
//| introduce large numerical errors in the algorithm. |
//| NOTE: MinLPSetBC() can be used to specify different constraints |
//| for different variables. |
//| NOTE: BndL > BndU will result in LP problem being recognized as |
//| infeasible. |
//+------------------------------------------------------------------+
void CMinLP::MinLPSetBCi(CMinLPState &State,int i,double bndl,double bndu)
{
int n=State.m_n;
//--- check
if(!CAp::Assert(i>=0 && i<n,__FUNCTION__+": I is outside of [0,N)"))
return;
if(!CAp::Assert(MathIsValidNumber(bndl) || IsNegInf(bndl),__FUNCTION__+": BndL is NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(bndu) || IsPosInf(bndu),__FUNCTION__+": BndU is NAN or -INF"))
return;
State.m_bndl.Set(i,bndl);
State.m_bndu.Set(i,bndu);
}
//+------------------------------------------------------------------+
//| This function sets one-sided linear constraints A*x ~ AU, where |
//| "~" can be a mix of "<=", "=" and ">=". |
//| IMPORTANT: this function is provided here for compatibility with |
//| the rest of ALGLIB optimizers which accept constraints|
//| in format like this one. Many real-life problems |
//| feature two-sided constraints like a0 <= a*x <= a1. It|
//| is really inefficient to add them as a pair of |
//| one-sided constraints. |
//| Use MinLPSetLC2Dense(), MinLPSetLC2(), MinLPAddLC2() (or its |
//| sparse version) wherever possible. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinLPCreate() |
//| call. |
//| A - linear constraints, array[K, N + 1]. Each row of A |
//| represents one constraint, with first N elements |
//| being linear coefficients, and last element being |
//| right side. |
//| CT - constraint types, array[K]: |
//| * if CT[i] > 0, then I-th constraint is |
//| A[i, *]*x >= A[i, n] |
//| * if CT[i] = 0, then I-th constraint is |
//| A[i, *] * x = A[i, n] |
//| * if CT[i] < 0, then I-th constraint is |
//| A[i, *] * x <= A[i, n] |
//| K - number of equality/inequality constraints, K >= 0; |
//| if not given, inferred from sizes of A and CT. |
//+------------------------------------------------------------------+
void CMinLP::MinLPSetLC(CMinLPState &State,CMatrixDouble &a,CRowInt &ct,
int k)
{
//--- create variables
CRowDouble al;
CRowDouble au;
int n=State.m_n;
//--- check
if(!CAp::Assert(k>=0,__FUNCTION__+": K<0"))
return;
if(!CAp::Assert(k==0 || CAp::Cols(a)>=n+1,__FUNCTION__+": Cols(A)<N+1"))
return;
if(!CAp::Assert(CAp::Rows(a)>=k,__FUNCTION__+": Rows(A)<K"))
return;
if(!CAp::Assert(CAp::Len(ct)>=k,__FUNCTION__+": Length(CT)<K"))
return;
if(!CAp::Assert(CApServ::IsFiniteMatrix(a,k,n+1),__FUNCTION__+": A contains infinite or NaN values!"))
return;
//--- Handle zero K
if(k==0)
{
State.m_m=0;
return;
}
//--- Convert constraints to two-sided storage format, call another function
al.Resize(k);
au.Resize(k);
for(int i=0; i<k; i++)
{
if(ct[i]>0)
{
al.Set(i,a.Get(i,n));
au.Set(i,AL_POSINF);
continue;
}
if(ct[i]<0)
{
al.Set(i,AL_NEGINF);
au.Set(i,a.Get(i,n));
continue;
}
al.Set(i,a.Get(i,n));
au.Set(i,a.Get(i,n));
}
MinLPSetLC2Dense(State,a,al,au,k);
}
//+------------------------------------------------------------------+
//| This function sets two-sided linear constraints AL <= A*x <= AU. |
//| This version accepts dense matrix as input; internally LP solver|
//| uses sparse storage anyway (most LP problems are sparse), but for|
//| your convenience it may accept dense inputs. This function |
//| overwrites linear constraints set by previous calls (if such |
//| calls were made). |
//| We recommend you to use sparse version of this function unless |
//| you solve small-scale LP problem (less than few hundreds of |
//| variables). |
//| NOTE: there also exist several versions of this function: |
//| * one-sided dense version which accepts constraints in the same|
//| format as one used by QP and NLP solvers |
//| * two-sided sparse version which accepts sparse matrix |
//| * two-sided dense version which allows you to add constraints |
//| row by row |
//| * two-sided sparse version which allows you to add constraints |
//| row by row |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinLPCreate() |
//| call. |
//| A - linear constraints, array[K, N]. Each row of A |
//| represents one constraint. One-sided inequality |
//| constraints, two-sided inequality constraints, |
//| equality constraints are supported (see below) |
//| AL, AU - lower and upper bounds, array[K]; |
//| * AL[i] = AU[i] => equality constraint Ai * x |
//| * AL[i]<AU[i] => two-sided constraint |
//| AL[i] <= Ai*x <= AU[i] |
//| * AL[i] = -INF => one-sided constraint |
//| Ai*x <= AU[i] |
//| * AU[i] = +INF => one-sided constraint |
//| AL[i] <= Ai*x |
//| * AL[i] = -INF, AU[i] = +INF => constraint is |
//| ignored |
//| K - number of equality/inequality constraints, K >= 0; |
//| if not given, inferred from sizes of A, AL, AU. |
//+------------------------------------------------------------------+
void CMinLP::MinLPSetLC2Dense(CMinLPState &State,CMatrixDouble &a,
CRowDouble &al,CRowDouble &au,int k)
{
//--- create variables
int n=State.m_n;
int i=0;
int j=0;
int nz=0;
CRowInt nrs;
//--- check
if(!CAp::Assert(k>=0,__FUNCTION__+": K<0"))
return;
if(!CAp::Assert(k==0 || CAp::Cols(a)>=n,__FUNCTION__+": Cols(A)<N"))
return;
if(!CAp::Assert(CAp::Rows(a)>=k,__FUNCTION__+": Rows(A)<K"))
return;
if(!CAp::Assert(CAp::Len(al)>=k,__FUNCTION__+": Length(AL)<K"))
return;
if(!CAp::Assert(CAp::Len(au)>=k,__FUNCTION__+": Length(AU)<K"))
return;
if(!CAp::Assert(CApServ::IsFiniteMatrix(a,k,n),__FUNCTION__+": A contains infinite or NaN values!"))
return;
//--- Count actual (different from -INF<=A*x<=+INF) constraints;
//--- count non-zero elements in each row.
nrs.Resize(k);
State.m_m=k;
if(State.m_m==0)
return;
for(i=0; i<k; i++)
{
if(!CAp::Assert(MathIsValidNumber(al[i]) || IsNegInf(al[i]),__FUNCTION__+": AL contains NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(au[i]) || IsPosInf(au[i]),__FUNCTION__+": AU contains NAN or -INF"))
return;
nz=0;
for(j=0; j<n; j++)
if(a.Get(i,j)!=0.0)
nz++;
nrs.Set(i,nz);
}
//--- Allocate storage, copy
State.m_al=al;
State.m_au=au;
CSparse::SparseCreateCRSBuf(State.m_m,n,nrs,State.m_a);
for(i=0; i<k; i++)
for(j=0; j<n; j++)
if(a.Get(i,j)!=0.0)
CSparse::SparseSet(State.m_a,i,j,a.Get(i,j));
}
//+------------------------------------------------------------------+
//| This function sets two-sided linear constraints AL <= A*x <= AU |
//| with sparse constraining matrix A. Recommended for large-scale |
//| problems. |
//| This function overwrites linear (non-box) constraints set by |
//| previous calls (if such calls were made). |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinLPCreate() |
//| call. |
//| A - sparse matrix with size [K, N] (exactly!). Each row|
//| of A represents one general linear constraint. A |
//| can be stored in any sparse storage format. |
//| AL, AU - lower and upper bounds, array[K]; |
//| * AL.Set(i, AU.Set(i,> equality constraint Ai*x |
//| * AL[i]<AU.Set(i,> two-sided constraint |
//| AL[i] <= Ai*x <= AU[i] |
//| * AL[i] = -INF => one-sided constraint |
//| Ai*x <= AU[i] |
//| * AU[i] = +INF => one-sided constraint |
//| AL[i] <= Ai*x |
//| * AL.Set(i, -INF, AU.Set(i, +INF => constraint is |
//| ignored |
//| K - number of equality/inequality constraints, K >= 0. |
//| If K = 0 is specified, A, AL, AU are ignored. |
//+------------------------------------------------------------------+
void CMinLP::MinLPSetLC2(CMinLPState &State,CSparseMatrix &a,
CRowDouble &al,CRowDouble &au,int k)
{
int n=State.m_n;
//--- Quick exit
if(k==0)
{
State.m_m=0;
return;
}
//--- Integrity checks
if(!CAp::Assert(k>0,__FUNCTION__+": K<0"))
return;
if(!CAp::Assert(CSparse::SparseGetNCols(a)==n,__FUNCTION__+": Cols(A)<>N"))
return;
if(!CAp::Assert(CSparse::SparseGetNRows(a)==k,__FUNCTION__+": Rows(A)<>K"))
return;
if(!CAp::Assert(CAp::Len(al)>=k,__FUNCTION__+": Length(AL)<K"))
return;
if(!CAp::Assert(CAp::Len(au)>=k,__FUNCTION__+": Length(AU)<K"))
return;
for(int i=0; i<k; i++)
{
if(!CAp::Assert(MathIsValidNumber(al[i]) || IsNegInf(al[i]),__FUNCTION__+": AL contains NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(au[i]) || IsPosInf(au[i]),__FUNCTION__+": AU contains NAN or -INF"))
return;
}
//--- Copy
State.m_m=k;
CSparse::SparseCopyToCRSBuf(a,State.m_a);
State.m_al=al;
State.m_au=au;
}
//+------------------------------------------------------------------+
//| This function appends two-sided linear constraint AL <= A*x <= AU|
//| to the list of currently present constraints. |
//| This version accepts dense constraint vector as input, but |
//| sparsifies it for internal storage and processing. Thus, time to |
//| add one constraint in is O(N) - we have to scan entire array of |
//| length N. Sparse version of this function is order of magnitude |
//| faster for constraints with just a few nonzeros per row. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinLPCreate() |
//| call. |
//| A - linear constraint coefficient, array[N], right side|
//| is NOT included. |
//| AL, AU - lower and upper bounds; |
//| * AL = AU => equality constraint Ai*x |
//| * AL<AU => two-sided constraint |
//| AL <= A*x <= AU |
//| * AL = -INF => one-sided constraint Ai*x <= AU |
//| * AU = +INF => one-sided constraint AL <= Ai*x |
//| * AL = -INF, AU = +INF => constraint is ignored |
//+------------------------------------------------------------------+
void CMinLP::MinLPAddLC2Dense(CMinLPState &State,CRowDouble &a,
double al,double au)
{
//--- create variables
int n=State.m_n;
int nnz=0;
//--- check
if(!CAp::Assert(CAp::Len(a)>=n,__FUNCTION__+": Length(A)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(a,n),__FUNCTION__+": A contains infinite or NaN values!"))
return;
if(!CAp::Assert(MathIsValidNumber(al) || IsNegInf(al),__FUNCTION__+": AL is NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(au) || IsPosInf(au),__FUNCTION__+": AU is NAN or -INF"))
return;
CApServ::IVectorSetLengthAtLeast(State.m_adddtmpi,n);
CApServ::RVectorSetLengthAtLeast(State.m_adddtmpr,n);
nnz=0;
for(int i=0; i<n; i++)
if(a[i]!=0.0)
{
State.m_adddtmpi.Set(nnz,i);
State.m_adddtmpr.Set(nnz,a[i]);
nnz++;
}
//--- function call
MinLPAddLC2(State,State.m_adddtmpi,State.m_adddtmpr,nnz,al,au);
}
//+------------------------------------------------------------------+
//| This function appends two-sided linear constraint AL <= A*x <= AU|
//| to the list of currently present constraints. |
//| Constraint is passed in compressed format: as list of non-zero |
//| entries of coefficient vector A. Such approach is more efficient |
//| than dense storage for highly sparse constraint vectors. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinLPCreate() |
//| call. |
//| IdxA - array[NNZ], indexes of non-zero elements of A: |
//| * can be unsorted |
//| * can include duplicate indexes (corresponding |
//| entries of ValA[] will be summed) |
//| ValA - array[NNZ], values of non-zero elements of A |
//| NNZ - number of non-zero coefficients in A |
//| AL, AU - lower and upper bounds; |
//| * AL = AU => equality constraint A*x |
//| * AL<AU => two-sided constraint AL <= A*x <= AU |
//| * AL = -INF => one-sided constraint A*x <= AU |
//| * AU = +INF => one-sided constraint AL <= A*x |
//| * AL = -INF, AU = +INF => constraint is ignored |
//+------------------------------------------------------------------+
void CMinLP::MinLPAddLC2(CMinLPState &State,CRowInt &idxa,
CRowDouble &vala,int nnz,double al,double au)
{
//--- create variables
int m=State.m_m;
int n=State.m_n;
int i=0;
int j=0;
int k=0;
int offs=0;
int offsdst=0;
int didx=0;
int uidx=0;
//--- Check inputs
if(!CAp::Assert(nnz>=0,__FUNCTION__+": NNZ<0"))
return;
if(!CAp::Assert(CAp::Len(idxa)>=nnz,__FUNCTION__+": Length(IdxA)<NNZ"))
return;
if(!CAp::Assert(CAp::Len(vala)>=nnz,__FUNCTION__+": Length(ValA)<NNZ"))
return;
for(i=0; i<nnz; i++)
if(!CAp::Assert(idxa[i]>=0 && idxa[i]<n,__FUNCTION__+": IdxA contains indexes outside of [0,N) range"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(vala,nnz),__FUNCTION__+": ValA contains infinite or NaN values!"))
return;
if(!CAp::Assert(MathIsValidNumber(al) || IsNegInf(al),__FUNCTION__+": AL is NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(au) || IsPosInf(au),__FUNCTION__+": AU is NAN or -INF"))
return;
//--- If M=0, it means that A is uninitialized.
//--- Prepare sparse matrix structure
if(m==0)
{
State.m_a.m_MatrixType=1;
State.m_a.m_M=0;
State.m_a.m_N=n;
State.m_a.m_NInitialized=0;
CApServ::IVectorSetLengthAtLeast(State.m_a.m_RIdx,1);
State.m_a.m_RIdx.Set(0,0);
}
//--- Reallocate storage
offs=State.m_a.m_RIdx[m];
State.m_a.m_Idx.Resize(offs+nnz);
State.m_a.m_Vals.Resize(offs+nnz);
State.m_a.m_DIdx.Resize(m+1);
State.m_a.m_UIdx.Resize(m+1);
State.m_a.m_RIdx.Resize(m+2);
State.m_al.Resize(m+1);
State.m_au.Resize(m+1);
//--- If NNZ=0, perform quick and simple row append.
if(nnz==0)
{
State.m_a.m_DIdx.Set(m,State.m_a.m_RIdx[m]);
State.m_a.m_UIdx.Set(m,State.m_a.m_RIdx[m]);
State.m_a.m_RIdx.Set(m+1,State.m_a.m_RIdx[m]);
State.m_al.Set(m,al);
State.m_au.Set(m,au);
State.m_a.m_M=m+1;
State.m_m=m+1;
return;
}
//--- Now we are sure that A contains properly initialized sparse
//--- matrix (or some appropriate dummy for M=0) and we have NNZ>0
//--- (no need to care about degenerate cases).
//--- Append rows to A:
//--- * append data
//--- * sort in place
//--- * merge duplicate indexes
//--- * compute DIdx and UIdx
for(i=0; i<nnz; i++)
{
State.m_a.m_Idx.Set(offs+i,idxa[i]);
State.m_a.m_Vals.Set(offs+i,vala[i]);
}
CTSort::TagSortMiddleIR(State.m_a.m_Idx,State.m_a.m_Vals,offs,nnz);
offsdst=offs;
for(i=1; i<nnz; i++)
{
if(State.m_a.m_Idx[offsdst]!=State.m_a.m_Idx[offs+i])
{
offsdst++;
State.m_a.m_Idx.Set(offsdst,State.m_a.m_Idx[offs+i]);
State.m_a.m_Vals.Set(offsdst,State.m_a.m_Vals[offs+i]);
}
else
State.m_a.m_Vals.Add(offsdst,State.m_a.m_Vals[offs+i]);
}
nnz=offsdst-offs+1;
uidx=-1;
didx=-1;
for(j=offs; j<=offsdst; j++)
{
k=State.m_a.m_Idx[j];
if(k==m)
didx=j;
else
if(k>m && uidx==-1)
{
uidx=j;
break;
}
}
if(uidx==-1)
uidx=offsdst+1;
if(didx==-1)
didx=uidx;
State.m_a.m_DIdx.Set(m,didx);
State.m_a.m_UIdx.Set(m,uidx);
State.m_a.m_RIdx.Set(m+1,offsdst+1);
State.m_a.m_M=m+1;
State.m_a.m_NInitialized+=nnz;
State.m_al.Set(m,al);
State.m_au.Set(m,au);
State.m_m=m+1;
}
//+------------------------------------------------------------------+
//| This function solves LP problem. |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| You should use MinLPResults() function to access results after |
//| calls to this function. |
//+------------------------------------------------------------------+
void CMinLP::MinLPOptimize(CMinLPState &State)
{
//--- create variables
int n=State.m_n;
int m=State.m_m;
int i=0;
double v=0;
bool badconstr=false;
CDualSimplexSettings Settings;
CMatrixDouble dummy;
CDualSimplexBasis dummybasis;
ClearReportFields(State);
//--- Most basic check for correctness of constraints
badconstr=false;
for(i=0; i<n; i++)
if((MathIsValidNumber(State.m_bndl[i]) && MathIsValidNumber(State.m_bndu[i])) && State.m_bndl[i]>State.m_bndu[i])
badconstr=true;
for(i=0; i<m; i++)
if((MathIsValidNumber(State.m_al[i]) && MathIsValidNumber(State.m_au[i])) && State.m_al[i]>State.m_au[i])
badconstr=true;
if(badconstr)
{
State.m_repterminationtype=-3;
State.m_repn=n;
State.m_repm=m;
CAblasF::RSetAllocV(n,0.0,State.m_xs);
CAblasF::RSetAllocV(n,0.0,State.m_lagbc);
CAblasF::RSetAllocV(m,0.0,State.m_laglc);
CAblasF::ISetAllocV(n+m,0,State.m_cs);
State.m_repf=0;
State.m_repprimalerror=0;
for(i=0; i<n; i++)
{
if(MathIsValidNumber(State.m_bndl[i]))
State.m_repprimalerror=MathMax(State.m_repprimalerror,State.m_bndl[i]-0);
if(MathIsValidNumber(State.m_bndu[i]))
State.m_repprimalerror=MathMax(State.m_repprimalerror,0-State.m_bndu[i]);
}
for(i=0; i<m; i++)
{
if(MathIsValidNumber(State.m_al[i]))
State.m_repprimalerror=MathMax(State.m_repprimalerror,State.m_al[i]-0);
if(MathIsValidNumber(State.m_au[i]))
State.m_repprimalerror=MathMax(State.m_repprimalerror,0-State.m_au[i]);
}
State.m_repdualerror=0;
for(i=0; i<n; i++)
{
State.m_repdualerror=MathMax(State.m_repdualerror,MathAbs(State.m_c[i]));
}
State.m_repslackerror=0;
return;
}
//--- Call current solver
if(State.m_algokind==1 || State.m_algokind==2)
{
//--- Call the solver
if(State.m_algokind==1)
{
//--- Dual simplex method with presolve
CLPQPPresolve::PresolveNoneScaleUser(State.m_s,State.m_c,State.m_bndl,State.m_bndu,n,State.m_a,State.m_al,State.m_au,m,State.m_presolver);
CRevisedDualSimplex::DSSSettingsInit(Settings);
Settings.m_xtolabs=State.m_dsseps;
Settings.m_dtolabs=State.m_dsseps;
CRevisedDualSimplex::DSSInit(State.m_presolver.m_newn,State.m_dss);
CRevisedDualSimplex::DSSSetProblem(State.m_dss,State.m_presolver.m_c,State.m_presolver.m_bndl,State.m_presolver.m_bndu,dummy,State.m_presolver.m_sparsea,1,State.m_presolver.m_al,State.m_presolver.m_au,State.m_presolver.m_newm,dummybasis,m_alllogicalsbasis,Settings);
CRevisedDualSimplex::DSSOptimize(State.m_dss,Settings);
//--- Export results, convert from presolve
CApServ::RVectorSetLengthAtLeast(State.m_xs,State.m_presolver.m_newn);
CApServ::RVectorSetLengthAtLeast(State.m_lagbc,State.m_presolver.m_newn);
CApServ::RVectorSetLengthAtLeast(State.m_laglc,State.m_presolver.m_newm);
CApServ::IVectorSetLengthAtLeast(State.m_cs,State.m_presolver.m_newn+State.m_presolver.m_newm);
State.m_xs=State.m_dss.m_repx;
State.m_lagbc=State.m_dss.m_replagbc;
State.m_laglc=State.m_dss.m_replaglc;
State.m_cs=State.m_dss.m_repstats;
State.m_repiterationscount=State.m_dss.m_repiterationscount;
State.m_repterminationtype=State.m_dss.m_repterminationtype;
CLPQPPresolve::PresolveBwd(State.m_presolver,State.m_xs,State.m_cs,State.m_lagbc,State.m_laglc);
State.m_repn=n;
State.m_repm=m;
}
if(State.m_algokind==2)
{
//--- Interior point method with presolve
CLPQPPresolve::PresolveNoneScaleUser(State.m_s,State.m_c,State.m_bndl,State.m_bndu,n,State.m_a,State.m_al,State.m_au,m,State.m_presolver);
CAblasF::RSetAllocV(State.m_presolver.m_newn,1.0,State.m_units);
CAblasF::RSetAllocV(State.m_presolver.m_newn,0.0,State.m_zeroorigin);
CSparse::SparseCreateSKSBandBuf(State.m_presolver.m_newn,State.m_presolver.m_newn,0,State.m_ipmquadratic);
for(i=0; i<State.m_presolver.m_newn; i++)
CSparse::SparseSet(State.m_ipmquadratic,i,i,State.m_ipmlambda);
CSparse::SparseConvertToCRS(State.m_ipmquadratic);
CVIPMSolver::VIPMInitSparse(State.m_ipm,State.m_units,State.m_zeroorigin,State.m_presolver.m_newn);
CVIPMSolver::VIPMSetQuadraticLinear(State.m_ipm,dummy,State.m_ipmquadratic,1,false,State.m_presolver.m_c);
CVIPMSolver::VIPMSetConstraints(State.m_ipm,State.m_presolver.m_bndl,State.m_presolver.m_bndu,State.m_presolver.m_sparsea,State.m_presolver.m_newm,dummy,0,State.m_presolver.m_al,State.m_presolver.m_au);
CVIPMSolver::VIPMSetCond(State.m_ipm,State.m_ipmeps,State.m_ipmeps,State.m_ipmeps);
CVIPMSolver::VIPMOptimize(State.m_ipm,true,State.m_xs,State.m_lagbc,State.m_laglc,State.m_repterminationtype);
//--- Export results, convert from presolve
CAblasF::ISetAllocV(State.m_presolver.m_newn+State.m_presolver.m_newm,0,State.m_cs);
CLPQPPresolve::PresolveBwd(State.m_presolver,State.m_xs,State.m_cs,State.m_lagbc,State.m_laglc);
State.m_repn=n;
State.m_repm=m;
State.m_repiterationscount=State.m_ipm.m_repiterationscount;
}
//--- Compute F, primal and dual errors
State.m_repf=CAblasF::RDotV(n,State.m_xs,State.m_c);
State.m_repprimalerror=0;
State.m_repdualerror=0;
State.m_repslackerror=0;
State.m_tmpg=State.m_c;
if(m>0)
{
CSparse::SparseMV(State.m_a,State.m_xs,State.m_tmpax);
CSparse::SparseGemV(State.m_a,1.0,1,State.m_laglc,0,1.0,State.m_tmpg,0);
}
CAblasF::RAddV(n,1.0,State.m_lagbc,State.m_tmpg);
for(i=0; i<n; i++)
{
if(MathIsValidNumber(State.m_bndl[i]))
{
State.m_repprimalerror=MathMax(State.m_repprimalerror,State.m_bndl[i]-State.m_xs[i]);
State.m_repslackerror=MathMax(State.m_repslackerror,MathMax(State.m_xs[i]-State.m_bndl[i],0.0)*MathMax(-State.m_lagbc[i],0.0));
}
if(MathIsValidNumber(State.m_bndu[i]))
{
State.m_repprimalerror=MathMax(State.m_repprimalerror,State.m_xs[i]-State.m_bndu[i]);
State.m_repslackerror=MathMax(State.m_repslackerror,MathMax(State.m_bndu[i]-State.m_xs[i],0.0)*MathMax(State.m_lagbc[i],0.0));
}
State.m_repdualerror=MathMax(State.m_repdualerror,MathAbs(State.m_tmpg[i]));
}
for(i=0; i<m; i++)
{
v=State.m_tmpax[i];
if(MathIsValidNumber(State.m_al[i]))
{
State.m_repprimalerror=MathMax(State.m_repprimalerror,State.m_al[i]-v);
State.m_repslackerror=MathMax(State.m_repslackerror,MathMax(v-State.m_al[i],0.0)*MathMax(-State.m_laglc[i],0.0));
}
if(MathIsValidNumber(State.m_au[i]))
{
State.m_repprimalerror=MathMax(State.m_repprimalerror,v-State.m_au[i]);
State.m_repslackerror=MathMax(State.m_repslackerror,MathMax(State.m_au[i]-v,0.0)*MathMax(State.m_laglc[i],0.0));
}
}
return;
}
//--- Integrity check failed - unknown solver
CAp::Assert(false,__FUNCTION__+": integrity check failed - unknown solver");
}
//+------------------------------------------------------------------+
//| LP solver results |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| X - array[N], solution (on failure: last trial point) |
//| Rep - optimization report. You should check |
//| Rep.TerminationType, which contains completion |
//| code, and you may check another fields which |
//| contain another information about algorithm |
//| functioning. |
//| Failure codes returned by algorithm are: |
//| * -4 LP problem is primal unbounded(dual infeasible) |
//| * -3 LP problem is primal infeasible(dual unbounded) |
//| * -2 IPM solver detected that problem is either infeasible |
//| or unbounded |
//| Success codes: |
//| * 1..4 successful completion |
//| * 5 MaxIts steps was taken |
//+------------------------------------------------------------------+
void CMinLP::MinLPResults(CMinLPState &State,CRowDouble &x,
CMinLPReport &rep)
{
x.Resize(0);
MinLPResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| LP results |
//| Buffered implementation of MinLPResults() which uses |
//| pre-allocated buffer to store X[]. If buffer size is too small, |
//| it resizes buffer. It is intended to be used in the inner cycles |
//| of performance critical algorithms where array reallocation |
//| penalty is too large to be ignored. |
//+------------------------------------------------------------------+
void CMinLP::MinLPResultsBuf(CMinLPState &State,
CRowDouble &x,CMinLPReport &rep)
{
//--- create variables
int i=0;
int repn=0;
int repm=0;
repn=State.m_repn;
repm=State.m_repm;
rep.m_y.Resize(repm);
rep.m_stats.Resize(repn+repm);
rep.m_f=State.m_repf;
rep.m_primalerror=State.m_repprimalerror;
rep.m_dualerror=State.m_repdualerror;
rep.m_slackerror=State.m_repslackerror;
rep.m_iterationscount=State.m_repiterationscount;
rep.m_terminationtype=State.m_repterminationtype;
rep.m_laglc=State.m_laglc;
rep.m_lagbc=State.m_lagbc;
x=State.m_xs;
rep.m_y=rep.m_laglc.ToVector()*(-1.0);
State.m_cs=rep.m_stats;
if(CAp::Len(x)!=repn)
x.Resize(repn);
}
//+------------------------------------------------------------------+
//| Clear report fields prior to the optimization. |
//+------------------------------------------------------------------+
void CMinLP::ClearReportFields(CMinLPState &State)
{
State.m_repf=0.0;
State.m_repprimalerror=0.0;
State.m_repdualerror=0.0;
State.m_repiterationscount=0;
State.m_repterminationtype=0;
State.m_repn=0;
State.m_repm=0;
}
//+------------------------------------------------------------------+
//| This object stores temporaries of SLP subsolver. |
//+------------------------------------------------------------------+
struct CMinSLPSubsolver
{
int m_curdcnt;
int m_hessiantype;
bool m_basispresent;
CSparseMatrix m_sparsedummy;
CSparseMatrix m_sparseefflc;
CSparseMatrix m_sparserawlc;
CRowInt m_cs;
CRowDouble m_cural;
CRowDouble m_curau;
CRowDouble m_curb;
CRowDouble m_curbndl;
CRowDouble m_curbndu;
CRowDouble m_lagbc;
CRowDouble m_laglc;
CRowDouble m_sk;
CRowDouble m_tmp0;
CRowDouble m_tmp1;
CRowDouble m_xs;
CRowDouble m_yk;
CPresolveInfo m_presolver;
CMatrixDouble m_curd;
CMatrixDouble m_curhd;
CMatrixDouble m_densedummy;
CMatrixDouble m_h;
CDualSimplexState m_dss;
CDualSimplexSettings m_dsssettings;
CDualSimplexBasis m_lastbasis;
//--- constructor / destructor
CMinSLPSubsolver(void);
~CMinSLPSubsolver(void) {}
void Copy(const CMinSLPSubsolver &obj);
//--- overloading
void operator=(const CMinSLPSubsolver &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinSLPSubsolver::CMinSLPSubsolver(void)
{
m_curdcnt=0;
m_hessiantype=0;
m_basispresent=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinSLPSubsolver::Copy(const CMinSLPSubsolver &obj)
{
m_curdcnt=obj.m_curdcnt;
m_hessiantype=obj.m_hessiantype;
m_basispresent=obj.m_basispresent;
m_sparsedummy=obj.m_sparsedummy;
m_sparseefflc=obj.m_sparseefflc;
m_sparserawlc=obj.m_sparserawlc;
m_cs=obj.m_cs;
m_cural=obj.m_cural;
m_curau=obj.m_curau;
m_curb=obj.m_curb;
m_curbndl=obj.m_curbndl;
m_curbndu=obj.m_curbndu;
m_lagbc=obj.m_lagbc;
m_laglc=obj.m_laglc;
m_sk=obj.m_sk;
m_tmp0=obj.m_tmp0;
m_tmp1=obj.m_tmp1;
m_xs=obj.m_xs;
m_yk=obj.m_yk;
m_presolver=obj.m_presolver;
m_curd=obj.m_curd;
m_curhd=obj.m_curhd;
m_densedummy=obj.m_densedummy;
m_h=obj.m_h;
m_dss=obj.m_dss;
m_dsssettings=obj.m_dsssettings;
m_lastbasis=obj.m_lastbasis;
}
//+------------------------------------------------------------------+
//| This object stores temporaries for LagrangianFG() function |
//+------------------------------------------------------------------+
struct CMinSLPTmpLagrangian
{
CRowDouble m_sclagtmp0;
CRowDouble m_sclagtmp1;
//--- constructor / destructor
CMinSLPTmpLagrangian(void) {}
~CMinSLPTmpLagrangian(void) {}
void Copy(const CMinSLPTmpLagrangian &obj);
//--- overloading
void operator=(const CMinSLPTmpLagrangian &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinSLPTmpLagrangian::Copy(const CMinSLPTmpLagrangian &obj)
{
m_sclagtmp0=obj.m_sclagtmp0;
m_sclagtmp1=obj.m_sclagtmp1;
}
//+------------------------------------------------------------------+
//| This object stores temporaries for LagrangianFG() function |
//+------------------------------------------------------------------+
struct CMinSLPTmpMerit
{
CRowDouble m_mftmp0;
CMinSLPTmpMerit(void) {}
~CMinSLPTmpMerit(void) {}
void Copy(const CMinSLPTmpMerit &obj);
//--- overloading
void operator=(const CMinSLPTmpMerit &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinSLPTmpMerit::Copy(const CMinSLPTmpMerit &obj)
{
m_mftmp0=obj.m_mftmp0;
}
//+------------------------------------------------------------------+
//| This object stores temporaries of Phase13 SLP subsolver. |
//+------------------------------------------------------------------+
struct CMinSLPPhase13State
{
bool m_usecorrection;
RCommState m_rphase13state;
CRowDouble m_d;
CRowDouble m_dummylagmult;
CRowDouble m_dx;
CRowDouble m_stepkfic;
CRowDouble m_stepkfin;
CRowDouble m_stepkxc;
CRowDouble m_stepkxn;
CMinSLPTmpMerit m_tmpmerit;
CMatrixDouble m_stepkjc;
CMatrixDouble m_stepkjn;
//--- constructor / destructor
CMinSLPPhase13State(void) { m_usecorrection=false; }
~CMinSLPPhase13State(void) {}
//---
void Copy(const CMinSLPPhase13State &obj);
//--- overloading
void operator=(const CMinSLPPhase13State &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinSLPPhase13State::Copy(const CMinSLPPhase13State &obj)
{
m_usecorrection=obj.m_usecorrection;
m_rphase13state=obj.m_rphase13state;
m_d=obj.m_d;
m_dummylagmult=obj.m_dummylagmult;
m_dx=obj.m_dx;
m_stepkfic=obj.m_stepkfic;
m_stepkfin=obj.m_stepkfin;
m_stepkxc=obj.m_stepkxc;
m_stepkxn=obj.m_stepkxn;
m_tmpmerit=obj.m_tmpmerit;
m_stepkjc=obj.m_stepkjc;
m_stepkjn=obj.m_stepkjn;
}
//+------------------------------------------------------------------+
//| This object stores temporaries of Phase13 SLP subsolver. |
//+------------------------------------------------------------------+
struct CMinSLPPhase2State
{
int m_lastlcidx;
int m_lastnlcidx;
double m_lastlcerr;
double m_lastnlcerr;
RCommState m_rphase2state;
CRowDouble m_d;
CRowDouble m_meritlagmult;
CRowDouble m_stepkfic;
CRowDouble m_stepkfin;
CRowDouble m_stepklaggrad;
CRowDouble m_stepknlaggrad;
CRowDouble m_stepknlagmult;
CRowDouble m_stepkxc;
CRowDouble m_stepkxn;
CRowDouble m_tmp0;
CMinSLPTmpMerit m_tmpmerit;
CMinSLPTmpLagrangian m_tmplagrangianfg;
CMatrixDouble m_stepkjc;
CMatrixDouble m_stepkjn;
CLinMinState m_mcstate;
//--- constructor / destructor
CMinSLPPhase2State(void);
~CMinSLPPhase2State(void) {}
void Copy(const CMinSLPPhase2State &obj);
//--- overloading
void operator=(const CMinSLPPhase2State &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinSLPPhase2State::CMinSLPPhase2State(void)
{
m_lastlcidx=0;
m_lastnlcidx=0;
m_lastlcerr=0;
m_lastnlcerr=0;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinSLPPhase2State::Copy(const CMinSLPPhase2State &obj)
{
m_lastlcidx=obj.m_lastlcidx;
m_lastnlcidx=obj.m_lastnlcidx;
m_lastlcerr=obj.m_lastlcerr;
m_lastnlcerr=obj.m_lastnlcerr;
m_rphase2state=obj.m_rphase2state;
m_d=obj.m_d;
m_meritlagmult=obj.m_meritlagmult;
m_stepkfic=obj.m_stepkfic;
m_stepkfin=obj.m_stepkfin;
m_stepklaggrad=obj.m_stepklaggrad;
m_stepknlaggrad=obj.m_stepknlaggrad;
m_stepknlagmult=obj.m_stepknlagmult;
m_stepkxc=obj.m_stepkxc;
m_stepkxn=obj.m_stepkxn;
m_tmp0=obj.m_tmp0;
m_tmpmerit=obj.m_tmpmerit;
m_tmplagrangianfg=obj.m_tmplagrangianfg;
m_stepkjc=obj.m_stepkjc;
m_stepkjn=obj.m_stepkjn;
m_mcstate=obj.m_mcstate;
}
//+------------------------------------------------------------------+
//| This object stores temporaries of SLP solver. |
//+------------------------------------------------------------------+
struct CMinSLPState
{
int m_fstagnationcnt;
int m_hessiantype;
int m_historylen;
int m_lpfailurecnt;
int m_maxits;
int m_n;
int m_nec;
int m_nic;
int m_nlec;
int m_nlic;
int m_repbcidx;
int m_repinneriterationscount;
int m_replcidx;
int m_repnlcidx;
int m_repouteriterationscount;
int m_repsimplexiterations1;
int m_repsimplexiterations2;
int m_repsimplexiterations3;
int m_repsimplexiterations;
int m_repterminationtype;
double m_bigc;
double m_epsx;
double m_f;
double m_repbcerr;
double m_replcerr;
double m_repnlcerr;
double m_trustrad;
bool m_HasBndL[];
bool m_HasBndU[];
bool m_needfij;
bool m_xupdated;
RCommState m_rstate;
CRowInt m_lcsrcidx;
CRowDouble m_backupfi;
CRowDouble m_backupx;
CRowDouble m_dummylagmult;
CRowDouble m_fi;
CRowDouble m_fscales;
CRowDouble m_maxlaghistory;
CRowDouble m_meritfunctionhistory;
CRowDouble m_meritlagmult;
CRowDouble m_s;
CRowDouble m_scaledbndl;
CRowDouble m_scaledbndu;
CRowDouble m_step0fi;
CRowDouble m_step0x;
CRowDouble m_stepkfi;
CRowDouble m_stepkx;
CRowDouble m_x;
CMinSLPTmpMerit m_tmpmerit;
CMinSLPSubsolver m_subsolver;
CMinSLPPhase2State m_state2;
CMinSLPPhase13State m_state13;
CMatrixDouble m_backupj;
CMatrixDouble m_j;
CMatrixDouble m_scaledcleic;
CMatrixDouble m_step0j;
CMatrixDouble m_stepkj;
//--- constructor / destructor
CMinSLPState(void);
~CMinSLPState(void) {}
//---
void Copy(const CMinSLPState &obj);
//--- overloading
void operator=(const CMinSLPState &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinSLPState::CMinSLPState(void)
{
m_fstagnationcnt=0;
m_hessiantype=0;
m_historylen=0;
m_lpfailurecnt=0;
m_maxits=0;
m_n=0;
m_nec=0;
m_nic=0;
m_nlec=0;
m_nlic=0;
m_repbcidx=0;
m_repinneriterationscount=0;
m_replcidx=0;
m_repnlcidx=0;
m_repouteriterationscount=0;
m_repsimplexiterations1=0;
m_repsimplexiterations2=0;
m_repsimplexiterations3=0;
m_repsimplexiterations=0;
m_repterminationtype=0;
m_bigc=0;
m_epsx=0;
m_f=0;
m_repbcerr=0;
m_replcerr=0;
m_repnlcerr=0;
m_trustrad=0;
m_needfij=false;
m_xupdated=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinSLPState::Copy(const CMinSLPState &obj)
{
m_fstagnationcnt=obj.m_fstagnationcnt;
m_hessiantype=obj.m_hessiantype;
m_historylen=obj.m_historylen;
m_lpfailurecnt=obj.m_lpfailurecnt;
m_maxits=obj.m_maxits;
m_n=obj.m_n;
m_nec=obj.m_nec;
m_nic=obj.m_nic;
m_nlec=obj.m_nlec;
m_nlic=obj.m_nlic;
m_repbcidx=obj.m_repbcidx;
m_repinneriterationscount=obj.m_repinneriterationscount;
m_replcidx=obj.m_replcidx;
m_repnlcidx=obj.m_repnlcidx;
m_repouteriterationscount=obj.m_repouteriterationscount;
m_repsimplexiterations1=obj.m_repsimplexiterations1;
m_repsimplexiterations2=obj.m_repsimplexiterations2;
m_repsimplexiterations3=obj.m_repsimplexiterations3;
m_repsimplexiterations=obj.m_repsimplexiterations;
m_repterminationtype=obj.m_repterminationtype;
m_bigc=obj.m_bigc;
m_epsx=obj.m_epsx;
m_f=obj.m_f;
m_repbcerr=obj.m_repbcerr;
m_replcerr=obj.m_replcerr;
m_repnlcerr=obj.m_repnlcerr;
m_trustrad=obj.m_trustrad;
ArrayCopy(m_HasBndL,obj.m_HasBndL);
ArrayCopy(m_HasBndU,obj.m_HasBndU);
m_needfij=obj.m_needfij;
m_xupdated=obj.m_xupdated;
m_rstate=obj.m_rstate;
m_lcsrcidx=obj.m_lcsrcidx;
m_backupfi=obj.m_backupfi;
m_backupx=obj.m_backupx;
m_dummylagmult=obj.m_dummylagmult;
m_fi=obj.m_fi;
m_fscales=obj.m_fscales;
m_maxlaghistory=obj.m_maxlaghistory;
m_meritfunctionhistory=obj.m_meritfunctionhistory;
m_meritlagmult=obj.m_meritlagmult;
m_s=obj.m_s;
m_scaledbndl=obj.m_scaledbndl;
m_scaledbndu=obj.m_scaledbndu;
m_step0fi=obj.m_step0fi;
m_step0x=obj.m_step0x;
m_stepkfi=obj.m_stepkfi;
m_stepkx=obj.m_stepkx;
m_x=obj.m_x;
m_tmpmerit=obj.m_tmpmerit;
m_subsolver=obj.m_subsolver;
m_state2=obj.m_state2;
m_state13=obj.m_state13;
m_backupj=obj.m_backupj;
m_j=obj.m_j;
m_scaledcleic=obj.m_scaledcleic;
m_step0j=obj.m_step0j;
m_stepkj=obj.m_stepkj;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CNLCSLP
{
public:
//--- constants
static const double m_slpstpclosetozero;
static const double m_slpdeltadecrease;
static const double m_slpdeltaincrease;
static const double m_slpstpclosetoone;
static const double m_maxtrustraddecay;
static const double m_maxtrustradgrowth;
static const double m_slpgtol;
static const double m_initbigc;
static const double m_maxbigc;
static const double m_bfgstol;
static const double m_defaultl1penalty;
static const double m_meritfunctionbase;
static const double m_meritfunctiongain;
static const double m_inequalitydampingfactor;
static const double m_augmentationfactor;
static const double m_inittrustrad;
static const double m_stagnationepsf;
static const int m_lpfailureslimit;
static const int m_fstagnationlimit;
static const int m_nondescentlimit;
static const int m_nonmonotonicphase2limit;
static const double m_slpbigscale;
static const double m_slpsmallscale;
static const double m_defaultmaglagdecay;
static void MinSLPInitBuf(CRowDouble &bndl,CRowDouble &bndu,CRowDouble &s,CRowDouble &x0,int n,CMatrixDouble &cleic,CRowInt &lcsrcidx,int nec,int nic,int nlec,int nlic,double epsx,int m_maxits,CMinSLPState &State);
static bool MinSLPIteration(CMinSLPState &State,CSmoothnessMonitor &smonitor,bool userterminationneeded);
private:
static void InitLPSubsolver(CMinSLPState &sstate,CMinSLPSubsolver &subsolver,int hessiantype);
static void LPSubproblemRestart(CMinSLPState &sstate,CMinSLPSubsolver &subsolver);
static void LPSubproblemUpdateHessian(CMinSLPState &sstate,CMinSLPSubsolver &subsolver,CRowDouble &x0,CRowDouble &g0,CRowDouble &x1,CRowDouble &g1);
static bool LPSubproblemSolve(CMinSLPState &State,CMinSLPSubsolver &subsolver,CRowDouble &x,CRowDouble &fi,CMatrixDouble &jac,int innerk,CRowDouble &d,CRowDouble &lagmult);
static void LPSubproblemAppendConjugacyConstraint(CMinSLPState &State,CMinSLPSubsolver &subsolver,CRowDouble &d);
static void Phase13Init(CMinSLPPhase13State &state13,int n,int nec,int nic,int nlec,int nlic,bool usecorrection);
static bool Phase13Iteration(CMinSLPState &State,CMinSLPPhase13State &state13,CSmoothnessMonitor &smonitor,bool userterminationneeded,CRowDouble &curx,CRowDouble &curfi,CMatrixDouble &curj,CRowDouble &lagmult,int &status,double &dnrm,double &stp);
static void Phase2Init(CMinSLPPhase2State &state2,int n,int nec,int nic,int nlec,int nlic,CRowDouble &meritlagmult);
static bool Phase2Iteration(CMinSLPState &State,CMinSLPPhase2State &state2,CSmoothnessMonitor &smonitor,bool userterminationneeded,CRowDouble &curx,CRowDouble &curfi,CMatrixDouble &curj,CRowDouble &lagmult,double &gammamax,int &status);
static void SLPSendX(CMinSLPState &State,CRowDouble &xs);
static bool SLPRetrieveFIJ(CMinSLPState &State,CRowDouble &fis,CMatrixDouble &js);
static void SLPCopyState(CMinSLPState &State,CRowDouble &x0,CRowDouble &fi0,CMatrixDouble &j0,CRowDouble &x1,CRowDouble &fi1,CMatrixDouble &j1);
static void LagrangianFG(CMinSLPState &State,CRowDouble &x,double trustrad,CRowDouble &fi,CMatrixDouble &j,CRowDouble &lagmult,CMinSLPTmpLagrangian &tmp,double &f,CRowDouble &g,double &lcerr,int &lcidx,double &nlcerr,int &nlcidx);
static double MeritFunction(CMinSLPState &State,CRowDouble &x,CRowDouble &fi,CRowDouble &lagmult,double mu,CMinSLPTmpMerit &tmp);
static double RawLagrangian(CMinSLPState &State,CRowDouble &x,CRowDouble &fi,CRowDouble &lagmult,CMinSLPTmpMerit &tmp);
static void MeritFunctionAndRawLagrangian(CMinSLPState &State,CRowDouble &x,CRowDouble &fi,CRowDouble &lagmult,double mu,CMinSLPTmpMerit &tmp,double &meritf,double &rawlag);
};
//+------------------------------------------------------------------+
//| Constants |
//+------------------------------------------------------------------+
const double CNLCSLP::m_slpstpclosetozero=0.001;
const double CNLCSLP::m_slpdeltadecrease=0.20;
const double CNLCSLP::m_slpdeltaincrease=0.80;
const double CNLCSLP::m_slpstpclosetoone=0.95;
const double CNLCSLP::m_maxtrustraddecay=0.1;
const double CNLCSLP::m_maxtrustradgrowth=1.333;
const double CNLCSLP::m_slpgtol=0.4;
const double CNLCSLP::m_initbigc=500.0;
const double CNLCSLP::m_maxbigc=1.0E5;
const double CNLCSLP::m_bfgstol=1.0E-5;
const double CNLCSLP::m_defaultl1penalty=0.1;
const double CNLCSLP::m_meritfunctionbase=0.0;
const double CNLCSLP::m_meritfunctiongain=2.0;
const double CNLCSLP::m_inequalitydampingfactor=10.0;
const double CNLCSLP::m_augmentationfactor=10.0;
const double CNLCSLP::m_inittrustrad=0.1;
const double CNLCSLP::m_stagnationepsf=1.0E-12;
const int CNLCSLP::m_lpfailureslimit=20;
const int CNLCSLP::m_fstagnationlimit=20;
const int CNLCSLP::m_nondescentlimit=99999;
const int CNLCSLP::m_nonmonotonicphase2limit=5;
const double CNLCSLP::m_slpbigscale=5.0;
const double CNLCSLP::m_slpsmallscale=0.2;
const double CNLCSLP::m_defaultmaglagdecay=0.85;
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CNLCSLP::MinSLPInitBuf(CRowDouble &bndl,
CRowDouble &bndu,
CRowDouble &s,
CRowDouble &x0,
int n,
CMatrixDouble &cleic,
CRowInt &lcsrcidx,
int nec,
int nic,
int nlec,
int nlic,
double epsx,
int m_maxits,
CMinSLPState &State)
{
//--- create variables
int i=0;
int j=0;
double v=0;
double vv=0;
State.m_n=n;
State.m_nec=nec;
State.m_nic=nic;
State.m_nlec=nlec;
State.m_nlic=nlic;
//--- Settings
State.m_hessiantype=2;
//--- Prepare RCOMM State
State.m_rstate.ia.Resize(10);
ArrayResize(State.m_rstate.ba,4);
State.m_rstate.ra.Resize(17);
State.m_rstate.stage=-1;
State.m_needfij=false;
State.m_xupdated=false;
State.m_x.Resize(n);
State.m_fi.Resize(1+nlec+nlic);
State.m_j.Resize(1+nlec+nlic,n);
//--- Allocate memory.
CApServ::RVectorSetLengthAtLeast(State.m_s,n);
CApServ::RVectorSetLengthAtLeast(State.m_step0x,n);
CApServ::RVectorSetLengthAtLeast(State.m_stepkx,n);
CApServ::RVectorSetLengthAtLeast(State.m_backupx,n);
CApServ::RVectorSetLengthAtLeast(State.m_step0fi,1+nlec+nlic);
CApServ::RVectorSetLengthAtLeast(State.m_stepkfi,1+nlec+nlic);
CApServ::RVectorSetLengthAtLeast(State.m_backupfi,1+nlec+nlic);
CApServ::RMatrixSetLengthAtLeast(State.m_step0j,1+nlec+nlic,n);
CApServ::RMatrixSetLengthAtLeast(State.m_stepkj,1+nlec+nlic,n);
CApServ::RMatrixSetLengthAtLeast(State.m_backupj,1+nlec+nlic,n);
CApServ::RVectorSetLengthAtLeast(State.m_fscales,1+nlec+nlic);
CApServ::RVectorSetLengthAtLeast(State.m_meritlagmult,nec+nic+nlec+nlic);
CApServ::RVectorSetLengthAtLeast(State.m_dummylagmult,nec+nic+nlec+nlic);
ArrayResize(State.m_HasBndL,n);
ArrayResize(State.m_HasBndU,n);
CApServ::RVectorSetLengthAtLeast(State.m_scaledbndl,n);
CApServ::RVectorSetLengthAtLeast(State.m_scaledbndu,n);
CApServ::RMatrixSetLengthAtLeast(State.m_scaledcleic,nec+nic,n+1);
CApServ::IVectorSetLengthAtLeast(State.m_lcsrcidx,nec+nic);
CApServ::RVectorSetLengthAtLeast(State.m_meritfunctionhistory,m_nonmonotonicphase2limit+1);
CApServ::RVectorSetLengthAtLeast(State.m_maxlaghistory,m_nonmonotonicphase2limit+1);
//--- Prepare scaled problem
for(i=0; i<n; i++)
{
State.m_HasBndL[i]=MathIsValidNumber(bndl[i]);
State.m_HasBndU[i]=MathIsValidNumber(bndu[i]);
if(State.m_HasBndL[i])
State.m_scaledbndl.Set(i,bndl[i]/s[i]);
if(State.m_HasBndU[i])
State.m_scaledbndu.Set(i,bndu[i]/s[i]);
if(State.m_HasBndL[i] && State.m_HasBndU[i])
if(!CAp::Assert(bndl[i]<=bndu[i],__FUNCTION__+": integrity check failed,box constraints are inconsistent"))
return;
State.m_step0x.Set(i,x0[i]/s[i]);
State.m_s.Set(i,s[i]);
}
for(i=0; i<nec+nic; i++)
{
//--- Permutation
State.m_lcsrcidx.Set(i,lcsrcidx[i]);
//--- Scale and normalize linear constraints
vv=0.0;
for(j=0; j<n; j++)
{
v=cleic.Get(i,j)*s[j];
State.m_scaledcleic.Set(i,j,v);
vv+=v*v;
}
vv=MathSqrt(vv);
State.m_scaledcleic.Set(i,n,cleic.Get(i,n));
if(vv>0.0)
State.m_scaledcleic.Row(i,State.m_scaledcleic[i]/vv);
}
//--- Initial enforcement of box constraints
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i])
State.m_step0x.Set(i,MathMax(State.m_step0x[i],State.m_scaledbndl[i]));
if(State.m_HasBndU[i])
State.m_step0x.Set(i,MathMin(State.m_step0x[i],State.m_scaledbndu[i]));
}
//--- Stopping criteria
State.m_epsx=epsx;
State.m_maxits=m_maxits;
//--- Report fields
State.m_repsimplexiterations=0;
State.m_repsimplexiterations1=0;
State.m_repsimplexiterations2=0;
State.m_repsimplexiterations3=0;
State.m_repterminationtype=0;
State.m_repbcerr=0;
State.m_repbcidx=-1;
State.m_replcerr=0;
State.m_replcidx=-1;
State.m_repnlcerr=0;
State.m_repnlcidx=-1;
State.m_repinneriterationscount=0;
State.m_repouteriterationscount=0;
//--- Integrity checks:
//--- * it is important that significant step length is large enough that
//--- we do not decrease trust regiod radius; it should also be small,
//--- so we won't treat large steps as insignificant
if(!CAp::Assert(m_slpstpclosetozero<m_slpdeltadecrease,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(m_slpdeltadecrease<m_slpdeltaincrease,__FUNCTION__+": integrity check failed"))
return;
if(!CAp::Assert(m_slpdeltaincrease<m_slpstpclosetoone,__FUNCTION__+": integrity check failed"))
return;
}
//+------------------------------------------------------------------+
//| This function performs actual processing for SLP algorithm. It |
//| expects that caller redirects its reverse communication requests |
//| NeedFiJ / XUpdated to external user who will provide analytic |
//| derivative (or handle reports about progress). |
//| In case external user does not have analytic derivative, it is |
//| responsibility of caller to intercept NeedFiJ request and replace|
//| it with appropriate numerical differentiation scheme. |
//| Results are stored: |
//| * point - in State.StepKX |
//| IMPORTANT: this function works with scaled problem formulation; |
//| it is responsibility of the caller to UnScale request |
//| and scale Jacobian. |
//| NOTE: SMonitor is expected to be correctly initialized smoothness|
//| monitor. |
//+------------------------------------------------------------------+
bool CNLCSLP::MinSLPIteration(CMinSLPState &State,
CSmoothnessMonitor &smonitor,
bool userterminationneeded)
{
//--- create variables
bool result=false;
int n=0;
int nslack=0;
int nec=0;
int nic=0;
int nlec=0;
int nlic=0;
int i=0;
int j=0;
int innerk=0;
double v=0;
double vv=0;
double mx=0;
bool lpstagesuccess=false;
double gammamax=0;
double f1=0;
double f2=0;
int status=0;
double stp=0;
double deltamax=0;
double multiplyby=0;
double setscaleto=0;
double prevtrustrad=0;
bool increasebigc=false;
double d1nrm=0;
double mu=0;
double expandedrad=0;
double tol=0;
bool dotrace=false;
bool dodetailedtrace=false;
double maxlag=0;
double maxhist=0;
int label=-1;
//--- Reverse communication preparations
//--- I know it looks ugly, but it works the same way
//--- anywhere from C++ to Python.
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(State.m_rstate.stage>=0)
{
n=State.m_rstate.ia[0];
nslack=State.m_rstate.ia[1];
nec=State.m_rstate.ia[2];
nic=State.m_rstate.ia[3];
nlec=State.m_rstate.ia[4];
nlic=State.m_rstate.ia[5];
i=State.m_rstate.ia[6];
j=State.m_rstate.ia[7];
innerk=State.m_rstate.ia[8];
status=State.m_rstate.ia[9];
lpstagesuccess=State.m_rstate.ba[0];
increasebigc=State.m_rstate.ba[1];
dotrace=State.m_rstate.ba[2];
dodetailedtrace=State.m_rstate.ba[3];
v=State.m_rstate.ra[0];
vv=State.m_rstate.ra[1];
mx=State.m_rstate.ra[2];
gammamax=State.m_rstate.ra[3];
f1=State.m_rstate.ra[4];
f2=State.m_rstate.ra[5];
stp=State.m_rstate.ra[6];
deltamax=State.m_rstate.ra[7];
multiplyby=State.m_rstate.ra[8];
setscaleto=State.m_rstate.ra[9];
prevtrustrad=State.m_rstate.ra[10];
d1nrm=State.m_rstate.ra[11];
mu=State.m_rstate.ra[12];
expandedrad=State.m_rstate.ra[13];
tol=State.m_rstate.ra[14];
maxlag=State.m_rstate.ra[15];
maxhist=State.m_rstate.ra[16];
}
else
{
n=359;
nslack=-58;
nec=-919;
nic=-909;
nlec=81;
nlic=255;
i=74;
j=-788;
innerk=809;
status=205;
lpstagesuccess=false;
increasebigc=true;
dotrace=false;
dodetailedtrace=true;
v=-541;
vv=-698;
mx=-900;
gammamax=-318;
f1=-940;
f2=1016;
stp=-229;
deltamax=-536;
multiplyby=487;
setscaleto=-115;
prevtrustrad=886;
d1nrm=346;
mu=-722;
expandedrad=-413;
tol=-461;
maxlag=927;
maxhist=201;
}
switch(State.m_rstate.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
case 3:
label=3;
break;
case 4:
label=4;
break;
default:
//--- Routine body
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
nlec=State.m_nlec;
nlic=State.m_nlic;
nslack=n+2*(nec+nlec)+(nic+nlic);
dotrace=CAp::IsTraceEnabled("SLP");
dodetailedtrace=dotrace && CAp::IsTraceEnabled("SLP.DETAILED");
//--- Prepare rcomm interface
State.m_needfij=false;
State.m_xupdated=false;
//--- Initialize algorithm data:
//---*Lagrangian and "Big C" estimates
//--- * trust region
//--- * initial function scales (vector of 1's)
//--- * current approximation of the Hessian matrix H (unit matrix)
//--- * initial linearized constraints
//--- * initial violation of linear/nonlinear constraints
State.m_lpfailurecnt=0;
State.m_fstagnationcnt=0;
State.m_bigc=m_initbigc;
State.m_trustrad=m_inittrustrad;
State.m_fscales.Fill(1.0);
State.m_meritfunctionhistory.Fill(CMath::m_maxrealnumber);
State.m_maxlaghistory.Fill(0.0);
State.m_historylen=0;
gammamax=0.0;
//--- Avoid spurious warnings about possibly uninitialized vars
status=0;
stp=0;
//--- Evaluate function vector and Jacobian at Step0X, send first location report.
//--- Compute initial violation of constraints.
SLPSendX(State,State.m_step0x);
State.m_needfij=true;
State.m_rstate.stage=0;
label=-1;
break;
}
//--- main loop
while(label>=0)
{
switch(label)
{
case 0:
State.m_needfij=false;
if(!SLPRetrieveFIJ(State,State.m_step0fi,State.m_step0j))
{
//--- Failed to retrieve function/Jaconian, infinities detected!
State.m_stepkx=State.m_step0x;
State.m_repterminationtype=-8;
return(false);
}
SLPCopyState(State,State.m_step0x,State.m_step0fi,State.m_step0j,State.m_stepkx,State.m_stepkfi,State.m_stepkj);
SLPSendX(State,State.m_stepkx);
State.m_f=State.m_stepkfi[0]*State.m_fscales[0];
State.m_xupdated=true;
State.m_rstate.stage=1;
label=-1;
break;
case 1:
State.m_xupdated=false;
COptServ::CheckLcViolation(State.m_scaledcleic,State.m_lcsrcidx,nec,nic,State.m_stepkx,n,State.m_replcerr,State.m_replcidx);
COptServ::UnScaleAndCheckNLcViolation(State.m_stepkfi,State.m_fscales,nlec,nlic,State.m_repnlcerr,State.m_repnlcidx);
//--- Trace output (if needed)
if(dotrace)
{
CAp::Trace("////////////////////////////////////////////////////////////////////////////////////////////////////\n");
CAp::Trace("//--- SLP SOLVER STARTED //\n");
CAp::Trace("////////////////////////////////////////////////////////////////////////////////////////////////////\n");
}
//--- Perform outer (NLC) iterations
InitLPSubsolver(State,State.m_subsolver,State.m_hessiantype);
case 5:
//--- Before beginning new outer iteration:
//--- * renormalize target function and/or constraints, if some of them have too large magnitudes
//--- * save initial point for the outer iteration
for(i=0; i<=nlec+nlic; i++)
{
//--- Determine (a) multiplicative coefficient applied to function value
//--- and Jacobian row, and (b) new value of the function scale.
mx=0;
for(j=0; j<n; j++)
mx=MathMax(mx,MathAbs(State.m_stepkj.Get(i,j)));
multiplyby=1.0;
setscaleto=State.m_fscales[i];
if(mx>=m_slpbigscale)
{
multiplyby=1/mx;
setscaleto=State.m_fscales[i]*mx;
}
if(mx<=m_slpsmallscale && State.m_fscales[i]>1.0)
{
if((State.m_fscales[i]*mx)>1.0)
{
multiplyby=1/mx;
setscaleto=State.m_fscales[i]*mx;
}
else
{
multiplyby=State.m_fscales[i];
setscaleto=1.0;
}
}
if(multiplyby!=1.0)
{
//--- Function #I needs renormalization:
//--- * update function vector element and Jacobian matrix row
//--- * update FScales[] array
State.m_stepkfi.Mul(i,multiplyby);
State.m_stepkj.Row(i,State.m_stepkj[i]*multiplyby);
State.m_fscales.Set(i,setscaleto);
}
}
//--- Save initial point for the outer iteration
SLPCopyState(State,State.m_stepkx,State.m_stepkfi,State.m_stepkj,State.m_step0x,State.m_step0fi,State.m_step0j);
//--- Trace output (if needed)
if(dotrace)
{
CAp::Trace(StringFormat("\n=== OUTER ITERATION %5d STARTED ==================================================================\n",State.m_repouteriterationscount));
if(dodetailedtrace)
{
CAp::Trace("> printing raw data (prior to applying variable and function scales)\n");
CAp::Trace("X (raw) = ");
CApServ::TraceVectoRunScaledUnshiftedAutopRec(State.m_step0x,n,State.m_s,true,State.m_s,false);
CAp::Trace("\n");
CAp::Trace("> printing scaled data (after applying variable and function scales)\n");
CAp::Trace("X (scaled) = ");
CApServ::TraceVectorAutopRec(State.m_step0x,0,n);
CAp::Trace("\n");
CAp::Trace("FScales = ");
CApServ::TraceVectorAutopRec(State.m_fscales,0,1+nlec+nlic);
CAp::Trace("\n");
CAp::Trace("Fi (scaled) = ");
CApServ::TraceVectorAutopRec(State.m_stepkfi,0,1+nlec+nlic);
CAp::Trace("\n");
CAp::Trace("|Ji| (scaled) = ");
CApServ::TraceRowNrm1AutopRec(State.m_stepkj,0,1+nlec+nlic,0,n);
CAp::Trace("\n");
}
mx=0;
for(i=1; i<=nlec; i++)
mx=MathMax(mx,MathAbs(State.m_stepkfi[i]));
for(i=nlec+1; i<=nlec+nlic; i++)
mx=MathMax(mx,State.m_stepkfi[i]);
CAp::Trace(StringFormat("trustRad = %.3E\n",State.m_trustrad));
CAp::Trace(StringFormat("lin.violation = %.3E (scaled violation of linear constraints)\n",State.m_replcerr));
CAp::Trace(StringFormat("nlc.violation = %.3E (scaled violation of nonlinear constraints)\n",mx));
CAp::Trace(StringFormat("gammaMax = %.3E\n",gammamax));
CAp::Trace(StringFormat("max|LagMult| = %.3E (maximum over %d last iterations)\n",CAblasF::RMaxAbsV(State.m_historylen,State.m_maxlaghistory),State.m_historylen));
}
//--- PHASE 1:
//--- * perform step using linear model with second order correction
//---*compute "reference" Lagrange multipliers
//--- * compute merit function at the end of the phase 1 and push it to the history queue
//--- NOTE: a second order correction helps to overcome Maratos effect - a tendency
//--- of L1 penalized merit function to reject nonzero steps along steepest
//--- descent direction.
//--- The idea (explained in more details in the Phase13Iteration() body)
//--- is to perform one look-ahead step and use updated constraint values
//--- back at the initial point.
Phase13Init(State.m_state13,n,nec,nic,nlec,nlic,false);
case 2:
case 7:
if(!Phase13Iteration(State,State.m_state13,smonitor,userterminationneeded,State.m_stepkx,State.m_stepkfi,State.m_stepkj,State.m_meritlagmult,status,d1nrm,stp))
{
label=8;
break;
}
State.m_rstate.stage=2;
label=-1;
break;
case 8:
if(status<0)
{
label=5;
break;
}
if(status==0)
{
label=6;
break;
}
maxlag=CAblasF::RMaxAbsV(nec+nic+nlec+nlic,State.m_meritlagmult);
maxhist=CAblasF::RMaxAbsV(State.m_historylen,State.m_maxlaghistory);
mu=CApServ::Coalesce(MathMax(maxhist,maxlag),m_defaultl1penalty);
for(i=State.m_historylen; i>=1; i--)
{
State.m_meritfunctionhistory.Set(i,State.m_meritfunctionhistory[i-1]);
State.m_maxlaghistory.Set(i,State.m_maxlaghistory[i-1]);
}
State.m_meritfunctionhistory.Set(0,MeritFunction(State,State.m_stepkx,State.m_stepkfi,State.m_meritlagmult,mu,State.m_tmpmerit));
State.m_maxlaghistory.Set(0,CApServ::Coalesce(maxlag,m_defaultmaglagdecay*maxhist));
State.m_historylen=MathMin(State.m_historylen+1,m_nonmonotonicphase2limit);
//--- Decide whether we need to increase BigC (penalty for the constraint violation that
//--- is used by the linear subsolver) or not. BigC is increased if all of the following
//--- holds true:
//--- * BigC can be increased (it is below upper limit)
//--- * a short step was performed (shorter than the current trust region)
//--- * at least one of the constraints is infeasible within current trust region
if((d1nrm*stp)<(0.99*State.m_trustrad) && State.m_bigc<(double)(0.9*m_maxbigc))
{
increasebigc=false;
expandedrad=1.1*State.m_trustrad;
tol=MathMax(MathSqrt(CMath::m_machineepsilon)*State.m_trustrad,1000*CMath::m_machineepsilon);
for(i=0; i<=nec+nic-1; i++)
{
v=0;
vv=0;
for(j=0; j<n; j++)
{
v+=State.m_scaledcleic.Get(i,j)*State.m_stepkx[j];
vv+=MathAbs(State.m_scaledcleic.Get(i,j)*expandedrad);
}
v-=State.m_scaledcleic.Get(i,n);
if(i>=nec)
v=MathMax(v,0.0);
increasebigc=increasebigc || MathAbs(v)>(vv+tol);
}
for(i=1; i<=nlec+nlic; i++)
{
v=State.m_stepkfi[i];
vv=0;
for(j=0; j<n; j++)
vv+=MathAbs(State.m_stepkj.Get(i,j)*expandedrad);
if(i>=nlec+1)
v=MathMax(v,0.0);
increasebigc=increasebigc || MathAbs(v)>(vv+tol);
}
if(increasebigc)
{
State.m_bigc=MathMin(10*State.m_bigc,m_maxbigc);
if(dotrace)
CAp::Trace(StringFormat("BigC = %.3E (trust radius is small,but some constraints are still infeasible - increasing constraint violation penalty)\n",State.m_bigc));
}
}
//--- PHASE 2: conjugate subiterations
//--- If step with second order correction is shorter than 1.0, it means
//--- that target is sufficiently nonlinear to use advanced iterations.
//--- * perform inner LP subiterations with additional conjugacy constraints
//--- * check changes in merit function, discard iteration results if merit function increased
if(stp>=(double)(m_slpstpclosetoone))
{
label=9;
break;
}
if(dotrace)
CAp::Trace("> linear model produced short step,starting conjugate-gradient-like phase\n");
SLPCopyState(State,State.m_stepkx,State.m_stepkfi,State.m_stepkj,State.m_backupx,State.m_backupfi,State.m_backupj);
//--- LP subiterations
Phase2Init(State.m_state2,n,nec,nic,nlec,nlic,State.m_meritlagmult);
case 3:
case 11:
if(Phase2Iteration(State,State.m_state2,smonitor,userterminationneeded,State.m_stepkx,State.m_stepkfi,State.m_stepkj,State.m_dummylagmult,gammamax,status))
{
State.m_rstate.stage=3;
label=-1;
break;
}
case 12:
if(status==0)
{
//--- Save progress so far and stop
label=6;
break;
}
//--- Evaluating step
//--- This step is essential because previous step (which minimizes Lagrangian) may fail
//--- to produce descent direction for L1-penalized merit function and will increase it
//--- instead of decreasing.
//--- During evaluation we compare merit function at new location with maximum computed
//--- over last NonmonotonicPhase2Limit+1 previous ones (as suggested in 'A Sequential
//--- Quadratic Programming Algorithm with Non-Monotone Line Search' by Yu-Hong Dai).
//--- Settings NonmonotonicPhase2Limit to 0 will result in strictly monotonic line search,
//--- whilst having nonzero limits means that we perform more robust nonmonotonic search.
if(!CAp::Assert(State.m_historylen>=1,__FUNCTION__+": integrity check 6559 failed"))
return(false);
f1=State.m_meritfunctionhistory[0];
for(i=1; i<=State.m_historylen; i++)
f1=MathMax(f1,State.m_meritfunctionhistory[i]);
f2=MeritFunction(State,State.m_stepkx,State.m_stepkfi,State.m_meritlagmult,mu,State.m_tmpmerit);
if(dotrace)
{
CAp::Trace(StringFormat("> evaluating changes in merit function (max over last %d values is used for reference):\n",m_nonmonotonicphase2limit + 1));
CAp::Trace(StringFormat("meritF: %14.6E -> %14.6E (delta=%11.3E)\n",f1,f2,f2 - f1));
}
if(f2<f1)
{
label=13;
break;
}
//--- Merit function does not decrease, discard phase results and report is as one
//--- more "fake" inner iteration.
//--- NOTE: it is important that F2=F1 is considered as "does not decrease"
if(dotrace)
CAp::Trace("> CG-like phase increased merit function,completely discarding phase (happens sometimes,but not too often)\n");
SLPCopyState(State,State.m_backupx,State.m_backupfi,State.m_backupj,State.m_stepkx,State.m_stepkfi,State.m_stepkj);
State.m_repinneriterationscount++;
SLPSendX(State,State.m_stepkx);
State.m_f=State.m_stepkfi[0]*State.m_fscales[0];
State.m_xupdated=true;
State.m_rstate.stage=4;
label=-1;
break;
case 4:
State.m_xupdated=false;
COptServ::CheckLcViolation(State.m_scaledcleic,State.m_lcsrcidx,nec,nic,State.m_stepkx,n,State.m_replcerr,State.m_replcidx);
COptServ::UnScaleAndCheckNLcViolation(State.m_stepkfi,State.m_fscales,nlec,nlic,State.m_repnlcerr,State.m_repnlcidx);
label=10;
break;
case 13:
//--- Merit function decreased, accept phase
State.m_meritfunctionhistory.Set(0,f2);
if(dotrace)
CAp::Trace("> CG-like phase decreased merit function,CG-like step accepted\n");
case 14:
label=10;
break;
case 9:
//--- No phase #2
if(dotrace)
{
if(stp>0.0)
CAp::Trace("> linear model produced long step,no need to start CG-like iterations\n");
else
CAp::Trace("> linear model produced zero step,maybe trust radius is too large\n");
}
case 10:
//--- Update trust region
prevtrustrad=State.m_trustrad;
deltamax=0;
for(i=0; i<n; i++)
deltamax=MathMax(deltamax,MathAbs(State.m_step0x[i]-State.m_stepkx[i])/State.m_trustrad);
if(deltamax<=m_slpdeltadecrease)
State.m_trustrad=State.m_trustrad*MathMax(deltamax/m_slpdeltadecrease,m_maxtrustraddecay);
if(deltamax>=m_slpdeltaincrease)
State.m_trustrad=State.m_trustrad*MathMin(deltamax/m_slpdeltaincrease,m_maxtrustradgrowth);
//--- Trace
if(dotrace)
{
CAp::Trace("\n--- outer iteration ends ---------------------------------------------------------------------------\n");
CAp::Trace(StringFormat("deltaMax = %.3f (ratio of step length to trust radius)\n",deltamax));
CAp::Trace(StringFormat("newTrustRad = %.3E",State.m_trustrad));
if(State.m_trustrad>prevtrustrad)
CAp::Trace(",trust radius increased");
if(State.m_trustrad<prevtrustrad)
CAp::Trace(",trust radius decreased");
CAp::Trace("\n");
}
//--- Advance outer iteration counter, test stopping criteria
State.m_repouteriterationscount++;
if(MathAbs(State.m_stepkfi[0]-State.m_step0fi[0])<=(m_stagnationepsf*MathAbs(State.m_step0fi[0])))
State.m_fstagnationcnt++;
else
State.m_fstagnationcnt=0;
if(State.m_trustrad<=State.m_epsx)
{
State.m_repterminationtype=2;
if(dotrace)
CAp::Trace(StringFormat("> stopping condition met: trust radius is smaller than %.3E\n",State.m_epsx));
label=6;
break;
}
if(State.m_maxits>0 && State.m_repinneriterationscount>=State.m_maxits)
{
State.m_repterminationtype=5;
if(dotrace)
CAp::Trace(StringFormat("> stopping condition met: %d iterations performed\n",State.m_repinneriterationscount));
label=6;
break;
}
if(State.m_fstagnationcnt>=m_fstagnationlimit)
{
State.m_repterminationtype=7;
if(dotrace)
CAp::Trace(StringFormat("> stopping criteria are too stringent: F stagnated for %d its,stopping\n",State.m_fstagnationcnt));
label=6;
break;
}
label=5;
break;
case 6:
COptServ::SmoothnessMonitorTraceStatus(smonitor,dotrace);
result=false;
return(result);
}
}
//--- Saving State
result=true;
State.m_rstate.ba[0]=lpstagesuccess;
State.m_rstate.ba[1]=increasebigc;
State.m_rstate.ba[2]=dotrace;
State.m_rstate.ba[3]=dodetailedtrace;
State.m_rstate.ia.Set(0,n);
State.m_rstate.ia.Set(1,nslack);
State.m_rstate.ia.Set(2,nec);
State.m_rstate.ia.Set(3,nic);
State.m_rstate.ia.Set(4,nlec);
State.m_rstate.ia.Set(5,nlic);
State.m_rstate.ia.Set(6,i);
State.m_rstate.ia.Set(7,j);
State.m_rstate.ia.Set(8,innerk);
State.m_rstate.ia.Set(9,status);
State.m_rstate.ra.Set(0,v);
State.m_rstate.ra.Set(1,vv);
State.m_rstate.ra.Set(2,mx);
State.m_rstate.ra.Set(3,gammamax);
State.m_rstate.ra.Set(4,f1);
State.m_rstate.ra.Set(5,f2);
State.m_rstate.ra.Set(6,stp);
State.m_rstate.ra.Set(7,deltamax);
State.m_rstate.ra.Set(8,multiplyby);
State.m_rstate.ra.Set(9,setscaleto);
State.m_rstate.ra.Set(10,prevtrustrad);
State.m_rstate.ra.Set(11,d1nrm);
State.m_rstate.ra.Set(12,mu);
State.m_rstate.ra.Set(13,expandedrad);
State.m_rstate.ra.Set(14,tol);
State.m_rstate.ra.Set(15,maxlag);
State.m_rstate.ra.Set(16,maxhist);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function initializes SLP subproblem. |
//| Should be called once in the beginning of the optimization. |
//| INPUT PARAMETERS: |
//| SState - solver State |
//| Subsolver - SLP subproblem to initialize |
//| HessianType - 0 for identity Hessian, 1 for BFGS update |
//| RETURN VALUE: |
//| True on success |
//| False on failure of the LP solver (unexpected... but possible |
//| due to numerical errors) |
//+------------------------------------------------------------------+
void CNLCSLP::InitLPSubsolver(CMinSLPState &sstate,
CMinSLPSubsolver &subsolver,
int hessiantype)
{
//--- create variables
int n=sstate.m_n;
int nec=sstate.m_nec;
int nic=sstate.m_nic;
int nlec=sstate.m_nlec;
int nlic=sstate.m_nlic;
int nslack=n+2*(nec+nlec)+(nic+nlic);
int lccnt=nec+nic+nlec+nlic;
int nnz=0;
int offs=0;
int i=0;
int j=0;
//--- Create simplex solver.
//--- NOTE: we disable DSE pricing because it interferes with our
//--- warm-start strategy.
CRevisedDualSimplex::DSSSettingsInit(subsolver.m_dsssettings);
subsolver.m_dsssettings.m_pricing=0;
//--- Allocate temporaries
CApServ::RVectorSetLengthAtLeast(subsolver.m_cural,lccnt+n);
CApServ::RVectorSetLengthAtLeast(subsolver.m_curau,lccnt+n);
CApServ::RMatrixSetLengthAtLeast(subsolver.m_curd,n,n);
CApServ::RMatrixSetLengthAtLeast(subsolver.m_curhd,n,n);
CApServ::RVectorSetLengthAtLeast(subsolver.m_curbndl,nslack);
CApServ::RVectorSetLengthAtLeast(subsolver.m_curbndu,nslack);
CApServ::RVectorSetLengthAtLeast(subsolver.m_curb,nslack);
CApServ::RVectorSetLengthAtLeast(subsolver.m_sk,n);
CApServ::RVectorSetLengthAtLeast(subsolver.m_yk,n);
//--- Initial State
subsolver.m_basispresent=false;
subsolver.m_curdcnt=0;
subsolver.m_hessiantype=hessiantype;
if(hessiantype==1 || hessiantype==2)
{
//--- Prepare Hessian matrix
subsolver.m_h=matrix<double>::Identity(n,n);
}
//--- Linear constraints do not change across subiterations, that's
//--- why we allocate storage for them at the start of the program.
//--- A full set of "raw" constraints is stored; later we will filter
//--- out inequality ones which are inactive anywhere in the current
//--- trust region.
//--- NOTE: because sparserawlc object stores only linear constraint
//--- (linearizations of nonlinear ones are not stored) we
//--- allocate only minimum necessary space.
nnz=0;
for(i=0; i<nec+nic; i++)
{
for(j=0; j<n; j++)
if(sstate.m_scaledcleic.Get(i,j)!=0.0)
nnz++;
}
CApServ::IVectorSetLengthAtLeast(subsolver.m_sparserawlc.m_RIdx,nec+nic+1);
CApServ::RVectorSetLengthAtLeast(subsolver.m_sparserawlc.m_Vals,nnz);
CApServ::IVectorSetLengthAtLeast(subsolver.m_sparserawlc.m_Idx,nnz);
CApServ::IVectorSetLengthAtLeast(subsolver.m_sparserawlc.m_DIdx,nec+nic);
CApServ::IVectorSetLengthAtLeast(subsolver.m_sparserawlc.m_UIdx,nec+nic);
offs=0;
subsolver.m_sparserawlc.m_RIdx.Set(0,0);
for(i=0; i<nec+nic; i++)
{
for(j=0; j<n; j++)
if(sstate.m_scaledcleic.Get(i,j)!=0.0)
{
//--- Primary part of the matrix
subsolver.m_sparserawlc.m_Vals.Set(offs,sstate.m_scaledcleic.Get(i,j));
subsolver.m_sparserawlc.m_Idx.Set(offs,j);
offs++;
}
subsolver.m_sparserawlc.m_RIdx.Set(i+1,offs);
}
subsolver.m_sparserawlc.m_MatrixType=1;
subsolver.m_sparserawlc.m_NInitialized=subsolver.m_sparserawlc.m_RIdx[nec+nic];
subsolver.m_sparserawlc.m_M=nec+nic;
subsolver.m_sparserawlc.m_N=n;
CSparse::SparseInitDUIdx(subsolver.m_sparserawlc);
}
//+------------------------------------------------------------------+
//| Restarts LP subproblem (cleans the matrix of internally stored |
//| directions) |
//| INPUT PARAMETERS: |
//| SState - solver State |
//| Subsolver - SLP subproblem to initialize |
//+------------------------------------------------------------------+
void CNLCSLP::LPSubproblemRestart(CMinSLPState &sstate,
CMinSLPSubsolver &subsolver)
{
subsolver.m_curdcnt=0;
}
//+------------------------------------------------------------------+
//| Updates Hessian estimate |
//| INPUT PARAMETERS: |
//| SState - solver State |
//| Subsolver - SLP subproblem to initialize |
//+------------------------------------------------------------------+
void CNLCSLP::LPSubproblemUpdateHessian(CMinSLPState &sstate,
CMinSLPSubsolver &subsolver,
CRowDouble &x0,
CRowDouble &g0,
CRowDouble &x1,
CRowDouble &g1)
{
//--- create variables
int n=sstate.m_n;
double vv=0;
double v=0;
double v0=0;
double v1=0;
double v2=0;
double gk=0;
double sk=0;
double yk=0;
//--- check
if(subsolver.m_hessiantype==1 || subsolver.m_hessiantype==2)
{
CApServ::RVectorSetLengthAtLeast(subsolver.m_tmp0,n);
v=0;
v0=0;
v1=0;
v2=0;
for(int i=0; i<n; i++)
{
sk=x1[i]-x0[i];
yk=g1[i]-g0[i];
gk=g0[i];
v+=sk*yk;
v0=v0+sk*sk;
v1=v1+yk*yk;
v2=v2+gk*gk;
subsolver.m_sk.Set(i,sk);
subsolver.m_yk.Set(i,yk);
}
if(MathSqrt(v0)>MathMax(sstate.m_epsx,m_bfgstol) && MathSqrt(v1)>(m_bfgstol*MathSqrt(v2)) && v>m_bfgstol*MathSqrt(v0)*MathSqrt(v1))
{
//--- Update Hessian if following criteria hold:
//--- * MCINFO=1 (good step)
//--- * step length is large enough
//--- * |Yk| is large enough when compared with |G|
//--- * (Sk,Yk) is large enough when compared with |S| and |G|
vv=CAblas::RMatrixSyvMVect(n,subsolver.m_h,0,0,true,subsolver.m_sk,0,subsolver.m_tmp0);
CAblas::RMatrixGemVect(n,n,1.0,subsolver.m_h,0,0,0,subsolver.m_sk,0,0.0,subsolver.m_tmp0,0);
CAblas::RMatrixGer(n,n,subsolver.m_h,0,0,1/v,subsolver.m_yk,0,subsolver.m_yk,0);
CAblas::RMatrixGer(n,n,subsolver.m_h,0,0,-(1/vv),subsolver.m_tmp0,0,subsolver.m_tmp0,0);
}
}
}
//+------------------------------------------------------------------+
//| This function solves LP subproblem given by initial point X, |
//| function vector Fi and Jacobian Jac, and returns estimates of |
//| Lagrangian multipliers and search direction D[]. |
//| This function does NOT append search direction D to conjugacy |
//| constraints, you have to use |
//| LPSubproblemAppendConjugacyConstraint(). |
//+------------------------------------------------------------------+
bool CNLCSLP::LPSubproblemSolve(CMinSLPState &State,
CMinSLPSubsolver &subsolver,
CRowDouble &x,
CRowDouble &fi,
CMatrixDouble &jac,
int innerk,
CRowDouble &d,
CRowDouble &lagmult)
{
//--- create variables
bool result=false;
int n=State.m_n;
int nec=State.m_nec;
int nic=State.m_nic;
int nlec=State.m_nlec;
int nlic=State.m_nlic;
int nslack=n+2*(nec+nlec)+(nic+nlic);
int lccnt=nec+nic+nlec+nlic;
//--- Locations of slack variables
int offsslackec=n;
int offsslacknlec=n+2*nec;
int offsslackic=n+2*nec+2*nlec;
int offsslacknlic=n+2*(nec+nlec)+nic;
int i=0;
int j=0;
int k=0;
double v=0;
double vv=0;
double vright=0;
double vmax=0;
int basisinittype=0;
int offs=0;
int nnz=0;
int j0=0;
int j1=0;
//--- Prepare temporary structures
subsolver.m_cural.Resize(lccnt+subsolver.m_curdcnt);
subsolver.m_curau.Resize(lccnt+subsolver.m_curdcnt);
//--- Prepare default solution: all zeros
result=true;
d.Fill(0.0);
lagmult.Fill(0);
//--- Linear term B
//--- NOTE: elements [N,NSlack) are equal to bigC + perturbation to improve numeric properties of LP problem
for(i=0; i<n; i++)
subsolver.m_curb.Set(i,jac.Get(0,i));
v=0;
for(i=0; i<n; i++)
v+=CMath::Sqr(jac.Get(0,i));
v=CApServ::Coalesce(MathSqrt(v),1.0);
for(i=n; i<nslack; i++)
subsolver.m_curb.Set(i,(State.m_bigc+1.0/(1+i))*v);
//--- Trust radius constraints for primary variables
for(i=0; i<n; i++)
{
subsolver.m_curbndl.Set(i,-State.m_trustrad);
subsolver.m_curbndu.Set(i,State.m_trustrad);
if(State.m_HasBndL[i])
subsolver.m_curbndl.Set(i,MathMax(subsolver.m_curbndl[i],State.m_scaledbndl[i]-x[i]));
if(State.m_HasBndU[i])
subsolver.m_curbndu.Set(i,MathMin(subsolver.m_curbndu[i],State.m_scaledbndu[i]-x[i]));
}
//--- Prepare storage for "effective" constraining matrix
nnz=subsolver.m_sparserawlc.m_RIdx[nec+nic];
for(i=0; i<nlec+nlic; i++)
{
for(j=0; j<n; j++)
if(jac.Get(1+i,j)!=0.0)
nnz++;
}
nnz=nnz+2*nec+nic;
nnz=nnz+2*nlec+nlic;
nnz=nnz+subsolver.m_curdcnt*n;
subsolver.m_sparseefflc.m_RIdx.Resize(lccnt+n+1);
subsolver.m_sparseefflc.m_Vals.Resize(nnz);
subsolver.m_sparseefflc.m_Idx.Resize(nnz);
CApServ::IVectorSetLengthAtLeast(subsolver.m_sparseefflc.m_DIdx,lccnt+n);
CApServ::IVectorSetLengthAtLeast(subsolver.m_sparseefflc.m_UIdx,lccnt+n);
subsolver.m_sparseefflc.m_M=0;
subsolver.m_sparseefflc.m_N=nslack;
subsolver.m_sparseefflc.m_MatrixType=1;
//--- Append linear equality/inequality constraints
//--- Scan sparsified linear constraints stored in sparserawlc[], skip ones
//--- which are inactive anywhere in the trust region.
CApServ::RVectorSetLengthAtLeast(subsolver.m_tmp0,nslack);
for(i=0; i<n; i++)
subsolver.m_tmp0.Set(i,x[i]);
for(i=n; i<nslack; i++)
subsolver.m_tmp0.Set(i,0);
for(i=0; i<nec+nic; i++)
{
//--- Calculate:
//--- * VRight - product of X[] (extended with zeros up to NSlack elements)
//--- and AR[i] - Ith row of sparserawlc matrix.
//--- * VMax - maximum value of X*ARi computed over trust region
vright=0;
vmax=0;
j0=subsolver.m_sparserawlc.m_RIdx[i];
j1=subsolver.m_sparserawlc.m_RIdx[i+1];
for(k=j0; k<j1; k++)
{
j=subsolver.m_sparserawlc.m_Idx[k];
v=subsolver.m_tmp0[j];
vv=subsolver.m_sparserawlc.m_Vals[k];
vright=vright+vv*v;
if(vv>=0)
vmax+=vv*(v+subsolver.m_curbndu[j]);
else
vmax+=vv*(v+subsolver.m_curbndl[j]);
}
//--- If constraint is an inequality one and guaranteed to be inactive
//--- within trust region, it is skipped (row itself is retained but
//--- filled by zeros).
if(i>=nec && vmax<=State.m_scaledcleic.Get(i,n))
{
offs=subsolver.m_sparseefflc.m_RIdx[i];
subsolver.m_sparseefflc.m_Vals.Set(offs,-1);
subsolver.m_sparseefflc.m_Idx.Set(offs,offsslackic+(i-nec));
subsolver.m_sparseefflc.m_RIdx.Set(i+1,offs+1);
subsolver.m_cural.Set(i,0.0);
subsolver.m_curau.Set(i,0.0);
subsolver.m_curbndl.Set(offsslackic+(i-nec),0);
subsolver.m_curbndu.Set(offsslackic+(i-nec),0);
continue;
}
//--- Start working on row I
offs=subsolver.m_sparseefflc.m_RIdx[i];
//--- Copy constraint from sparserawlc[] to sparseefflc[]
j0=subsolver.m_sparserawlc.m_RIdx[i];
j1=subsolver.m_sparserawlc.m_RIdx[i+1];
for(k=j0; k<j1; k++)
{
subsolver.m_sparseefflc.m_Idx.Set(offs,subsolver.m_sparserawlc.m_Idx[k]);
subsolver.m_sparseefflc.m_Vals.Set(offs,subsolver.m_sparserawlc.m_Vals[k]);
offs++;
}
//--- Set up slack variables
if(i<nec)
{
subsolver.m_sparseefflc.m_Vals.Set(offs,-1);
subsolver.m_sparseefflc.m_Vals.Set(offs+1,1);
subsolver.m_sparseefflc.m_Idx.Set(offs,offsslackec+2*i);
subsolver.m_sparseefflc.m_Idx.Set(offs+1,offsslackec+2*i+1);
offs+=2;
}
else
{
//--- Slack variables for inequality constraints
subsolver.m_sparseefflc.m_Vals.Set(offs,-1);
subsolver.m_sparseefflc.m_Idx.Set(offs,offsslackic+(i-nec));
offs++;
}
//--- Finalize row
subsolver.m_sparseefflc.m_RIdx.Set(i+1,offs);
//--- Set up bounds.
//--- NOTE: bounds for equality and inequality constraints are
//--- handled differently
v=vright-State.m_scaledcleic.Get(i,n);
if(i<nec)
{
subsolver.m_cural.Set(i,-v);
subsolver.m_curau.Set(i,-v);
subsolver.m_curbndl.Set(offsslackec+2*i,0);
subsolver.m_curbndl.Set(offsslackec+2*i+1,0);
subsolver.m_curbndu.Set(offsslackec+2*i,MathAbs(v));
subsolver.m_curbndu.Set(offsslackec+2*i+1,MathAbs(v));
}
else
{
subsolver.m_cural.Set(i,AL_NEGINF);
subsolver.m_curau.Set(i,-v);
subsolver.m_curbndl.Set(offsslackic+(i-nec),0);
subsolver.m_curbndu.Set(offsslackic+(i-nec),MathMax(v,0));
}
}
subsolver.m_sparseefflc.m_M+=(nec+nic);
//--- Append nonlinear equality/inequality constraints
for(i=0; i<nlec+nlic; i++)
{
//--- Calculate scale coefficient
vv=0;
for(j=0; j<n; j++)
{
v=jac.Get(1+i,j);
vv+=v*v;
}
vv=1/CApServ::Coalesce(MathSqrt(vv),1);
//--- Copy scaled row
offs=subsolver.m_sparseefflc.m_RIdx[subsolver.m_sparseefflc.m_M+i];
for(j=0; j<n; j++)
{
if(jac.Get(1+i,j)!=0.0)
{
subsolver.m_sparseefflc.m_Vals.Set(offs,vv*jac.Get(1+i,j));
subsolver.m_sparseefflc.m_Idx.Set(offs,j);
offs++;
}
}
if(i<nlec)
{
//--- Add slack terms for equality constraints
subsolver.m_sparseefflc.m_Vals.Set(offs,-1);
subsolver.m_sparseefflc.m_Vals.Set(offs+1,1);
subsolver.m_sparseefflc.m_Idx.Set(offs,offsslacknlec+2*i);
subsolver.m_sparseefflc.m_Idx.Set(offs+1,offsslacknlec+2*i+1);
offs+=2;
}
else
{
//--- Add slack terms for inequality constraints
subsolver.m_sparseefflc.m_Vals.Set(offs,-1);
subsolver.m_sparseefflc.m_Idx.Set(offs,offsslacknlic+(i-nlec));
offs++;
}
subsolver.m_sparseefflc.m_RIdx.Set(subsolver.m_sparseefflc.m_M+i+1,offs);
//--- Set box constraints on slack variables and bounds on linear equality/inequality constraints
v=vv*fi[1+i];
if(i<nlec)
{
//--- Equality constraint
subsolver.m_cural.Set(subsolver.m_sparseefflc.m_M+i,-v);
subsolver.m_curau.Set(subsolver.m_sparseefflc.m_M+i,-v);
subsolver.m_curbndl.Set(offsslacknlec+2*i,0);
subsolver.m_curbndl.Set(offsslacknlec+2*i+1,0);
subsolver.m_curbndu.Set(offsslacknlec+2*i,MathAbs(v));
subsolver.m_curbndu.Set(offsslacknlec+2*i+1,MathAbs(v));
}
else
{
//--- Inequality constraint
subsolver.m_cural.Set(subsolver.m_sparseefflc.m_M+i,AL_NEGINF);
subsolver.m_curau.Set(subsolver.m_sparseefflc.m_M+i,-v);
subsolver.m_curbndl.Set(offsslacknlic+(i-nlec),0);
subsolver.m_curbndu.Set(offsslacknlic+(i-nlec),MathMax(v,0));
}
}
subsolver.m_sparseefflc.m_M+=(nlec+nlic);
//--- Append conjugacy constraints
for(i=0; i<subsolver.m_curdcnt; i++)
{
//--- Copy N elements of CurHD
//--- NOTE: we expect product of D and H to be dense, so we copy all N elements
v=0;
for(j=0; j<n; j++)
{
vv=subsolver.m_curhd.Get(i,j);
v+=vv*vv;
}
v=1.0/CApServ::Coalesce(MathSqrt(v),1.0);
offs=subsolver.m_sparseefflc.m_RIdx[subsolver.m_sparseefflc.m_M];
for(j=0; j<n; j++)
{
vv=subsolver.m_curhd.Get(i,j);
subsolver.m_sparseefflc.m_Vals.Set(offs,v*vv);
subsolver.m_sparseefflc.m_Idx.Set(offs,j);
offs++;
}
subsolver.m_sparseefflc.m_RIdx.Set(subsolver.m_sparseefflc.m_M+1,offs);
//--- Set bounds on linear constraints
subsolver.m_cural.Set(subsolver.m_sparseefflc.m_M,0);
subsolver.m_curau.Set(subsolver.m_sparseefflc.m_M,0);
//--- Increase row count
subsolver.m_sparseefflc.m_M ++;
}
//--- Finalize sparse matrix structure
if(!CAp::Assert(subsolver.m_sparseefflc.m_RIdx[subsolver.m_sparseefflc.m_M]<=CAp::Len(subsolver.m_sparseefflc.m_Idx),"LPSubproblemSolve: critical integrity check failed"))
return(false);
if(!CAp::Assert(subsolver.m_sparseefflc.m_RIdx[subsolver.m_sparseefflc.m_M]<=CAp::Len(subsolver.m_sparseefflc.m_Vals),"LPSubproblemSolve: critical integrity check failed"))
return(false);
subsolver.m_sparseefflc.m_NInitialized=subsolver.m_sparseefflc.m_RIdx[subsolver.m_sparseefflc.m_M];
CSparse::SparseInitDUIdx(subsolver.m_sparseefflc);
//--- Choose dual simplex method basis initialization type
if(innerk==1 && subsolver.m_basispresent)
basisinittype=2;
else
basisinittype=1;
//--- Solve linear program
CApServ::RVectorSetLengthAtLeast(subsolver.m_tmp0,nslack);
subsolver.m_tmp0.Fill(State.m_trustrad);
CLPQPPresolve::PresolveNoneScaleUser(subsolver.m_tmp0,subsolver.m_curb,subsolver.m_curbndl,subsolver.m_curbndu,nslack,subsolver.m_sparseefflc,subsolver.m_cural,subsolver.m_curau,subsolver.m_sparseefflc.m_M,subsolver.m_presolver);
CRevisedDualSimplex::DSSInit(subsolver.m_presolver.m_newn,subsolver.m_dss);
CRevisedDualSimplex::DSSSetProblem(subsolver.m_dss,subsolver.m_presolver.m_c,subsolver.m_presolver.m_bndl,subsolver.m_presolver.m_bndu,subsolver.m_densedummy,subsolver.m_presolver.m_sparsea,1,subsolver.m_presolver.m_al,subsolver.m_presolver.m_au,subsolver.m_presolver.m_newm,subsolver.m_lastbasis,basisinittype,subsolver.m_dsssettings);
CRevisedDualSimplex::DSSOptimize(subsolver.m_dss,subsolver.m_dsssettings);
subsolver.m_xs=subsolver.m_dss.m_repx;
subsolver.m_lagbc=subsolver.m_dss.m_replagbc;
subsolver.m_laglc=subsolver.m_dss.m_replaglc;
subsolver.m_cs=subsolver.m_dss.m_repstats;
CLPQPPresolve::PresolveBwd(subsolver.m_presolver,subsolver.m_xs,subsolver.m_cs,subsolver.m_lagbc,subsolver.m_laglc);
State.m_repsimplexiterations+=subsolver.m_dss.m_repiterationscount;
State.m_repsimplexiterations1+=subsolver.m_dss.m_repiterationscount1;
State.m_repsimplexiterations2+=subsolver.m_dss.m_repiterationscount2;
State.m_repsimplexiterations3+=subsolver.m_dss.m_repiterationscount3;
if(subsolver.m_dss.m_repterminationtype<=0)
{
//--- LP solver failed due to numerical errors; exit
result=false;
return(result);
}
if(innerk==1)
{
//--- Store basis
CRevisedDualSimplex::DSSExportBasis(subsolver.m_dss,subsolver.m_lastbasis);
subsolver.m_basispresent=true;
}
//--- Extract direction D[] and Lagrange multipliers
d=subsolver.m_xs;
lagmult=subsolver.m_laglc;
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function appends last search direction D to conjugacy |
//| constraints of the LP subproblem. |
//+------------------------------------------------------------------+
void CNLCSLP::LPSubproblemAppendConjugacyConstraint(CMinSLPState &State,
CMinSLPSubsolver &subsolver,
CRowDouble &d)
{
int n=State.m_n;
//--- Update matrix of products H*Dprev
if(!CAp::Assert(subsolver.m_curdcnt<CAp::Rows(subsolver.m_curd),__FUNCTION__+": CurD is too small"))
return;
//--- Store direction and default conjugacy constraint d'*I*Dprev=0
subsolver.m_curd.Row(subsolver.m_curdcnt,d);
subsolver.m_curhd.Row(subsolver.m_curdcnt,d);
subsolver.m_curdcnt++;
if(State.m_hessiantype==1)
{
//--- Conjugacy constraint d*H*Dprev=0, full recomputation of (H*Dprev)
CAblas::RMatrixGemm(subsolver.m_curdcnt,n,n,1.0,subsolver.m_curd,0,0,0,subsolver.m_h,0,0,0,0.0,subsolver.m_curhd,0,0);
}
if(State.m_hessiantype==2)
{
//--- Conjugacy constraint d*H*Dprev=0, only last row of (H*Dprev) is recomputed
CApServ::RVectorSetLengthAtLeast(subsolver.m_tmp0,n);
CAblas::RMatrixGemVect(n,n,1.0,subsolver.m_h,0,0,0,d,0,0.0,subsolver.m_tmp0,0);
subsolver.m_curhd.Row(subsolver.m_curdcnt-1,subsolver.m_tmp0);
}
}
//+------------------------------------------------------------------+
//| This function initializes Phase13 temporaries. It should be |
//| called before beginning of each new iteration. You may call it |
//| multiple times for the same instance of Phase13 temporaries. |
//| INPUT PARAMETERS: |
//| State13 - instance to be initialized. |
//| N - problem dimensionality |
//| NEC, NIC - linear equality / inequality constraint count |
//| NLEC, NLIC - nonlinear equality / inequality constraint count|
//| UseCorrection - True if we want to perform second order |
//| correction |
//| OUTPUT PARAMETERS: |
//| State13 - instance being initialized |
//+------------------------------------------------------------------+
void CNLCSLP::Phase13Init(CMinSLPPhase13State &state13,
int n,
int nec,
int nic,
int nlec,
int nlic,
bool usecorrection)
{
//--- create variables
int nslack=n+2*(nec+nlec)+(nic+nlic);
state13.m_usecorrection=usecorrection;
//--- allocate
CApServ::RVectorSetLengthAtLeast(state13.m_d,nslack);
CApServ::RVectorSetLengthAtLeast(state13.m_dx,nslack);
CApServ::RVectorSetLengthAtLeast(state13.m_stepkxc,n);
CApServ::RVectorSetLengthAtLeast(state13.m_stepkxn,n);
CApServ::RVectorSetLengthAtLeast(state13.m_stepkfic,1+nlec+nlic);
CApServ::RVectorSetLengthAtLeast(state13.m_stepkfin,1+nlec+nlic);
CApServ::RMatrixSetLengthAtLeast(state13.m_stepkjc,1+nlec+nlic,n);
CApServ::RMatrixSetLengthAtLeast(state13.m_stepkjn,1+nlec+nlic,n);
CApServ::RVectorSetLengthAtLeast(state13.m_dummylagmult,nec+nic+nlec+nlic);
state13.m_rphase13state.ia.Resize(8+1);
ArrayResize(state13.m_rphase13state.ba,2+1);
state13.m_rphase13state.ra.Resize(6+1);
state13.m_rphase13state.stage=-1;
}
//+------------------------------------------------------------------+
//| This function tries to perform either phase #1 or phase #3 step. |
//| Former corresponds to linear model step (without conjugacy |
//| constraints) with correction for nonlinearity ("second order |
//| correction"). Such correction helps to overcome Maratos effect |
//| (a tendency of L1 penalized merit functions to reject nonzero |
//| steps). |
//| Latter is a step using linear model with no second order |
//| correction. |
//| INPUT PARAMETERS: |
//| State - SLP solver State |
//| SMonitor - smoothness monitor |
//| UserTerminationNeeded - True if user requested termination |
//| CurX - current point, array[N] |
//| CurFi - function vector at CurX, array[1 + NLEC + NLIC] |
//| CurJ - Jacobian at CurX, array[1 + NLEC + NLIC, N] |
//| LagMult - array[NEC + NIC + NLEC + NLIC], contents ignored |
//| on input. |
//| OUTPUT PARAMETERS: |
//| State - RepTerminationType is set to current termination |
//| code (if Status = 0). |
//| CurX - advanced to new point |
//| CurFi - updated with function vector at CurX[] |
//| CurJ - updated with Jacobian at CurX[] |
//| LagMult - filled with current Lagrange multipliers |
//| Status - when reverse communication is done, Status is set |
//| to: |
//| * negative value, if we have to restart outer |
//| iteration |
//| * positive value, if we can proceed to the next |
//| stage of the outer iteration |
//| * zero, if algorithm is terminated |
//| (RepTerminationType is set to appropriate value) |
//| DNrm - inf-norm of the proposed step vector D |
//| Stp - step length(multiplier for D), in [0, 1] |
//+------------------------------------------------------------------+
bool CNLCSLP::Phase13Iteration(CMinSLPState &State,
CMinSLPPhase13State &state13,
CSmoothnessMonitor &smonitor,
bool userterminationneeded,
CRowDouble &curx,
CRowDouble &curfi,
CMatrixDouble &curj,
CRowDouble &lagmult,
int &status,
double &dnrm,
double &stp)
{
//--- create variables
bool result=false;
int n=0;
int nslack=0;
int nec=0;
int nic=0;
int nlec=0;
int nlic=0;
int innerk=0;
int i=0;
int j=0;
double v=0;
double mx=0;
double f0=0;
double f1=0;
double nu=0;
double localstp=0;
double mu=0;
bool dotrace=false;
bool doprobing=false;
bool dotracexd=false;
int label=-1;
//--- Reverse communication preparations
//--- I know it looks ugly, but it works the same way
//--- anywhere from C++ to Python.
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(state13.m_rphase13state.stage>=0)
{
n=state13.m_rphase13state.ia[0];
nslack=state13.m_rphase13state.ia[1];
nec=state13.m_rphase13state.ia[2];
nic=state13.m_rphase13state.ia[3];
nlec=state13.m_rphase13state.ia[4];
nlic=state13.m_rphase13state.ia[5];
innerk=state13.m_rphase13state.ia[6];
i=state13.m_rphase13state.ia[7];
j=state13.m_rphase13state.ia[8];
dotrace=state13.m_rphase13state.ba[0];
doprobing=state13.m_rphase13state.ba[1];
dotracexd=state13.m_rphase13state.ba[2];
v=state13.m_rphase13state.ra[0];
mx=state13.m_rphase13state.ra[1];
f0=state13.m_rphase13state.ra[2];
f1=state13.m_rphase13state.ra[3];
nu=state13.m_rphase13state.ra[4];
localstp=state13.m_rphase13state.ra[5];
mu=state13.m_rphase13state.ra[6];
}
else
{
n=922;
nslack=-154;
nec=306;
nic=-1011;
nlec=951;
nlic=-463;
innerk=88;
i=-861;
j=-678;
dotrace=true;
doprobing=true;
dotracexd=true;
v=-233;
mx=-936;
f0=-279;
f1=94;
nu=-812;
localstp=427;
mu=178;
}
switch(state13.m_rphase13state.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
case 3:
label=3;
break;
default:
//--- Routine body
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
nlec=State.m_nlec;
nlic=State.m_nlic;
nslack=n+2*(nec+nlec)+(nic+nlic);
innerk=1;
dotrace=CAp::IsTraceEnabled("SLP");
dotracexd=dotrace && CAp::IsTraceEnabled("SLP.DETAILED");
doprobing=CAp::IsTraceEnabled("SLP.PROBING");
if(!CAp::Assert(CAp::Len(lagmult)>=nec+nic+nlec+nlic,"Phase13Iteration: integrity check failed"))
return(false);
//--- Report iteration beginning
if(dotrace)
{
if(state13.m_usecorrection)
CAp::Trace("\n--- linear step with second-order correction -------------------------------------------------------\n");
else
CAp::Trace("\n--- linear step without second-order correction ----------------------------------------------------\n");
}
//--- Default decision is to continue algorithm
status=1;
stp=0;
dnrm=0;
//--- Determine step direction using linearized model with no conjugacy terms
LPSubproblemRestart(State,State.m_subsolver);
if(!LPSubproblemSolve(State,State.m_subsolver,curx,curfi,curj,innerk,state13.m_d,lagmult))
{
if(dotrace)
CAp::Trace("> [WARNING] initial phase #1 LP subproblem failed\n");
//--- Increase failures counter.
//--- Stop after too many subsequent failures
State.m_lpfailurecnt++;
if(State.m_lpfailurecnt>=m_lpfailureslimit)
{
State.m_repterminationtype=7;
status=0;
if(dotrace)
CAp::Trace("> stopping condition met: too many phase #1 LP failures\n");
result=false;
return(result);
}
//--- Can not solve LP subproblem, decrease trust radius
State.m_trustrad=0.5*State.m_trustrad;
if(dotrace)
CAp::Trace(StringFormat("> trust radius was decreased to {0,0:E4}\n",State.m_trustrad));
if(State.m_trustrad<State.m_epsx)
{
State.m_repterminationtype=2;
status=0;
if(dotrace)
CAp::Trace(StringFormat("> stopping condition met: trust radius is smaller than %.3E\n",State.m_epsx));
}
else
status=-1;
result=false;
return(result);
}
mu=MathMax(CAblasF::RMaxAbsV(State.m_historylen,State.m_maxlaghistory),CAblasF::RMaxAbsV(nec+nic+nlec+nlic,lagmult));
mu=CApServ::Coalesce(mu,m_defaultl1penalty);
//--- Compute second order correction if required. The issue we address here
//--- is a tendency of L1 penalized function to reject steps built using simple
//--- linearized model when nonlinear constraints change faster than the target.
//--- The idea is that we perform trial step (stp=1) using simple linearized model,
//--- compute constraint vector at the new trial point - and use these updated
//--- constraint linearizations back at the initial point.
if(!state13.m_usecorrection)
{
label=4;
break;
}
//--- Perform trial step using vector D to StepKXC
state13.m_stepkxc=curx+state13.m_d+0;
SLPSendX(State,state13.m_stepkxc);
State.m_needfij=true;
state13.m_rphase13state.stage=0;
label=-1;
break;
}
while(label>=0)
{
switch(label)
{
case 0:
State.m_needfij=false;
if(!SLPRetrieveFIJ(State,state13.m_stepkfic,state13.m_stepkjc))
{
//--- Failed to retrieve func/Jac, infinities detected
State.m_repterminationtype=-8;
status=0;
if(dotrace)
CAp::Trace("[ERROR] infinities in target/constraints are detected\n");
result=false;
return(result);
}
//--- Move back to point CurX[], restore original linearization of the target
state13.m_stepkfic.Set(0,curfi[0]);
state13.m_stepkxc=curx;
state13.m_stepkjc.Row(0,curj[0]+0);
//--- Extrapolate linearization of nonlinear constraints back to origin
for(i=1; i<=nlec+nlic; i++)
{
v=state13.m_d.Dot(state13.m_stepkjc[i]+0);
state13.m_stepkfic.Add(i,-v);
}
//--- Solve linearized problem one more time, now with new linearization of constraints
//--- (but still old linearization of the target), obtain DX
//--- NOTE: because LPSubproblemRestart() call resets set of conjugate constraints, we
//--- have to re-add it after solve.
LPSubproblemRestart(State,State.m_subsolver);
if(!LPSubproblemSolve(State,State.m_subsolver,state13.m_stepkxc,state13.m_stepkfic,state13.m_stepkjc,innerk,state13.m_dx,state13.m_dummylagmult))
{
//--- Second LP subproblem failed.
//--- Noncritical failure, can be ignored,
if(dotrace)
CAp::Trace("> [WARNING] second phase #1 LP subproblem failed\n");
if(dotrace)
CAp::Trace("> using step without second order correction\n");
}
else
{
//--- Set D to new direction
state13.m_d=state13.m_dx;
}
case 4:
//--- Now we have search direction in D:
//--- * compute DNrm
//--- * append D to the list of the conjugacy constraints, so next time when we use the solver we will
//--- automatically produce conjugate direction
dnrm=CAblasF::RMaxAbsV(n,state13.m_d);
LPSubproblemAppendConjugacyConstraint(State,State.m_subsolver,state13.m_d);
//--- Perform merit function backtracking line search, with trial point being
//--- computed as XN = XK + Stp*D, with Stp in [0,1]
//--- NOTE: we use MeritLagMult - Lagrange multipliers computed for initial,
//--- uncorrected task - for the merit function model.
//--- Using DummyLagMult can destabilize algorithm.
localstp=1.0;
nu=0.5;
f0=MeritFunction(State,curx,curfi,lagmult,mu,state13.m_tmpmerit);
f1=f0;
COptServ::SmoothnessMonitorStartLineSearch(smonitor,curx,curfi,curj);
case 6:
for(i=0; i<n; i++)
state13.m_stepkxn.Set(i,curx[i]+state13.m_d[i]*localstp);
SLPSendX(State,state13.m_stepkxn);
State.m_needfij=true;
state13.m_rphase13state.stage=1;
label=-1;
break;
case 1:
State.m_needfij=false;
if(!SLPRetrieveFIJ(State,state13.m_stepkfin,state13.m_stepkjn))
{
//--- Failed to retrieve func/Jac, infinities detected
State.m_repterminationtype=-8;
status=0;
if(dotrace)
CAp::Trace("[ERROR] infinities in target/constraints are detected\n");
result=false;
return(result);
}
COptServ::SmoothnessMonitorEnqueuePoint(smonitor,state13.m_d,localstp,state13.m_stepkxn,state13.m_stepkfin,state13.m_stepkjn);
f1=MeritFunction(State,state13.m_stepkxn,state13.m_stepkfin,lagmult,mu,state13.m_tmpmerit);
if(f1<f0)
{
//--- Step is found!
label=7;
break;
}
if(localstp<0.001)
{
//--- Step is shorter than 0.001 times current search direction,
//--- it means that no good step can be found.
localstp=0;
SLPCopyState(State,curx,curfi,curj,state13.m_stepkxn,state13.m_stepkfin,state13.m_stepkjn);
label=7;
break;
}
localstp=nu*localstp;
nu=MathMax(0.1,0.5*nu);
label=6;
break;
case 7:
COptServ::SmoothnessMonitorFinalizeLineSearch(smonitor);
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i])
state13.m_stepkxn.Set(i,MathMax(state13.m_stepkxn[i],State.m_scaledbndl[i]));
if(State.m_HasBndU[i])
state13.m_stepkxn.Set(i,MathMin(state13.m_stepkxn[i],State.m_scaledbndu[i]));
}
if(userterminationneeded)
{
//--- User requested termination, break before we move to new point
State.m_repterminationtype=8;
status=0;
if(dotrace)
CAp::Trace("> user requested termination\n");
result=false;
return(result);
}
//--- Trace
if(!dotrace)
{
label=8;
break;
}
if(!doprobing)
{
label=10;
break;
}
COptServ::SmoothnessMonitorStartProbing(smonitor,1.0,2,State.m_trustrad);
case 12:
if(!COptServ::SmoothnessMonitorProbe(smonitor))
{
label=13;
break;
}
for(j=0; j<n; j++)
{
state13.m_stepkxc.Set(j,curx[j]+state13.m_d[j]*smonitor.m_probingstp);
if(State.m_HasBndL[j])
state13.m_stepkxc.Set(j,MathMax(state13.m_stepkxc[j],State.m_scaledbndl[j]));
if(State.m_HasBndU[j])
state13.m_stepkxc.Set(j,MathMin(state13.m_stepkxc[j],State.m_scaledbndu[j]));
}
SLPSendX(State,state13.m_stepkxc);
State.m_needfij=true;
state13.m_rphase13state.stage=2;
label=-1;
break;
case 2:
State.m_needfij=false;
if(!SLPRetrieveFIJ(State,state13.m_stepkfic,state13.m_stepkjc))
{
label=13;
break;
}
smonitor.m_probingf.Set(0,RawLagrangian(State,state13.m_stepkxc,state13.m_stepkfic,lagmult,state13.m_tmpmerit));
smonitor.m_probingf.Set(1,state13.m_stepkfic[0]);
label=12;
break;
case 13:
CAp::Trace("*** ------------------------------------------------------------\n");
CAp::Trace("*** | probing search direction suggested by LP subproblem |\n");
CAp::Trace("*** ------------------------------------------------------------\n");
CAp::Trace("*** | Step | Lagrangian (unaugmentd)| Target function |\n");
CAp::Trace("*** |along D| must be smooth | must be smooth |\n");
CAp::Trace("*** | | function | slope | function | slope |\n");
COptServ::SmoothnessMonitorTraceProbingResults(smonitor);
case 10:
mx=0;
for(i=0; i<n; i++)
mx=MathMax(mx,MathAbs(state13.m_d[i])/State.m_trustrad);
if(localstp>0.0)
CAp::Trace("> nonzero linear step was performed\n");
else
CAp::Trace("> zero linear step was performed\n");
CAp::Trace(StringFormat("max(|Di|)/TrustRad = %.6f\n",mx));
CAp::Trace(StringFormat("stp = %.6f\n",localstp));
if(dotracexd)
{
CAp::Trace("X0 (scaled) = ");
CApServ::TraceVectorAutopRec(curx,0,n);
CAp::Trace("\n");
CAp::Trace("D (scaled) = ");
CApServ::TraceVectorAutopRec(state13.m_d,0,n);
CAp::Trace("\n");
CAp::Trace("X1 (scaled) = ");
CApServ::TraceVectorAutopRec(state13.m_stepkxn,0,n);
CAp::Trace("\n");
}
CAp::Trace(StringFormat("meritF: %14.6E -> %14.6E (delta=%11.3E)\n",f0,f1,f1 - f0));
CAp::Trace(StringFormat("scaled-targetF: %14.6E -> %14.6E (delta=%11.3E)\n",curfi[0],state13.m_stepkfin[0],state13.m_stepkfin[0] - curfi[0]));
case 8:
//--- Move to new point
stp=localstp;
SLPCopyState(State,state13.m_stepkxn,state13.m_stepkfin,state13.m_stepkjn,curx,curfi,curj);
if(localstp<=0.0)
{
label=14;
break;
}
//--- Report one more inner iteration
State.m_repinneriterationscount++;
SLPSendX(State,curx);
State.m_f=curfi[0]*State.m_fscales[0];
State.m_xupdated=true;
state13.m_rphase13state.stage=3;
label=-1;
break;
case 3:
State.m_xupdated=false;
//--- Update constraint violations
COptServ::CheckLcViolation(State.m_scaledcleic,State.m_lcsrcidx,nec,nic,curx,n,State.m_replcerr,State.m_replcidx);
COptServ::UnScaleAndCheckNLcViolation(curfi,State.m_fscales,nlec,nlic,State.m_repnlcerr,State.m_repnlcidx);
case 14:
result=false;
return(result);
}
}
//--- Saving State
result=true;
state13.m_rphase13state.ba[0]=dotrace;
state13.m_rphase13state.ba[1]=doprobing;
state13.m_rphase13state.ba[2]=dotracexd;
state13.m_rphase13state.ia.Set(0,n);
state13.m_rphase13state.ia.Set(1,nslack);
state13.m_rphase13state.ia.Set(2,nec);
state13.m_rphase13state.ia.Set(3,nic);
state13.m_rphase13state.ia.Set(4,nlec);
state13.m_rphase13state.ia.Set(5,nlic);
state13.m_rphase13state.ia.Set(6,innerk);
state13.m_rphase13state.ia.Set(7,i);
state13.m_rphase13state.ia.Set(8,j);
state13.m_rphase13state.ra.Set(0,v);
state13.m_rphase13state.ra.Set(1,mx);
state13.m_rphase13state.ra.Set(2,f0);
state13.m_rphase13state.ra.Set(3,f1);
state13.m_rphase13state.ra.Set(4,nu);
state13.m_rphase13state.ra.Set(5,localstp);
state13.m_rphase13state.ra.Set(6,mu);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function initializes Phase2 temporaries. It should be called|
//| before beginning of each new iteration. You may call it multiple |
//| times for the same instance of Phase2 temporaries. |
//| INPUT PARAMETERS: |
//| State2 - instance to be initialized. |
//| N - problem dimensionality |
//| NEC, NIC - linear equality / inequality constraint count |
//| NLEC, NLIC - nonlinear equality / inequality constraint count|
//| MeritLagMult - Lagrange multiplier estimates used by merit |
//| function (we could use ones computed during |
//| phase #2, but these may differ from ones |
//| computed initially at the beginning of the outer|
//| iteration, so it may confuse algorithm) |
//| OUTPUT PARAMETERS: |
//| State2 - instance being initialized |
//+------------------------------------------------------------------+
void CNLCSLP::Phase2Init(CMinSLPPhase2State &state2,
int n,
int nec,
int nic,
int nlec,
int nlic,
CRowDouble &meritlagmult)
{
int nslack=n+2*(nec+nlec)+(nic+nlic);
//--- allocate
CApServ::RVectorSetLengthAtLeast(state2.m_d,nslack);
CApServ::RVectorSetLengthAtLeast(state2.m_tmp0,nslack);
CApServ::RVectorSetLengthAtLeast(state2.m_stepkxn,n);
CApServ::RVectorSetLengthAtLeast(state2.m_stepkxc,n);
CApServ::RVectorSetLengthAtLeast(state2.m_stepkfin,1+nlec+nlic);
CApServ::RVectorSetLengthAtLeast(state2.m_stepkfic,1+nlec+nlic);
CApServ::RMatrixSetLengthAtLeast(state2.m_stepkjn,1+nlec+nlic,n);
CApServ::RMatrixSetLengthAtLeast(state2.m_stepkjc,1+nlec+nlic,n);
CApServ::RVectorSetLengthAtLeast(state2.m_stepklaggrad,n);
CApServ::RVectorSetLengthAtLeast(state2.m_stepknlaggrad,n);
CApServ::RVectorSetLengthAtLeast(state2.m_stepknlagmult,nec+nic+nlec+nlic);
CApServ::RVectorSetLengthAtLeast(state2.m_meritlagmult,nec+nic+nlec+nlic);
state2.m_meritlagmult=meritlagmult;
state2.m_rphase2state.ia.Resize(12+1);
ArrayResize(state2.m_rphase2state.ba,3);
state2.m_rphase2state.ra.Resize(10);
state2.m_rphase2state.stage=-1;
}
//+------------------------------------------------------------------+
//| This function tries to perform phase #2 iterations. |
//| Phase #2 is a sequence of linearized steps minimizing |
//| L2-penalized Lagrangian performed with successively increasing |
//| set of conjugacy constraints (which make algorithm behavior |
//| similar to that of CG). |
//| INPUT PARAMETERS: |
//| State - SLP solver State |
//| SMonitor - smoothness monitor |
//| UserTerminationNeeded - True if user requested termination |
//| CurX - current point, array[N] |
//| CurFi - function vector at CurX, array[1 + NLEC + NLIC] |
//| CurJ - Jacobian at CurX, array[1 + NLEC + NLIC, N] |
//| LagMult - array[NEC + NIC + NLEC + NLIC], contents ignored |
//| on input. |
//| GammaMax - current estimate of the Hessian norm |
//| OUTPUT PARAMETERS: |
//| State - RepTerminationType is set to current termination |
//| code (if Status = 0). |
//| CurX - advanced to new point |
//| CurFi - updated with function vector at CurX[] |
//| CurJ - updated with Jacobian at CurX[] |
//| LagMult - filled with current Lagrange multipliers |
//| GammaMax - updated estimate of the Hessian norm |
//| Status - when reverse communication is done, Status is set |
//| to: |
//| * negative value, if we have to restart outer |
//| iteration |
//| * positive value, if we can proceed to the next |
//| stage of the outer iteration |
//| * zero, if algorithm is terminated |
//| (RepTerminationType is set to appropriate value) |
//+------------------------------------------------------------------+
bool CNLCSLP::Phase2Iteration(CMinSLPState &State,
CMinSLPPhase2State &state2,
CSmoothnessMonitor &smonitor,
bool userterminationneeded,
CRowDouble &curx,
CRowDouble &curfi,
CMatrixDouble &curj,
CRowDouble &lagmult,
double &gammamax,
int &status)
{
//--- create variables
bool result=false;
int n=0;
int nslack=0;
int nec=0;
int nic=0;
int nlec=0;
int nlic=0;
double stp=0;
int mcinfo=0;
int mcnfev=0;
int mcstage=0;
int i=0;
int j=0;
int innerk=0;
double v=0;
double vv=0;
double mx=0;
int nondescentcnt=0;
double stepklagval=0;
double stepknlagval=0;
double gammaprev=0;
double f0=0;
double f1=0;
double mu=0;
bool dotrace=false;
bool doprobing=false;
bool dotracexd=false;
int label=-1;
//--- Reverse communication preparations
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(state2.m_rphase2state.stage>=0)
{
n=state2.m_rphase2state.ia[0];
nslack=state2.m_rphase2state.ia[1];
nec=state2.m_rphase2state.ia[2];
nic=state2.m_rphase2state.ia[3];
nlec=state2.m_rphase2state.ia[4];
nlic=state2.m_rphase2state.ia[5];
mcinfo=state2.m_rphase2state.ia[6];
mcnfev=state2.m_rphase2state.ia[7];
mcstage=state2.m_rphase2state.ia[8];
i=state2.m_rphase2state.ia[9];
j=state2.m_rphase2state.ia[10];
innerk=state2.m_rphase2state.ia[11];
nondescentcnt=state2.m_rphase2state.ia[12];
dotrace=state2.m_rphase2state.ba[0];
doprobing=state2.m_rphase2state.ba[1];
dotracexd=state2.m_rphase2state.ba[2];
stp=state2.m_rphase2state.ra[0];
v=state2.m_rphase2state.ra[1];
vv=state2.m_rphase2state.ra[2];
mx=state2.m_rphase2state.ra[3];
stepklagval=state2.m_rphase2state.ra[4];
stepknlagval=state2.m_rphase2state.ra[5];
gammaprev=state2.m_rphase2state.ra[6];
f0=state2.m_rphase2state.ra[7];
f1=state2.m_rphase2state.ra[8];
mu=state2.m_rphase2state.ra[9];
}
else
{
n=-819;
nslack=-826;
nec=667;
nic=692;
nlec=84;
nlic=529;
mcinfo=14;
mcnfev=386;
mcstage=-908;
i=577;
j=289;
innerk=317;
nondescentcnt=476;
dotrace=true;
doprobing=false;
dotracexd=true;
stp=-962;
v=161;
vv=-447;
mx=-799;
stepklagval=508;
stepknlagval=-153;
gammaprev=-450;
f0=769;
f1=638;
mu=-361;
}
switch(state2.m_rphase2state.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
default:
//--- Routine body
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
nlec=State.m_nlec;
nlic=State.m_nlic;
nslack=n+2*(nec+nlec)+(nic+nlic);
dotrace=CAp::IsTraceEnabled("SLP");
dotracexd=dotrace && CAp::IsTraceEnabled("SLP.DETAILED");
doprobing=CAp::IsTraceEnabled("SLP.PROBING");
if(!CAp::Assert(CAp::Len(lagmult)>=nec+nic+nlec+nlic,"Phase13Iteration: integrity check failed"))
return(false);
//--- Report iteration beginning
if(dotrace)
CAp::Trace("\n--- linear step with conjugate constraints (CG-like convergence) -----------------------------------\n");
//--- The default decision is to continue iterations
status=1;
//--- Perform inner LP subiterations.
//--- During this process we maintain information about several points:
//---*point #0, initial one, with "step0" prefix
//---*point #K, last one of current LP session, with "stepk" prefix
//--- * additionally we have point #KN, current candidate during line search at step K.
//--- For each point we store:
//--- * location X (scaled coordinates)
//--- * function vector Fi (target function + nonlinear constraints)
//--- * scaled Jacobian J
mu=MathMax(CAblasF::RMaxAbsV(State.m_historylen,State.m_maxlaghistory),CAblasF::RMaxAbsV(nec+nic+nlec+nlic,State.m_meritlagmult));
mu=CApServ::Coalesce(mu,m_defaultl1penalty);
nondescentcnt=0;
LPSubproblemRestart(State,State.m_subsolver);
innerk=1;
label=3;
break;
}
while(label>=0)
{
switch(label)
{
case 3:
if(innerk>n)
{
label=5;
break;
}
//--- Formulate LP subproblem and solve it
if(!LPSubproblemSolve(State,State.m_subsolver,curx,curfi,curj,innerk,state2.m_d,lagmult))
{
//--- LP solver failed due to numerical errors; exit.
//--- It may happen when we solve problem with LOTS of conjugacy constraints.
if(innerk==1)
{
//--- The very first iteration failed, really strange.
if(dotrace)
CAp::Trace("[WARNING] the very first LP subproblem failed to produce descent direction\n");
}
else
{
//--- Quite a normal, the problem is overconstrained by conjugacy constraints now
if(dotrace)
CAp::Trace("> LP subproblem is overconstrained (happens after too many iterations),time to stop\n");
}
result=false;
return(result);
}
mx=0;
for(i=0; i<n; i++)
mx=MathMax(mx,MathAbs(state2.m_d[i])/State.m_trustrad);
if(mx==0.0)
{
//--- Nearly-zero direction is suggested (maybe we arrived exactly to the solution), stop iterations
status=1;
SLPCopyState(State,curx,curfi,curj,state2.m_stepkxn,state2.m_stepkfin,state2.m_stepkjn);
if(dotrace)
CAp::Trace("> LP subproblem suggested nearly zero step\n");
if(dotrace)
CAp::Trace(StringFormat("max(|Di|)/TrustRad = %.6f\n",mx));
if(dotrace)
CAp::Trace("> stopping CG-like iterations\n");
result=false;
return(result);
}
LPSubproblemAppendConjugacyConstraint(State,State.m_subsolver,state2.m_d);
//--- Perform line search to minimize Lagrangian along D.
//--- Post-normalize StepKXN with respect to box constraints.
//--- MCSRCH can fail in the following cases:
//--- * rounding errors prevent optimization
//--- * non-descent direction is specified (MCINFO=0 is returned)
//--- In the latter case we proceed to minimization of merit function.
//--- NOTE: constraint violation reports are updated during Lagrangian computation
state2.m_lastlcerr=0;
state2.m_lastlcidx=-1;
state2.m_lastnlcerr=0;
state2.m_lastnlcidx=-1;
CApServ::RVectorSetLengthAtLeast(state2.m_tmp0,n);
LagrangianFG(State,curx,State.m_trustrad,curfi,curj,lagmult,state2.m_tmplagrangianfg,stepklagval,state2.m_stepklaggrad,state2.m_lastlcerr,state2.m_lastlcidx,state2.m_lastnlcerr,state2.m_lastnlcidx);
SLPCopyState(State,curx,curfi,curj,state2.m_stepkxn,state2.m_stepkfin,state2.m_stepkjn);
state2.m_stepknlaggrad=state2.m_stepklaggrad;
v=CAblasF::RDotV(n,state2.m_d,state2.m_stepklaggrad);
if(v>=0.0)
{
//--- Non-descent direction D was specified; it may happen because LP subproblem favors
//--- directions which decrease L1 penalty and default augmentation of Lagrangian involves
//--- only L2 term.
//--- Append direction to the conjugacy constraints and retry direction generation.
//--- We make several retries with conjugate directions before giving up.
if(dotrace)
CAp::Trace(StringFormat("> LP subproblem suggested nondescent step,skipping it (dLag=%.3E)\n",v));
nondescentcnt++;
if(m_nondescentlimit>0 && nondescentcnt>m_nondescentlimit)
{
if(dotrace)
CAp::Trace("> too many nondescent steps,stopping CG-like iterations\n");
status=1;
result=false;
return(result);
}
label=4;
break;
}
COptServ::SmoothnessMonitorStartLineSearch(smonitor,curx,curfi,curj);
stepknlagval=stepklagval;
mcnfev=0;
mcstage=0;
stp=1.0;
CLinMin::MCSrch(n,state2.m_stepkxn,stepknlagval,state2.m_stepknlaggrad,state2.m_d,stp,1.0,m_slpgtol,mcinfo,mcnfev,state2.m_tmp0,state2.m_mcstate,mcstage);
case 6:
if(mcstage==0)
{
label=7;
break;
}
SLPSendX(State,state2.m_stepkxn);
State.m_needfij=true;
state2.m_rphase2state.stage=0;
label=-1;
break;
case 0:
State.m_needfij=false;
if(!SLPRetrieveFIJ(State,state2.m_stepkfin,state2.m_stepkjn))
{
//--- Failed to retrieve func/Jac, infinities detected
status=0;
State.m_repterminationtype=-8;
if(dotrace)
CAp::Trace("[ERROR] infinities in target/constraints are detected\n");
result=false;
return(result);
}
COptServ::SmoothnessMonitorEnqueuePoint(smonitor,state2.m_d,stp,state2.m_stepkxn,state2.m_stepkfin,state2.m_stepkjn);
LagrangianFG(State,state2.m_stepkxn,State.m_trustrad,state2.m_stepkfin,state2.m_stepkjn,lagmult,state2.m_tmplagrangianfg,stepknlagval,state2.m_stepknlaggrad,state2.m_lastlcerr,state2.m_lastlcidx,state2.m_lastnlcerr,state2.m_lastnlcidx);
CLinMin::MCSrch(n,state2.m_stepkxn,stepknlagval,state2.m_stepknlaggrad,state2.m_d,stp,1.0,m_slpgtol,mcinfo,mcnfev,state2.m_tmp0,state2.m_mcstate,mcstage);
label=6;
break;
case 7:
COptServ::SmoothnessMonitorFinalizeLineSearch(smonitor);
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i])
state2.m_stepkxn.Set(i,MathMax(state2.m_stepkxn[i],State.m_scaledbndl[i]));
if(State.m_HasBndU[i])
state2.m_stepkxn.Set(i,MathMin(state2.m_stepkxn[i],State.m_scaledbndu[i]));
}
if(mcinfo<=0)
{
//--- Line search failed miserably, terminate
status=1;
if(innerk==1)
{
//--- The very first iteration failed, really strange.
//--- Let's decrease trust radius and try one more time.
State.m_trustrad=0.5*State.m_trustrad;
if(dotrace)
CAp::Trace("> line search failed miserably for unknown reason,decreasing trust radius\n");
if(State.m_trustrad<State.m_epsx)
{
State.m_repterminationtype=2;
status=0;
if(dotrace)
CAp::Trace(StringFormat("> stopping condition met: trust radius is smaller than %.3E\n",State.m_epsx));
}
}
else
{
//--- Well, it can be normal
if(dotrace)
CAp::Trace("> line search failed miserably for unknown reason,proceeding further\n");
}
result=false;
return(result);
}
if(mcinfo==1)
LPSubproblemUpdateHessian(State,State.m_subsolver,curx,state2.m_stepklaggrad,state2.m_stepkxn,state2.m_stepknlaggrad);
//--- Update GammaMax - estimate of the function Hessian norm
v=0;
vv=0;
mx=0;
for(i=0; i<n; i++)
{
mx=MathMax(mx,MathAbs(state2.m_stepkxn[i]-curx[i]));
v+=CMath::Sqr(state2.m_stepkxn[i]-curx[i]);
vv+=(state2.m_stepkjn.Get(0,i)-curj.Get(0,i))*(state2.m_stepkxn[i]-curx[i]);
}
gammaprev=gammamax;
if(mx>m_bfgstol)
gammamax=MathMax(gammamax,MathAbs(vv/v));
//--- Trace
if(!dotrace)
{
label=8;
break;
}
if(!doprobing)
{
label=10;
break;
}
COptServ::SmoothnessMonitorStartProbing(smonitor,1.0,2,State.m_trustrad);
case 12:
if(!COptServ::SmoothnessMonitorProbe(smonitor))
{
label=13;
break;
}
for(j=0; j<n; j++)
{
state2.m_stepkxc.Set(j,curx[j]+state2.m_d[j]*smonitor.m_probingstp);
if(State.m_HasBndL[j])
state2.m_stepkxc.Set(j,MathMax(state2.m_stepkxc[j],State.m_scaledbndl[j]));
if(State.m_HasBndU[j])
state2.m_stepkxc.Set(j,MathMin(state2.m_stepkxc[j],State.m_scaledbndu[j]));
}
SLPSendX(State,state2.m_stepkxc);
State.m_needfij=true;
state2.m_rphase2state.stage=1;
label=-1;
break;
case 1:
State.m_needfij=false;
if(!SLPRetrieveFIJ(State,state2.m_stepkfic,state2.m_stepkjc))
{
label=13;
break;
}
smonitor.m_probingf.Set(0,RawLagrangian(State,state2.m_stepkxc,state2.m_stepkfic,lagmult,state2.m_tmpmerit));
smonitor.m_probingf.Set(1,state2.m_stepkfic[0]);
label=12;
break;
case 13:
CAp::Trace("*** ------------------------------------------------------------\n");
CAp::Trace("*** | probing search direction suggested by LP subproblem |\n");
CAp::Trace("*** ------------------------------------------------------------\n");
CAp::Trace("*** | Step | Lagrangian (unaugmentd)| Target function |\n");
CAp::Trace("*** |along D| must be smooth | must be smooth |\n");
CAp::Trace("*** | | function | slope | function | slope |\n");
COptServ::SmoothnessMonitorTraceProbingResults(smonitor);
case 10:
mx=0;
for(i=0; i<n; i++)
mx=MathMax(mx,MathAbs(state2.m_d[i])/State.m_trustrad);
f0=MeritFunction(State,curx,curfi,state2.m_meritlagmult,mu,state2.m_tmpmerit);
f1=MeritFunction(State,state2.m_stepkxn,state2.m_stepkfin,state2.m_meritlagmult,mu,state2.m_tmpmerit);
CAp::Trace("> LP subproblem produced good direction,minimization was performed\n");
CAp::Trace(StringFormat("max(|Di|)/TrustRad = %.6f\n",mx));
CAp::Trace(StringFormat("stp = %.6f\n",stp));
if(dotracexd)
{
CAp::Trace("X0 = ");
CApServ::TraceVectorAutopRec(curx,0,n);
CAp::Trace("\n");
CAp::Trace("D = ");
CApServ::TraceVectorAutopRec(state2.m_d,0,n);
CAp::Trace("\n");
CAp::Trace("X1 = X0 + stp*D\n");
CAp::Trace(" = ");
CApServ::TraceVectorAutopRec(state2.m_stepkxn,0,n);
CAp::Trace("\n");
}
CAp::Trace(StringFormat("meritF: %14.6E -> %14.6E (delta=%11.3E)\n",f0,f1,f1 - f0));
CAp::Trace(StringFormat("scaled-targetF: %14.6E -> %14.6E (delta=%11.3E)\n",curfi[0],state2.m_stepkfin[0],state2.m_stepkfin[0] - curfi[0]));
CAp::Trace(StringFormat("aug.Lagrangian: %14.6E -> %14.6E (delta=%11.3E)\n",stepklagval,stepknlagval,stepknlagval - stepklagval));
if(gammamax>gammaprev)
CAp::Trace(StringFormat("|H| = %.3E (Hessian norm increased)\n",gammamax));
case 8:
//--- Check status of the termination request
//--- Update current point
//--- Update constraint status.
//--- Report iteration.
if(userterminationneeded)
{
//--- User requested termination, break before we move to new point
status=0;
State.m_repterminationtype=8;
if(dotrace)
CAp::Trace("# user requested termination\n");
result=false;
return(result);
}
SLPCopyState(State,state2.m_stepkxn,state2.m_stepkfin,state2.m_stepkjn,curx,curfi,curj);
State.m_replcerr=state2.m_lastlcerr;
State.m_replcidx=state2.m_lastlcidx;
State.m_repnlcerr=state2.m_lastnlcerr;
State.m_repnlcidx=state2.m_lastnlcidx;
State.m_repinneriterationscount++;
SLPSendX(State,curx);
State.m_f=curfi[0]*State.m_fscales[0];
State.m_xupdated=true;
state2.m_rphase2state.stage=2;
label=-1;
break;
case 2:
State.m_xupdated=false;
//--- Terminate inner LP subiterations
if(State.m_maxits>0 && State.m_repinneriterationscount>=State.m_maxits)
{
//--- Iteration limit exhausted
status=1;
if(dotrace)
CAp::Trace("# stopping criteria met (MaxIts iterations performed)\n");
result=false;
return(result);
}
if(stp>=m_slpstpclosetoone)
{
//--- Step is close to 1.0, either of two is likely:
//--- * we move through nearly linear region of F()
//--- * we try to enforce some strongly violated constraint
//--- In any case, authors of the original algorithm recommend to break inner LP
//--- iteration and proceed to test of sufficient decrease of merit function.
status=1;
if(dotrace)
CAp::Trace("> step is close to 1,stopping iterations\n");
result=false;
return(result);
}
if((mcinfo!=1 && mcinfo!=3) && mcinfo!=5)
{
//--- Line search ended with "bad" MCINFO
//--- (neither sufficient decrease, neither maximum step);
//--- terminate.
status=1;
if(dotrace)
CAp::Trace("> line search ended with bad MCINFO,no more CG-like iterations\n");
result=false;
return(result);
}
case 4:
innerk++;
label=3;
break;
case 5:
result=false;
return(result);
}
}
//--- Saving State
result=true;
state2.m_rphase2state.ba[0]=dotrace;
state2.m_rphase2state.ba[1]=doprobing;
state2.m_rphase2state.ba[2]=dotracexd;
state2.m_rphase2state.ia.Set(0,n);
state2.m_rphase2state.ia.Set(1,nslack);
state2.m_rphase2state.ia.Set(2,nec);
state2.m_rphase2state.ia.Set(3,nic);
state2.m_rphase2state.ia.Set(4,nlec);
state2.m_rphase2state.ia.Set(5,nlic);
state2.m_rphase2state.ia.Set(6,mcinfo);
state2.m_rphase2state.ia.Set(7,mcnfev);
state2.m_rphase2state.ia.Set(8,mcstage);
state2.m_rphase2state.ia.Set(9,i);
state2.m_rphase2state.ia.Set(10,j);
state2.m_rphase2state.ia.Set(11,innerk);
state2.m_rphase2state.ia.Set(12,nondescentcnt);
state2.m_rphase2state.ra.Set(0,stp);
state2.m_rphase2state.ra.Set(1,v);
state2.m_rphase2state.ra.Set(2,vv);
state2.m_rphase2state.ra.Set(3,mx);
state2.m_rphase2state.ra.Set(4,stepklagval);
state2.m_rphase2state.ra.Set(5,stepknlagval);
state2.m_rphase2state.ra.Set(6,gammaprev);
state2.m_rphase2state.ra.Set(7,f0);
state2.m_rphase2state.ra.Set(8,f1);
state2.m_rphase2state.ra.Set(9,mu);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Copies X to State.X |
//+------------------------------------------------------------------+
void CNLCSLP::SLPSendX(CMinSLPState &State,CRowDouble &xs)
{
int n=State.m_n;
//--- copy
for(int i=0; i<n; i++)
{
if(State.m_HasBndL[i] && xs[i]<=State.m_scaledbndl[i])
{
State.m_x.Set(i,State.m_scaledbndl[i]);
continue;
}
if(State.m_HasBndU[i] && xs[i]>=State.m_scaledbndu[i])
{
State.m_x.Set(i,State.m_scaledbndu[i]);
continue;
}
State.m_x.Set(i,xs[i]);
}
}
//+------------------------------------------------------------------+
//| Retrieves F-vector and scaled Jacobian, copies them to FiS and JS|
//| Returns True on success, False on failure(when F or J are not |
//| finite numbers). |
//+------------------------------------------------------------------+
bool CNLCSLP::SLPRetrieveFIJ(CMinSLPState &State,
CRowDouble &fis,
CMatrixDouble &js)
{
//--- create variables
bool result=false;
int n=State.m_n;
int nlec=State.m_nlec;
int nlic=State.m_nlic;
int i=0;
int j=0;
double v=0;
double vv=0;
for(i=0; i<=nlec+nlic; i++)
{
vv=1/State.m_fscales[i];
fis.Set(i,vv*State.m_fi[i]);
v=0.1*v+fis[i];
for(j=0; j<n; j++)
{
js.Set(i,j,vv*State.m_j.Get(i,j));
v=0.1*v+js.Get(i,j);
}
}
result=MathIsValidNumber(v);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Copies State (X point, Fi vector, J jacobian) to preallocated |
//| storage. |
//+------------------------------------------------------------------+
void CNLCSLP::SLPCopyState(CMinSLPState &State,
CRowDouble &x0,
CRowDouble &fi0,
CMatrixDouble &j0,
CRowDouble &x1,
CRowDouble &fi1,
CMatrixDouble &j1)
{
x1=x0;
fi1=fi0;
j1=j0;
}
//+------------------------------------------------------------------+
//| This function calculates Lagrangian of the problem (in scaled |
//| variables): its value and gradient. |
//| Additionally it also estimates violation of linear constraints at|
//| the point as well as index of the most violated constraint |
//+------------------------------------------------------------------+
void CNLCSLP::LagrangianFG(CMinSLPState &State,
CRowDouble &x,
double trustrad,
CRowDouble &fi,
CMatrixDouble &j,
CRowDouble &lagmult,
CMinSLPTmpLagrangian &tmp,
double &f,
CRowDouble &g,
double &lcerr,
int &lcidx,
double &nlcerr,
int &nlcidx)
{
//--- create variables
int n=State.m_n;
int nec=State.m_nec;
int nic=State.m_nic;
int nlec=State.m_nlec;
int nlic=State.m_nlic;
int i=0;
double v=0;
double vlag=0;
double vact=0;
double vd=0;
double vviolate=0;
bool usesparsegemv=false;
double dampingfactor=m_inequalitydampingfactor/trustrad;
f=0;
//--- Prepare constraint violation report
lcerr=0;
lcidx=-1;
nlcerr=0;
nlcidx=-1;
//--- Target function
f=fi[0];
g=j[0]+0;
//--- Lagrangian terms for linear constraints, constraint violations
if(nec+nic>0)
{
usesparsegemv=State.m_subsolver.m_sparserawlc.m_RIdx[nec+nic]<CApServ::SparseLevel2Density()*n*(nec+nic);
CApServ::RVectorSetLengthAtLeast(tmp.m_sclagtmp0,MathMax(nec+nic,n));
CApServ::RVectorSetLengthAtLeast(tmp.m_sclagtmp1,MathMax(nec+nic,n));
if(usesparsegemv)
CSparse::SparseMV(State.m_subsolver.m_sparserawlc,x,tmp.m_sclagtmp0);
else
CAblas::RMatrixGemVect(nec+nic,n,1.0,State.m_scaledcleic,0,0,0,x,0,0.0,tmp.m_sclagtmp0,0);
for(i=0; i<=nec+nic-1; i++)
{
//--- Estimate constraint value at the point, update violation Info
//--- NOTE: here we expect that scaledCLEIC[] has normalized rows
v=tmp.m_sclagtmp0[i]-State.m_scaledcleic.Get(i,n);
if(i<nec || v>0)
{
//--- Either equality constraint or violated inequality one.
//--- Update violation report.
vviolate=MathAbs(v);
if(vviolate>lcerr)
{
lcerr=vviolate;
lcidx=State.m_lcsrcidx[i];
}
}
//--- Prepare
vlag=lagmult[i];
tmp.m_sclagtmp1.Set(i,0);
//--- Primary Lagrangian term
if(i<nec || v>0)
{
vact=v;
vd=1;
}
else
{
vd=1/(1-dampingfactor*v);
vact=v*vd;
vd=vd*vd;
}
f+=vlag*vact;
tmp.m_sclagtmp1.Add(i,vlag*vd);
//--- Quadratic augmentation term
if(i<nec || v>0)
vact=v;
else
vact=0;
f+=0.5*m_augmentationfactor*vact*vact;
tmp.m_sclagtmp1.Add(i,m_augmentationfactor*vact);
}
if(usesparsegemv)
{
CSparse::SparseMTV(State.m_subsolver.m_sparserawlc,tmp.m_sclagtmp1,tmp.m_sclagtmp0);
g+=tmp.m_sclagtmp0;
}
else
CAblas::RMatrixGemVect(n,nec+nic,1.0,State.m_scaledcleic,0,0,1,tmp.m_sclagtmp1,0,1.0,g,0);
}
//--- Lagrangian terms for nonlinear constraints
tmp.m_sclagtmp1=vector<double>::Zeros(nlec+nlic);
for(i=0; i<nlec+nlic; i++)
{
v=fi[1+i];
if(i<nlec || v>0)
{
//--- Either equality constraint or violated inequality one.
//--- Update violation report.
vviolate=MathAbs(v)*State.m_fscales[1+i];
if(vviolate>nlcerr)
{
nlcerr=vviolate;
nlcidx=i;
}
}
vlag=lagmult[nec+nic+i];
//--- Lagrangian term
if(i<nlec || v>0)
{
vact=v;
vd=1;
}
else
{
vd=1/(1-dampingfactor*v);
vact=v*vd;
vd=vd*vd;
}
f+=vlag*vact;
tmp.m_sclagtmp1.Add(i,vlag*vd);
//--- Augmentation term
if(i<nlec || v>0)
vact=v;
else
vact=0;
f+=0.5*m_augmentationfactor*vact*vact;
tmp.m_sclagtmp1.Add(i,m_augmentationfactor*vact);
}
CAblas::RMatrixGemVect(n,nlec+nlic,1.0,j,1,0,1,tmp.m_sclagtmp1,0,1.0,g,0);
}
//+------------------------------------------------------------------+
//| This function calculates L1 - penalized merit function |
//+------------------------------------------------------------------+
double CNLCSLP::MeritFunction(CMinSLPState &State,
CRowDouble &x,
CRowDouble &fi,
CRowDouble &lagmult,
double mu,
CMinSLPTmpMerit &tmp)
{
//--- create variables
double tmp0=0;
double tmp1=0;
//--- function call
MeritFunctionAndRawLagrangian(State,x,fi,lagmult,mu,tmp,tmp0,tmp1);
//--- return result
return(tmp0);
}
//+------------------------------------------------------------------+
//| This function calculates raw(unaugmented and smooth) Lagrangian |
//+------------------------------------------------------------------+
double CNLCSLP::RawLagrangian(CMinSLPState &State,
CRowDouble &x,
CRowDouble &fi,
CRowDouble &lagmult,
CMinSLPTmpMerit &tmp)
{
//--- create variables
double tmp0=0;
double tmp1=0;
//--- function call
MeritFunctionAndRawLagrangian(State,x,fi,lagmult,0.0,tmp,tmp0,tmp1);
//--- return result
return(tmp1);
}
//+------------------------------------------------------------------+
//| This function calculates L1-penalized merit function and raw |
//| (smooth and un-augmented) Lagrangian |
//+------------------------------------------------------------------+
void CNLCSLP::MeritFunctionAndRawLagrangian(CMinSLPState &State,
CRowDouble &x,
CRowDouble &fi,
CRowDouble &lagmult,
double mu,
CMinSLPTmpMerit &tmp,
double &meritf,
double &rawlag)
{
//--- create variables
int n=State.m_n;
int nec=State.m_nec;
int nic=State.m_nic;
int nlec=State.m_nlec;
int nlic=State.m_nlic;
int i=0;
double v=0;
meritf=0;
rawlag=0;
//--- Merit function and Lagrangian: primary term
meritf=fi[0];
rawlag=fi[0];
//--- Merit function: augmentation and penalty for linear constraints
CApServ::RVectorSetLengthAtLeast(tmp.m_mftmp0,nec+nic);
CAblas::RMatrixGemVect(nec+nic,n,1.0,State.m_scaledcleic,0,0,0,x,0,0.0,tmp.m_mftmp0,0);
for(i=0; i<nec+nic; i++)
{
v=tmp.m_mftmp0[i]-State.m_scaledcleic.Get(i,n);
if(i<nec)
{
//--- Merit function: augmentation term + L1 penalty term
meritf+=m_meritfunctionbase*MathAbs(v)+m_meritfunctiongain*mu*MathAbs(v);
//--- Raw Lagrangian
rawlag+=lagmult[i]*v;
}
else
{
//--- Merit function: augmentation term + L1 penalty term
meritf+=m_meritfunctionbase*MathMax(v,0)+m_meritfunctiongain*mu*MathMax(v,0);
//--- Raw Lagrangian
rawlag+=lagmult[i]*v;
}
}
//--- Merit function: augmentation and penalty for nonlinear constraints
for(i=0; i<nlec+nlic; i++)
{
v=fi[1+i];
if(i<nlec)
{
//--- Merit function: augmentation term + L1 penalty term
meritf+=(m_meritfunctionbase+m_meritfunctiongain*mu)*MathAbs(v);
//--- Raw Lagrangian
rawlag+=lagmult[nec+nic+i]*v;
}
else
{
//--- Merit function: augmentation term + L1 penalty term
meritf+=(m_meritfunctionbase+m_meritfunctiongain*mu)*MathMax(v,0);
//--- Raw Lagrangian
rawlag+=lagmult[nec+nic+i]*v;
}
}
}
//+------------------------------------------------------------------+
//| This object stores nonlinear optimizer State. |
//| You should use functions provided by MinNLC subpackage to work |
//| with this object |
//+------------------------------------------------------------------+
struct CMinNLCState
{
int m_aulitscnt;
int m_maxits;
int m_n;
int m_nec;
int m_ng;
int m_nh;
int m_nic;
int m_prectype;
int m_repbcidx;
int m_repdbgphase0its;
int m_repinneriterationscount;
int m_replcidx;
int m_repnfev;
int m_repnlcidx;
int m_repouteriterationscount;
int m_repterminationtype;
int m_smoothnessguardlevel;
int m_solvertype;
int m_updatefreq;
double m_diffstep;
double m_epsx;
double m_f;
double m_gammak;
double m_initialinequalitymultiplier;
double m_repbcerr;
double m_replcerr;
double m_repnlcerr;
double m_rho;
double m_stabilizingpoint;
double m_stpmax;
double m_teststep;
bool m_HasBndL[];
bool m_HasBndU[];
bool m_needfi;
bool m_needfij;
bool m_userterminationneeded;
bool m_xkpresent;
bool m_xrep;
bool m_xupdated;
RCommState m_rstate;
RCommState m_rstateaul;
RCommState m_rstateslp;
CSmoothnessMonitor m_smonitor;
CRowInt m_lcsrcidx;
CRowDouble m_bndl;
CRowDouble m_bndu;
CRowDouble m_bufc;
CRowDouble m_bufd;
CRowDouble m_dfbase;
CRowDouble m_dfm1;
CRowDouble m_dfp1;
CRowDouble m_fbase;
CRowDouble m_fi;
CRowDouble m_fm1;
CRowDouble m_fm2;
CRowDouble m_fp1;
CRowDouble m_fp2;
CRowDouble m_gk1;
CRowDouble m_gk;
CRowDouble m_lastscaleused;
CRowDouble m_nubc;
CRowDouble m_nulc;
CRowDouble m_nunlc;
CRowDouble m_s;
CRowDouble m_scaledbndl;
CRowDouble m_scaledbndu;
CRowDouble m_tmp0;
CRowDouble m_x;
CRowDouble m_xbase;
CRowDouble m_xc;
CRowDouble m_xk1;
CRowDouble m_xk;
CRowDouble m_xstart;
CMinSQPState m_sqpsolverstate;
CMinSLPState m_slpsolverstate;
CMinLBFGSState m_auloptimizer;
CMinLBFGSReport m_aulreport;
CMatrixDouble m_bufw;
CMatrixDouble m_bufz;
CMatrixDouble m_cleic;
CMatrixDouble m_j;
CMatrixDouble m_scaledcleic;
//--- constructor / destructor
CMinNLCState(void);
~CMinNLCState(void) {}
//---
void Copy(const CMinNLCState &obj);
//--- overloading
void operator=(const CMinNLCState &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinNLCState::CMinNLCState(void)
{
m_aulitscnt=0;
m_maxits=0;
m_n=0;
m_nec=0;
m_ng=0;
m_nh=0;
m_nic=0;
m_prectype=0;
m_repbcidx=0;
m_repdbgphase0its=0;
m_repinneriterationscount=0;
m_replcidx=0;
m_repnfev=0;
m_repnlcidx=0;
m_repouteriterationscount=0;
m_repterminationtype=0;
m_smoothnessguardlevel=0;
m_solvertype=0;
m_updatefreq=0;
m_diffstep=0;
m_epsx=0;
m_f=0;
m_gammak=0;
m_initialinequalitymultiplier=0;
m_repbcerr=0;
m_replcerr=0;
m_repnlcerr=0;
m_rho=0;
m_stabilizingpoint=0;
m_stpmax=0;
m_teststep=0;
m_needfi=false;
m_needfij=false;
m_userterminationneeded=false;
m_xkpresent=false;
m_xrep=false;
m_xupdated=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinNLCState::Copy(const CMinNLCState &obj)
{
m_aulitscnt=obj.m_aulitscnt;
m_maxits=obj.m_maxits;
m_n=obj.m_n;
m_nec=obj.m_nec;
m_ng=obj.m_ng;
m_nh=obj.m_nh;
m_nic=obj.m_nic;
m_prectype=obj.m_prectype;
m_repbcidx=obj.m_repbcidx;
m_repdbgphase0its=obj.m_repdbgphase0its;
m_repinneriterationscount=obj.m_repinneriterationscount;
m_replcidx=obj.m_replcidx;
m_repnfev=obj.m_repnfev;
m_repnlcidx=obj.m_repnlcidx;
m_repouteriterationscount=obj.m_repouteriterationscount;
m_repterminationtype=obj.m_repterminationtype;
m_smoothnessguardlevel=obj.m_smoothnessguardlevel;
m_solvertype=obj.m_solvertype;
m_updatefreq=obj.m_updatefreq;
m_diffstep=obj.m_diffstep;
m_epsx=obj.m_epsx;
m_f=obj.m_f;
m_gammak=obj.m_gammak;
m_initialinequalitymultiplier=obj.m_initialinequalitymultiplier;
m_repbcerr=obj.m_repbcerr;
m_replcerr=obj.m_replcerr;
m_repnlcerr=obj.m_repnlcerr;
m_rho=obj.m_rho;
m_stabilizingpoint=obj.m_stabilizingpoint;
m_stpmax=obj.m_stpmax;
m_teststep=obj.m_teststep;
ArrayCopy(m_HasBndL,obj.m_HasBndL);
ArrayCopy(m_HasBndU,obj.m_HasBndU);
m_needfi=obj.m_needfi;
m_needfij=obj.m_needfij;
m_userterminationneeded=obj.m_userterminationneeded;
m_xkpresent=obj.m_xkpresent;
m_xrep=obj.m_xrep;
m_xupdated=obj.m_xupdated;
m_rstate=obj.m_rstate;
m_rstateaul=obj.m_rstateaul;
m_rstateslp=obj.m_rstateslp;
m_smonitor=obj.m_smonitor;
m_lcsrcidx=obj.m_lcsrcidx;
m_bndl=obj.m_bndl;
m_bndu=obj.m_bndu;
m_bufc=obj.m_bufc;
m_bufd=obj.m_bufd;
m_dfbase=obj.m_dfbase;
m_dfm1=obj.m_dfm1;
m_dfp1=obj.m_dfp1;
m_fbase=obj.m_fbase;
m_fi=obj.m_fi;
m_fm1=obj.m_fm1;
m_fm2=obj.m_fm2;
m_fp1=obj.m_fp1;
m_fp2=obj.m_fp2;
m_gk1=obj.m_gk1;
m_gk=obj.m_gk;
m_lastscaleused=obj.m_lastscaleused;
m_nubc=obj.m_nubc;
m_nulc=obj.m_nulc;
m_nunlc=obj.m_nunlc;
m_s=obj.m_s;
m_scaledbndl=obj.m_scaledbndl;
m_scaledbndu=obj.m_scaledbndu;
m_tmp0=obj.m_tmp0;
m_x=obj.m_x;
m_xbase=obj.m_xbase;
m_xc=obj.m_xc;
m_xk1=obj.m_xk1;
m_xk=obj.m_xk;
m_xstart=obj.m_xstart;
m_sqpsolverstate=obj.m_sqpsolverstate;
m_slpsolverstate=obj.m_slpsolverstate;
m_auloptimizer=obj.m_auloptimizer;
m_aulreport=obj.m_aulreport;
m_bufw=obj.m_bufw;
m_bufz=obj.m_bufz;
m_cleic=obj.m_cleic;
m_j=obj.m_j;
m_scaledcleic=obj.m_scaledcleic;
}
//+------------------------------------------------------------------+
//| These fields store optimization report: |
//| * iterationscount total number of inner iterations |
//| * nfev number of gradient evaluations |
//| * terminationtype termination type(see below) |
//| Scaled constraint violations are reported: |
//| * bcerr maximum violation of the box constraints |
//| * bcidx index of the most violated box constraint|
//| (or -1, if all box constraints are |
//| satisfied or there is no box constraint) |
//| * lcerr maximum violation of the linear |
//| constraints, computed as maximum scaled |
//| distance between final point and |
//| constraint boundary. |
//| * lcidx index of the most violated linear |
//| constraint (or -1, if all constraints are |
//| satisfied or there is no general linear |
//| constraints) |
//| * nlcerr maximum violation of the nonlinear |
//| constraints |
//| * nlcidx index of the most violated nonlinear |
//| constraint (or -1, if all constraints are |
//| satisfied or there is no nonlinear |
//| constraints) |
//| Violations of box constraints are scaled on per-component basis |
//| according to the scale vector s[] as specified by |
//| MinNLCSetScale(). Violations of the general linear constraints |
//| are also computed using user - supplied variable scaling. |
//| Violations of nonlinear constraints are computed "as is" |
//| TERMINATION CODES |
//| TerminationType field contains completion code, which can be |
//| either: |
//| === FAILURE CODE === |
//| -8 internal integrity control detected infinite or NAN |
//| values in function / gradient. Abnormal termination |
//| signaled. |
//| -3 box constraints are infeasible. |
//| Note: infeasibility of non-box constraints does NOT trigger |
//| emergency completion; you have to examine bcerr/lcerr/ |
//| nlcerr to detect possibly inconsistent constraints. |
//| === SUCCESS CODE === |
//| 2 relative step is no more than EpsX. |
//| 5 MaxIts steps was taken |
//| 7 stopping conditions are too stringent, further |
//| improvement is impossible, X contains best point |
//| found so far. |
//| 8 user requested algorithm termination via |
//| MinNLCRequestTermination(), last accepted point is |
//| returned |
//| Other fields of this structure are not documented and should not |
//| be used! |
//+------------------------------------------------------------------+
struct CMinNLCReport
{
int m_bcidx;
int m_dbgphase0its;
int m_iterationscount;
int m_lcidx;
int m_nfev;
int m_nlcidx;
int m_terminationtype;
double m_bcerr;
double m_lcerr;
double m_nlcerr;
//--- constructor / destructor
CMinNLCReport(void) { ZeroMemory(this); }
~CMinNLCReport(void) {}
//---
void Copy(const CMinNLCReport &obj);
//--- overloading
void operator=(const CMinNLCReport &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinNLCReport::Copy(const CMinNLCReport &obj)
{
m_bcidx=obj.m_bcidx;
m_dbgphase0its=obj.m_dbgphase0its;
m_iterationscount=obj.m_iterationscount;
m_lcidx=obj.m_lcidx;
m_nfev=obj.m_nfev;
m_nlcidx=obj.m_nlcidx;
m_terminationtype=obj.m_terminationtype;
m_bcerr=obj.m_bcerr;
m_lcerr=obj.m_lcerr;
m_nlcerr=obj.m_nlcerr;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CMinNLC
{
public:
//--- constants
static const int m_lbfgsfactor;
static const double m_aulmaxgrowth;
static const double m_maxlagmult;
static const double m_hessesttol;
static const double m_initgamma;
static const double m_regprec;
static void MinNLCCreate(int n,CRowDouble &x,CMinNLCState &State);
static void MinNLCCreateF(int n,CRowDouble &x,double diffstep,CMinNLCState &State);
static void MinNLCSetBC(CMinNLCState &State,CRowDouble &bndl,CRowDouble &bndu);
static void MinNLCSetLC(CMinNLCState &State,CMatrixDouble &c,CRowInt &ct,int k);
static void MinNLCSetNLC(CMinNLCState &State,int nlec,int nlic);
static void MinNLCSetCond(CMinNLCState &State,double epsx,int m_maxits);
static void MinNLCSetScale(CMinNLCState &State,CRowDouble &s);
static void MinNLCSetPrecInexact(CMinNLCState &State);
static void MinNLCSetPrecExactLowRank(CMinNLCState &State,int updatefreq);
static void MinNLCSetPrecExactRobust(CMinNLCState &State,int updatefreq);
static void MinNLCSetPrecNone(CMinNLCState &State);
static void MinNLCSetSTPMax(CMinNLCState &State,double stpmax);
static void MinNLCSetAlgoAUL(CMinNLCState &State,double rho,int itscnt);
static void MinNLCSetAlgoSLP(CMinNLCState &State);
static void MinNLCSetAlgoSQP(CMinNLCState &State);
static void MinNLCSetXRep(CMinNLCState &State,bool needxrep);
static bool MinNLCIteration(CMinNLCState &State);
static void MinNLCOptGuardGradient(CMinNLCState &State,double teststep);
static void MinNLCOptGuardSmoothness(CMinNLCState &State,int level);
static void MinNLCOptGuardResults(CMinNLCState &State,COptGuardReport &rep);
static void MinNLCOptGuardNonC1Test0Results(CMinNLCState &State,COptGuardNonC1Test0Report &strrep,COptGuardNonC1Test0Report &lngrep);
static void MinNLCOptGuardNonC1Test1Results(CMinNLCState &State,COptGuardNonC1Test1Report &strrep,COptGuardNonC1Test1Report &lngrep);
static void MinNLCResults(CMinNLCState &State,CRowDouble &x,CMinNLCReport &rep);
static void MinNLCResultsBuf(CMinNLCState &State,CRowDouble &x,CMinNLCReport &rep);
static void MinNLCRequestTermination(CMinNLCState &State);
static void MinNLCRestartFrom(CMinNLCState &State,CRowDouble &x);
static void MinNLCEqualityPenaltyFunction(double alpha,double &f,double &df,double &d2f);
static void MinNLCInequalityPenaltyFunction(double alpha,double stabilizingpoint,double &f,double &df,double &d2f);
static void MinNLCInequalityShiftFunction(double alpha,double &f,double &df,double &d2f);
private:
static void ClearRequestFields(CMinNLCState &State);
static void MinNLCInitInternal(int n,CRowDouble &x,double diffstep,CMinNLCState &State);
static void ClearPreconditioner(CMinLBFGSState &auloptimizer);
static void UpdatePreconditioner(int prectype,int updatefreq,int &preccounter,CMinLBFGSState &auloptimizer,CRowDouble &x,double rho,double gammak,CRowDouble &bndl,bool &HasBndL[],CRowDouble &bndu,bool &HasBndU[],CRowDouble &nubc,CMatrixDouble &cleic,
CRowDouble &nulc,CRowDouble &fi,CMatrixDouble &jac,CRowDouble &nunlc,CRowDouble &bufd,CRowDouble &bufc,CMatrixDouble &bufw,CMatrixDouble &bufz,CRowDouble &tmp0,int n,int nec,int nic,int ng,int nh);
static void PenaltyBC(CRowDouble &x,CRowDouble &bndl,bool &HasBndL[],CRowDouble &bndu,bool &HasBndU[],CRowDouble &nubc,int n,double rho,double stabilizingpoint,double &f,CRowDouble &g);
static void PenaltyLC(CRowDouble &x,CMatrixDouble &cleic,CRowDouble &nulc,int n,int nec,int nic,double rho,double stabilizingpoint,double &f,CRowDouble &g);
static void PenaltyNLC(CRowDouble &fi,CMatrixDouble &j,CRowDouble &nunlc,int n,int ng,int nh,double rho,double stabilizingpoint,double &f,CRowDouble &g);
static bool AULIteration(CMinNLCState &State,CSmoothnessMonitor &smonitor);
static void UnScale(CMinNLCState &State,CRowDouble &xs,CRowDouble &scaledbndl,CRowDouble &scaledbndu,CRowDouble &xu);
};
//+------------------------------------------------------------------+
//| Constants |
//+------------------------------------------------------------------+
const double CMinNLC::m_aulmaxgrowth=10.0;
const double CMinNLC::m_maxlagmult=1.0E7;
const int CMinNLC::m_lbfgsfactor=10;
const double CMinNLC::m_hessesttol=1.0E-6;
const double CMinNLC::m_initgamma=1.0E-6;
const double CMinNLC::m_regprec=1.0E-6;
//+------------------------------------------------------------------+
//| NONLINEARLY CONSTRAINED OPTIMIZATION WITH PRECONDITIONED |
//| AUGMENTED LAGRANGIAN ALGORITHM |
//| DESCRIPTION: |
//| The subroutine minimizes function F(x) of N arguments subject to |
//| any combination of: |
//| * bound constraints |
//| * linear inequality constraints |
//| * linear equality constraints |
//| * nonlinear equality constraints Gi(x) = 0 |
//| * nonlinear inequality constraints Hi(x) <= 0 |
//| REQUIREMENTS: |
//| * user must provide function value and gradient for F(), H(), |
//| G() |
//| * starting point X0 must be feasible or not too far away from |
//| the feasible set |
//| * F(), G(), H() are continuously differentiable on the |
//| feasible set and its neighborhood |
//| * nonlinear constraints G() and H() must have non-zero |
//| gradient at G(x) = 0 and at H(x) = 0. Say, constraint like |
//| x^2 >= 1 is supported, but x^2 >= 0 is NOT supported. |
//| USAGE: |
//| Constrained optimization if far more complex than the |
//| unconstrained one. Nonlinearly constrained optimization is one |
//| of the most esoteric numerical procedures. |
//| Here we give very brief outline of the MinNLC optimizer. We |
//| strongly recommend you to study examples in the ALGLIB Reference |
//| Manual and to read ALGLIB User Guide on optimization, which is |
//| available at http://www.alglib.net/optimization/ |
//| 1. User initializes algorithm State with MinNLCCreate() call |
//| and chooses what NLC solver to use. There is some solver |
//| which is used by default, with default Settings, but you |
//| should NOT rely on default choice. It may change in future |
//| releases of ALGLIB without notice, and no one can guarantee |
//| that new solver will be able to solve your problem with |
//| default Settings. |
//| From the other side, if you choose solver explicitly, you can be |
//| pretty sure that it will work with new ALGLIB releases. |
//| In the current release following solvers can be used: |
//| * SQP solver, recommended for medium-scale problems (less than|
//| thousand of variables) with hard-to-evaluate target |
//| functions. Requires less function evaluations than other |
//| solvers but each step involves solution of QP subproblem, |
//| so running time may be higher than that of AUL (another |
//| recommended option). Activated with MinNLCSetAlgoSQP() |
//| function. |
//| * AUL solver with dense preconditioner, recommended for |
//| large-scale problems or for problems with cheap target |
//| function. Needs more function evaluations that SQP (about |
//| 5x - 10x times more), but its iterations are much |
//| cheaper that that of SQP. Activated with MinNLCSetAlgoAUL() |
//| function. |
//| * SLP solver, successive linear programming. The slowest one, |
//| requires more target function evaluations that SQP and AUL. |
//| However, it is somewhat more robust in tricky cases, so |
//| it can be used as a backup plan. Activated with |
//| MinNLCSetAlgoSLP() function. |
//| 2. [optional] user activates OptGuard integrity checker which |
//| tries to detect possible errors in the user - supplied |
//| callbacks: |
//| * discontinuity/nonsmoothness of the target/nonlinear |
//| constraints |
//| * errors in the analytic gradient provided by user. |
//| This feature is essential for early prototyping stages because it|
//| helps to catch common coding and problem statement errors. |
//| OptGuard can be activated with following functions (one per each |
//| check performed): |
//| * MinNLCOptGuardSmoothness() |
//| * MinNLCOptGuardGradient() |
//| 3. User adds boundary and/or linear and/or nonlinear |
//| constraints by means of calling one of the following |
//| functions: |
//| a) MinNLCSetBC() for boundary constraints |
//| b) MinNLCSetLC() for linear constraints |
//| c) MinNLCSetNLC() for nonlinear constraints |
//| You may combine(a), (b) and (c) in one optimization problem. |
//| 4. User sets scale of the variables with MinNLCSetScale() |
//| function. It is VERY important to set scale of the |
//| variables, because nonlinearly constrained problems are |
//| hard to solve when variables are badly scaled. |
//| 5. User sets stopping conditions with MinNLCSetCond(). If |
//| NLC solver uses inner/outer iteration layout, this |
//| function sets stopping conditions for INNER iterations. |
//| 6. Finally, user calls MinNLCOptimize() function which takes |
//| algorithm State and pointer (delegate, etc.) to callback |
//| function which calculates F / G / H. |
//| 7. User calls MinNLCResults() to get solution; additionally you|
//| can retrieve OptGuard report with MinNLCOptGuardResults(), |
//| and get detailed report about purported errors in the target|
//| function with: |
//| * MinNLCOptGuardNonC1Test0Results() |
//| * MinNLCOptGuardNonC1Test1Results() |
//| 8. Optionally user may call MinNLCRestartFrom() to solve |
//| another problem with same N but another starting point. |
//| MinNLCRestartFrom() allows to reuse already initialized |
//| structure. |
//| INPUT PARAMETERS: |
//| N - problem dimension, N > 0 : |
//| * if given, only leading N elements of X are used |
//| * if not given, automatically determined from size |
//| of X |
//| X - starting point, array[N]: |
//| * it is better to set X to a feasible point |
//| * but X can be infeasible, in which case algorithm |
//| will try to find feasible point first, using X as|
//| initial approximation. |
//| OUTPUT PARAMETERS: |
//| State - structure stores algorithm State |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCCreate(int n,CRowDouble &x,CMinNLCState &State)
{
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values"))
return;
MinNLCInitInternal(n,x,0.0,State);
}
//+------------------------------------------------------------------+
//| This subroutine is a finite difference variant of MinNLCCreate().|
//| It uses finite differences in order to differentiate target |
//| function. |
//| Description below contains information which is specific to this |
//| function only. We recommend to read comments on MinNLCCreate() |
//| in order to get more information about creation of NLC optimizer.|
//| INPUT PARAMETERS: |
//| N - problem dimension, N > 0: |
//| * if given, only leading N elements of X are used |
//| * if not given, automatically determined from size |
//| of X |
//| X - starting point, array[N]: |
//| * it is better to set X to a feasible point |
//| * but X can be infeasible, in which case algorithm |
//| will try to find feasible point first, using X as|
//| initial approximation. |
//| DiffStep - differentiation step, > 0 |
//| OUTPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| NOTES: |
//| 1. algorithm uses 4-point central formula for differentiation. |
//| 2. differentiation step along I-th axis is equal to |
//| DiffStep*S[I] where S[] is scaling vector which can be set |
//| by MinNLCSetScale() call. |
//| 3. we recommend you to use moderate values of differentiation |
//| step. Too large step will result in too large TRUNCATION |
//| errors, while too small step will result in too large |
//| NUMERICAL errors. 1.0E-4 can be good value to start from. |
//| 4. Numerical differentiation is very inefficient - one gradient|
//| calculation needs 4 * N function evaluations. This function |
//| will work for any N - either small(1...10), moderate |
//| (10...100) or large(100...). However, performance penalty |
//| will be too severe for any N's except for small ones. We |
//| should also say that code which relies on numerical |
//| differentiation is less robust and precise. Imprecise |
//| gradient may slow down convergence, especially on highly |
//| nonlinear problems. Thus we recommend to use this function |
//| for fast prototyping on small-dimensional problems only, |
//| and to implement analytical gradient as soon as possible. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCCreateF(int n,
CRowDouble &x,
double diffstep,
CMinNLCState &State)
{
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values"))
return;
if(!CAp::Assert(MathIsValidNumber(diffstep),__FUNCTION__+": DiffStep is infinite or NaN!"))
return;
if(!CAp::Assert(diffstep>0.0,__FUNCTION__+": DiffStep is non-positive!"))
return;
//--- function call
MinNLCInitInternal(n,x,diffstep,State);
}
//+------------------------------------------------------------------+
//| This function sets boundary constraints for NLC optimizer. |
//| Boundary constraints are inactive by default (after initial |
//| creation). They are preserved after algorithm restart with |
//| MinNLCRestartFrom(). |
//| You may combine boundary constraints with general linear ones -|
//| and with nonlinear ones! Boundary constraints are handled more |
//| efficiently than other types. Thus, if your problem has mixed |
//| constraints, you may explicitly specify some of them as boundary |
//| and save some time / space. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| BndL - lower bounds, array[N]. If some (all) variables |
//| are unbounded, you may specify very small number|
//| or -INF. |
//| BndU - upper bounds, array[N]. If some (all) variables |
//| are unbounded, you may specify very large number|
//| or +INF. |
//| NOTE 1: it is possible to specify BndL.Set(i, BndU[i]. In this |
//| case I-th variable will be "frozen" at |
//| X.Set(i, BndL.Set(i, BndU[i]. |
//| NOTE 2: when you solve your problem with augmented Lagrangian |
//| solver, boundary constraints are satisfied only |
//| approximately! It is possible that algorithm will |
//| evaluate function outside of feasible area! |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetBC(CMinNLCState &State,
CRowDouble &bndl,
CRowDouble &bndu)
{
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(bndl)>=n,__FUNCTION__+": Length(BndL)<N"))
return;
if(!CAp::Assert(CAp::Len(bndu)>=n,__FUNCTION__+": Length(BndU)<N"))
return;
for(int i=0; i<n; i++)
{
if(!CAp::Assert(MathIsValidNumber(bndl[i]) || IsNegInf(bndl[i]),__FUNCTION__+": BndL contains NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(bndu[i]) || IsPosInf(bndu[i]),__FUNCTION__+": BndL contains NAN or -INF"))
return;
State.m_bndl.Set(i,bndl[i]);
State.m_HasBndL[i]=MathIsValidNumber(bndl[i]);
State.m_bndu.Set(i,bndu[i]);
State.m_HasBndU[i]=MathIsValidNumber(bndu[i]);
}
}
//+------------------------------------------------------------------+
//| This function sets linear constraints for MinNLC optimizer. |
//| Linear constraints are inactive by default (after initial |
//| creation). They are preserved after algorithm restart with |
//| MinNLCRestartFrom(). |
//| You may combine linear constraints with boundary ones - and with |
//| nonlinear ones! If your problem has mixed constraints, you may |
//| explicitly specify some of them as linear. It may help optimizer |
//| to handle them more efficiently. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinNLCCreate |
//| call. |
//| C - linear constraints, array[K, N + 1]. Each row of C |
//| represents one constraint, either equality or |
//| inequality (see below): |
//| * first N elements correspond to coefficients, |
//| * last element corresponds to the right part. |
//| All elements of C (including right part) must be |
//| finite. |
//| CT - type of constraints, array[K]: |
//| * if CT[i] > 0, then I-th constraint is |
//| C[i, *] * x >= C[i, n + 1] |
//| * if CT[i] = 0, then I-th constraint is |
//| C[i, *] * x = C[i, n + 1] |
//| * if CT[i] < 0, then I-th constraint is |
//| C[i, *] * x <= C[i, n + 1] |
//| K - number of equality/inequality constraints, K >= 0: |
//| * if given, only leading K elements of C/CT are |
//| used |
//| * if not given, automatically determined from sizes|
//| of C/CT |
//| NOTE 1: when you solve your problem with augmented Lagrangian |
//| solver, linear constraints are satisfied only |
//| approximately! It is possible that algorithm will |
//| evaluate function outside of feasible area! |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetLC(CMinNLCState &State,
CMatrixDouble &c,
CRowInt &ct,
int k)
{
//--- create variables
int n=State.m_n;
int i=0;
//--- First, check for errors in the inputs
if(!CAp::Assert(k>=0,__FUNCTION__+": K<0"))
return;
if(!CAp::Assert(CAp::Cols(c)>=n+1 || k==0,__FUNCTION__+": Cols(C)<N+1"))
return;
if(!CAp::Assert(CAp::Rows(c)>=k,__FUNCTION__+": Rows(C)<K"))
return;
if(!CAp::Assert(CAp::Len(ct)>=k,__FUNCTION__+": Length(CT)<K"))
return;
if(!CAp::Assert(CApServ::IsFiniteMatrix(c,k,n+1),__FUNCTION__+": C contains infinite or NaN values!"))
return;
//--- Handle zero K
if(k==0)
{
State.m_nec=0;
State.m_nic=0;
return;
}
//--- Equality constraints are stored first, in the upper
//--- NEC rows of State.CLEIC matrix. Inequality constraints
//--- are stored in the next NIC rows.
//--- NOTE: we convert inequality constraints to the form
//--- A*x<=b before copying them.
CApServ::RMatrixSetLengthAtLeast(State.m_cleic,k,n+1);
CApServ::IVectorSetLengthAtLeast(State.m_lcsrcidx,k);
State.m_nec=0;
State.m_nic=0;
for(i=0; i<k; i++)
{
if(ct[i]==0)
{
State.m_cleic.Row(State.m_nec,c,i);
State.m_lcsrcidx.Set(State.m_nec,i);
State.m_nec=State.m_nec+1;
}
}
for(i=0; i<k; i++)
{
if(ct[i]!=0)
{
if(ct[i]>0)
State.m_cleic.Row(State.m_nec+State.m_nic,c[i]*(-1.0));
else
State.m_cleic.Row(State.m_nec+State.m_nic,c,i);
State.m_lcsrcidx.Set(State.m_nec+State.m_nic,i);
State.m_nic++;
}
}
}
//+------------------------------------------------------------------+
//| This function sets nonlinear constraints for MinNLC optimizer. |
//| In fact, this function sets NUMBER of nonlinear constraints. |
//| Constraints itself (constraint functions) are passed to |
//| MinNLCOptimize() method. This method requires user-defined vector|
//| function F[] and its Jacobian J[], where: |
//| * first component of F[] and first row of Jacobian J[] |
//| corresponds to function being minimized |
//| * next NLEC components of F[] (and rows of J) correspond to |
//| nonlinear equality constraints G_i(x) = 0 |
//| * next NLIC components of F[] (and rows of J) correspond to |
//| nonlinear inequality constraints H_i(x) <= 0 |
//| NOTE: you may combine nonlinear constraints with linear/boundary |
//| ones. If your problem has mixed constraints, you may |
//| explicitly specify some of them as linear ones. It may help|
//| optimizer to handle them more efficiently. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinNLCCreate|
//| call. |
//| NLEC - number of Non-Linear Equality Constraints(NLEC),|
//| >= 0 |
//| NLIC - number of Non-Linear Inquality Constraints(NLIC)|
//| >= 0 |
//| NOTE 1: when you solve your problem with augmented Lagrangian |
//| solver, nonlinear constraints are satisfied only |
//| approximately! It is possible that algorithm will |
//| evaluate function outside of feasible area! |
//| NOTE 2: algorithm scales variables according to scale specified |
//| by MinNLCSetScale() function, so it can handle problems |
//| with badly scaled variables (as long as we KNOW their |
//| scales). |
//| However, there is no way to automatically scale nonlinear |
//| constraints Gi(x) and Hi(x). Inappropriate scaling of Gi/Hi may |
//| ruin convergence. Solving problem with constraint "1000*G0(x)=0" |
//| is NOT same as solving it with constraint "0.001*G0(x)=0". |
//| It means that YOU are the one who is responsible for correct |
//| scaling of nonlinear constraints Gi(x) and Hi(x). We recommend |
//| you to scale nonlinear constraints in such way that I-th |
//| component of dG/dX (or dH/dx) has approximately unit magnitude |
//| (for problems with unit scale) or has magnitude approximately |
//| equal to 1/S[i] (where S is a scale set by MinNLCSetScale() |
//| function). |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetNLC(CMinNLCState &State,int nlec,int nlic)
{
if(!CAp::Assert(nlec>=0,__FUNCTION__+": NLEC<0"))
return;
if(!CAp::Assert(nlic>=0,__FUNCTION__+": NLIC<0"))
return;
State.m_ng=nlec;
State.m_nh=nlic;
State.m_fi.Resize(1+State.m_ng+State.m_nh);
State.m_j.Resize(1+State.m_ng+State.m_nh,State.m_n);
}
//+------------------------------------------------------------------+
//| This function sets stopping conditions for inner iterations of |
//| optimizer. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| EpsX - >= 0, The subroutine finishes its work if on k+1-th|
//| iteration the condition |v| <= EpsX is fulfilled, |
//| where: |
//| * | . | means Euclidian norm |
//| * v - scaled step vector, v[i] = dx[i] / s[i] |
//| * dx - step vector, dx = X(k + 1) - X(k) |
//| * s - scaling coefficients set by MinNLCSetScale() |
//| MaxIts - maximum number of iterations. If MaxIts = 0, the |
//| number of iterations is unlimited. |
//| Passing EpsX = 0 and MaxIts = 0(simultaneously) will lead to |
//| automatic selection of the stopping condition. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetCond(CMinNLCState &State,
double epsx,
int m_maxits)
{
if(!CAp::Assert(MathIsValidNumber(epsx),__FUNCTION__+": EpsX is not finite number"))
return;
if(!CAp::Assert((double)(epsx)>=0.0,__FUNCTION__+": negative EpsX"))
return;
if(!CAp::Assert(m_maxits>=0,__FUNCTION__+": negative MaxIts!"))
return;
if(epsx==0.0 && m_maxits==0)
epsx=1.0E-8;
State.m_epsx=epsx;
State.m_maxits=m_maxits;
}
//+------------------------------------------------------------------+
//| This function sets scaling coefficients for NLC optimizer. |
//| ALGLIB optimizers use scaling matrices to test stopping |
//| conditions (step size and gradient are scaled before comparison |
//| with tolerances). Scale of the I-th variable is a translation |
//| invariant measure of: |
//| a) "how large" the variable is |
//| b) how large the step should be to make significant changes in |
//| the function |
//| Scaling is also used by finite difference variant of the |
//| optimizer - step along I-th axis is equal to DiffStep*S[I]. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| S - array[N], non-zero scaling coefficients S[i] may |
//| be negative, sign doesn't matter. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetScale(CMinNLCState &State,CRowDouble &s)
{
//--- check
if(!CAp::Assert(CAp::Len(s)>=State.m_n,__FUNCTION__+": Length(S)<N"))
return;
for(int i=0; i<State.m_n; i++)
{
if(!CAp::Assert(MathIsValidNumber(s[i]),__FUNCTION__+": S contains infinite or NAN elements"))
return;
if(!CAp::Assert(s[i]!=0.0,__FUNCTION__+": S contains zero elements"))
return;
}
State.m_s=s.Abs()+0;
}
//+------------------------------------------------------------------+
//| This function sets preconditioner to "inexact LBFGS-based" mode. |
//| Preconditioning is very important for convergence of Augmented |
//| Lagrangian algorithm because presence of penalty term makes |
//| problem ill-conditioned. Difference between performance of |
//| preconditioned and unpreconditioned methods can be as large |
//| as 100x! |
//| MinNLC optimizer may use following preconditioners, each with its|
//| own benefits and drawbacks: |
//| a) inexact LBFGS-based, with O(N * K) evaluation time |
//| b) exact low rank one, with O(N * K ^ 2) evaluation time |
//| c) exact robust one, with O(N ^ 3 + K * N ^ 2) evaluation |
//| time where K is a total number of general linear and |
//| nonlinear constraints (box ones are not counted). |
//| Inexact LBFGS-based preconditioner uses L-BFGS formula combined |
//| with orthogonality assumption to perform very fast updates. For a|
//| N-dimensional problem with K general linear or nonlinear |
//| constraints (boundary ones are not counted) it has O(N * K) cost|
//| per iteration. This preconditioner has best quality (less |
//| iterations) when general linear and nonlinear constraints |
//| are orthogonal to each other (orthogonality with respect to |
//| boundary constraints is not required). Number of iterations |
//| increases when constraints are non - orthogonal, because |
//| algorithm assumes orthogonality, but still it is better than no |
//| preconditioner at all. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetPrecInexact(CMinNLCState &State)
{
State.m_updatefreq=0;
State.m_prectype=1;
}
//+------------------------------------------------------------------+
//| This function sets preconditioner to "exact low rank" mode. |
//| Preconditioning is very important for convergence of Augmented |
//| Lagrangian algorithm because presence of penalty term makes |
//| problem ill-conditioned. Difference between performance of |
//| preconditioned and unpreconditioned methods can be as large |
//| as 100x! |
//| MinNLC optimizer may use following preconditioners, each with its|
//| own benefits and drawbacks: |
//| a) inexact LBFGS-based, with O(N * K) evaluation time |
//| b) exact low rank one, with O(N * K ^ 2) evaluation time |
//| c) exact robust one, with O(N ^ 3 + K * N ^ 2) evaluation |
//| time where K is a total number of general linear and |
//| nonlinear constraints (box ones are not counted). |
//| It also provides special unpreconditioned mode of operation which|
//| can be used for test purposes. Comments below discuss low rank |
//| preconditioner. |
//| Exact low-rank preconditioner uses Woodbury matrix identity to |
//| build quadratic model of the penalized function. It has following|
//| features: |
//| * no special assumptions about orthogonality of constraints |
//| * preconditioner evaluation is optimized for K << N. Its cost |
//| is O(N * K ^ 2), so it may become prohibitively slow for |
//| K >= N. |
//| * finally, stability of the process is guaranteed only for |
//| K << N. Woodbury update often fail for K >= N due to |
//| degeneracy of intermediate matrices. |
//| That's why we recommend to use "exact robust" preconditioner for |
//| such cases. |
//| RECOMMENDATIONS: |
//| We recommend to choose between "exact low rank" and "exact |
//| robust" preconditioners, with "low rank" version being chosen |
//| when you know in advance that total count of non-box constraints |
//| won't exceed N, and "robust" version being chosen when you need |
//| bulletproof solution. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| UpdateFreq - update frequency. Preconditioner is rebuilt |
//| after every UpdateFreq iterations. Recommended |
//| value : 10 or higher. Zero value means that good|
//| default value will be used. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetPrecExactLowRank(CMinNLCState &State,
int updatefreq)
{
if(!CAp::Assert(updatefreq>=0,__FUNCTION__+": UpdateFreq<0"))
return;
if(updatefreq==0)
updatefreq=10;
State.m_prectype=2;
State.m_updatefreq=updatefreq;
}
//+------------------------------------------------------------------+
//| This function sets preconditioner to "exact robust" mode. |
//| Preconditioning is very important for convergence of Augmented |
//| Lagrangian algorithm because presence of penalty term makes |
//| problem ill-conditioned. Difference between performance of |
//| preconditioned and unpreconditioned methods can be as large |
//| as 100x! |
//| MinNLC optimizer may use following preconditioners, each with its|
//| own benefits and drawbacks: |
//| a) inexact LBFGS-based, with O(N * K) evaluation time |
//| b) exact low rank one, with O(N * K ^ 2) evaluation time |
//| c) exact robust one, with O(N ^ 3 + K * N ^ 2) evaluation |
//| time where K is a total number of general linear and |
//| nonlinear constraints (box ones are not counted). |
//| It also provides special unpreconditioned mode of operation which|
//| can be used for test purposes. Comments below discuss robust |
//| preconditioner. |
//| Exact robust preconditioner uses Cholesky decomposition to invert|
//| approximate Hessian matrix H = D + W'*C*W (where D stands for |
//| diagonal terms of Hessian, combined result of initial scaling |
//| matrix and penalty from box constraints; W stands for general |
//| linear constraints and linearization of nonlinear ones; C stands |
//| for diagonal matrix of penalty coefficients). |
//| This preconditioner has following features: |
//| * no special assumptions about constraint structure |
//| *preconditioner is optimized for stability; unlike "exact |
//| low rank" version which fails for K >= N, this one works well|
//| for any value of K. |
//| * the only drawback is that is takes O(N ^ 3 + K * N ^ 2) time |
//| to build it. No economical Woodbury update is applied even |
//| when it makes sense, thus there are exist situations (K << N)|
//| when "exact low rank" preconditioner outperforms this one. |
//| RECOMMENDATIONS: |
//| We recommend to choose between "exact low rank" and "exact |
//| robust" preconditioners, with "low rank" version being chosen |
//| when you know in advance that total count of non-box constraints |
//| won't exceed N, and "robust" version being chosen when you need |
//| bulletproof solution. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| UpdateFreq - update frequency. Preconditioner is rebuilt |
//| after every UpdateFreq iterations. Recommended |
//| value: 10 or higher. Zero value means that good |
//| default value will be used. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetPrecExactRobust(CMinNLCState &State,
int updatefreq)
{
if(!CAp::Assert(updatefreq>=0,__FUNCTION__+": UpdateFreq<0"))
return;
if(updatefreq==0)
updatefreq=10;
State.m_prectype=3;
State.m_updatefreq=updatefreq;
}
//+------------------------------------------------------------------+
//| This function sets preconditioner to "turned off" mode. |
//| Preconditioning is very important for convergence of Augmented |
//| Lagrangian algorithm because presence of penalty term makes |
//| problem ill-conditioned. Difference between performance of |
//| preconditioned and unpreconditioned methods can be as large |
//| as 100x! |
//| MinNLC optimizer may utilize two preconditioners, each with its |
//| own benefits and drawbacks: |
//| a) inexact LBFGS-based, and b) exact low rank one. |
//| It also provides special unpreconditioned mode of operation which|
//| can be used for test purposes. |
//| This function activates this test mode. Do not use it in |
//| production code to solve real-life problems. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetPrecNone(CMinNLCState &State)
{
State.m_updatefreq=0;
State.m_prectype=0;
}
//+------------------------------------------------------------------+
//| This function sets maximum step length (after scaling of step |
//| vector with respect to variable scales specified by |
//| MinNLCSetScale() call). |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| StpMax - maximum step length, >= 0. Set StpMax to 0.0 |
//| (default), if you don't want to limit step |
//| length. |
//| Use this subroutine when you optimize target function which |
//| contains exp() or other fast growing functions, and optimization |
//| algorithm makes too large steps which leads to overflow. This |
//| function allows us to reject steps that are too large (and |
//| therefore expose us to the possible overflow) without actually |
//| calculating function value at the x + stp*d. |
//| NOTE: different solvers employed by MinNLC optimizer use |
//| different norms for step; AUL solver uses 2-norm, whilst |
//| SLP solver uses INF-norm. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetSTPMax(CMinNLCState &State,double stpmax)
{
if(!CAp::Assert(MathIsValidNumber(stpmax),__FUNCTION__+": StpMax is not finite!"))
return;
if(!CAp::Assert((double)(stpmax)>=0.0,__FUNCTION__+": StpMax<0!"))
return;
State.m_stpmax=stpmax;
}
//+------------------------------------------------------------------+
//| This function tells MinNLC unit to use Augmented Lagrangian |
//| algorithm for nonlinearly constrained optimization. This |
//| algorithm is a slight modification of one described in |
//| "A Modified Barrier-Augmented Lagrangian Method for Constrained |
//| Minimization(1999)" by D.GOLDFARB, R.POLYAK, K. SCHEINBERG, |
//| I.YUZEFOVICH. |
//| AUL solver can be significantly faster than SQP on easy problems |
//| due to cheaper iterations, although it needs more function |
//| evaluations. |
//| Augmented Lagrangian algorithm works by converting problem of |
//| minimizing F(x) subject to equality/inequality constraints to |
//| unconstrained problem of the form |
//| min[ f(x) + |
//| + Rho * PENALTY_EQ(x) + SHIFT_EQ(x, Nu1) + |
//| + Rho * PENALTY_INEQ(x) + SHIFT_INEQ(x, Nu2) ] |
//| where: |
//| * Rho is a fixed penalization coefficient |
//| * PENALTY_EQ(x) is a penalty term, which is used to |
//| APPROXIMATELY enforce equality constraints |
//| *SHIFT_EQ(x) is a special "shift" term which is used to |
//| "fine-tune" equality constraints, greatly increasing |
//| precision |
//| * PENALTY_INEQ(x) is a penalty term which is used to |
//| approximately enforce inequality constraints |
//| *SHIFT_INEQ(x) is a special "shift" term which is used to |
//| "fine-tune" inequality constraints, greatly increasing |
//| precision |
//| * Nu1/Nu2 are vectors of Lagrange coefficients which are fine- |
//| tuned during outer iterations of algorithm |
//| This version of AUL algorithm uses preconditioner, which |
//| greatly accelerates convergence. Because this algorithm is |
//| similar to penalty methods, it may perform steps into infeasible |
//| area. All kinds of constraints (boundary, linear and nonlinear |
//| ones) may be violated in intermediate points - and in the |
//| solution. However, properly configured AUL method is |
//| significantly better at handling constraints than barrier and/or |
//| penalty methods. |
//| The very basic outline of algorithm is given below: |
//| 1) first outer iteration is performed with "default" values of |
//| Lagrange multipliers Nu1/Nu2. Solution quality is low |
//| (candidate point can be too far away from true solution; |
//| large violation of constraints is possible) and is |
//| comparable with that of penalty methods. |
//| 2) subsequent outer iterations refine Lagrange multipliers and |
//| improve quality of the solution. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| Rho - penalty coefficient, Rho > 0: |
//| * large enough that algorithm converges with |
//| desired precision. Minimum value is |
//| 10 * max(S'*diag(H)*S), where S is a scale matrix|
//| (set by MinNLCSetScale) and H is a Hessian of the|
//| function being minimized. If you can not easily |
//| estimate Hessian norm, see our recommendations |
//| below. |
//| * not TOO large to prevent ill - conditioning |
//| * for unit - scale problems(variables and Hessian |
//| have unit magnitude), Rho = 100 or Rho = 1000 can|
//| be used. |
//| * it is important to note that Rho is internally |
//| multiplied by scaling matrix, i.e. optimum value |
//| of Rho depends on scale of variables specified |
//| by MinNLCSetScale(). |
//| ItsCnt - number of outer iterations: |
//| * ItsCnt = 0 means that small number of outer |
//| iterations is automatically chosen (10 iterations|
//| in current version). |
//| * ItsCnt = 1 means that AUL algorithm performs just|
//| as usual barrier method. |
//| * ItsCnt > 1 means that AUL algorithm performs |
//| specified number of outer iterations |
//| HOW TO CHOOSE PARAMETERS |
//| Nonlinear optimization is a tricky area and Augmented Lagrangian |
//| algorithm is sometimes hard to tune. Good values of Rho and |
//| ItsCnt are problem - specific. In order to help you we prepared |
//| following set of recommendations: |
//| * for unit-scale problems (variables and Hessian have unit |
//| magnitude), Rho = 100 or Rho = 1000 can be used. |
//| * start from some small value of Rho and solve problem with |
//| just one outer iteration (ItcCnt = 1). In this case algorithm|
//| behaves like penalty method. Increase Rho in 2x or 10x steps |
//| until you see that one outer iteration returns point which is|
//| "rough approximation to solution". |
//| It is very important to have Rho so large that penalty term |
//| becomes constraining i.e. modified function becomes highly convex|
//| in constrained directions. |
//| From the other side, too large Rho may prevent you from |
//| converging to the solution. You can diagnose it by studying |
//| number of inner iterations performed by algorithm: too few (5-10 |
//| on 1000-dimensional problem) or too many (orders of magnitude |
//| more than dimensionality) usually means that Rho is too large. |
//| * with just one outer iteration you usually have low-quality |
//| solution. Some constraints can be violated with very large |
//| margin, while other ones (which are NOT violated in the true |
//| solution) can push final point too far in the inner area of |
//| the feasible set. |
//| For example, if you have constraint x0 >= 0 and true solution |
//| x0=1, then merely a presence of "x0>=0" will introduce a bias |
//| towards larger values of x0. Say, algorithm may stop at x0 = 1.5 |
//| instead of 1.0. |
//| * after you found good Rho, you may increase number of outer |
//| iterations. ItsCnt = 10 is a good value. Subsequent outer |
//| iteration will refine values of Lagrange multipliers. |
//| Constraints which were violated will be enforced, inactive |
//| constraints will be dropped(corresponding multipliers will be|
//| decreased). Ideally, you should see 10-1000x improvement in |
//| constraint handling(constraint violation is reduced). |
//| * if you see that algorithm converges to vicinity of solution, |
//| but additional outer iterations do not refine solution, it |
//| may mean that algorithm is unstable - it wanders around true |
//| solution, but can not approach it. Sometimes algorithm may be|
//| stabilized by increasing Rho one more time, making it 5x or |
//| 10x larger. |
//| SCALING OF CONSTRAINTS [IMPORTANT] |
//| AUL optimizer scales variables according to scale specified by |
//| MinNLCSetScale() function, so it can handle problems with badly |
//| scaled variables (as long as we KNOW their scales). However, |
//| because function being optimized is a mix of original function |
//| and constraint - dependent penalty functions, it is important to |
//| rescale both variables AND constraints. |
//| Say, if you minimize f(x) = x^2 subject to 1000000*x >= 0, then |
//| you have constraint whose scale is different from that of target |
//| function (another example is 0.000001*x >= 0). It is also |
//| possible to have constraints whose scales are misaligned: |
//| 1000000*x0 >= 0, 0.000001*x1 <= 0. Inappropriate scaling may ruin|
//| convergence because minimizing x^2 subject to x >= 0 is NOT same |
//| as minimizing it subject to 1000000*x >= 0. |
//| Because we know coefficients of boundary/linear constraints, we |
//| can automatically rescale and normalize them. However, there is |
//| no way to automatically rescale nonlinear constraints Gi(x) and |
//| Hi(x) - they are black boxes. |
//| It means that YOU are the one who is responsible for correct |
//| scaling of nonlinear constraints Gi(x) and Hi(x). We recommend |
//| you to rescale nonlinear constraints in such way that I-th |
//| component of dG/dX (or dH/dx) has magnitude approximately equal |
//| to 1/S[i] (where S is a scale set by MinNLCSetScale() function). |
//| WHAT IF IT DOES NOT CONVERGE? |
//| It is possible that AUL algorithm fails to converge to precise |
//| values of Lagrange multipliers. It stops somewhere around true |
//| solution, but candidate point is still too far from solution, |
//| and some constraints are violated. Such kind of failure is |
//| specific for Lagrangian algorithms - technically, they stop at |
//| some point, but this point is not constrained solution. |
//| There are exist several reasons why algorithm may fail to |
//| converge: |
//| a) too loose stopping criteria for inner iteration |
//| b) degenerate, redundant constraints |
//| c) target function has unconstrained extremum exactly at the |
//| boundary of some constraint |
//| d) numerical noise in the target function |
//| In all these cases algorithm is unstable - each outer iteration |
//| results in large and almost random step which improves handling |
//| of some constraints, but violates other ones (ideally outer |
//| iterations should form a sequence of progressively decreasing |
//| steps towards solution). |
//| First reason possible is that too loose stopping criteria for |
//| inner iteration were specified. Augmented Lagrangian algorithm |
//| solves a sequence of intermediate problems, and requries each of |
//| them to be solved with high precision. Insufficient precision |
//| results in incorrect update of Lagrange multipliers. |
//| Another reason is that you may have specified degenerate |
//| constraints: say, some constraint was repeated twice. In most |
//| cases AUL algorithm gracefully handles such situations, but |
//| sometimes it may spend too much time figuring out subtle |
//| degeneracies in constraint matrix. |
//| Third reason is tricky and hard to diagnose. Consider situation |
//| when you minimize f = x^2 subject to constraint x >= 0. |
//| Unconstrained extremum is located exactly at the boundary of |
//| constrained area. In this case algorithm will tend to oscillate |
//| between negative and positive x. Each time it stops at x<0 it |
//| "reinforces" constraint x >= 0, and each time it is bounced to |
//| x>0 it "relaxes" constraint( and is attracted to x < 0). |
//| Such situation sometimes happens in problems with hidden |
//| symetries. Algorithm is got caught in a loop with Lagrange |
//| multipliers being continuously increased / decreased. Luckily, |
//| such loop forms after at least three iterations, so this problem|
//| can be solved by DECREASING number of outer iterations down |
//| to 1-2 and increasing penalty coefficient Rho as much as possible|
//| Final reason is numerical noise. AUL algorithm is robust against |
//| moderate noise (more robust than, say, active set methods), but |
//| large noise may destabilize algorithm. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetAlgoAUL(CMinNLCState &State,double rho,int itscnt)
{
if(!CAp::Assert(itscnt>=0,__FUNCTION__+": negative ItsCnt"))
return;
if(!CAp::Assert(MathIsValidNumber(rho),__FUNCTION__+": Rho is not finite"))
return;
if(!CAp::Assert(rho>0.0,__FUNCTION__+": Rho<=0"))
return;
if(itscnt==0)
itscnt=10;
State.m_aulitscnt=itscnt;
State.m_rho=rho;
State.m_solvertype=0;
}
//+------------------------------------------------------------------+
//| This function tells MinNLC optimizer to use SLP (Successive |
//| Linear Programming) algorithm for nonlinearly constrained |
//| optimization. This algorithm is a slight modification of one |
//| described in "A Linear programming - based optimization algorithm|
//| for solving nonlinear programming problems" (2010) by Claus Still|
//| and Tapio Westerlund. |
//| This solver is the slowest one in ALGLIB, it requires more target|
//| function evaluations that SQP and AUL. However it is somewhat |
//| more robust in tricky cases, so it can be used as a backup plan. |
//| We recommend to use this algo when SQP/AUL do not work (does not |
//| return the solution you expect). If trying different approach |
//| gives same results, then MAYBE something is wrong with your |
//| optimization problem. |
//| Despite its name ("linear" = "first order method") this algorithm|
//| performs steps similar to that of conjugate gradients method; |
//| internally it uses orthogonality/conjugacy requirement for |
//| subsequent steps which makes it closer to second order methods in|
//| terms of convergence speed. |
//| Convergence is proved for the following case: |
//| * function and constraints are continuously differentiable (C1|
//| class) |
//| * extended Mangasarian-Fromovitz constraint qualification |
//| (EMFCQ) holds; in the context of this algorithm EMFCQ means |
//| that one can, for any infeasible point, find a search |
//| direction such that the constraint infeasibilities are |
//| reduced. |
//| This algorithm has following nice properties: |
//| * no parameters to tune |
//| * no convexity requirements for target function or constraints|
//| * initial point can be infeasible |
//| * algorithm respects box constraints in all intermediate |
//| points (it does not even evaluate function outside of box |
//| constrained area) |
//| * once linear constraints are enforced, algorithm will not |
//| violate them |
//| * no such guarantees can be provided for nonlinear |
//| constraints, but once nonlinear constraints are enforced, |
//| algorithm will try to respect them as much as possible |
//| * numerical differentiation does not violate box constraints |
//| (although general linear and nonlinear ones can be violated |
//| during differentiation) |
//| * from our experience, this algorithm is somewhat more robust |
//| in really difficult cases |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| ===== TRACING SLP SOLVER ======================================= |
//| SLP solver supports advanced tracing capabilities. You can trace |
//| algorithm output by specifying following trace symbols (case- |
//| insensitive) by means of trace_file() call: |
//| * 'SLP' - for basic trace of algorithm steps and decisions. |
//| Only short scalars(function values and deltas) are |
//| printed. |
//| N-dimensional quantities like search directions are|
//| NOT printed. |
//| It also prints OptGuard integrity checker report |
//| when nonsmoothness of target / constraints is |
//| suspected. |
//| * 'SLP.DETAILED' - for output of points being visited and |
//| search directions. |
//| This symbol also implicitly defines 'SLP'. You can |
//| control output format by additionally specifying: |
//| * nothing to output in 6-digit exponential format |
//| * 'PREC.E15' to output in 15-digit exponential |
//| format |
//| * 'PREC.F6' to output in 6-digit fixed-point format|
//| * 'SLP.PROBING' - to let algorithm insert additional function |
//| evaluations before line search in order to build |
//| human-readable chart of the raw Lagrangian |
//| (~40 additional function evaluations is performed |
//| for each line search). This symbol also implicitly |
//| defines 'SLP'. Definition of this symbol also |
//| automatically activates OptGuard smoothness monitor|
//| * 'OPTGUARD' - for report of smoothness/continuity violations |
//| in target and/or constraints. This kind of |
//| reporting is included in 'SLP', but it comes with |
//| lots of additional Info. If you need just |
//| smoothness monitoring, specify this setting. |
//| NOTE: this tag merely directs OptGuard output to log file. Even |
//| if you specify it, you still have to configure OptGuard by |
//| calling MinNLCOptGuard...() family of functions. |
//| By default trace is disabled and adds no overhead to the |
//| optimization process. However, specifying any of the symbols adds|
//| some formatting and output - related overhead. Specifying |
//| 'SLP.PROBING' adds even larger overhead due to additional |
//| function evaluations being performed. |
//| You may specify multiple symbols by separating them with commas: |
//| > |
//| >CAlglib::Trace_File("SLP,SLP.PROBING,PREC.F6", |
//| "path/to/trace.log") |
//| > |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetAlgoSLP(CMinNLCState &State)
{
State.m_solvertype=1;
}
//+------------------------------------------------------------------+
//| This function tells MinNLC optimizer to use SQP (Successive |
//| Quadratic Programming) algorithm for nonlinearly constrained |
//| optimization. |
//| This algorithm needs order of magnitude (5x-10x) less function |
//| evaluations than AUL solver, but has higher overhead because each|
//| iteration involves solution of quadratic programming problem. |
//| Convergence is proved for the following case: |
//| * function and constraints are continuously differentiable |
//| (C1 class) |
//| This algorithm has following nice properties: |
//| * no parameters to tune |
//| * no convexity requirements for target function or constraints|
//| * initial point can be infeasible |
//| * algorithm respects box constraints in all intermediate |
//| points (it does not even evaluate function outside of box |
//| constrained area) |
//| * once linear constraints are enforced, algorithm will not |
//| violate them |
//| * no such guarantees can be provided for nonlinear |
//| constraints, but once nonlinear constraints are enforced, |
//| algorithm will try to respect them as much as possible |
//| * numerical differentiation does not violate box constraints |
//| (although general linear and nonlinear ones can be violated |
//| during differentiation) |
//| We recommend this algorithm as a default option for medium scale |
//| problems (less than thousand of variables) or problems with |
//| target function being hard to evaluate. |
//| For large-scale problems or ones with very cheap target function|
//| AUL solver can be better option. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| ===== INTERACTION WITH OPTGUARD ================================ |
//| OptGuard integrity checker allows us to catch problems like |
//| errors in gradients and discontinuity/nonsmoothness of the |
//| target/constraints. The latter kind of problems can be detected |
//| by looking upon line searches performed during optimization and |
//| searching for signs of nonsmoothness. |
//| The problem with SQP is that it is too good for OptGuard to work-|
//| it does not perform line searches. It typically needs 1-2 |
//| function evaluations per step, and it is not enough for OptGuard |
//| to detect nonsmoothness. |
//| So, if you suspect that your problem is nonsmooth and if you want|
//| to confirm or deny it, we recommend you to either: |
//| * use AUL or SLP solvers, which can detect nonsmoothness of |
//| the problem |
//| * or, alternatively, activate 'SQP.PROBING' trace tag that |
//| will insert additional function evaluations (~40 per line |
//| step) that will help OptGuard integrity checker to study |
//| properties of your problem |
//| ===== TRACING SQP SOLVER ======================================= |
//| SQP solver supports advanced tracing capabilities. You can trace |
//| algorithm output by specifying following trace symbols (case- |
//| insensitive) by means of trace_file() call: |
//| * 'SQP' - for basic trace of algorithm steps and |
//| decisions. Only short scalars (function values |
//| and deltas) are printed. |
//| N-dimensional quantities like search directions |
//| are NOT printed. |
//| It also prints OptGuard integrity checker report|
//| when nonsmoothness of target/constraints is |
//| suspected. |
//| * 'SQP.DETAILED' - for output of points being visited and |
//| search directions. This symbol also implicitly |
//| defines 'SQP'. You can control output format by |
//| additionally specifying: |
//| * nothing to output in 6-digit exponential |
//| format |
//| * 'PREC.E15' to output in 15-digit exponential |
//| format |
//| * 'PREC.F6' to output in 6-digit fixed-point |
//| format |
//| * 'SQP.PROBING' - to let algorithm insert additional function |
//| evaluations before line search in order to build|
//| human-readable chart of the raw Lagrangian (~40 |
//| additional function evaluations is performed for|
//| each line search). This symbol also implicitly |
//| defines 'SQP' and activates OptGuard integrity |
//| checker which detects continuity and smoothness |
//| violations. An OptGuard log is printed at the |
//| end of the file. |
//| By default trace is disabled and adds no overhead to the |
//| optimization process. However, specifying any of the symbols adds|
//| some formatting and output-related overhead. Specifying |
//| 'SQP.PROBING' adds even larger overhead due to additional |
//| function evaluations being performed. |
//| You may specify multiple symbols by separating them with commas: |
//| > |
//| >CAlglib::Trace_File("SQP,SQP.PROBING,PREC.F6", |
//| "path/to/trace.log") |
//| > |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetAlgoSQP(CMinNLCState &State)
{
State.m_solvertype=2;
}
//+------------------------------------------------------------------+
//| This function turns on / off reporting. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NeedXRep - whether iteration reports are needed or not |
//| If NeedXRep is True, algorithm will call rep() callback function |
//| if it is provided to MinNLCOptimize(). |
//| NOTE: algorithm passes two parameters to rep() callback - |
//| current point and penalized function value at current |
//| point. Important - function value which is returned is |
//| NOT function being minimized. It is sum of the value of |
//| the function being minimized - and penalty term. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCSetXRep(CMinNLCState &State,
bool needxrep)
{
State.m_xrep=needxrep;
}
//+------------------------------------------------------------------+
//| NOTES: |
//| 1. This function has two different implementations: one which |
//| uses exact (analytical) user-supplied Jacobian, and one |
//| which uses only function vector and numerically |
//| differentiates function in order to obtain gradient. |
//| Depending on the specific function used to create optimizer |
//| object you should choose appropriate variant of MinNLCOptimize()-|
//| one which accepts function AND Jacobian or one which accepts ONLY|
//| function. |
//| Be careful to choose variant of MinNLCOptimize() which |
//| corresponds to your optimization scheme! Table below lists |
//| different combinations of callback (function/gradient) passed to |
//| MinNLCOptimize() and specific function used to create optimizer. |
//| | USER PASSED TO MinNLCOptimize() |
//| CREATED WITH | function only | function and gradient |
//| ------------------------------------------------------------ |
//| MinNLCCreateF() | works FAILS |
//| MinNLCCreate() | FAILS works |
//| Here "FAILS" denotes inappropriate combinations of optimizer |
//| creation function and MinNLCOptimize() version. Attemps to use |
//| such combination will lead to exception. Either you did not pass |
//| gradient when it WAS needed or you passed gradient when it was |
//| NOT needed. |
//+------------------------------------------------------------------+
bool CMinNLC::MinNLCIteration(CMinNLCState &State)
{
//--- create variables
bool result=false;
int i=0;
int k=0;
int n=0;
int ng=0;
int nh=0;
double vleft=0;
double vright=0;
bool b=false;
int i_=0;
int label=-1;
//--- Reverse communication preparations
//--- I know it looks ugly, but it works the same way
//--- anywhere from C++ to Python.
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(State.m_rstate.stage>=0)
{
i=State.m_rstate.ia[0];
k=State.m_rstate.ia[1];
n=State.m_rstate.ia[2];
ng=State.m_rstate.ia[3];
nh=State.m_rstate.ia[4];
b=State.m_rstate.ba[0];
vleft=State.m_rstate.ra[0];
vright=State.m_rstate.ra[1];
}
else
{
i=359;
k=-58;
n=-919;
ng=-909;
nh=81;
b=true;
vleft=74;
vright=-788;
}
switch(State.m_rstate.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
case 3:
label=3;
break;
case 4:
label=4;
break;
case 5:
label=5;
break;
case 6:
label=6;
break;
case 7:
label=7;
break;
case 8:
label=8;
break;
case 9:
label=9;
break;
case 10:
label=10;
break;
case 11:
label=11;
break;
case 12:
label=12;
break;
case 13:
label=13;
break;
case 14:
label=14;
break;
case 15:
label=15;
break;
case 16:
label=16;
break;
case 17:
label=17;
break;
case 18:
label=18;
break;
case 19:
label=19;
break;
case 20:
label=20;
break;
case 21:
label=21;
break;
case 22:
label=22;
break;
case 23:
label=23;
break;
case 24:
label=24;
break;
default:
//--- Routine body
//--- Init
State.m_userterminationneeded=false;
State.m_repterminationtype=0;
State.m_repinneriterationscount=0;
State.m_repouteriterationscount=0;
State.m_repnfev=0;
State.m_repdbgphase0its=0;
State.m_repbcerr=0;
State.m_repbcidx=-1;
State.m_replcerr=0;
State.m_replcidx=-1;
State.m_repnlcerr=0;
State.m_repnlcidx=-1;
n=State.m_n;
ng=State.m_ng;
nh=State.m_nh;
ClearRequestFields(State);
if(!CAp::Assert(State.m_smoothnessguardlevel==0 || State.m_smoothnessguardlevel==1,__FUNCTION__+": integrity check failed"))
return(false);
b=State.m_smoothnessguardlevel>0;
b=b || (State.m_solvertype==1 && CAp::IsTraceEnabled("SLP.PROBING"));
b=b || (State.m_solvertype==2 && CAp::IsTraceEnabled("SQP.PROBING"));
COptServ::SmoothnessMonitorInit(State.m_smonitor,State.m_s,n,1+ng+nh,b);
State.m_lastscaleused=State.m_s;
//--- Check correctness of box constraints
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i] && State.m_HasBndU[i])
{
if(State.m_bndl[i]>State.m_bndu[i])
{
State.m_repterminationtype=-3;
State.m_repbcerr=State.m_bndl[i]-State.m_bndu[i];
State.m_repbcidx=i;
result=false;
return(result);
}
}
}
//--- Test gradient
if(!(State.m_diffstep==0.0 && State.m_teststep>0.0))
label=25;
else
label=27;
break;
}
//---main loop
while(label>=0)
{
switch(label)
{
case 27:
if(!COptServ::SmoothnessMonitorCheckGradientATX0(State.m_smonitor,State.m_xstart,State.m_s,State.m_bndl,State.m_bndu,true,State.m_teststep))
{
label=28;
break;
}
State.m_x=State.m_smonitor.m_x;
State.m_needfij=true;
State.m_rstate.stage=0;
label=-1;
break;
case 0:
State.m_needfij=false;
State.m_smonitor.m_fi=State.m_fi;
State.m_smonitor.m_j=State.m_j;
label=27;
break;
case 28:
case 25:
//--- AUL solver
if(State.m_solvertype!=0)
{
label=29;
break;
}
if(State.m_diffstep!=0.0)
{
CApServ::RVectorSetLengthAtLeast(State.m_xbase,n);
CApServ::RVectorSetLengthAtLeast(State.m_fbase,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fm2,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fm1,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fp1,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fp2,1+ng+nh);
}
State.m_rstateaul.ia.Resize(9);
State.m_rstateaul.ra.Resize(8);
State.m_rstateaul.stage=-1;
case 6:
case 31:
if(!AULIteration(State,State.m_smonitor))
{
label=32;
break;
}
//--- Numerical differentiation (if needed) - intercept NeedFiJ
//--- request and replace it by sequence of NeedFi requests
if(!(State.m_diffstep!=0.0 && State.m_needfij))
{
label=33;
break;
}
State.m_needfij=false;
State.m_needfi=true;
State.m_xbase=State.m_x;
k=0;
case 35:
if(k>n-1)
{
label=37;
break;
}
State.m_x=State.m_xbase;
State.m_x.Add(k,- State.m_s[k]*State.m_diffstep);
State.m_rstate.stage=1;
label=-1;
break;
case 1:
State.m_fm2=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Add(k,- 0.5*State.m_s[k]*State.m_diffstep);
State.m_rstate.stage=2;
label=-1;
break;
case 2:
State.m_fm1=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Add(k,0.5*State.m_s[k]*State.m_diffstep);
State.m_rstate.stage=3;
label=-1;
break;
case 3:
State.m_fp1=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Add(k,State.m_s[k]*State.m_diffstep);
State.m_rstate.stage=4;
label=-1;
break;
case 4:
State.m_fp2=State.m_fi;
State.m_j.Col(k,((State.m_fp1-State.m_fm1)*8-(State.m_fp2-State.m_fm2))/(6*State.m_diffstep*State.m_s[k]));
k++;
label=35;
break;
case 37:
State.m_x=State.m_xbase;
State.m_rstate.stage=5;
label=-1;
break;
case 5:
//--- Restore previous values of fields and continue
State.m_needfi=false;
State.m_needfij=true;
label=31;
break;
case 33:
//--- Forward request to caller
State.m_rstate.stage=6;
label=-1;
break;
case 32:
result=false;
return(result);
case 29:
//--- SLP solver
if(State.m_solvertype!=1)
{
label=38;
break;
}
if(State.m_diffstep!=0.0)
{
CApServ::RVectorSetLengthAtLeast(State.m_xbase,n);
CApServ::RVectorSetLengthAtLeast(State.m_fbase,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fm2,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fm1,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fp1,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fp2,1+ng+nh);
}
CNLCSLP::MinSLPInitBuf(State.m_bndl,State.m_bndu,State.m_s,State.m_xstart,n,State.m_cleic,State.m_lcsrcidx,State.m_nec,State.m_nic,State.m_ng,State.m_nh,State.m_epsx,State.m_maxits,State.m_slpsolverstate);
case 40:
if(!CNLCSLP::MinSLPIteration(State.m_slpsolverstate,State.m_smonitor,State.m_userterminationneeded))
{
label=41;
break;
}
//--- Forward request to caller
if(!State.m_slpsolverstate.m_needfij)
{
label=42;
break;
}
//--- Evaluate target function/Jacobian
if(State.m_diffstep!=0.0)
{
label=44;
break;
}
//--- Analytic Jacobian is provided
UnScale(State,State.m_slpsolverstate.m_x,State.m_slpsolverstate.m_scaledbndl,State.m_slpsolverstate.m_scaledbndu,State.m_x);
State.m_needfij=true;
State.m_rstate.stage=7;
label=-1;
break;
case 7:
State.m_needfij=false;
State.m_slpsolverstate.m_fi=State.m_fi;
for(i=0; i<=ng+nh; i++)
State.m_slpsolverstate.m_j.Row(i,State.m_s.ToVector()*State.m_j[i]);
k=n;
label=45;
break;
case 44:
//--- Numerical differentiation
State.m_needfij=false;
State.m_needfi=true;
UnScale(State,State.m_slpsolverstate.m_x,State.m_slpsolverstate.m_scaledbndl,State.m_slpsolverstate.m_scaledbndu,State.m_xbase);
k=0;
case 46:
if(k>n-1)
{
label=48;
break;
}
vleft=State.m_xbase[k]-State.m_s[k]*State.m_diffstep;
vright=State.m_xbase[k]+State.m_s[k]*State.m_diffstep;
if(!((State.m_HasBndL[k] && (double)(vleft)<(double)(State.m_bndl[k])) || (State.m_HasBndU[k] && (double)(vright)>(double)(State.m_bndu[k]))))
{
label=49;
break;
}
//--- Box constraint is violated by 4-point centered formula, use 2-point uncentered one
if(State.m_HasBndL[k] && vleft<State.m_bndl[k])
vleft=State.m_bndl[k];
if(State.m_HasBndU[k] && vright>State.m_bndu[k])
vright=State.m_bndu[k];
if(!CAp::Assert(vleft<=vright,__FUNCTION__+": integrity check failed"))
return(false);
if(vleft==vright)
{
//--- Fixed variable
State.m_j.Col(k,vector<double>::Zeros(ng+nh+1));
label=47;
break;
}
State.m_x=State.m_xbase;
State.m_x.Set(k,vleft);
State.m_rstate.stage=8;
label=-1;
break;
case 8:
State.m_fm1=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Set(k,vright);
State.m_rstate.stage=9;
label=-1;
break;
case 9:
State.m_fp1=State.m_fi;
State.m_j.Col(k,(State.m_fp1-State.m_fm1)/(vright-vleft));
label=50;
break;
case 49:
//--- 4-point centered formula does not violate box constraints
State.m_x=State.m_xbase;
State.m_x.Add(k,-State.m_s[k]*State.m_diffstep);
State.m_rstate.stage=10;
label=-1;
break;
case 10:
State.m_fm2=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Add(k,-0.5*State.m_s[k]*State.m_diffstep);
State.m_rstate.stage=11;
label=-1;
break;
case 11:
State.m_fm1=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Add(k,0.5*State.m_s[k]*State.m_diffstep);
State.m_rstate.stage=12;
label=-1;
break;
case 12:
State.m_fp1=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Add(k,State.m_s[k]*State.m_diffstep);
State.m_rstate.stage=13;
label=-1;
break;
case 13:
State.m_fp2=State.m_fi;
State.m_j.Col(k,((State.m_fp1-State.m_fm1)*8-(State.m_fp2-State.m_fm2))/(6*State.m_diffstep*State.m_s[k]));
case 50:
case 47:
k++;
label=46;
break;
case 48:
State.m_x=State.m_xbase;
State.m_rstate.stage=14;
label=-1;
break;
case 14:
State.m_needfi=false;
State.m_needfij=true;
State.m_slpsolverstate.m_fi=State.m_fi;
for(i=0; i<=ng+nh; i++)
State.m_slpsolverstate.m_j.Row(i,State.m_s.ToVector()*State.m_j[i]);
k=n;
case 45:
State.m_repnfev++;
label=40;
break;
case 42:
if(!State.m_slpsolverstate.m_xupdated)
{
label=51;
break;
}
//--- Report current point
if(!State.m_xrep)
{
label=40;
break;
}
UnScale(State,State.m_slpsolverstate.m_x,State.m_slpsolverstate.m_scaledbndl,State.m_slpsolverstate.m_scaledbndu,State.m_x);
State.m_f=State.m_slpsolverstate.m_f;
State.m_xupdated=true;
State.m_rstate.stage=15;
label=-1;
break;
case 15:
State.m_xupdated=false;
case 53:
label=40;
break;
case 51:
if(!CAp::Assert(State.m_slpsolverstate.m_needfij,__FUNCTION__+":SLP:request"))
return(false);
label=40;
break;
case 41:
State.m_repterminationtype=State.m_slpsolverstate.m_repterminationtype;
State.m_repouteriterationscount=State.m_slpsolverstate.m_repouteriterationscount;
State.m_repinneriterationscount=State.m_slpsolverstate.m_repinneriterationscount;
State.m_repbcerr=State.m_slpsolverstate.m_repbcerr;
State.m_repbcidx=State.m_slpsolverstate.m_repbcidx;
State.m_replcerr=State.m_slpsolverstate.m_replcerr;
State.m_replcidx=State.m_slpsolverstate.m_replcidx;
State.m_repnlcerr=State.m_slpsolverstate.m_repnlcerr;
State.m_repnlcidx=State.m_slpsolverstate.m_repnlcidx;
UnScale(State,State.m_slpsolverstate.m_stepkx,State.m_slpsolverstate.m_scaledbndl,State.m_slpsolverstate.m_scaledbndu,State.m_xc);
result=false;
return(result);
case 38:
//--- SQP solver
if(State.m_solvertype!=2)
{
label=55;
break;
}
if(State.m_diffstep!=0.0)
{
CApServ::RVectorSetLengthAtLeast(State.m_xbase,n);
CApServ::RVectorSetLengthAtLeast(State.m_fbase,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fm2,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fm1,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fp1,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fp2,1+ng+nh);
}
CNLCSQP::MinSQPInitBuf(State.m_bndl,State.m_bndu,State.m_s,State.m_xstart,n,State.m_cleic,State.m_lcsrcidx,State.m_nec,State.m_nic,State.m_ng,State.m_nh,State.m_epsx,State.m_maxits,State.m_sqpsolverstate);
case 57:
if(!CNLCSQP::MinSQPIteration(State.m_sqpsolverstate,State.m_smonitor,State.m_userterminationneeded))
{
label=58;
break;
}
//--- Forward request to caller
if(!State.m_sqpsolverstate.m_needfij)
{
label=59;
break;
}
//--- Evaluate target function/Jacobian
if(State.m_diffstep!=0.0)
{
label=61;
break;
}
//--- Analytic Jacobian is provided
UnScale(State,State.m_sqpsolverstate.m_x,State.m_sqpsolverstate.m_scaledbndl,State.m_sqpsolverstate.m_scaledbndu,State.m_x);
State.m_needfij=true;
State.m_rstate.stage=16;
label=-1;
break;
case 16:
State.m_needfij=false;
State.m_sqpsolverstate.m_fi=State.m_fi;
for(k=0; k<n; k++)
State.m_sqpsolverstate.m_j.Col(k,State.m_j.Col(k)*State.m_s[k]);
label=62;
break;
case 61:
//--- Numerical differentiation
State.m_needfij=false;
State.m_needfi=true;
UnScale(State,State.m_sqpsolverstate.m_x,State.m_sqpsolverstate.m_scaledbndl,State.m_sqpsolverstate.m_scaledbndu,State.m_xbase);
k=0;
case 63:
if(k>n-1)
{
label=65;
break;
}
vleft=State.m_xbase[k]-State.m_s[k]*State.m_diffstep;
vright=State.m_xbase[k]+State.m_s[k]*State.m_diffstep;
if(!((State.m_HasBndL[k] && vleft<State.m_bndl[k]) || (State.m_HasBndU[k] && vright>State.m_bndu[k])))
{
label=66;
break;
}
//--- Box constraint is violated by 4-point centered formula, use 2-point uncentered one
if(State.m_HasBndL[k] && vleft<State.m_bndl[k])
vleft=State.m_bndl[k];
if(State.m_HasBndU[k] && vright>State.m_bndu[k])
vright=State.m_bndu[k];
if(!CAp::Assert(vleft<=vright,__FUNCTION__+": integrity check failed"))
return(false);
if(vleft==vright)
{
//--- Fixed variable
State.m_j.Col(k,vector<double>::Zeros(ng+nh+1));
label=64;
break;
}
State.m_x=State.m_xbase;
State.m_x.Set(k,vleft);
State.m_rstate.stage=17;
label=-1;
break;
case 17:
State.m_fm1=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Set(k,vright);
State.m_rstate.stage=18;
label=-1;
break;
case 18:
State.m_fp1=State.m_fi;
State.m_j.Col(k,(State.m_fp1-State.m_fm1)/(vright-vleft));
label=67;
break;
case 66:
//--- 4-point centered formula does not violate box constraints
State.m_x=State.m_xbase;
State.m_x.Add(k,-State.m_s[k]*State.m_diffstep);
State.m_rstate.stage=19;
label=-1;
break;
case 19:
State.m_fm2=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Add(k,-0.5*State.m_s[k]*State.m_diffstep);
State.m_rstate.stage=20;
label=-1;
break;
case 20:
State.m_fm1=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Add(k,0.5*State.m_s[k]*State.m_diffstep);
State.m_rstate.stage=21;
label=-1;
break;
case 21:
State.m_fp1=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Add(k,State.m_s[k]*State.m_diffstep);
State.m_rstate.stage=22;
label=-1;
break;
case 22:
State.m_fp2=State.m_fi;
State.m_j.Col(k,((State.m_fp1-State.m_fm1)*8-(State.m_fp2-State.m_fm2))/(6*State.m_diffstep*State.m_s[k]));
case 67:
case 64:
k++;
label=63;
break;
case 65:
State.m_x=State.m_xbase;
State.m_rstate.stage=23;
label=-1;
break;
case 23:
State.m_needfi=false;
State.m_needfij=true;
State.m_sqpsolverstate.m_fi=State.m_fi;
for(k=0; k<n; k++)
State.m_sqpsolverstate.m_j.Col(k,State.m_j.Col(k)*State.m_s[k]);
case 62:
State.m_repnfev++;
label=57;
break;
case 59:
if(!State.m_sqpsolverstate.m_xupdated)
{
label=68;
break;
}
//--- Report current point
if(!State.m_xrep)
{
label=57;
break;
}
UnScale(State,State.m_sqpsolverstate.m_x,State.m_sqpsolverstate.m_scaledbndl,State.m_sqpsolverstate.m_scaledbndu,State.m_x);
State.m_f=State.m_sqpsolverstate.m_f;
State.m_xupdated=true;
State.m_rstate.stage=24;
label=-1;
break;
case 24:
State.m_xupdated=false;
case 70:
label=57;
break;
case 68:
if(!CAp::Assert(State.m_sqpsolverstate.m_needfij,__FUNCTION__+":SQP:request"))
return(false);
label=57;
break;
case 58:
State.m_repterminationtype=State.m_sqpsolverstate.m_repterminationtype;
State.m_repouteriterationscount=State.m_sqpsolverstate.m_repiterationscount;
State.m_repinneriterationscount=State.m_sqpsolverstate.m_repiterationscount;
State.m_repbcerr=State.m_sqpsolverstate.m_repbcerr;
State.m_repbcidx=State.m_sqpsolverstate.m_repbcidx;
State.m_replcerr=State.m_sqpsolverstate.m_replcerr;
State.m_replcidx=State.m_sqpsolverstate.m_replcidx;
State.m_repnlcerr=State.m_sqpsolverstate.m_repnlcerr;
State.m_repnlcidx=State.m_sqpsolverstate.m_repnlcidx;
UnScale(State,State.m_sqpsolverstate.m_stepkx,State.m_sqpsolverstate.m_scaledbndl,State.m_sqpsolverstate.m_scaledbndu,State.m_xc);
result=false;
return(result);
case 55:
result=false;
return(result);
}
}
//--- Saving State
result=true;
State.m_rstate.ba[0]=b;
State.m_rstate.ia.Set(0,i);
State.m_rstate.ia.Set(1,k);
State.m_rstate.ia.Set(2,n);
State.m_rstate.ia.Set(3,ng);
State.m_rstate.ia.Set(4,nh);
State.m_rstate.ra.Set(0,vleft);
State.m_rstate.ra.Set(1,vright);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function activates/deactivates verification of the user- |
//| supplied analytic gradient/Jacobian. |
//| Upon activation of this option OptGuard integrity checker |
//| performs numerical differentiation of your target function |
//| (constraints) at the initial point (note: future versions may |
//| also perform check at the final point) and compares numerical |
//| gradient/Jacobian with analytic one provided by you. |
//| If difference is too large, an error flag is set and optimization|
//| session continues. After optimization session is over, you can |
//| retrieve the report which stores both gradients/Jacobians, and |
//| specific components highlighted as suspicious by the OptGuard. |
//| The primary OptGuard report can be retrieved with |
//| MinNLCOptGuardResults(). |
//| IMPORTANT: gradient check is a high-overhead option which will |
//| cost you about 3*N additional function evaluations. |
//| In many cases it may cost as much as the rest of the |
//| optimization session. |
//| YOU SHOULD NOT USE IT IN THE PRODUCTION CODE UNLESS YOU WANT TO |
//| CHECK DERIVATIVES PROVIDED BY SOME THIRD PARTY. |
//| NOTE: unlike previous incarnation of the gradient checking code, |
//| OptGuard does NOT interrupt optimization even if it |
//| discovers bad gradient. |
//| INPUT PARAMETERS: |
//| State - structure used to store algorithm State |
//| TestStep - verification step used for numerical |
//| differentiation: |
//| * TestStep = 0 turns verification off |
//| * TestStep > 0 activates verification |
//| You should carefully choose TestStep. Value |
//| which is too large (so large that function |
//| behavior is non- cubic at this scale) will |
//| lead to false alarms. Too short step will |
//| result in rounding errors dominating numerical |
//| derivative. |
//| You may use different step for different |
//| parameters by means of setting scale with |
//| MinNLCSetScale(). |
//| === EXPLANATION ================================================ |
//| In order to verify gradient algorithm performs following steps: |
//| * two trial steps are made to |
//| X[i] - TestStep * S[i] and X[i] + TestStep * S[i], |
//| where X[i] is i-th component of the initial point and S[i] |
//| is a scale of i-th parameter |
//| * F(X) is evaluated at these trial points |
//| * we perform one more evaluation in the middle point of the |
//| interval |
//| * we build cubic model using function values and derivatives |
//| at trial points and we compare its prediction with actual |
//| value in the middle point |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCOptGuardGradient(CMinNLCState &State,
double teststep)
{
if(!CAp::Assert(MathIsValidNumber(teststep),__FUNCTION__+": TestStep contains NaN or INF"))
return;
if(!CAp::Assert(teststep>=0.0,__FUNCTION__+": invalid argument TestStep(TestStep<0)"))
return;
State.m_teststep=teststep;
}
//+------------------------------------------------------------------+
//| This function activates/deactivates nonsmoothness monitoring |
//| option of the OptGuard integrity checker. Smoothness monitor |
//| silently observes solution process and tries to detect ill-posed |
//| problems, i.e. ones with: |
//| a) discontinuous target function(non-C0) and/or constraints |
//| b) nonsmooth target function(non-C1) and/or constraints |
//| Smoothness monitoring does NOT interrupt optimization even if it |
//| suspects that your problem is nonsmooth. It just sets |
//| corresponding flags in the OptGuard report which can be retrieved|
//| after optimization is over. |
//| Smoothness monitoring is a moderate overhead option which often |
//| adds less than 1% to the optimizer running time. Thus, you can |
//| use it even for large scale problems. |
//| NOTE: OptGuard does NOT guarantee that it will always detect |
//| C0/C1 continuity violations. |
//| First, minor errors are hard to catch-say, a 0.0001 difference |
//| in the model values at two sides of the gap may be due to |
//| discontinuity of the model - or simply because the model has |
//| changed. |
//| Second, C1-violations are especially difficult to detect in a |
//| noninvasive way. The optimizer usually performs very short steps |
//| near the nonsmoothness, and differentiation usually introduces |
//| a lot of numerical noise. It is hard to tell whether some tiny |
//| discontinuity in the slope is due to real nonsmoothness or just |
//| due to numerical noise alone. |
//| Our top priority was to avoid false positives, so in some rare |
//| cases minor errors may went unnoticed (however, in most cases |
//| they can be spotted with restart from different initial point). |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| level - monitoring level: |
//| * 0 - monitoring is disabled |
//| * 1 - noninvasive low - overhead monitoring; |
//| function values and/or gradients are |
//| recorded, but OptGuard does not try to |
//| perform additional evaluations in order |
//| to get more information about suspicious |
//| locations. |
//| This kind of monitoring does not work well with SQP because SQP |
//| solver needs just 1-2 function evaluations per step, which is not|
//| enough for OptGuard to make any conclusions. |
//| === EXPLANATION ================================================ |
//| One major source of headache during optimization is the |
//| possibility of the coding errors in the target function/ |
//| constraints (or their gradients). Such errors most often |
//| manifest themselves as discontinuity or nonsmoothness of the |
//| target/constraints. |
//| Another frequent situation is when you try to optimize something |
//| involving lots of min() and max() operations, i.e. nonsmooth |
//| target. Although not a coding error, it is nonsmoothness anyway -|
//| and smooth optimizers usually stop right after encountering |
//| nonsmoothness, well before reaching solution. |
//| OptGuard integrity checker helps you to catch such situations: |
//| it monitors function values/gradients being passed to the |
//| optimizer and tries to errors. Upon discovering suspicious pair |
//| of points it raises appropriate flag (and allows you to continue |
//| optimization). When optimization is done, you can study OptGuard |
//| result. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCOptGuardSmoothness(CMinNLCState &State,int level)
{
if(!CAp::Assert(level==0 || level==1,__FUNCTION__+": unexpected value of level parameter"))
return;
State.m_smoothnessguardlevel=level;
}
//+------------------------------------------------------------------+
//| Results of OptGuard integrity check, should be called after |
//| optimization session is over. |
//| === PRIMARY REPORT ============================================= |
//| OptGuard performs several checks which are intended to catch |
//| common errors in the implementation of nonlinear function/ |
//| gradient: |
//| * incorrect analytic gradient |
//| * discontinuous (non-C0) target functions (constraints) |
//| * nonsmooth (non-C1) target functions (constraints) |
//| Each of these checks is activated with appropriate function: |
//| * MinNLCOptGuardGradient() for gradient verification |
//| * MinNLCOptGuardSmoothness() for C0/C1 checks |
//| Following flags are set when these errors are suspected: |
//| * rep.badgradsuspected, and additionally: |
//| * rep.badgradfidx for specific function (Jacobian row) |
//| suspected |
//| * rep.badgradvidx for specific variable (Jacobian column)|
//| suspected |
//| * rep.badgradxbase, a point where gradient/Jacobian is |
//| tested |
//| * rep.badgraduser, user-provided gradient/Jacobian |
//| * rep.badgradnum, reference gradient/Jacobian obtained |
//| via numerical differentiation |
//| * rep.nonc0suspected, and additionally: |
//| * rep.nonc0fidx - an index of specific function violating|
//| C0 continuity |
//| * rep.nonc1suspected, and additionally |
//| * rep.nonc1fidx - an index of specific function violating|
//| C1 continuity |
//| Here function index 0 means target function, index 1 or higher |
//| denotes nonlinear constraints. |
//| === ADDITIONAL REPORTS / LOGS ================================== |
//| Several different tests are performed to catch C0/C1 errors, you |
//| can find out specific test signaled error by looking to: |
//| * rep.nonc0test0positive, for non-C0 test #0 |
//| * rep.nonc1test0positive, for non-C1 test #0 |
//| * rep.nonc1test1positive, for non-C1 test #1 |
//| Additional information (including line search logs) can be |
//| obtained by means of: |
//| * MinNLCOptGuardNonC1Test0Results() |
//| * MinNLCOptGuardNonC1Test1Results() |
//| which return detailed error reports, specific points where |
//| discontinuities were found, and so on. |
//| ================================================================ |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| rep - generic OptGuard report; more detailed reports can|
//| be retrieved with other functions. |
//| NOTE: false negatives (nonsmooth problems are not identified |
//| as nonsmooth ones) are possible although unlikely. |
//| The reason is that you need to make several evaluations around |
//| nonsmoothness in order to accumulate enough information about |
//| function curvature. Say, if you start right from the nonsmooth |
//| point, optimizer simply won't get enough data to understand what |
//| is going wrong before it terminates due to abrupt changes in the |
//| derivative. It is also possible that "unlucky" step will move us |
//| to the termination too quickly. |
//| Our current approach is to have less than 0.1 % false negatives |
//| in our test examples (measured with multiple restarts from random|
//| points), and to have exactly 0 % false positives. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCOptGuardResults(CMinNLCState &State,COptGuardReport &rep)
{
COptServ::SmoothnessMonitorExportReport(State.m_smonitor,rep);
}
//+------------------------------------------------------------------+
//| Detailed results of the OptGuard integrity check for |
//| nonsmoothness test #0 |
//| Nonsmoothness(non-C1) test #0 studies function values (not |
//| gradient!) obtained during line searches and monitors behavior |
//| of the directional derivative estimate. |
//| This test is less powerful than test #1, but it does not depend |
//| on the gradient values and thus it is more robust against |
//| artifacts introduced by numerical differentiation. |
//| Two reports are returned: |
//| *a "strongest" one, corresponding to line search which had |
//| highest value of the nonsmoothness indicator |
//| *a "longest" one, corresponding to line search which had more |
//| function evaluations, and thus is more detailed |
//| In both cases following fields are returned: |
//| * positive - is TRUE when test flagged suspicious point; |
//| FALSE if test did not notice anything (in the |
//| latter cases fields below are empty). |
//| * fidx-is an index of the function (0 for target function, 1 or|
//| higher for nonlinear constraints) which is |
//| suspected of being "non-C1" |
//| * x0[], d[] - arrays of length N which store initial point and |
//| direction for line search (d[] can be normalized, |
//| but does not have to) |
//| * stp[], f[] - arrays of length CNT which store step lengths |
//| and function values at these points; f[i] is |
//| evaluated in x0 + stp[i]*d. |
//| * stpidxa, stpidxb - we suspect that function violates C1 |
//| continuity between steps #stpidxa and #stpidxb |
//| (usually we have stpidxb = stpidxa + 3, with most |
//| likely position of the violation between |
//| stpidxa + 1 and stpidxa + 2. |
//| ================================================================ |
//| = SHORTLY SPEAKING: build a 2D plot of(stp, f) and look at it - |
//| = you will see where C1 continuity is violated.|
//| ================================================================ |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| strrep - C1 test #0 "strong" report |
//| lngrep - C1 test #0 "long" report |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCOptGuardNonC1Test0Results(CMinNLCState &State,
COptGuardNonC1Test0Report &strrep,
COptGuardNonC1Test0Report &lngrep)
{
COptGuardApi::SmoothnessMonitorExportC1Test0Report(State.m_smonitor.m_nonc1test0strrep,State.m_lastscaleused,strrep);
COptGuardApi::SmoothnessMonitorExportC1Test0Report(State.m_smonitor.m_nonc1test0lngrep,State.m_lastscaleused,lngrep);
}
//+------------------------------------------------------------------+
//| Detailed results of the OptGuard integrity check for |
//| nonsmoothness test #1 |
//| Nonsmoothness(non-C1) test #1 studies individual components of |
//| the gradient computed during line search. |
//| When precise analytic gradient is provided this test is more |
//| powerful than test #0 which works with function values and |
//| ignores user-provided gradient. However, test #0 becomes more |
//| powerful when numerical differentiation is employed (in such |
//| cases test #1 detects higher levels of numerical noise and |
//| becomes too conservative). |
//| This test also tells specific components of the gradient which |
//| violate C1 continuity, which makes it more informative than #0, |
//| which just tells that continuity is violated. |
//| Two reports are returned: |
//| *a "strongest" one, corresponding to line search which had |
//| highest value of the nonsmoothness indicator |
//| *a "longest" one, corresponding to line search which had more |
//| function evaluations, and thus is more detailed |
//| In both cases following fields are returned: |
//| * positive - is TRUE when test flagged suspicious point; FALSE |
//| if test did not notice anything(in the latter cases fields |
//| below are empty). |
//| * fidx-is an index of the function(0 for target function, 1 or |
//| higher for nonlinear constraints) which is suspected of |
//| being "non-C1" |
//| * vidx - is an index of the variable in [0, N) with nonsmooth |
//| derivative |
//| * x0[], d[] - arrays of length N which store initial point and |
//| direction for line search(d[] can be normalized, but does not|
//| have to) |
//| * stp[], g[]-arrays of length CNT which store step lengths and |
//| gradient values at these points; g[i] is evaluated in |
//| x0 + stp[i]*d and contains vidx-th component of the gradient.|
//| * stpidxa, stpidxb - we suspect that function violates C1 |
//| continuity between steps #stpidxa and #stpidxb (usually we |
//| have stpidxb = stpidxa + 3, with most likely position of the |
//| violation between stpidxa + 1 and stpidxa + 2. |
//| ================================================================ |
//| = SHORTLY SPEAKING: build a 2D plot of (stp, f) and look at it - |
//| = you will see where C1 continuity is violated.|
//| ================================================================ |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| strrep - C1 test #1 "strong" report |
//| lngrep - C1 test #1 "long" report |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCOptGuardNonC1Test1Results(CMinNLCState &State,
COptGuardNonC1Test1Report &strrep,
COptGuardNonC1Test1Report &lngrep)
{
COptGuardApi::SmoothnessMonitorExportC1Test1Report(State.m_smonitor.m_nonc1test1strrep,State.m_lastscaleused,strrep);
COptGuardApi::SmoothnessMonitorExportC1Test1Report(State.m_smonitor.m_nonc1test1lngrep,State.m_lastscaleused,lngrep);
}
//+------------------------------------------------------------------+
//| MinNLC results: the solution found, completion codes and |
//| additional information. |
//| If you activated OptGuard integrity checking functionality and |
//| want to get OptGuard report, it can be retrieved with: |
//| * MinNLCOptGuardResults() - for a primary report about(a) |
//| suspected C0/C1 continuity |
//| violations and (b) errors in the |
//| analytic gradient. |
//| * MinNLCOptGuardNonC1Test0Results() - for C1 continuity |
//| violation test #0, detailed line |
//| search log |
//| * MinNLCOptGuardNonC1Test1Results() - for C1 continuity |
//| violation test #1, detailed line |
//| search log |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| X - array[0..N - 1], solution |
//| Rep - optimization report, contains information about |
//| completion code, constraint violation at the |
//| solution and so on. |
//| You should check rep.m_terminationtype in order to distinguish|
//| successful termination from unsuccessful one: |
//| === FAILURE CODES === |
//| * -8 internal integrity control detected infinite or NAN |
//| values in function/gradient. Abnormal termination |
//| signalled. |
//| * -3 box constraints are infeasible. |
//| Note: infeasibility of non-box constraints does NOT trigger |
//| emergency completion; you have to examine rep.m_bcerr/ |
//| rep.m_lcerr/rep.m_nlcerr to detect possibly inconsistent |
//| constraints. |
//| === SUCCESS CODES === |
//| * 2 scaled step is no more than EpsX. |
//| * 5 MaxIts steps were taken. |
//| * 8 user requested algorithm termination via |
//| MinNLCRequestTermination(), last accepted point is |
//| returned. |
//| More information about fields of this structure can be found in |
//| the comments on CMinNLCReport datatype. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCResults(CMinNLCState &State,CRowDouble &x,
CMinNLCReport &rep)
{
x.Resize(0);
MinNLCResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| NLC results |
//| Buffered implementation of MinNLCResults() which uses pre- |
//| allocated buffer to store X[]. If buffer size is too small, it |
//| resizes buffer. It is intended to be used in the inner cycles of |
//| performance critical algorithms where array reallocation penalty |
//| is too large to be ignored. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCResultsBuf(CMinNLCState &State,
CRowDouble &x,
CMinNLCReport &rep)
{
//--- copy
rep.m_iterationscount=State.m_repinneriterationscount;
rep.m_nfev=State.m_repnfev;
rep.m_terminationtype=State.m_repterminationtype;
rep.m_bcerr=State.m_repbcerr;
rep.m_bcidx=State.m_repbcidx;
rep.m_lcerr=State.m_replcerr;
rep.m_lcidx=State.m_replcidx;
rep.m_nlcerr=State.m_repnlcerr;
rep.m_nlcidx=State.m_repnlcidx;
rep.m_dbgphase0its=State.m_repdbgphase0its;
if(State.m_repterminationtype>0)
x=State.m_xc;
else
x=vector<double>::Full(State.m_n,AL_NaN);
}
//+------------------------------------------------------------------+
//| This subroutine submits request for termination of running |
//| optimizer. It should be called from user - supplied callback |
//| when user decides that it is time to "smoothly" terminate |
//| optimization process. As result, optimizer stops at point which |
//| was "current accepted" when termination request was submitted |
//| and returns error code 8(successful termination). |
//| INPUT PARAMETERS: |
//| State - optimizer structure |
//| NOTE: after request for termination optimizer may perform |
//| several additional calls to user-supplied callbacks. |
//| It does NOT guarantee to stop immediately - it just |
//| guarantees that these additional calls will be discarded |
//| later. |
//| NOTE: calling this function on optimizer which is NOT running |
//| will have no effect. |
//| NOTE: multiple calls to this function are possible. First call |
//| is counted, subsequent calls are silently ignored. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCRequestTermination(CMinNLCState &State)
{
State.m_userterminationneeded=true;
}
//+------------------------------------------------------------------+
//| This subroutine restarts algorithm from new point. |
//| All optimization parameters (including constraints) are left |
//| unchanged. |
//| This function allows to solve multiple optimization problems |
//| (which must have same number of dimensions) without object |
//| reallocation penalty. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinNLCCreate |
//| call. |
//| X - new starting point. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCRestartFrom(CMinNLCState &State,CRowDouble &x)
{
int n=State.m_n;
//--- First, check for errors in the inputs
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- Set XC
State.m_xstart=x;
//--- prepare RComm facilities
State.m_rstate.ia.Resize(5);
ArrayResize(State.m_rstate.ba,1);
State.m_rstate.ra.Resize(2);
State.m_rstate.stage=-1;
ClearRequestFields(State);
}
//+------------------------------------------------------------------+
//| Penalty function for equality constraints. |
//| INPUT PARAMETERS: |
//| Alpha - function argument. Penalty function becomes large |
//| when Alpha approaches - 1 or + 1. It is defined for|
//| Alpha <= -1 or Alpha >= +1 - in this case infinite |
//| value is returned. |
//| OUTPUT PARAMETERS: |
//| F - depending on Alpha: |
//| * for Alpha in (-1 + eps, +1 - eps), F = F(Alpha) |
//| * for Alpha outside of interval, F is some very |
//| large number |
//| DF - depending on Alpha: |
//| * for Alpha in (-1 + eps, +1 - eps), |
//| DF = dF(Alpha) / dAlpha, exact numerical |
//| derivative. |
//| * otherwise, it is zero |
//| D2F - second derivative |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCEqualityPenaltyFunction(double alpha,double &f,
double &df,double &d2f)
{
f=0;
df=0;
d2f=0;
f=0.5*alpha*alpha;
df=alpha;
d2f=1.0;
}
//+------------------------------------------------------------------+
//|"Penalty" function for inequality constraints, which is multiplied|
//| by penalty coefficient Rho. |
//| "Penalty" function plays only supplementary role - it helps to |
//| stabilize algorithm when solving non - convex problems. Because |
//| it is multiplied by fixed and large Rho - not Lagrange multiplier|
//| Nu which may become arbitrarily small! - it enforces convexity of|
//| the problem behind the boundary of the feasible area. |
//| This function is zero at the feasible area and in the close |
//| neighborhood, it becomes non-zero only at some distance (scaling |
//| is essential!) and grows quadratically. |
//| Penalty function must enter augmented Lagrangian as Rho * PENALTY|
//| (x - lowerbound) with corresponding changes being made for upper |
//| bound or other kinds of constraints. |
//| INPUT PARAMETERS: |
//| Alpha - function argument. Typically, if we have active |
//| constraint with precise Lagrange multiplier, we |
//| have Alpha around 1. Large positive Alpha's |
//| correspond to inner area of the feasible set. |
//| Alpha < 1 corresponds to outer area of the feasible|
//| set. |
//| StabilizingPoint - point where F becomes non-zero. Must be |
//| negative value, at least -1, large values(hundreds)|
//| are possible. |
//| OUTPUT PARAMETERS: |
//| F - F(Alpha) |
//| DF - DF = dF(Alpha) / dAlpha, exact derivative |
//| D2F - second derivative |
//| NOTE: it is important to have significantly non-zero |
//| StabilizingPoint, because when it is large, shift term |
//| does not interfere with Lagrange multipliers converging |
//| to their final values. Thus, convergence of such modified|
//| AUL algorithm is still guaranteed by same set of theorems|
//+------------------------------------------------------------------+
void CMinNLC::MinNLCInequalityPenaltyFunction(double alpha,
double stabilizingpoint,
double &f,
double &df,
double &d2f)
{
if(alpha>=stabilizingpoint)
{
f=0.0;
df=0.0;
d2f=0.0;
}
else
{
alpha=alpha-stabilizingpoint;
f=0.5*alpha*alpha;
df=alpha;
d2f=1.0;
}
}
//+------------------------------------------------------------------+
//| "Shift" function for inequality constraints, which is multiplied |
//| by corresponding Lagrange multiplier. |
//| "Shift" function is a main factor which enforces inequality |
//| constraints. Inequality penalty function plays only supplementary|
//| role - it prevents accidental step deep into infeasible area when|
//| working with non-convex problems (read comments on corresponding |
//| function for more information). |
//| Shift function must enter augmented Lagrangian as |
//| Nu / Rho * SHIFT((x - lowerbound)*Rho + 1) |
//| with corresponding changes being made for upper bound or other |
//| kinds of constraints. |
//| INPUT PARAMETERS: |
//| Alpha - function argument. Typically, if we have active |
//| constraint with precise Lagrange multiplier, we |
//| have Alpha around 1. Large positive Alpha's |
//| correspond to inner area of the feasible set. |
//| Alpha < 1 corresponds to outer area of the feasible|
//| set. |
//| OUTPUT PARAMETERS: |
//| F - F(Alpha) |
//| DF - DF = dF(Alpha) / dAlpha, exact derivative |
//| D2F - second derivative |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCInequalityShiftFunction(double alpha,double &f,
double &df,double &d2f)
{
f=0;
df=0;
d2f=0;
if(alpha>=0.5)
{
f=-MathLog(alpha);
df=-(1/alpha);
d2f=1/(alpha*alpha);
}
else
{
f=2*alpha*alpha-4*alpha+(MathLog(2)+1.5);
df=4*alpha-4;
d2f=4;
}
}
//+------------------------------------------------------------------+
//| Clears request fileds (to be sure that we don't forget to clear |
//| something) |
//+------------------------------------------------------------------+
void CMinNLC::ClearRequestFields(CMinNLCState &State)
{
State.m_needfi=false;
State.m_needfij=false;
State.m_xupdated=false;
}
//+------------------------------------------------------------------+
//| Internal initialization subroutine. |
//| Sets default NLC solver with default criteria. |
//+------------------------------------------------------------------+
void CMinNLC::MinNLCInitInternal(int n,CRowDouble &x,double diffstep,
CMinNLCState &State)
{
//--- create variables
CMatrixDouble c;
CRowInt ct;
//--- Default params
State.m_stabilizingpoint=-2.0;
State.m_initialinequalitymultiplier=1.0;
//--- Smoothness monitor, default init
State.m_teststep=0;
State.m_smoothnessguardlevel=0;
COptServ::SmoothnessMonitorInit(State.m_smonitor,State.m_s,0,0,false);
//--- Initialize other params
State.m_n=n;
State.m_diffstep=diffstep;
State.m_userterminationneeded=false;
State.m_xstart=x;
State.m_xc=x;
State.m_bndl=vector<double>::Full(n,AL_NEGINF);
ArrayResize(State.m_HasBndL,n);
State.m_bndu=vector<double>::Full(n,AL_POSINF);
ArrayResize(State.m_HasBndU,n);
State.m_s=vector<double>::Ones(n);
State.m_lastscaleused=vector<double>::Ones(n);
State.m_xstart.Resize(n);
State.m_xc.Resize(n);
State.m_x.Resize(n);
ArrayInitialize(State.m_HasBndL,false);
ArrayInitialize(State.m_HasBndU,false);
MinNLCSetLC(State,c,ct,0);
MinNLCSetNLC(State,0,0);
MinNLCSetCond(State,0.0,0);
MinNLCSetXRep(State,false);
MinNLCSetAlgoSQP(State);
MinNLCSetPrecExactRobust(State,0);
MinNLCSetSTPMax(State,0.0);
CMinLBFGS::MinLBFGSCreate(n,MathMin(m_lbfgsfactor,n),x,State.m_auloptimizer);
MinNLCRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| This function clears preconditioner for L-BFGS optimizer (sets it|
//| do default State); |
//| Parameters: |
//| AULOptimizer - optimizer to tune |
//+------------------------------------------------------------------+
void CMinNLC::ClearPreconditioner(CMinLBFGSState &auloptimizer)
{
CMinLBFGS::MinLBFGSSetPrecDefault(auloptimizer);
}
//+------------------------------------------------------------------+
//| This function updates preconditioner for L-BFGS optimizer. |
//| Parameters: |
//| PrecType - preconditioner type: |
//| * 0 for unpreconditioned iterations |
//| * 1 for inexact LBFGS |
//| * 2 for exact low rank preconditioner update |
//| after each UpdateFreq its |
//| * 3 for exact robust preconditioner update after|
//| each UpdateFreq its |
//| UpdateFreq - update frequency |
//| PrecCounter - iterations counter, must be zero on the first |
//| call, automatically increased by this function. |
//| This counter is used to implement |
//| "update-once-in-X-iterations" scheme. |
//| AULOptimizer - optimizer to tune |
//| X - current point |
//| Rho - penalty term |
//| GammaK - current estimate of Hessian norm (used for |
//| initialization of preconditioner). Can be zero, |
//| in which case Hessian is assumed to be unit. |
//+------------------------------------------------------------------+
void CMinNLC::UpdatePreconditioner(int prectype,int updatefreq,
int &preccounter,
CMinLBFGSState &auloptimizer,
CRowDouble &x,double rho,
double gammak,CRowDouble &bndl,
bool &HasBndL[],CRowDouble &bndu,
bool &HasBndU[],CRowDouble &nubc,
CMatrixDouble &cleic,
CRowDouble &nulc,CRowDouble &fi,
CMatrixDouble &jac,
CRowDouble &nunlc,CRowDouble &bufd,
CRowDouble &bufc,
CMatrixDouble &bufw,
CMatrixDouble &bufz,
CRowDouble &tmp0,
int n,int nec,int nic,
int ng,int nh)
{
//--- create variables
int i=0;
int j=0;
int k=0;
double v=0;
double p=0;
double dp=0;
double d2p=0;
bool bflag=false;
int i_=0;
//--- check
if(!CAp::Assert(rho>0.0,__FUNCTION__+": integrity check failed"))
return;
CApServ::RVectorSetLengthAtLeast(bufd,n);
CApServ::RVectorSetLengthAtLeast(bufc,nec+nic+ng+nh);
CApServ::RVectorSetLengthAtLeast(tmp0,n);
//--- Preconditioner before update from barrier/penalty functions
if(gammak==0.0)
gammak=1;
bufd.Fill(gammak);
//--- Update diagonal Hessian using nonlinearity from boundary constraints:
//--- * penalty term from equality constraints
//--- * shift term from inequality constraints
//--- NOTE: penalty term for inequality constraints is ignored because it
//--- is large only in exceptional cases.
for(i=0; i<n; i++)
{
if((HasBndL[i] && HasBndU[i]) && bndl[i]==bndu[i])
{
MinNLCEqualityPenaltyFunction((x[i]-bndl[i])*rho,p,dp,d2p);
bufd.Add(i,d2p*rho);
continue;
}
if(HasBndL[i])
{
MinNLCInequalityShiftFunction((x[i]-bndl[i])*rho+1,p,dp,d2p);
bufd.Add(i,nubc[2*i+0]*d2p*rho);
}
if(HasBndU[i])
{
MinNLCInequalityShiftFunction((bndu[i]-x[i])*rho+1,p,dp,d2p);
bufd.Add(i,nubc[2*i+1]*d2p*rho);
}
}
//--- Process linear constraints
bufw=cleic;
bufw.Resize(nec+nic+ng+nh,n);
for(i=0; i<nec+nic; i++)
{
v=x.DotR(bufw,i)-cleic.Get(i,n);
if(i<nec)
{
//--- Equality constraint
MinNLCEqualityPenaltyFunction(v*rho,p,dp,d2p);
bufc.Set(i,d2p*rho);
}
else
{
//--- Inequality constraint
MinNLCInequalityShiftFunction(-(v*rho)+1,p,dp,d2p);
bufc.Set(i,nulc[i]*d2p*rho);
}
}
//--- Process nonlinear constraints
for(i=0; i<ng+nh; i++)
{
bufw.Row(nec+nic+i,jac,1+i);
v=fi[1+i];
if(i<ng)
{
//--- Equality constraint
MinNLCEqualityPenaltyFunction(v*rho,p,dp,d2p);
bufc.Set(nec+nic+i,d2p*rho);
}
else
{
//--- Inequality constraint
MinNLCInequalityShiftFunction(-(v*rho)+1,p,dp,d2p);
bufc.Set(nec+nic+i,nunlc[i]*d2p*rho);
}
}
//--- Add regularizer (large Rho often result in nearly-degenerate matrices;
//--- sometimes Cholesky decomposition fails without regularization).
//--- We use RegPrec*diag(W'*W) as preconditioner.
k=nec+nic+ng+nh;
tmp0.Fill(0.0);
for(i=0; i<k; i++)
{
v=bufc[i];
for(j=0; j<n; j++)
tmp0.Add(j,v*CMath::Sqr(bufw.Get(i,j)));
}
for(j=0; j<=n-1; j++)
bufd.Add(j,tmp0[j]*m_regprec);
//--- Apply preconditioner
switch(prectype)
{
case 1:
CMinLBFGS::MinLBFGSSetPrecRankKLBFGSFast(auloptimizer,bufd,bufc,bufw,nec+nic+ng+nh);
break;
case 2:
if(preccounter%updatefreq==0)
CMinLBFGS::MinLBFGSSetPrecLowRankExact(auloptimizer,bufd,bufc,bufw,nec+nic+ng+nh);
break;
case 3:
if(preccounter%updatefreq==0)
{
//--- Generate full NxN dense Hessian
bufz=matrix<double>::Zeros(n,n);
bufz.Diag(bufd);
if(nec+nic+ng+nh>0)
{
for(i=0; i<nec+nic+ng+nh; i++)
{
if(!CAp::Assert(bufc[i]>=0.0,__FUNCTION__+": updatepreconditioner() integrity failure"))
return;
v=MathSqrt(bufc[i]);
bufw.Row(i,bufw[i]*v);
}
CAblas::RMatrixSyrk(n,nec+nic+ng+nh,1.0,bufw,0,0,2,1.0,bufz,0,0,true);
}
//--- Evaluate Cholesky decomposition, set preconditioner
bflag=CTrFac::SPDMatrixCholeskyRec(bufz,0,n,true,bufd);
if(!CAp::Assert(bflag,__FUNCTION__+": updatepreconditioner() failure,Cholesky failed"))
return;
CMinLBFGS::MinLBFGSSetPrecCholesky(auloptimizer,bufz,true);
}
break;
}
preccounter++;
}
//+------------------------------------------------------------------+
//| This subroutine adds penalty from boundary constraints to target |
//| function and its gradient. Penalty function is one which is used |
//| for main AUL cycle - with Lagrange multipliers and infinite at |
//| the barrier and beyond. |
//| Parameters: |
//| X[] - current point |
//| BndL[], BndU[] - boundary constraints |
//| HasBndL[], HasBndU[] - I-th element is True if corresponding |
//| constraint is present |
//| NuBC[] - Lagrange multipliers corresponding to |
//| constraints |
//| Rho - penalty term |
//| StabilizingPoint - branch point for inequality stabilizing term|
//| F - function value to modify |
//| G - gradient to modify |
//+------------------------------------------------------------------+
void CMinNLC::PenaltyBC(CRowDouble &x,CRowDouble &bndl,bool &HasBndL[],
CRowDouble &bndu,bool &HasBndU[],CRowDouble &nubc,
int n,double rho,double stabilizingpoint,
double &f,CRowDouble &g)
{
//--- create variables
double p=0;
double dp=0;
double d2p=0;
for(int i=0; i<n; i++)
{
if((HasBndL[i] && HasBndU[i]) && bndl[i]==bndu[i])
{
//--- I-th boundary constraint is of equality-type
MinNLCEqualityPenaltyFunction((x[i]-bndl[i])*rho,p,dp,d2p);
f=f+p/rho-nubc[2*i+0]*(x[i]-bndl[i]);
g.Add(i,dp-nubc[2*i+0]);
continue;
}
if(HasBndL[i])
{
//--- Handle lower bound
MinNLCInequalityPenaltyFunction(x[i]-bndl[i],stabilizingpoint,p,dp,d2p);
f=f+rho*p;
g.Add(i,rho*dp);
MinNLCInequalityShiftFunction((x[i]-bndl[i])*rho+1,p,dp,d2p);
f=f+p/rho*nubc[2*i+0];
g.Add(i,dp*nubc[2*i+0]);
}
if(HasBndU[i])
{
//--- Handle upper bound
MinNLCInequalityPenaltyFunction(bndu[i]-x[i],stabilizingpoint,p,dp,d2p);
f=f+rho*p;
g.Add(i,-rho*dp);
MinNLCInequalityShiftFunction((bndu[i]-x[i])*rho+1,p,dp,d2p);
f=f+p/rho*nubc[2*i+1];
g.Add(i,-dp*nubc[2*i+1]);
}
}
}
//+------------------------------------------------------------------+
//| This subroutine adds penalty from linear constraints to target |
//| function and its gradient. Penalty function is one which is used |
//| for main AUL cycle - with Lagrange multipliers and infinite at |
//| the barrier and beyond. |
//| Parameters: |
//| X[] - current point |
//| CLEIC[] - constraints matrix, first NEC rows are equality |
//| ones, next NIC rows are inequality ones. |
//| array[NEC + NIC, N + 1] |
//| NuLC[] - Lagrange multipliers corresponding to |
//| constraints, array[NEC + NIC] |
//| N - dimensionalty |
//| NEC - number of equality constraints |
//| NIC - number of inequality constraints. |
//| Rho - penalty term |
//| StabilizingPoint - branch point for inequality stabilizing term|
//| F - function value to modify |
//| G - gradient to modify |
//+------------------------------------------------------------------+
void CMinNLC::PenaltyLC(CRowDouble &x,CMatrixDouble &cleic,
CRowDouble &nulc,int n,int nec,int nic,
double rho,double stabilizingpoint,
double &f,CRowDouble &g)
{
//--- create variables
double v=0;
double p=0;
double dp=0;
double d2p=0;
double fupd=0;
double gupd=0;
int i_=0;
for(int i=0; i<nec+nic; i++)
{
v=-cleic.Get(i,n);
for(i_=0; i_<n; i_++)
v+=cleic.Get(i,i_)*x[i_];
fupd=0;
gupd=0;
if(i<nec)
{
//--- Equality constraint
MinNLCEqualityPenaltyFunction(v*rho,p,dp,d2p);
fupd=fupd+p/rho;
gupd=gupd+dp;
fupd=fupd-nulc[i]*v;
gupd=gupd-nulc[i];
}
else
{
//--- Inequality constraint
MinNLCInequalityPenaltyFunction(-v,stabilizingpoint,p,dp,d2p);
fupd=fupd+p*rho;
gupd=gupd-dp*rho;
MinNLCInequalityShiftFunction(-(v*rho)+1,p,dp,d2p);
fupd=fupd+p/rho*nulc[i];
gupd=gupd-dp*nulc[i];
}
f=f+fupd;
for(i_=0; i_<n; i_++)
g.Add(i_,gupd*cleic.Get(i,i_));
}
}
//+------------------------------------------------------------------+
//| This subroutine adds penalty from nonlinear constraints to target|
//| function and its gradient. Penalty function is one which is used |
//| for main AUL cycle - with Lagrange multipliers and infinite at |
//| the barrier and beyond. |
//| Parameters: |
//| Fi[] - function vector: |
//| * 1 component for function being minimized |
//| * NG components for equality constraints |
//| G_i(x) = 0 |
//| * NH components for inequality constraints |
//| H_i(x) <= 0 |
//| J[] - Jacobian matrix, array[1 + NG + NH, N] |
//| NuNLC[] - Lagrange multipliers corresponding to |
//| constraints, array[NG + NH] |
//| N - number of dimensions |
//| NG - number of equality constraints |
//| NH - number of inequality constraints |
//| Rho - penalty term |
//| StabilizingPoint - branch point for inequality stabilizing term |
//| F - function value to modify |
//| G - gradient to modify |
//+------------------------------------------------------------------+
void CMinNLC::PenaltyNLC(CRowDouble &fi,CMatrixDouble &j,
CRowDouble &nunlc,int n,int ng,int nh,
double rho,double stabilizingpoint,
double &f,CRowDouble &g)
{
//--- create variables
double v=0;
double p=0;
double dp=0;
double d2p=0;
double fupd=0;
double gupd=0;
int i_=0;
//--- IMPORTANT: loop starts from 1, not zero!
for(int i=1; i<=ng+nh; i++)
{
v=fi[i];
fupd=0;
gupd=0;
if(i<=ng)
{
//--- Equality constraint
MinNLCEqualityPenaltyFunction(v*rho,p,dp,d2p);
fupd=fupd+p/rho;
gupd=gupd+dp;
fupd=fupd-nunlc[i-1]*v;
gupd=gupd-nunlc[i-1];
}
else
{
//--- Inequality constraint
MinNLCInequalityPenaltyFunction(-v,stabilizingpoint,p,dp,d2p);
fupd=fupd+p*rho;
gupd=gupd-dp*rho;
MinNLCInequalityShiftFunction(-(v*rho)+1,p,dp,d2p);
fupd=fupd+p/rho*nunlc[i-1];
gupd=gupd-dp*nunlc[i-1];
}
f+=fupd;
for(i_=0; i_<n; i_++)
g.Add(i_,gupd*j.Get(i,i_));
}
}
//+------------------------------------------------------------------+
//| This function performs actual processing for AUL algorithm. It |
//| expects that caller redirects its reverse communication requests |
//| NeedFiJ / XUpdated to external user who will provide analytic |
//| derivative (or handle reports about progress). |
//| In case external user does not have analytic derivative, it is |
//| responsibility of caller to intercept NeedFiJ request and replace|
//| it with appropriate numerical differentiation scheme. |
//+------------------------------------------------------------------+
bool CMinNLC::AULIteration(CMinNLCState &State,CSmoothnessMonitor &smonitor)
{
//--- create variables
bool result=false;
int n=0;
int nec=0;
int nic=0;
int ng=0;
int nh=0;
int i=0;
int j=0;
int outerit=0;
int preccounter=0;
double v=0;
double vv=0;
double p=0;
double dp=0;
double d2p=0;
double v0=0;
double v1=0;
double v2=0;
int i_=0;
int label=-1;
//--- Reverse communication preparations
//--- I know it looks ugly, but it works the same way
//--- anywhere from C++ to Python.
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(State.m_rstateaul.stage>=0)
{
n=State.m_rstateaul.ia[0];
nec=State.m_rstateaul.ia[1];
nic=State.m_rstateaul.ia[2];
ng=State.m_rstateaul.ia[3];
nh=State.m_rstateaul.ia[4];
i=State.m_rstateaul.ia[5];
j=State.m_rstateaul.ia[6];
outerit=State.m_rstateaul.ia[7];
preccounter=State.m_rstateaul.ia[8];
v=State.m_rstateaul.ra[0];
vv=State.m_rstateaul.ra[1];
p=State.m_rstateaul.ra[2];
dp=State.m_rstateaul.ra[3];
d2p=State.m_rstateaul.ra[4];
v0=State.m_rstateaul.ra[5];
v1=State.m_rstateaul.ra[6];
v2=State.m_rstateaul.ra[7];
}
else
{
n=809;
nec=205;
nic=-838;
ng=939;
nh=-526;
i=763;
j=-541;
outerit=-698;
preccounter=-900;
v=-318;
vv=-940;
p=1016;
dp=-229;
d2p=-536;
v0=487;
v1=-115;
v2=886;
}
switch(State.m_rstateaul.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
default:
//--- Routine body
if(!CAp::Assert(State.m_solvertype==0,__FUNCTION__+": internal error"))
return(false);
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
ng=State.m_ng;
nh=State.m_nh;
//--- Prepare scaled problem
CApServ::RVectorSetLengthAtLeast(State.m_scaledbndl,n);
CApServ::RVectorSetLengthAtLeast(State.m_scaledbndu,n);
CApServ::RMatrixSetLengthAtLeast(State.m_scaledcleic,nec+nic,n+1);
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i])
State.m_scaledbndl.Set(i,State.m_bndl[i]/State.m_s[i]);
if(State.m_HasBndU[i])
State.m_scaledbndu.Set(i,State.m_bndu[i]/State.m_s[i]);
State.m_xc.Set(i,State.m_xstart[i]/State.m_s[i]);
}
for(i=0; i<nec+nic; i++)
{
//--- Scale and normalize linear constraints
vv=0.0;
for(j=0; j<n; j++)
{
v=State.m_cleic.Get(i,j)*State.m_s[j];
State.m_scaledcleic.Set(i,j,v);
vv+=v*v;
}
vv=MathSqrt(vv);
State.m_scaledcleic.Set(i,n,State.m_cleic.Get(i,n));
if(vv>0.0)
{
for(j=0; j<=n; j++)
State.m_scaledcleic.Set(i,j,State.m_scaledcleic.Get(i,j)/vv);
}
}
//--- Prepare stopping criteria
CMinLBFGS::MinLBFGSSetCond(State.m_auloptimizer,0,0,State.m_epsx,State.m_maxits);
CMinLBFGS::MinLBFGSSetStpMax(State.m_auloptimizer,State.m_stpmax);
//--- Main AUL cycle:
//--- * prepare Lagrange multipliers NuNB/NuLC
//--- * set GammaK (current estimate of Hessian norm) to InitGamma and XKPresent to False
State.m_nubc=vector<double>::Zeros(2*n);
State.m_nulc=vector<double>::Zeros(nec+nic);
State.m_nunlc=vector<double>::Zeros(ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_xk,n);
CApServ::RVectorSetLengthAtLeast(State.m_gk,n);
CApServ::RVectorSetLengthAtLeast(State.m_xk1,n);
CApServ::RVectorSetLengthAtLeast(State.m_gk1,n);
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i] && State.m_HasBndU[i] && State.m_bndl[i]==State.m_bndu[i])
continue;
if(State.m_HasBndL[i])
State.m_nubc.Set(2*i,State.m_initialinequalitymultiplier);
if(State.m_HasBndU[i])
State.m_nubc.Set(2*i+1,State.m_initialinequalitymultiplier);
}
for(i=0; i<nic; i++)
State.m_nulc.Set(nec+i,State.m_initialinequalitymultiplier);
for(i=0; i<nh; i++)
State.m_nunlc.Set(ng+i,State.m_initialinequalitymultiplier);
State.m_gammak=m_initgamma;
State.m_xkpresent=false;
if(!CAp::Assert(State.m_aulitscnt>0,__FUNCTION__+": integrity check failed"))
return(false);
ClearPreconditioner(State.m_auloptimizer);
outerit=0;
label=3;
break;
}
//--- main loop
while(label>=0)
{
switch(label)
{
case 3:
if(outerit>State.m_aulitscnt-1)
{
label=5;
break;
}
//--- Optimize with current Lagrange multipliers
//--- NOTE: this code expects and checks that line search ends in the
//--- point which is used as beginning for the next search. Such
//--- guarantee is given by MCSRCH function. L-BFGS optimizer
//--- does not formally guarantee it, but it follows same rule.
//--- Below we a) rely on such property of the optimizer, and b)
//--- assert that it is true, in order to fail loudly if it is
//--- not true.
//--- NOTE: security check for NAN/INF in F/G is responsibility of
//--- LBFGS optimizer. AUL optimizer checks for NAN/INF only
//--- when we update Lagrange multipliers.
preccounter=0;
CMinLBFGS::MinLBFGSSetXRep(State.m_auloptimizer,true);
CMinLBFGS::MinLBFGSRestartFrom(State.m_auloptimizer,State.m_xc);
case 6:
if(!CMinLBFGS::MinLBFGSIteration(State.m_auloptimizer))
{
label=7;
break;
}
if(!State.m_auloptimizer.m_needfg)
{
label=8;
break;
}
//--- Un-scale X, evaluate F/G/H, re-scale Jacobian
for(i=0; i<n; i++)
State.m_x.Set(i,State.m_auloptimizer.m_x[i]*State.m_s[i]);
State.m_needfij=true;
State.m_rstateaul.stage=0;
label=-1;
break;
case 0:
State.m_needfij=false;
for(i=0; i<=ng+nh; i++)
for(j=0; j<n; j++)
State.m_j.Mul(i,j,State.m_s[j]);
//--- Store data for estimation of Hessian norm:
//--- * current point (re-scaled)
State.m_xk1=State.m_auloptimizer.m_x;
State.m_gk1=State.m_j[0]+0;
//--- Function being optimized
State.m_auloptimizer.m_f=State.m_fi[0];
State.m_auloptimizer.m_g=State.m_j[0]+0;
//--- Send information to OptGuard monitor
COptServ::SmoothnessMonitorEnqueuePoint(smonitor,State.m_auloptimizer.m_d,State.m_auloptimizer.m_stp,State.m_auloptimizer.m_x,State.m_fi,State.m_j);
//--- Penalty for violation of boundary/linear/nonlinear constraints
PenaltyBC(State.m_auloptimizer.m_x,State.m_scaledbndl,State.m_HasBndL,State.m_scaledbndu,State.m_HasBndU,State.m_nubc,n,State.m_rho,State.m_stabilizingpoint,State.m_auloptimizer.m_f,State.m_auloptimizer.m_g);
PenaltyLC(State.m_auloptimizer.m_x,State.m_scaledcleic,State.m_nulc,n,nec,nic,State.m_rho,State.m_stabilizingpoint,State.m_auloptimizer.m_f,State.m_auloptimizer.m_g);
PenaltyNLC(State.m_fi,State.m_j,State.m_nunlc,n,ng,nh,State.m_rho,State.m_stabilizingpoint,State.m_auloptimizer.m_f,State.m_auloptimizer.m_g);
//--- Forward termination request if needed
if(State.m_userterminationneeded)
CMinLBFGS::MinLBFGSRequestTermination(State.m_auloptimizer);
//--- To optimizer
label=6;
break;
case 8:
if(!State.m_auloptimizer.m_xupdated)
{
label=10;
break;
}
//--- Report current point (if needed)
if(!State.m_xrep)
{
label=12;
break;
}
State.m_x=State.m_auloptimizer.m_x*State.m_s+0;
State.m_f=State.m_auloptimizer.m_f;
State.m_xupdated=true;
State.m_rstateaul.stage=1;
label=-1;
break;
case 1:
State.m_xupdated=false;
case 12:
//--- Send information to OptGuard monitor
COptServ::SmoothnessMonitorFinalizeLineSearch(smonitor);
COptServ::SmoothnessMonitorStartLineSearch(smonitor,State.m_auloptimizer.m_x,State.m_fi,State.m_j);
//--- Forward termination request if needed
if(State.m_userterminationneeded)
CMinLBFGS::MinLBFGSRequestTermination(State.m_auloptimizer);
//--- Update constraints violation
COptServ::CheckBcViolation(State.m_HasBndL,State.m_scaledbndl,State.m_HasBndU,State.m_scaledbndu,State.m_auloptimizer.m_x,n,State.m_s,false,State.m_repbcerr,State.m_repbcidx);
COptServ::CheckLcViolation(State.m_scaledcleic,State.m_lcsrcidx,nec,nic,State.m_auloptimizer.m_x,n,State.m_replcerr,State.m_replcidx);
COptServ::CheckNLcViolation(State.m_fi,ng,nh,State.m_repnlcerr,State.m_repnlcidx);
//--- Update GammaK
if(State.m_xkpresent)
{
//--- XK/GK store beginning of current line search, and XK1/GK1
//--- store data for the end of the line search:
//--- * first, we Assert() that XK1 (last point where function
//--- was evaluated) is same as AULOptimizer.X (what is
//--- reported by RComm interface
//--- * calculate step length V2.
//--- If V2>HessEstTol, then:
//--- * calculate V0 - directional derivative at XK,
//--- and V1 - directional derivative at XK1
//--- * set GammaK to Max(GammaK, |V1-V0|/V2)
for(i=0; i<n; i++)
{
if(!CAp::Assert(MathAbs(State.m_auloptimizer.m_x[i]-State.m_xk1[i])<=(100*CMath::m_machineepsilon) || !(MathIsValidNumber(State.m_auloptimizer.m_x[i]) && MathIsValidNumber(State.m_xk1[i])),__FUNCTION__+": integrity check failed,unexpected behavior of LBFGS optimizer"))
return(false);
}
v2=0.0;
for(i=0; i<n; i++)
v2+=CMath::Sqr(State.m_xk[i]-State.m_xk1[i]);
v2=MathSqrt(v2);
if(v2>m_hessesttol)
{
v0=0.0;
v1=0.0;
for(i=0; i<n; i++)
{
v=(State.m_xk[i]-State.m_xk1[i])/v2;
v0+=State.m_gk[i]*v;
v1+=State.m_gk1[i]*v;
}
State.m_gammak=MathMax(State.m_gammak,MathAbs(v1-v0)/v2);
}
}
else
{
//--- Beginning of the first line search, XK is not yet initialized.
State.m_xk=State.m_xk1;
State.m_gk=State.m_gk1;
State.m_xkpresent=true;
}
//--- Update preconsitioner using current GammaK
UpdatePreconditioner(State.m_prectype,State.m_updatefreq,preccounter,State.m_auloptimizer,State.m_auloptimizer.m_x,State.m_rho,State.m_gammak,State.m_scaledbndl,State.m_HasBndL,State.m_scaledbndu,State.m_HasBndU,State.m_nubc,State.m_scaledcleic,State.m_nulc,State.m_fi,State.m_j,State.m_nunlc,State.m_bufd,State.m_bufc,State.m_bufw,State.m_bufz,State.m_tmp0,n,nec,nic,ng,nh);
label=6;
break;
case 10:
CAp::Assert(false,__FUNCTION__+": integrity check failed");
return(false);
case 7:
CMinLBFGS::MinLBFGSResultsBuf(State.m_auloptimizer,State.m_xc,State.m_aulreport);
State.m_repinneriterationscount+=State.m_aulreport.m_iterationscount;
State.m_repnfev+=State.m_aulreport.m_nfev;
State.m_repterminationtype=State.m_aulreport.m_terminationtype;
State.m_repouteriterationscount++;
if(State.m_repterminationtype<=0 || State.m_repterminationtype==8)
{
label=5;
break;
}
//--- 1. Evaluate F/J
//--- 2. Check for NAN/INF in F/J: we just calculate sum of their
//--- components, it should be enough to reduce vector/matrix to
//--- just one value which either "normal" (all summands were "normal")
//--- or NAN/INF (at least one summand was NAN/INF).
//--- 3. Update Lagrange multipliers
State.m_x=State.m_xc*State.m_s+0;
State.m_needfij=true;
State.m_rstateaul.stage=2;
label=-1;
break;
case 2:
State.m_needfij=false;
v=0.0;
for(i=0; i<=ng+nh; i++)
{
v=0.1*v+State.m_fi[i];
for(j=0; j<n; j++)
v=0.1*v+State.m_j.Get(i,j);
}
if(!MathIsValidNumber(v))
{
//--- Abnormal termination - infinities in function/gradient
State.m_repterminationtype=-8;
result=false;
return(result);
}
for(i=0; i<=ng+nh; i++)
State.m_j.Row(i,State.m_s.ToVector()*State.m_j[i]);
for(i=0; i<n; i++)
{
//--- Process coefficients corresponding to equality-type
//--- constraints.
if(State.m_HasBndL[i] && State.m_HasBndU[i] && State.m_bndl[i]==State.m_bndu[i])
{
MinNLCEqualityPenaltyFunction((State.m_xc[i]-State.m_scaledbndl[i])*State.m_rho,p,dp,d2p);
State.m_nubc.Set(2*i,CApServ::BoundVal(State.m_nubc[2*i]-dp,-m_maxlagmult,m_maxlagmult));
continue;
}
//--- Process coefficients corresponding to inequality-type
//--- constraints. These coefficients have limited growth/decay
//--- per iteration which helps to stabilize algorithm.
if(!CAp::Assert(m_aulmaxgrowth>1.0,__FUNCTION__+": integrity error"))
return(false);
if(State.m_HasBndL[i])
{
MinNLCInequalityShiftFunction((State.m_xc[i]-State.m_scaledbndl[i])*State.m_rho+1,p,dp,d2p);
v=MathAbs(dp);
v=MathMin(v,m_aulmaxgrowth);
v=MathMax(v,1/m_aulmaxgrowth);
State.m_nubc.Set(2*i,CApServ::BoundVal(State.m_nubc[2*i]*v,-m_maxlagmult,m_maxlagmult));
}
if(State.m_HasBndU[i])
{
MinNLCInequalityShiftFunction((State.m_scaledbndu[i]-State.m_xc[i])*State.m_rho+1,p,dp,d2p);
v=MathAbs(dp);
v=MathMin(v,m_aulmaxgrowth);
v=MathMax(v,1/m_aulmaxgrowth);
State.m_nubc.Set(2*i+1,CApServ::BoundVal(State.m_nubc[2*i+1]*v,-m_maxlagmult,m_maxlagmult));
}
}
for(i=0; i<=nec+nic-1; i++)
{
v=-State.m_scaledcleic.Get(i,n);
for(i_=0; i_<n; i_++)
v+=State.m_scaledcleic.Get(i,i_)*State.m_xc[i_];
if(i<nec)
{
MinNLCEqualityPenaltyFunction(v*State.m_rho,p,dp,d2p);
State.m_nulc.Set(i,CApServ::BoundVal(State.m_nulc[i]-dp,-m_maxlagmult,m_maxlagmult));
}
else
{
MinNLCInequalityShiftFunction(-(v*State.m_rho)+1,p,dp,d2p);
v=MathAbs(dp);
v=MathMin(v,m_aulmaxgrowth);
v=MathMax(v,1/m_aulmaxgrowth);
State.m_nulc.Set(i,CApServ::BoundVal(State.m_nulc[i]*v,-m_maxlagmult,m_maxlagmult));
}
}
for(i=1; i<=ng+nh; i++)
{
//--- NOTE: loop index must start from 1, not zero!
v=State.m_fi[i];
if(i<=ng)
{
MinNLCEqualityPenaltyFunction(v*State.m_rho,p,dp,d2p);
State.m_nunlc.Set(i-1,CApServ::BoundVal(State.m_nunlc[i-1]-dp,-m_maxlagmult,m_maxlagmult));
}
else
{
MinNLCInequalityShiftFunction(-(v*State.m_rho)+1,p,dp,d2p);
v=MathAbs(dp);
v=MathMin(v,m_aulmaxgrowth);
v=MathMax(v,1/m_aulmaxgrowth);
State.m_nunlc.Set(i-1,CApServ::BoundVal(State.m_nunlc[i-1]*v,-m_maxlagmult,m_maxlagmult));
}
}
outerit++;
label=3;
break;
case 5:
for(i=0; i<n; i++)
State.m_xc.Set(i,State.m_xc[i]*State.m_s[i]);
result=false;
return(result);
}
}
//--- Saving State
result=true;
State.m_rstateaul.ia.Set(0,n);
State.m_rstateaul.ia.Set(1,nec);
State.m_rstateaul.ia.Set(2,nic);
State.m_rstateaul.ia.Set(3,ng);
State.m_rstateaul.ia.Set(4,nh);
State.m_rstateaul.ia.Set(5,i);
State.m_rstateaul.ia.Set(6,j);
State.m_rstateaul.ia.Set(7,outerit);
State.m_rstateaul.ia.Set(8,preccounter);
State.m_rstateaul.ra.Set(0,v);
State.m_rstateaul.ra.Set(1,vv);
State.m_rstateaul.ra.Set(2,p);
State.m_rstateaul.ra.Set(3,dp);
State.m_rstateaul.ra.Set(4,d2p);
State.m_rstateaul.ra.Set(5,v0);
State.m_rstateaul.ra.Set(6,v1);
State.m_rstateaul.ra.Set(7,v2);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| Unscales X (converts from scaled variables to original ones), |
//| paying special attention to box constraints (output is always |
//| feasible; active constraints are mapped to active ones). |
//+------------------------------------------------------------------+
void CMinNLC::UnScale(CMinNLCState &State,CRowDouble &xs,
CRowDouble &scaledbndl,CRowDouble &scaledbndu,
CRowDouble &xu)
{
int n=State.m_n;
for(int i=0; i<n; i++)
{
if(State.m_HasBndL[i] && xs[i]<=scaledbndl[i])
{
xu.Set(i,State.m_bndl[i]);
continue;
}
if(State.m_HasBndU[i] && xs[i]>=scaledbndu[i])
{
xu.Set(i,State.m_bndu[i]);
continue;
}
xu.Set(i,xs[i]*State.m_s[i]);
if(State.m_HasBndL[i] && xu[i]<State.m_bndl[i])
xu.Set(i,State.m_bndl[i]);
if(State.m_HasBndU[i] && xu[i]>State.m_bndu[i])
xu.Set(i,State.m_bndu[i]);
}
}
//+------------------------------------------------------------------+
//| This object stores temporaries for internal QP solver. |
//+------------------------------------------------------------------+
struct CMinNSQP
{
double m_fc;
double m_fn;
bool m_tmpb[];
CSNNLSSolver m_nnls;
CRowInt m_tmpidx;
CRowDouble m_d;
CRowDouble m_gc;
CRowDouble m_invutc;
CRowDouble m_tmp0;
CRowDouble m_tmpc;
CRowDouble m_tmpd;
CRowDouble m_tmplambdas;
CRowDouble m_x0;
CRowDouble m_xc;
CRowDouble m_xn;
CMatrixDouble m_ch;
CMatrixDouble m_rk;
CMatrixDouble m_tmpc2;
CMatrixDouble m_uh;
//--- constructor / destructor
CMinNSQP(void) { m_fc=0; m_fn=0; }
~CMinNSQP(void) {}
//---
void Copy(const CMinNSQP &obj);
//--- overloading
void operator=(const CMinNSQP &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinNSQP::Copy(const CMinNSQP &obj)
{
m_fc=obj.m_fc;
m_fn=obj.m_fn;
ArrayCopy(m_tmpb,obj.m_tmpb);
m_nnls=obj.m_nnls;
m_tmpidx=obj.m_tmpidx;
m_d=obj.m_d;
m_gc=obj.m_gc;
m_invutc=obj.m_invutc;
m_tmp0=obj.m_tmp0;
m_tmpc=obj.m_tmpc;
m_tmpd=obj.m_tmpd;
m_tmplambdas=obj.m_tmplambdas;
m_x0=obj.m_x0;
m_xc=obj.m_xc;
m_xn=obj.m_xn;
m_ch=obj.m_ch;
m_rk=obj.m_rk;
m_tmpc2=obj.m_tmpc2;
m_uh=obj.m_uh;
}
//+------------------------------------------------------------------+
//| This object stores nonlinear optimizer State. |
//| You should use functions provided by MinNS subpackage to work |
//| with this object |
//+------------------------------------------------------------------+
struct CMinNSState
{
int m_agsmaxbacktrack;
int m_agsmaxbacktracknonfull;
int m_agsmaxraddecays;
int m_agsminupdate;
int m_agssamplesize;
int m_agsshortlimit;
int m_dbgncholesky;
int m_maxits;
int m_n;
int m_nec;
int m_ng;
int m_nh;
int m_nic;
int m_repfuncidx;
int m_repinneriterationscount;
int m_repnfev;
int m_repouteriterationscount;
int m_repterminationtype;
int m_repvaridx;
int m_solvertype;
double m_agsalphadecay;
double m_agsdecrease;
double m_agsinitstp;
double m_agspenaltyincrease;
double m_agspenaltylevel;
double m_agsraddecay;
double m_agsradius;
double m_agsrhononlinear;
double m_agsshortf;
double m_agsshortstpabs;
double m_agsshortstprel;
double m_agsstattold;
double m_diffstep;
double m_epsx;
double m_f;
double m_meritf;
double m_rawf;
double m_replcerr;
double m_repnlcerr;
double m_rholinear;
bool m_HasBndL[];
bool m_HasBndU[];
bool m_needfi;
bool m_needfij;
bool m_userterminationneeded;
bool m_xrep;
bool m_xupdated;
RCommState m_rstate;
RCommState m_rstateags;
CRowInt m_tmp3;
CRowDouble m_bndl;
CRowDouble m_bndu;
CRowDouble m_colmax;
CRowDouble m_d;
CRowDouble m_diagh;
CRowDouble m_fbase;
CRowDouble m_fi;
CRowDouble m_fm;
CRowDouble m_fp;
CRowDouble m_meritg;
CRowDouble m_rawg;
CRowDouble m_s;
CRowDouble m_samplef;
CRowDouble m_scaledbndl;
CRowDouble m_scaledbndu;
CRowDouble m_signmax;
CRowDouble m_signmin;
CRowDouble m_tmp0;
CRowDouble m_tmp1;
CRowDouble m_x;
CRowDouble m_xbase;
CRowDouble m_xc;
CRowDouble m_xn;
CRowDouble m_xscaled;
CRowDouble m_xstart;
CMinNSQP m_nsqp;
CMatrixDouble m_cleic;
CMatrixDouble m_j;
CMatrixDouble m_samplegm;
CMatrixDouble m_samplegmbc;
CMatrixDouble m_samplex;
CMatrixDouble m_scaledcleic;
CMatrixDouble m_tmp2;
CHighQualityRandState m_agsrs;
//--- constructor / destructor
CMinNSState(void);
~CMinNSState(void) {}
//---
void Copy(const CMinNSState &obj);
//--- overloading
void operator=(const CMinNSState &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinNSState::CMinNSState(void)
{
m_agsmaxbacktrack=0;
m_agsmaxbacktracknonfull=0;
m_agsmaxraddecays=0;
m_agsminupdate=0;
m_agssamplesize=0;
m_agsshortlimit=0;
m_dbgncholesky=0;
m_maxits=0;
m_n=0;
m_nec=0;
m_ng=0;
m_nh=0;
m_nic=0;
m_repfuncidx=0;
m_repinneriterationscount=0;
m_repnfev=0;
m_repouteriterationscount=0;
m_repterminationtype=0;
m_repvaridx=0;
m_solvertype=0;
m_agsalphadecay=0;
m_agsdecrease=0;
m_agsinitstp=0;
m_agspenaltyincrease=0;
m_agspenaltylevel=0;
m_agsraddecay=0;
m_agsradius=0;
m_agsrhononlinear=0;
m_agsshortf=0;
m_agsshortstpabs=0;
m_agsshortstprel=0;
m_agsstattold=0;
m_diffstep=0;
m_epsx=0;
m_f=0;
m_meritf=0;
m_rawf=0;
m_replcerr=0;
m_repnlcerr=0;
m_rholinear=0;
m_needfi=false;
m_needfij=false;
m_userterminationneeded=false;
m_xrep=false;
m_xupdated=false;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinNSState::Copy(const CMinNSState &obj)
{
m_agsmaxbacktrack=obj.m_agsmaxbacktrack;
m_agsmaxbacktracknonfull=obj.m_agsmaxbacktracknonfull;
m_agsmaxraddecays=obj.m_agsmaxraddecays;
m_agsminupdate=obj.m_agsminupdate;
m_agssamplesize=obj.m_agssamplesize;
m_agsshortlimit=obj.m_agsshortlimit;
m_dbgncholesky=obj.m_dbgncholesky;
m_maxits=obj.m_maxits;
m_n=obj.m_n;
m_nec=obj.m_nec;
m_ng=obj.m_ng;
m_nh=obj.m_nh;
m_nic=obj.m_nic;
m_repfuncidx=obj.m_repfuncidx;
m_repinneriterationscount=obj.m_repinneriterationscount;
m_repnfev=obj.m_repnfev;
m_repouteriterationscount=obj.m_repouteriterationscount;
m_repterminationtype=obj.m_repterminationtype;
m_repvaridx=obj.m_repvaridx;
m_solvertype=obj.m_solvertype;
m_agsalphadecay=obj.m_agsalphadecay;
m_agsdecrease=obj.m_agsdecrease;
m_agsinitstp=obj.m_agsinitstp;
m_agspenaltyincrease=obj.m_agspenaltyincrease;
m_agspenaltylevel=obj.m_agspenaltylevel;
m_agsraddecay=obj.m_agsraddecay;
m_agsradius=obj.m_agsradius;
m_agsrhononlinear=obj.m_agsrhononlinear;
m_agsshortf=obj.m_agsshortf;
m_agsshortstpabs=obj.m_agsshortstpabs;
m_agsshortstprel=obj.m_agsshortstprel;
m_agsstattold=obj.m_agsstattold;
m_diffstep=obj.m_diffstep;
m_epsx=obj.m_epsx;
m_f=obj.m_f;
m_meritf=obj.m_meritf;
m_rawf=obj.m_rawf;
m_replcerr=obj.m_replcerr;
m_repnlcerr=obj.m_repnlcerr;
m_rholinear=obj.m_rholinear;
ArrayCopy(m_HasBndL,obj.m_HasBndL);
ArrayCopy(m_HasBndU,obj.m_HasBndU);
m_needfi=obj.m_needfi;
m_needfij=obj.m_needfij;
m_userterminationneeded=obj.m_userterminationneeded;
m_xrep=obj.m_xrep;
m_xupdated=obj.m_xupdated;
m_rstate=obj.m_rstate;
m_rstateags=obj.m_rstateags;
m_tmp3=obj.m_tmp3;
m_bndl=obj.m_bndl;
m_bndu=obj.m_bndu;
m_colmax=obj.m_colmax;
m_d=obj.m_d;
m_diagh=obj.m_diagh;
m_fbase=obj.m_fbase;
m_fi=obj.m_fi;
m_fm=obj.m_fm;
m_fp=obj.m_fp;
m_meritg=obj.m_meritg;
m_rawg=obj.m_rawg;
m_s=obj.m_s;
m_samplef=obj.m_samplef;
m_scaledbndl=obj.m_scaledbndl;
m_scaledbndu=obj.m_scaledbndu;
m_signmax=obj.m_signmax;
m_signmin=obj.m_signmin;
m_tmp0=obj.m_tmp0;
m_tmp1=obj.m_tmp1;
m_x=obj.m_x;
m_xbase=obj.m_xbase;
m_xc=obj.m_xc;
m_xn=obj.m_xn;
m_xscaled=obj.m_xscaled;
m_xstart=obj.m_xstart;
m_nsqp=obj.m_nsqp;
m_cleic=obj.m_cleic;
m_j=obj.m_j;
m_samplegm=obj.m_samplegm;
m_samplegmbc=obj.m_samplegmbc;
m_samplex=obj.m_samplex;
m_scaledcleic=obj.m_scaledcleic;
m_tmp2=obj.m_tmp2;
m_agsrs=obj.m_agsrs;
}
//+------------------------------------------------------------------+
//| This structure stores optimization report: |
//| * IterationsCount total number of inner iterations |
//| * NFEV number of gradient evaluations |
//| * TerminationType termination type(see below) |
//| * CErr maximum violation of all types of |
//| constraints |
//| * LCErr maximum violation of linear |
//| constraints |
//| * NLCErr maximum violation of nonlinear |
//| constraints |
//| TERMINATION CODES |
//| TerminationType field contains completion code, which can be: |
//| -8 internal integrity control detected infinite or NAN |
//| values in function / gradient. Abnormal termination |
//| signalled. |
//| -3 box constraints are inconsistent |
//| -1 inconsistent parameters were passed: |
//| * penalty parameter for MinNSSetAlgoAGS() is zero, but|
//| we have nonlinear constraints set by MinNSSetNLC() |
//| 2 sampling radius decreased below epsx |
//| 5 MaxIts steps was taken |
//| 7 stopping conditions are too stringent, further |
//| improvement is impossible, X contains best point |
//| found so far. |
//| 8 User requested termination via |
//| MinNSRequestTermination() |
//| Other fields of this structure are not documented and should not |
//| be used! |
//+------------------------------------------------------------------+
struct CMinNSReport
{
int m_funcidx;
int m_iterationscount;
int m_nfev;
int m_terminationtype;
int m_varidx;
double m_cerr;
double m_lcerr;
double m_nlcerr;
//--- constructor / destructor
CMinNSReport(void) { ZeroMemory(this); }
~CMinNSReport(void) {}
//---
void Copy(const CMinNSReport &obj);
//--- overloading
void operator=(const CMinNSReport &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinNSReport::Copy(const CMinNSReport &obj)
{
m_funcidx=obj.m_funcidx;
m_iterationscount=obj.m_iterationscount;
m_nfev=obj.m_nfev;
m_terminationtype=obj.m_terminationtype;
m_varidx=obj.m_varidx;
m_cerr=obj.m_cerr;
m_lcerr=obj.m_lcerr;
m_nlcerr=obj.m_nlcerr;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CMinNS
{
public:
static void MinNSCreate(int n,CRowDouble &x,CMinNSState &State);
static void MinNSCreateF(int n,CRowDouble &x,double diffstep,CMinNSState &State);
static void MinNSSetBC(CMinNSState &State,CRowDouble &bndl,CRowDouble &bndu);
static void MinNSSetLC(CMinNSState &State,CMatrixDouble &c,CRowInt &ct,int k);
static void MinNSSetNLC(CMinNSState &State,int nlec,int nlic);
static void MinNSSetCond(CMinNSState &State,double epsx,int m_maxits);
static void MinNSSetScale(CMinNSState &State,CRowDouble &s);
static void MinNSSetAlgoAGS(CMinNSState &State,double radius,double penalty);
static void MinNSSetXRep(CMinNSState &State,bool needxrep);
static void MinNSRequestTermination(CMinNSState &State);
static bool MinNSIteration(CMinNSState &State);
static void MinNSResults(CMinNSState &State,CRowDouble &x,CMinNSReport &rep);
static void MinNSResultsBuf(CMinNSState &State,CRowDouble &x,CMinNSReport &rep);
static void MinNSRestartFrom(CMinNSState &State,CRowDouble &x);
private:
static void ClearRequestFields(CMinNSState &State);
static void MinNSInitInternal(int n,CRowDouble &x,double diffstep,CMinNSState &State);
static bool AGSIteration(CMinNSState &State);
static void UnscalePointBC(CMinNSState &State,CRowDouble &x);
static void SolveQP(CMatrixDouble &sampleg,CRowDouble &diagh,int nsample,int nvars,CRowDouble &coeffs,int &dbgncholesky,CMinNSQP &State);
static void QPCalculateGradFunc(CMatrixDouble &sampleg,CRowDouble &diagh,int nsample,int nvars,CRowDouble &coeffs,CRowDouble &g,double &f,CRowDouble &tmp);
static void QPCalculateFunc(CMatrixDouble &sampleg,CRowDouble &diagh,int nsample,int nvars,CRowDouble &coeffs,double &f,CRowDouble &tmp);
static void QPSolveU(CMatrixDouble &a,int n,CRowDouble &x);
static void QPSolveUT(CMatrixDouble &a,int n,CRowDouble &x);
};
//+------------------------------------------------------------------+
//| NONSMOOTH NONCONVEX OPTIMIZATION |
//| SUBJECT TO BOX / LINEAR / NONLINEAR - NONSMOOTH CONSTRAINTS |
//| DESCRIPTION: |
//| The subroutine minimizes function F(x) of N arguments subject to|
//| any combination of: |
//| * bound constraints |
//| * linear inequality constraints |
//| * linear equality constraints |
//| * nonlinear equality constraints Gi(x) = 0 |
//| * nonlinear inequality constraints Hi(x) <= 0 |
//| IMPORTANT: see MinNSSetAlgoAGS for important information on |
//| performance restrictions of AGS solver. |
//| REQUIREMENTS: |
//| * starting point X0 must be feasible or not too far away from |
//| the feasible set |
//| * F(), G(), H() are continuous, locally Lipschitz and |
//| continuously (but not necessarily twice) differentiable in an|
//| open dense subset of R^N. |
//| Functions F(), G() and H() may be nonsmooth and non-convex. |
//| Informally speaking, it means that functions are composed of |
//| large differentiable "patches" with nonsmoothness having place |
//| only at the boundaries between these "patches". Most real-life |
//| nonsmooth functions satisfy these requirements. Say, anything |
//| which involves finite number of abs(), min() and max() is very |
//| likely to pass the test. Say, it is possible to optimize anything|
//| of the following: |
//| * f = abs(x0) + 2 * abs(x1) |
//| * f = max(x0, x1) |
//| * f = sin(max(x0, x1) + abs(x2)) |
//| * for nonlinearly constrained problems: F() must be bounded |
//| from below without nonlinear constraints (this requirement |
//| is due to the fact that, contrary to box and linear |
//| constraints, nonlinear ones require special handling). |
//| * user must provide function value and gradient for F(), H(), |
//| G() at all points where function / gradient can be calculated|
//| If optimizer requires value exactly at the boundary between |
//| "patches"(say, at x = 0 for f = abs(x)), where gradient is |
//| not defined, user may resolve tie arbitrarily (in our case - |
//| return +1 or -1 at its discretion). |
//| * NS solver supports numerical differentiation, i.e. it may |
//| differentiate your function for you, but it results in 2N |
//| increase of function evaluations. Not recommended unless you |
//| solve really small problems. See MinNSCreateF() for more |
//| information on this functionality. |
//| USAGE: |
//| 1. User initializes algorithm State with MinNSCreate() call and |
//| chooses what NLC solver to use. There is some solver which is |
//| used by default, with default Settings, but you should NOT |
//| rely on default choice. It may change in future releases of |
//| ALGLIB without notice, and no one can guarantee that new |
//| solver will be able to solve your problem with default |
//| Settings. |
//| From the other side, if you choose solver explicitly, you can be |
//| pretty sure that it will work with new ALGLIB releases. |
//| In the current release following solvers can be used: |
//| * AGS solver (activated with MinNSSetAlgoAGS() function) |
//| 2. User adds boundary and/or linear and/or nonlinear constraints |
//| by means of calling one of the following functions: |
//| a) MinNSSetBC() for boundary constraints |
//| b) MinNSSetLC() for linear constraints |
//| c) MinNSSetNLC() for nonlinear constraints |
//| You may combine(a), (b) and (c) in one optimization problem. |
//| 3. User sets scale of the variables with MinNSSetScale() function|
//| It is VERY important to set scale of the variables, because |
//| nonlinearly constrained problems are hard to solve when |
//| variables are badly scaled. |
//| 4. User sets stopping conditions with MinNSSetCond(). |
//| 5. Finally, user calls MinNSOptimize() function which takes |
//| algorithm State and pointer (delegate, etc) to callback |
//| function which calculates F / G / H. |
//| 6. User calls MinNSResults() to get solution |
//| 7. Optionally user may call MinNSRestartFrom() to solve another |
//| problem with same N but another starting point. |
//| MinNSRestartFrom() allows to reuse already initialized |
//| structure. |
//| INPUT PARAMETERS: |
//| N - problem dimension, N > 0: |
//| * if given, only leading N elements of X are used |
//| * if not given, automatically determined from size |
//| of X |
//| X - starting point, array[N]: |
//| * it is better to set X to a feasible point |
//| * but X can be infeasible, in which case algorithm |
//| will try to find feasible point first, using X as|
//| initial approximation. |
//| OUTPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| NOTE: MinNSCreateF() function may be used if you do not have |
//| analytic gradient. This function creates solver which |
//| uses numerical differentiation with user-specified step. |
//+------------------------------------------------------------------+
void CMinNS::MinNSCreate(int n,CRowDouble &x,CMinNSState &State)
{
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values"))
return;
MinNSInitInternal(n,x,0.0,State);
}
//+------------------------------------------------------------------+
//| Version of MinNSCreateF() which uses numerical differentiation. |
//| I.e., you do not have to calculate derivatives yourself. However,|
//| this version needs 2N times more function evaluations. |
//| 2-point differentiation formula is used, because more precise |
//| 4-point formula is unstable when used on non - smooth functions. |
//| INPUT PARAMETERS: |
//| N - problem dimension, N > 0: |
//| * if given, only leading N elements of X are used |
//| * if not given, automatically determined from size |
//| of X |
//| X - starting point, array[N]: |
//| * it is better to set X to a feasible point |
//| * but X can be infeasible, in which case algorithm |
//| will try to find feasible point first, using X as|
//| initial approximation. |
//| DiffStep - differentiation step, DiffStep > 0. Algorithm |
//| performs numerical differentiation with step for|
//| I-th variable being equal to DiffStep*S[I] (here |
//| S[] is a scale vector, set by MinNSSetScale() |
//| function). Do not use too small steps, because |
//| it may lead to catastrophic cancellation during |
//| intermediate calculations. |
//| OUTPUT PARAMETERS: |
//| State - structure stores algorithm State |
//+------------------------------------------------------------------+
void CMinNS::MinNSCreateF(int n,CRowDouble &x,double diffstep,
CMinNSState &State)
{
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values"))
return;
if(!CAp::Assert(MathIsValidNumber(diffstep),__FUNCTION__+": DiffStep is infinite or NaN!"))
return;
if(!CAp::Assert((double)(diffstep)>0.0,__FUNCTION__+": DiffStep is non-positive!"))
return;
MinNSInitInternal(n,x,diffstep,State);
}
//+------------------------------------------------------------------+
//| This function sets boundary constraints. |
//| Boundary constraints are inactive by default (after initial |
//| creation). They are preserved after algorithm restart with |
//| MinNSRestartFrom(). |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| BndL - lower bounds, array[N]. If some (all) variables |
//| are unbounded, you may specify very small number |
//| or -INF. |
//| BndU - upper bounds, array[N]. If some (all) variables are|
//| unbounded, you may specify very large number |
//| or +INF. |
//| NOTE 1: it is possible to specify BndL[i]=BndU[i]. In this case |
//| I-th variable will be "frozen" at X[i]=BndL[i]=BndU[i]. |
//| NOTE 2: AGS solver has following useful properties: |
//| * bound constraints are always satisfied exactly |
//| * function is evaluated only INSIDE area specified by |
//| bound constraints, even when numerical |
//| differentiation is used(algorithm adjusts nodes |
//| according to boundary constraints) |
//+------------------------------------------------------------------+
void CMinNS::MinNSSetBC(CMinNSState &State,CRowDouble &bndl,CRowDouble &bndu)
{
int n=State.m_n;
//--- check
if(!CAp::Assert(CAp::Len(bndl)>=n,__FUNCTION__+": Length(BndL)<N"))
return;
if(!CAp::Assert(CAp::Len(bndu)>=n,__FUNCTION__+": Length(BndU)<N"))
return;
for(int i=0; i<n; i++)
{
if(!CAp::Assert(MathIsValidNumber(bndl[i]) || IsNegInf(bndl[i]),__FUNCTION__+": BndL contains NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(bndu[i]) || IsPosInf(bndu[i]),__FUNCTION__+": BndL contains NAN or -INF"))
return;
State.m_bndl.Set(i,bndl[i]);
State.m_HasBndL[i]=MathIsValidNumber(bndl[i]);
State.m_bndu.Set(i,bndu[i]);
State.m_HasBndU[i]=MathIsValidNumber(bndu[i]);
}
}
//+------------------------------------------------------------------+
//| This function sets linear constraints. |
//| Linear constraints are inactive by default (after initial |
//| creation). They are preserved after algorithm restart with |
//| MinNSRestartFrom(). |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinNSCreate() |
//| call. |
//| C - linear constraints, array[K, N + 1]. Each row of C |
//| represents one constraint, either equality or |
//| inequality(see below): |
//| * first N elements correspond to coefficients, |
//| * last element corresponds to the right part. |
//| All elements of C (including right part) must be |
//| finite. |
//| CT - type of constraints, array[K]: |
//| * if CT[i] > 0, then I-th constraint is |
//| C[i, *] * x >= C[i, n + 1] |
//| * if CT[i] = 0, then I-th constraint is |
//| C[i, *] * x = C[i, n + 1] |
//| * if CT[i] < 0, then I-th constraint is |
//| C[i, *] * x <= C[i, n + 1] |
//| K - number of equality / inequality constraints, K>=0: |
//| * if given, only leading K elements of C/CT are |
//| used |
//| * if not given, automatically determined from sizes|
//| of C/CT |
//| NOTE: linear (non-bound) constraints are satisfied only |
//| approximately: |
//| * there always exists some minor violation(about current |
//| sampling radius in magnitude during optimization, about|
//| EpsX in the solution) due to use of penalty method to |
//| handle constraints. |
//| * numerical differentiation, if used, may lead to |
//| function evaluations outside of the feasible area, |
//| because algorithm does NOT change numerical |
//| differentiation formula according to linear constraints|
//| If you want constraints to be satisfied exactly, try to |
//| reformulate your problem in such manner that all constraints will|
//| become boundary ones (this kind of constraints is always |
//| satisfied exactly, both in the final solution and in all |
//| intermediate points). |
//+------------------------------------------------------------------+
void CMinNS::MinNSSetLC(CMinNSState &State,CMatrixDouble &c,
CRowInt &ct,int k)
{
//--- create variables
int n=State.m_n;
int i=0;
//--- First, check for errors in the inputs
if(!CAp::Assert(k>=0,__FUNCTION__+": K<0"))
return;
if(!CAp::Assert(CAp::Cols(c)>=n+1 || k==0,__FUNCTION__+": Cols(C)<N+1"))
return;
if(!CAp::Assert(CAp::Rows(c)>=k,__FUNCTION__+": Rows(C)<K"))
return;
if(!CAp::Assert(CAp::Len(ct)>=k,__FUNCTION__+": Length(CT)<K"))
return;
if(!CAp::Assert(CApServ::IsFiniteMatrix(c,k,n+1),__FUNCTION__+": C contains infinite or NaN values!"))
return;
//--- Handle zero K
if(k==0)
{
State.m_nec=0;
State.m_nic=0;
return;
}
//--- Equality constraints are stored first, in the upper
//--- NEC rows of State.CLEIC matrix. Inequality constraints
//--- are stored in the next NIC rows.
//--- NOTE: we convert inequality constraints to the form
//--- A*x<=b before copying them.
CApServ::RMatrixSetLengthAtLeast(State.m_cleic,k,n+1);
State.m_nec=0;
State.m_nic=0;
for(i=0; i<k; i++)
{
if(ct[i]==0)
{
State.m_cleic.Row(State.m_nec,c,i);
State.m_nec ++;
}
}
for(i=0; i<k; i++)
{
if(ct[i]!=0)
{
if(ct[i]>0)
State.m_cleic.Row(State.m_nec+State.m_nic,c[i]*(-1.0));
else
State.m_cleic.Row(State.m_nec+State.m_nic,c,i);
State.m_nic++;
}
}
}
//+------------------------------------------------------------------+
//| This function sets nonlinear constraints. |
//| In fact, this function sets NUMBER of nonlinear constraints. |
//| Constraints itself (constraint functions) are passed to |
//| MinNSOptimize() method. This method requires user-defined vector |
//| function F[] and its Jacobian J[], where: |
//| * first component of F[] and first row of Jacobian J[] |
//| correspond to function being minimized |
//| * next NLEC components of F[] (and rows of J) correspond to |
//| nonlinear equality constraints G_i(x) = 0 |
//| * next NLIC components of F[] (and rows of J) correspond to |
//| nonlinear inequality constraints H_i(x) <= 0 |
//| NOTE: you may combine nonlinear constraints with linear/boundary |
//| ones. If your problem has mixed constraints, you may |
//| explicitly specify some of them as linear ones. It may help|
//| optimizer to handle them more efficiently. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with |
//| MinNSCreate() call. |
//| NLEC - number of Non-Linear Equality Constraints(NLEC),|
//| >= 0 |
//| NLIC - number of Non-Linear Inquality Constraints(NLIC)|
//| >= 0 |
//| NOTE 1: nonlinear constraints are satisfied only approximately! |
//| It is possible that algorithm will evaluate function |
//| outside of the feasible area! |
//| NOTE 2: algorithm scales variables according to scale specified |
//| by MinNSSetScale() function, so it can handle problems |
//| with badly scaled variables (as long as we KNOW their |
//| scales). |
//| However, there is no way to automatically scale nonlinear |
//| constraints Gi(x) and Hi(x). Inappropriate scaling of Gi/Hi may |
//| ruin convergence. Solving problem with constraint "1000*G0(x)=0" |
//| is NOT same as solving it with constraint "0.001*G0(x)=0". |
//| It means that YOU are the one who is responsible for correct |
//| scaling of nonlinear constraints Gi(x) and Hi(x). We recommend |
//| you to scale nonlinear constraints in such way that I-th |
//| component of dG/dX (or dH/dx) has approximately unit magnitude |
//| (for problems with unit scale) or has magnitude approximately |
//| equal to 1/S[i] (where S is a scale set by MinNSSetScale() |
//| function). |
//| NOTE 3: nonlinear constraints are always hard to handle, no |
//| matter what algorithm you try to use. Even basic box/ |
//| linear constraints modify function curvature by adding |
//| valleys and ridges. However, nonlinear constraints add |
//| valleys which are very hard to follow due to their |
//| "curved" nature. |
//| It means that optimization with single nonlinear constraint may |
//| be significantly slower than optimization with multiple linear |
//| ones. It is normal situation, and we recommend you to carefully |
//| choose Rho parameter of MinNSSetAlgoAGS(), because too large |
//| value may slow down convergence. |
//+------------------------------------------------------------------+
void CMinNS::MinNSSetNLC(CMinNSState &State,int nlec,int nlic)
{
if(!CAp::Assert(nlec>=0,__FUNCTION__+": NLEC<0"))
return;
if(!CAp::Assert(nlic>=0,__FUNCTION__+": NLIC<0"))
return;
State.m_ng=nlec;
State.m_nh=nlic;
State.m_fi.Resize(1+State.m_ng+State.m_nh);
State.m_j.Resize(1+State.m_ng+State.m_nh,State.m_n);
}
//+------------------------------------------------------------------+
//| This function sets stopping conditions for iterations of |
//| optimizer. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| EpsX - >= 0, The AGS solver finishes its work if on |
//| k+1-th iteration sampling radius decreases |
//| below EpsX. |
//| MaxIts - maximum number of iterations. If MaxIts = 0, |
//| the number of iterations is unlimited. |
//| Passing EpsX = 0 and MaxIts = 0 (simultaneously) will lead to |
//| automatic stopping criterion selection. We do not recommend you |
//| to rely on default choice in production code. |
//+------------------------------------------------------------------+
void CMinNS::MinNSSetCond(CMinNSState &State,double epsx,int m_maxits)
{
if(!CAp::Assert(MathIsValidNumber(epsx),__FUNCTION__+": EpsX is not finite number"))
return;
if(!CAp::Assert(epsx>=0.0,__FUNCTION__+": negative EpsX"))
return;
if(!CAp::Assert(m_maxits>=0,__FUNCTION__+": negative MaxIts!"))
return;
if(epsx==0.0 && m_maxits==0)
epsx=1.0E-6;
State.m_epsx=epsx;
State.m_maxits=m_maxits;
}
//+------------------------------------------------------------------+
//| This function sets scaling coefficients for NLC optimizer. |
//| ALGLIB optimizers use scaling matrices to test stopping |
//| conditions (step size and gradient are scaled before comparison |
//| with tolerances). Scale of the I-th variable is a translation |
//| invariant measure of: |
//| a) "how large" the variable is |
//| b) how large the step should be to make significant changes in |
//| the function |
//| Scaling is also used by finite difference variant of the |
//| optimizer - step along I-th axis is equal to DiffStep*S[I]. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| S - array[N], non-zero scaling coefficients S[i] |
//| may be negative, sign doesn't matter. |
//+------------------------------------------------------------------+
void CMinNS::MinNSSetScale(CMinNSState &State,CRowDouble &s)
{
if(!CAp::Assert(CAp::Len(s)>=State.m_n,__FUNCTION__+": Length(S)<N"))
return;
for(int i=0; i<State.m_n; i++)
{
if(!CAp::Assert(MathIsValidNumber(s[i]),__FUNCTION__+": S contains infinite or NAN elements"))
return;
if(!CAp::Assert(s[i]!=0.0,__FUNCTION__+": S contains zero elements"))
return;
State.m_s.Set(i,MathAbs(s[i]));
}
}
//+------------------------------------------------------------------+
//| This function tells MinNS unit to use AGS (adaptive gradient |
//| sampling) algorithm for nonsmooth constrained optimization. This |
//| algorithm is a slight modification of one described in "An |
//| Adaptive Gradient Sampling Algorithm for Nonsmooth Optimization" |
//| by Frank E. Curtisy and Xiaocun Quez. |
//| This optimizer has following benefits and drawbacks: |
//| + robustness; it can be used with nonsmooth and nonconvex |
//| functions. |
//| + relatively easy tuning; most of the metaparameters are easy |
//| to select. |
//| - it has convergence of steepest descent, slower than CG/LBFGS.|
//| - each iteration involves evaluation of ~2N gradient values and|
//| solution of 2Nx2N quadratic programming problem, which limits|
//| applicability of algorithm by small-scale problems (up |
//| to 50 - 100). |
//| IMPORTANT: this algorithm has convergence guarantees, i.e. it |
//| will steadily move towards some stationary point of |
//| the function. |
//| However, "stationary point" does not always mean "solution". |
//| Nonsmooth problems often have "flat spots", i.e. areas where |
//| function do not change at all. Such "flat spots" are stationary |
//| points by definition, and algorithm may be caught here. |
//| Nonsmooth CONVEX tasks are not prone to this problem. Say, if |
//| your function has form f() = MAX(f0, f1, ...), and f_i are convex|
//| then f() is convex too and you have guaranteed convergence to |
//| solution. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| Radius - initial sampling radius, >= 0. Internally |
//| multiplied by vector of per-variable scales |
//| specified by MinNSSetScale(). |
//| You should select relatively large sampling |
//| radius, roughly proportional to scaled length |
//| of the first steps of the algorithm. Something |
//| close to 0.1 in magnitude should be good for |
//| most problems. |
//| AGS solver can automatically decrease radius, |
//| so too large radius is not a problem (assuming |
//| that you won't choose so large radius that |
//| algorithm will sample function in too far away |
//| points, where gradient value is irrelevant). |
//| Too small radius won't cause algorithm to fail, |
//| but it may slow down algorithm (it may have to |
//| perform too short steps). |
//| Penalty - penalty coefficient for nonlinear constraints: |
//| * for problem with nonlinear constraints should |
//| be some problem - specific positive value, |
//| large enough that penalty term changes shape |
//| of the function. Starting from some problem - |
//| specific value penalty coefficient becomes |
//| large enough to exactly enforce nonlinear |
//| constraints; larger values do not improve |
//| precision. Increasing it too much may slow |
//| down convergence, so you should choose it |
//| carefully. |
//| * can be zero for problems WITHOUT nonlinear |
//| constraints (i.e. for unconstrained ones or |
//| ones with just box or linear constraints) |
//| * if you specify zero value for problem with at |
//| least one nonlinear constraint, algorithm will|
//| terminate with error code - 1. |
//| ALGORITHM OUTLINE |
//| The very basic outline of unconstrained AGS algorithm is given |
//| below: |
//| 0. If sampling radius is below EpsX or we performed more then |
//| MaxIts iterations - STOP. |
//| 1. sample O(N) gradient values at random locations around current|
//| point; informally speaking, this sample is an implicit |
//| piecewise linear model of the function, although algorithm |
//| formulation does not mention that explicitly |
//| 2. solve quadratic programming problem in order to find descent |
//| direction |
//| 3. if QP solver tells us that we are near solution, decrease |
//| sampling radius and move to(0) |
//| 4. perform backtracking line search |
//| 5. after moving to new point, goto(0) |
//| Constraint handling details: |
//| * box constraints are handled exactly by algorithm |
//| * linear/nonlinear constraints are handled by adding L1 |
//| penalty. Because our solver can handle nonsmoothness, we can |
//| use L1 penalty function, which is an exact one (i.e. exact |
//| solution is returned under such penalty). |
//| * penalty coefficient for linear constraints is chosen |
//| automatically; however, penalty coefficient for nonlinear |
//| constraints must be specified by user. |
//| ===== TRACING AGS SOLVER ======================================= |
//| AGS solver supports advanced tracing capabilities. You can trace |
//| algorithm output by specifying following trace symbols (case- |
//| insensitive) by means of trace_file() call: |
//| * 'AGS' - for basic trace of algorithm steps and |
//| decisions. Only short scalars (function |
//| values and deltas) are printed. N-dimensional|
//| quantities like search directions are NOT |
//| printed. |
//| * 'AGS.DETAILED' - for output of points being visited and |
//| search directions. This symbol also |
//| implicitly defines 'AGS'. You can control |
//| output format by additionally specifying: |
//| * nothing to output in 6-digit exponential |
//| format |
//| * 'PREC.E15' to output in 15-digit |
//| exponential format |
//| * 'PREC.F6' to output in 6-digit fixed-point |
//| format |
//| * 'AGS.DETAILED.SAMPLE' - for output of points being visited, |
//| search directions and gradient sample. May |
//| take a LOT of space, do not use it on |
//| problems with more that several tens of vars.|
//| This symbol also implicitly defines 'AGS' and|
//| 'AGS.DETAILED'. |
//| By default trace is disabled and adds no overhead to the |
//| optimization process. However, specifying any of the symbols adds|
//| some formatting and output-related overhead. |
//| You may specify multiple symbols by separating them with commas: |
//| > |
//| >CAlglib::Trace_File("AGS,PREC.F6", "path/to/trace.log") |
//| > |
//+------------------------------------------------------------------+
void CMinNS::MinNSSetAlgoAGS(CMinNSState &State,double radius,double penalty)
{
if(!CAp::Assert(MathIsValidNumber(radius),__FUNCTION__+": Radius is not finite"))
return;
if(!CAp::Assert(radius>0.0,__FUNCTION__+": Radius<=0"))
return;
if(!CAp::Assert(MathIsValidNumber(penalty),__FUNCTION__+": Penalty is not finite"))
return;
if(!CAp::Assert(penalty>=0.0,__FUNCTION__+": Penalty<0"))
return;
State.m_agsrhononlinear=penalty;
State.m_agsradius=radius;
State.m_solvertype=0;
}
//+------------------------------------------------------------------+
//| This function turns on / off reporting. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NeedXRep - whether iteration reports are needed or not |
//| If NeedXRep is True, algorithm will call rep() callback function |
//| if it is provided to MinNSOptimize(). |
//+------------------------------------------------------------------+
void CMinNS::MinNSSetXRep(CMinNSState &State,bool needxrep)
{
State.m_xrep=needxrep;
}
//+------------------------------------------------------------------+
//| This subroutine submits request for termination of running |
//| optimizer. It should be called from user-supplied callback when |
//| user decides that it is time to "smoothly" terminate optimization|
//| process. As result, optimizer stops at point which was "current |
//| accepted" when termination request was submitted and returns |
//| error code 8 (successful termination). |
//| INPUT PARAMETERS: |
//| State - optimizer structure |
//| NOTE: after request for termination optimizer may perform several|
//| additional calls to user-supplied callbacks. It does NOT |
//| guarantee to stop immediately - it just guarantees that |
//| these additional calls will be discarded later. |
//| NOTE: calling this function on optimizer which is NOT running |
//| will have no effect. |
//| NOTE: multiple calls to this function are possible. First call is|
//| counted, subsequent calls are silently ignored. |
//+------------------------------------------------------------------+
void CMinNS::MinNSRequestTermination(CMinNSState &State)
{
State.m_userterminationneeded=true;
}
//+------------------------------------------------------------------+
//| NOTES: |
//| 1. This function has two different implementations: one which |
//| uses exact (analytical) user-supplied Jacobian, and one which |
//| uses only function vector and numerically differentiates |
//| function in order to obtain gradient. |
//| Depending on the specific function used to create optimizer |
//| object you should choose appropriate variant of |
//| MinNSOptimize() - one which accepts function AND Jacobian or |
//| one which accepts ONLY function. |
//| Be careful to choose variant of MinNSOptimize() which |
//| corresponds to your optimization scheme! Table below lists |
//| different combinations of callback (function/gradient) passed |
//| to MinNSOptimize() and specific function used to create |
//| optimizer. |
//| | USER PASSED TO MinNSOptimize() |
//| CREATED WITH | function only | function and gradient |
//| ------------------------------------------------------------ |
//| MinNSCreateF() | works FAILS |
//| MinNSCreate() | FAILS works |
//| Here "FAILS" denotes inappropriate combinations of optimizer |
//| creation function and MinNSOptimize() version. Attemps to use |
//| such combination will lead to exception. Either you did not pass |
//| gradient when it WAS needed or you passed gradient when it was |
//| NOT needed. |
//+------------------------------------------------------------------+
bool CMinNS::MinNSIteration(CMinNSState &State)
{
//--- create variables
bool result=false;
int i=0;
int j=0;
int k=0;
int n=0;
int nec=0;
int nic=0;
int ng=0;
int nh=0;
double v=0;
double xp=0;
double xm=0;
int i_=0;
int label=-1;
//--- Reverse communication preparations
//--- I know it looks ugly, but it works the same way
//--- anywhere from C++ to Python.
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(State.m_rstate.stage>=0)
{
i=State.m_rstate.ia[0];
j=State.m_rstate.ia[1];
k=State.m_rstate.ia[2];
n=State.m_rstate.ia[3];
nec=State.m_rstate.ia[4];
nic=State.m_rstate.ia[5];
ng=State.m_rstate.ia[6];
nh=State.m_rstate.ia[7];
v=State.m_rstate.ra[0];
xp=State.m_rstate.ra[1];
xm=State.m_rstate.ra[2];
}
else
{
i=359;
j=-58;
k=-919;
n=-909;
nec=81;
nic=255;
ng=74;
nh=-788;
v=809;
xp=205;
xm=-838;
}
switch(State.m_rstate.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
case 3:
label=3;
break;
default:
//--- Routine body
//--- Init
State.m_replcerr=0.0;
State.m_repnlcerr=0.0;
State.m_repterminationtype=0;
State.m_repinneriterationscount=0;
State.m_repouteriterationscount=0;
State.m_repnfev=0;
State.m_repvaridx=0;
State.m_repfuncidx=0;
State.m_userterminationneeded=false;
State.m_dbgncholesky=0;
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
ng=State.m_ng;
nh=State.m_nh;
ClearRequestFields(State);
//--- AGS solver
if(State.m_solvertype!=0)
{
label=4;
break;
}
if(State.m_diffstep!=0.0)
{
CApServ::RVectorSetLengthAtLeast(State.m_xbase,n);
CApServ::RVectorSetLengthAtLeast(State.m_fbase,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fm,1+ng+nh);
CApServ::RVectorSetLengthAtLeast(State.m_fp,1+ng+nh);
}
CApServ::RVectorSetLengthAtLeast(State.m_xscaled,n);
CApServ::RVectorSetLengthAtLeast(State.m_rawg,n);
CApServ::RVectorSetLengthAtLeast(State.m_meritg,n);
State.m_rstateags.ia.Resize(14);
ArrayResize(State.m_rstateags.ba,6);
State.m_rstateags.ra.Resize(11);
State.m_rstateags.stage=-1;
label=6;
break;
}
//--- main loop
while(label>=0)
{
switch(label)
{
case 6:
if(!AGSIteration(State))
{
label=7;
break;
}
State.m_xscaled=State.m_x;
UnscalePointBC(State,State.m_x);
//--- Numerical differentiation (if needed) - intercept NeedFiJ
//--- request and replace it by sequence of NeedFi requests
if(!(State.m_diffstep!=0.0 && State.m_needfij))
{
label=8;
break;
}
State.m_needfij=false;
State.m_needfi=true;
State.m_xbase=State.m_x;
State.m_rstate.stage=0;
label=-1;
break;
case 0:
State.m_fbase=State.m_fi;
State.m_repnfev++;
k=0;
case 10:
if(k>n-1)
{
label=12;
break;
}
v=State.m_xbase[k];
xm=v-State.m_diffstep*State.m_s[k];
xp=v+State.m_diffstep*State.m_s[k];
if(State.m_HasBndL[k] && xm<State.m_bndl[k])
xm=State.m_bndl[k];
if(State.m_HasBndU[k] && xp>State.m_bndu[k])
xp=State.m_bndu[k];
if(!CAp::Assert(xm<=xp,__FUNCTION__+": integrity check failed (3y634)"))
return(false);
if(xm==xp)
{
label=13;
break;
}
//--- Compute F(XM) and F(XP)
State.m_x=State.m_xbase;
State.m_x.Set(k,xm);
State.m_rstate.stage=1;
label=-1;
break;
case 1:
State.m_fm=State.m_fi;
State.m_x=State.m_xbase;
State.m_x.Set(k,xp);
State.m_rstate.stage=2;
label=-1;
break;
case 2:
State.m_fp=State.m_fi;
//--- Compute subgradient at XBase
CAblasF::RCopyMulVC(1+ng+nh,1/(xp-xm),State.m_fp,State.m_j,k);
CAblasF::RAddVC(1+ng+nh,-(1/(xp-xm)),State.m_fm,State.m_j,k);
State.m_repnfev+=2;
label=14;
break;
case 13:
CAblasF::RSetC(1+ng+nh,0.0,State.m_j,k);
case 14:
k++;
label=10;
break;
case 12:
//--- Restore previous values of fields and continue
State.m_x=State.m_xscaled;
State.m_fi=State.m_fbase;
State.m_needfi=false;
State.m_needfij=true;
label=9;
break;
case 8:
//--- Forward request to caller
State.m_rstate.stage=3;
label=-1;
break;
case 3:
State.m_repnfev++;
State.m_x=State.m_xscaled;
case 9:
//--- Postprocess Jacobian: scale and produce 'raw' and 'merit' functions
for(i=0; i<=ng+nh; i++)
CAblasF::RMergeMulVR(n,State.m_s,State.m_j,i);
State.m_rawf=State.m_fi[0];
State.m_meritf=State.m_fi[0];
State.m_rawg=State.m_j[0]+0;
State.m_meritg=State.m_j[0]+0;
for(i=0; i<=nec+nic-1; i++)
{
v=CAblasF::RDotVR(n,State.m_x,State.m_scaledcleic,i)-State.m_scaledcleic.Get(i,n);
if(i>=nec && v<0.0)
continue;
State.m_meritf+=State.m_rholinear*MathAbs(v);
CAblasF::RAddRV(n,State.m_rholinear*MathSign(v),State.m_scaledcleic,i,State.m_meritg);
}
for(i=1; i<=ng+nh; i++)
{
v=State.m_fi[i];
if(i<=ng && v==0.0)
continue;
if(i>ng && v<=0.0)
continue;
State.m_meritf+=State.m_agsrhononlinear*MathAbs(v);
CAblasF::RAddRV(n,State.m_agsrhononlinear*MathSign(v),State.m_j,i,State.m_meritg);
}
//--- Done
label=6;
break;
case 7:
case 4:
result=false;
return(result);
}
}
//--- Saving State
result=true;
State.m_rstate.ia.Set(0,i);
State.m_rstate.ia.Set(1,j);
State.m_rstate.ia.Set(2,k);
State.m_rstate.ia.Set(3,n);
State.m_rstate.ia.Set(4,nec);
State.m_rstate.ia.Set(5,nic);
State.m_rstate.ia.Set(6,ng);
State.m_rstate.ia.Set(7,nh);
State.m_rstate.ra.Set(0,v);
State.m_rstate.ra.Set(1,xp);
State.m_rstate.ra.Set(2,xm);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| MinNS results |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| X - array[0..N - 1], solution |
//| Rep - optimization report. You should check |
//| Rep.TerminationType in order to distinguish |
//| successful termination from unsuccessful one: |
//| * -8 internal integrity control detected infinite or NAN |
//| values in function/gradient. Abnormal termination |
//| signalled. |
//| * -3 box constraints are inconsistent |
//| * -1 inconsistent parameters were passed: |
//| * penalty parameter for MinNSSetAlgoAGS() is zero, but|
//| we have nonlinear constraints set by MinNSSetNLC() |
//| * 2 sampling radius decreased below epsx |
//| * 7 stopping conditions are too stringent, further |
//| improvement is impossible, X contains best point found|
//| so far. |
//| * 8 User requested termination via |
//| MinNSRequestTermination() |
//+------------------------------------------------------------------+
void CMinNS::MinNSResults(CMinNSState &State,CRowDouble &x,CMinNSReport &rep)
{
x.Resize(0);
MinNSResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| Buffered implementation of MinNSResults() which uses pre- |
//| allocated buffer to store X[]. If buffer size is too small, it |
//| resizes buffer. It is intended to be used in the inner cycles of |
//| performance critical algorithms where array reallocation penalty |
//| is too large to be ignored. |
//+------------------------------------------------------------------+
void CMinNS::MinNSResultsBuf(CMinNSState &State,CRowDouble &x,CMinNSReport &rep)
{
//--- copy
rep.m_iterationscount=State.m_repinneriterationscount;
rep.m_nfev=State.m_repnfev;
rep.m_varidx=State.m_repvaridx;
rep.m_funcidx=State.m_repfuncidx;
rep.m_terminationtype=State.m_repterminationtype;
rep.m_cerr=MathMax(State.m_replcerr,State.m_repnlcerr);
rep.m_lcerr=State.m_replcerr;
rep.m_nlcerr=State.m_repnlcerr;
if(State.m_repterminationtype>0)
x=State.m_xc;
else
x=vector<double>::Full(State.m_n,AL_NaN);
}
//+------------------------------------------------------------------+
//| This subroutine restarts algorithm from new point. |
//| All optimization parameters (including constraints) are left |
//| unchanged. |
//| This function allows to solve multiple optimization problems |
//| (which must have same number of dimensions) without object |
//| reallocation penalty. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with |
//| MinNSCreate() call. |
//| X - new starting point. |
//+------------------------------------------------------------------+
void CMinNS::MinNSRestartFrom(CMinNSState &State,CRowDouble &x)
{
int n=State.m_n;
//--- First, check for errors in the inputs
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- Set XC
State.m_xstart=x;
//--- prepare RComm facilities
State.m_rstate.ia.Resize(8);
State.m_rstate.ra.Resize(3);
State.m_rstate.stage=-1;
ClearRequestFields(State);
}
//+------------------------------------------------------------------+
//| Clears request fileds (to be sure that we don't forget to clear |
//| something) |
//+------------------------------------------------------------------+
void CMinNS::ClearRequestFields(CMinNSState &State)
{
State.m_needfi=false;
State.m_needfij=false;
State.m_xupdated=false;
}
//+------------------------------------------------------------------+
//| Internal initialization subroutine. |
//| Sets default NLC solver with default criteria. |
//+------------------------------------------------------------------+
void CMinNS::MinNSInitInternal(int n,CRowDouble &x,double diffstep,
CMinNSState &State)
{
//--- create variables
CMatrixDouble c;
CRowInt ct;
State.m_agsinitstp=0.2;
State.m_agsstattold=MathSqrt(CMath::m_machineepsilon);
State.m_agsshortstpabs=1.0E-10;
State.m_agsshortstprel=0.75;
State.m_agsshortf=10*CMath::m_machineepsilon;
State.m_agsrhononlinear=0.0;
State.m_agsraddecay=0.2;
State.m_agsalphadecay=0.5;
State.m_agsdecrease=0.1;
State.m_agsmaxraddecays=50;
State.m_agsmaxbacktrack=20;
State.m_agsmaxbacktracknonfull=8;
State.m_agspenaltylevel=50.0;
State.m_agspenaltyincrease=100.0;
State.m_agsminupdate=MathMax(5,n/2);
State.m_agssamplesize=MathMax(2*n+1,State.m_agsminupdate+1);
State.m_agsshortlimit=4+State.m_agssamplesize/State.m_agsminupdate;
//--- Initialize other params
State.m_n=n;
State.m_diffstep=diffstep;
State.m_xstart=x;
State.m_xc=x;
State.m_bndl=vector<double>::Full(n,AL_NEGINF);
ArrayResize(State.m_HasBndL,n);
State.m_bndu=vector<double>::Full(n,AL_POSINF);
ArrayResize(State.m_HasBndU,n);
State.m_s=vector<double>::Ones(n);
State.m_xstart.Resize(n);
State.m_xc.Resize(n);
State.m_xn.Resize(n);
State.m_d.Resize(n);
State.m_x.Resize(n);
ArrayInitialize(State.m_HasBndL,false);
ArrayInitialize(State.m_HasBndU,false);
MinNSSetLC(State,c,ct,0);
MinNSSetNLC(State,0,0);
MinNSSetCond(State,0.0,0);
MinNSSetXRep(State,false);
MinNSSetAlgoAGS(State,0.1,1000.0);
MinNSRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| This function performs actual processing for AUL algorith. It |
//| expects that caller redirects its reverse communication requests |
//| NeedFiJ / XUpdated to external user who will provide analytic |
//| derivative (or handle reports about progress). |
//| In case external user does not have analytic derivative, it is |
//| responsibility of caller to intercept NeedFiJ request and replace|
//| it with appropriate numerical differentiation scheme. |
//+------------------------------------------------------------------+
bool CMinNS::AGSIteration(CMinNSState &State)
{
//--- create variables
bool result=false;
int n=0;
int nec=0;
int nic=0;
int ng=0;
int nh=0;
int i=0;
int j=0;
int k=0;
double radius0=0;
double radius=0;
int radiusdecays=0;
double alpha=0;
double recommendedstep=0;
double dhd=0;
double dnrminf=0;
double v=0;
double vv=0;
int maxsamplesize=0;
int cursamplesize=0;
double v0=0;
double v1=0;
bool b=false;
bool alphadecreased=false;
int shortstepscnt=0;
int backtrackits=0;
int maxbacktrackits=0;
bool fullsample=false;
double currentf0=0;
bool dotrace=false;
bool dodetailedtrace=false;
bool dotracesample=false;
int i_=0;
int label=-1;
//--- Reverse communication preparations
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(State.m_rstateags.stage>=0)
{
n=State.m_rstateags.ia[0];
nec=State.m_rstateags.ia[1];
nic=State.m_rstateags.ia[2];
ng=State.m_rstateags.ia[3];
nh=State.m_rstateags.ia[4];
i=State.m_rstateags.ia[5];
j=State.m_rstateags.ia[6];
k=State.m_rstateags.ia[7];
radiusdecays=State.m_rstateags.ia[8];
maxsamplesize=State.m_rstateags.ia[9];
cursamplesize=State.m_rstateags.ia[10];
shortstepscnt=State.m_rstateags.ia[11];
backtrackits=State.m_rstateags.ia[12];
maxbacktrackits=State.m_rstateags.ia[13];
b=State.m_rstateags.ba[0];
alphadecreased=State.m_rstateags.ba[1];
fullsample=State.m_rstateags.ba[2];
dotrace=State.m_rstateags.ba[3];
dodetailedtrace=State.m_rstateags.ba[4];
dotracesample=State.m_rstateags.ba[5];
radius0=State.m_rstateags.ra[0];
radius=State.m_rstateags.ra[1];
alpha=State.m_rstateags.ra[2];
recommendedstep=State.m_rstateags.ra[3];
dhd=State.m_rstateags.ra[4];
dnrminf=State.m_rstateags.ra[5];
v=State.m_rstateags.ra[6];
vv=State.m_rstateags.ra[7];
v0=State.m_rstateags.ra[8];
v1=State.m_rstateags.ra[9];
currentf0=State.m_rstateags.ra[10];
}
else
{
n=939;
nec=-526;
nic=763;
ng=-541;
nh=-698;
i=-900;
j=-318;
k=-940;
radiusdecays=1016;
maxsamplesize=-229;
cursamplesize=-536;
shortstepscnt=487;
backtrackits=-115;
maxbacktrackits=886;
b=false;
alphadecreased=false;
fullsample=true;
dotrace=true;
dodetailedtrace=true;
dotracesample=true;
radius0=922;
radius=-154;
alpha=306;
recommendedstep=-1011;
dhd=951;
dnrminf=-463;
v=88;
vv=-861;
v0=-678;
v1=-731;
currentf0=-675;
}
switch(State.m_rstateags.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
case 3:
label=3;
break;
default:
//--- Routine body
if(!CAp::Assert(State.m_solvertype==0,__FUNCTION__+": internal error"))
return(false);
n=State.m_n;
nec=State.m_nec;
nic=State.m_nic;
ng=State.m_ng;
nh=State.m_nh;
dotrace=CAp::IsTraceEnabled("AGS");
dodetailedtrace=dotrace && CAp::IsTraceEnabled("AGS.DETAILED");
dotracesample=dodetailedtrace && CAp::IsTraceEnabled("AGS.DETAILED.SAMPLE");
//--- Trace output (if needed)
if(dotrace)
{
CAp::Trace("////////////////////////////////////////////////////////////////////////////////////////////////////\n");
CAp::Trace("//--- AGS SOLVER STARTED //\n");
CAp::Trace("////////////////////////////////////////////////////////////////////////////////////////////////////\n");
}
//--- Check consistency of parameters
if(ng+nh>0 && State.m_agsrhononlinear==0.0)
{
if(dotrace)
CAp::Trace("> inconsistent parameters detected,stopping\n\n");
State.m_repterminationtype=-1;
result=false;
return(result);
}
//--- Allocate arrays.
CApServ::RVectorSetLengthAtLeast(State.m_colmax,n);
CApServ::RVectorSetLengthAtLeast(State.m_diagh,n);
CApServ::RVectorSetLengthAtLeast(State.m_signmin,n);
CApServ::RVectorSetLengthAtLeast(State.m_signmax,n);
maxsamplesize=State.m_agssamplesize;
CApServ::RMatrixSetLengthAtLeast(State.m_samplex,maxsamplesize+1,n);
CApServ::RMatrixSetLengthAtLeast(State.m_samplegm,maxsamplesize+1,n);
CApServ::RMatrixSetLengthAtLeast(State.m_samplegmbc,maxsamplesize+1,n);
CApServ::RVectorSetLengthAtLeast(State.m_samplef,maxsamplesize+1);
//--- Prepare optimizer
State.m_tmp0=vector<double>::Zeros(maxsamplesize);
State.m_tmp1=vector<double>::Full(maxsamplesize,AL_POSINF);
CApServ::IVectorSetLengthAtLeast(State.m_tmp3,1);
CApServ::RMatrixSetLengthAtLeast(State.m_tmp2,1,maxsamplesize+1);
//--- Prepare RNG, seed it with fixed values so
//--- that each run on same problem yeilds same results
CHighQualityRand::HQRndSeed(7235,98532,State.m_agsrs);
//--- Prepare initial point subject to current bound constraints and
//--- perform scaling of bound constraints, linear constraints, point itself
CApServ::RVectorSetLengthAtLeast(State.m_scaledbndl,n);
CApServ::RVectorSetLengthAtLeast(State.m_scaledbndu,n);
for(i=0; i<n; i++)
{
//--- Check and scale constraints
if(State.m_HasBndL[i] && State.m_HasBndU[i] && State.m_bndu[i]<State.m_bndl[i])
{
if(dotrace)
CAp::Trace("> inconsistent box constraints detected,stopping\n\n");
State.m_repterminationtype=-3;
result=false;
return(result);
}
if(State.m_HasBndL[i])
State.m_scaledbndl.Set(i,State.m_bndl[i]/State.m_s[i]);
else
State.m_scaledbndl.Set(i,AL_NEGINF);
if(State.m_HasBndU[i])
State.m_scaledbndu.Set(i,State.m_bndu[i]/State.m_s[i]);
else
State.m_scaledbndu.Set(i,AL_POSINF);
if(State.m_HasBndL[i] && State.m_HasBndU[i])
{
if(!CAp::Assert(State.m_scaledbndl[i]<=State.m_scaledbndu[i],__FUNCTION__+": integrity check failed (dfdf)"))
return(false);
}
if(State.m_HasBndL[i] && State.m_HasBndU[i] && State.m_bndl[i]==State.m_bndu[i])
{
if(!CAp::Assert(State.m_scaledbndl[i]==State.m_scaledbndu[i],__FUNCTION__+": integrity check failed (dsgh)"))
return(false);
}
//--- Scale and constrain point
State.m_xc.Set(i,State.m_xstart[i]);
if(State.m_HasBndL[i] && State.m_xc[i]<=State.m_bndl[i])
{
State.m_xc.Set(i,State.m_scaledbndl[i]);
continue;
}
if(State.m_HasBndU[i] && State.m_xc[i]>=State.m_bndu[i])
{
State.m_xc.Set(i,State.m_scaledbndu[i]);
continue;
}
State.m_xc.Mul(i,1.0/State.m_s[i]);
if(State.m_HasBndL[i] && State.m_xc[i]<=State.m_scaledbndl[i])
State.m_xc.Set(i,State.m_scaledbndl[i]);
if(State.m_HasBndU[i] && State.m_xc[i]>=State.m_scaledbndu[i])
State.m_xc.Set(i,State.m_scaledbndu[i]);
}
CApServ::RMatrixSetLengthAtLeast(State.m_scaledcleic,nec+nic,n+1);
for(i=0; i<nec+nic; i++)
{
//--- Scale and normalize linear constraints
vv=0.0;
for(j=0; j<n; j++)
{
v=State.m_cleic.Get(i,j)*State.m_s[j];
State.m_scaledcleic.Set(i,j,v);
vv+=v*v;
}
vv=MathSqrt(vv);
State.m_scaledcleic.Set(i,n,State.m_cleic.Get(i,n));
if(vv>0.0)
State.m_scaledcleic.Row(i,State.m_scaledcleic[i]/vv);
}
//--- Main cycle
//--- We maintain several variables during iteration:
//--- * RecommendedStep- current estimate of recommended step length;
//--- must be Radius0 on first entry
//--- * Radius - current sampling radius
//--- * CurSampleSize - current sample size (may change in future versions)
//--- * FullSample - whether we have full sample, or only partial one
//--- * RadiusDecays - total number of decreases performed for sampling radius
radius=State.m_agsradius;
radius0=radius;
recommendedstep=MathMin(radius0,State.m_agsinitstp);
cursamplesize=1;
radiusdecays=0;
shortstepscnt=0;
fullsample=false;
State.m_rholinear=0.0;
label=4;
break;
}
//--- main loop
while(label>=0)
{
switch(label)
{
case 4:
if(dotrace)
CAp::Trace(StringFormat("\n=== ITERATION %5d STARTED ========================================================================\n",State.m_repinneriterationscount));
//--- First phase of iteration - central point:
//--- 1. evaluate function at central point - first entry in sample.
//--- Its status is ignored, it is always recalculated.
//--- 2. report point and check gradient/function value for NAN/INF
//--- 3. check penalty coefficients for linear terms; increase them
//--- if directional derivative of function being optimized (not
//--- merit function!) is larger than derivative of penalty.
//--- 4. update report on constraint violation
cursamplesize=MathMax(cursamplesize,1);
State.m_samplex.Row(0,State.m_xc);
State.m_x=State.m_xc;
ClearRequestFields(State);
State.m_needfij=true;
State.m_rstateags.stage=0;
label=-1;
break;
case 0:
State.m_needfij=false;
currentf0=State.m_rawf;
State.m_replcerr=0.0;
for(i=0; i<nec+nic; i++)
{
v=-State.m_scaledcleic.Get(i,n);
for(j=0; j<n; j++)
v+=State.m_scaledcleic.Get(i,j)*State.m_xc[j];
if(i>=nec && v<=0.0)
continue;
State.m_replcerr=MathMax(State.m_replcerr,MathAbs(v));
}
State.m_repnlcerr=0.0;
for(i=1; i<=ng+nh; i++)
{
v=State.m_fi[i];
if(i>ng && v<=0.0)
continue;
State.m_repnlcerr=MathMax(State.m_repnlcerr,MathAbs(v));
}
State.m_samplef.Set(0,State.m_meritf);
CAblasF::RCopyVR(n,State.m_meritg,State.m_samplegm,0);
if(!State.m_xrep)
{
label=6;
break;
}
State.m_x=State.m_xc;
State.m_f=currentf0;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstateags.stage=1;
label=-1;
break;
case 1:
State.m_xupdated=false;
case 6:
if(State.m_userterminationneeded)
{
//--- User requested termination
if(dotrace)
CAp::Trace("> termination requested by user\n\n");
State.m_repterminationtype=8;
label=5;
break;
}
v=0;
for(i=0; i<n; i++)
v+=CMath::Sqr(State.m_samplegm.Get(0,i));
if(!MathIsValidNumber(v) || !MathIsValidNumber(State.m_samplef[0]))
{
//--- Abnormal termination - infinities in function/gradient
if(dotrace)
CAp::Trace("> NAN/INF detected in function/gradient,termination\n\n");
State.m_repterminationtype=-8;
label=5;
break;
}
if(!CAp::Assert(State.m_agspenaltylevel>1.0,__FUNCTION__+": integrity error"))
return(false);
if(!CAp::Assert(State.m_agspenaltyincrease>State.m_agspenaltylevel,__FUNCTION__+": integrity error"))
return(false);
if(MathSqrt(CAblasF::RDotV2(n,State.m_rawg))*State.m_agspenaltylevel>State.m_rholinear)
{
State.m_rholinear=MathSqrt(CAblasF::RDotV2(n,State.m_rawg))*State.m_agspenaltyincrease;
if(dotrace)
CAp::Trace("> penalty parameter needs increase,iteration restarted\n\n");
cursamplesize=0;
label=4;
break;
}
//--- Trace if needed
if(dotrace)
{
if(dodetailedtrace)
{
CAp::Trace("> printing raw data (prior to applying variable and function scales)\n");
CAp::Trace("X (raw) = ");
CApServ::TraceVectoRunScaledUnshiftedAutopRec(State.m_xc,n,State.m_s,true,State.m_s,false);
CAp::Trace("\n");
CAp::Trace("> printing scaled data (after applying variable and function scales)\n");
CAp::Trace("X (scaled) = ");
CApServ::TraceVectorAutopRec(State.m_xc,0,n);
CAp::Trace("\n");
}
CAp::Trace(StringFormat("sampleRad = %.3E\n",radius));
CAp::Trace(StringFormat("lin.violation = %.3E (scaled violation of linear constraints)\n",State.m_replcerr));
CAp::Trace(StringFormat("nlc.violation = %.3E (scaled violation of nonlinear constraints)\n",State.m_repnlcerr));
CAp::Trace(StringFormat("targetF = %.3E (target function)\n",currentf0));
CAp::Trace(StringFormat("meritF = %.3E (merit function)\n",State.m_samplef[0]));
CAp::Trace(StringFormat("Rho linear = %.3E\n",State.m_rholinear));
CAp::Trace(StringFormat("Rho nonlinear = %.3E\n",State.m_agsrhononlinear));
CAp::Trace("----------------------------------------------------------------------------------------------------\n");
}
//--- Check stopping conditions.
if(radiusdecays>=State.m_agsmaxraddecays)
{
//--- Too many attempts to decrease radius
if(dotrace)
CAp::Trace("> stopping condition met: too many attempts to decrease radius\n\n");
State.m_repterminationtype=7;
label=5;
break;
}
if(State.m_repinneriterationscount>=State.m_maxits && State.m_maxits>0)
{
//--- Too many iterations
if(dotrace)
CAp::Trace(StringFormat("> stopping condition met: %d iterations performed\n\n",State.m_repinneriterationscount));
State.m_repterminationtype=5;
label=5;
break;
}
if(radius<=(State.m_epsx*State.m_agsraddecay))
{
//--- Radius is smaller than required step tolerance multiplied by radius decay.
//--- Additional decay is required in order to make sure that optimization session
//--- with radius equal to EpsX was successfully done.
if(dotrace)
CAp::Trace(StringFormat("> stopping condition met: sampling radius is smaller than %.3E\n\n",State.m_epsx));
State.m_repterminationtype=2;
label=5;
break;
}
//--- Update sample:
//--- 1. invalidate entries which are too far away from XC
//--- and move all valid entries to beginning of the sample.
//--- 2. add new entries until we have AGSSampleSize
//--- items in our sample. We remove oldest entries from
//--- sample until we have enough place to add at least
//--- AGSMinUpdate items.
//--- 3. prepare "modified" gradient sample with respect to
//--- boundary constraints.
if(!CAp::Assert(cursamplesize>=1,__FUNCTION__+": integrity check failed (2367)"))
return(false);
k=1;
for(i=1; i<cursamplesize; i++)
{
//--- If entry is outside of Radius-ball around XC, discard it.
v=0.0;
for(j=0; j<n; j++)
v=MathMax(v,MathAbs(State.m_samplex.Get(i,j)-State.m_xc[j]));
if(v>radius)
continue;
//--- Move to the beginning
CAblasF::RCopyRR(n,State.m_samplex,i,State.m_samplex,k);
CAblasF::RCopyRR(n,State.m_samplegm,i,State.m_samplegm,k);
State.m_samplef.Set(k,State.m_samplef[i]);
k++;
}
cursamplesize=k;
if(State.m_agssamplesize-cursamplesize<State.m_agsminupdate)
{
//--- Remove oldest entries
k=State.m_agsminupdate-(State.m_agssamplesize-cursamplesize);
if(!CAp::Assert(k<=cursamplesize-1,__FUNCTION__+": integrity check failed (2662)"))
return(false);
for(i=1; i<cursamplesize-k; i++)
{
CAblasF::RCopyRR(n,State.m_samplex,i+k,State.m_samplex,i);
CAblasF::RCopyRR(n,State.m_samplegm,i+k,State.m_samplegm,i);
State.m_samplef.Set(i,State.m_samplef[i+k]);
}
cursamplesize=cursamplesize-k;
}
k=0;
i=cursamplesize;
case 8:
if(i>MathMin(cursamplesize+State.m_agsminupdate,State.m_agssamplesize)-1)
{
label=10;
break;
}
for(j=0; j<n; j++)
{
//--- Undistorted position
State.m_samplex.Set(i,j,State.m_xc[j]);
//--- Do not apply distortion if the variable is fixed
if(State.m_HasBndL[j] && State.m_HasBndU[j] && State.m_scaledbndl[j]==State.m_scaledbndu[j])
continue;
//--- Apply distortion
if(CHighQualityRand::HQRndUniformR(State.m_agsrs)>=0.5)
{
//--- Sample at the left side with 50% probability
v0=State.m_samplex.Get(i,j)-radius;
v1=State.m_samplex.Get(i,j);
if(State.m_HasBndL[j])
v0=MathMax(State.m_scaledbndl[j],v0);
}
else
{
//--- Sample at the right side with 50% probability
v0=State.m_samplex.Get(i,j);
v1=State.m_samplex.Get(i,j)+radius;
if(State.m_HasBndU[j])
v1=MathMin(State.m_scaledbndu[j],v1);
}
if(!CAp::Assert(v1>=v0,__FUNCTION__+": integrity check failed (9743)"))
return(false);
State.m_samplex.Set(i,j,CApServ::BoundVal(v0+(v1-v0)*CHighQualityRand::HQRndUniformR(State.m_agsrs),v0,v1));
}
State.m_x=State.m_samplex[i]+0;
ClearRequestFields(State);
State.m_needfij=true;
State.m_rstateags.stage=2;
label=-1;
break;
case 2:
State.m_needfij=false;
State.m_samplef.Set(i,State.m_meritf);
CAblasF::RCopyVR(n,State.m_meritg,State.m_samplegm,i);
k++;
i++;
label=8;
break;
case 10:
cursamplesize+=k;
fullsample=cursamplesize==State.m_agssamplesize;
for(j=0; j<cursamplesize; j++)
{
//--- For J-th element in gradient sample, process all of its components
//--- and modify them according to status of box constraints
for(i=0; i<n; i++)
{
if(!CAp::Assert(!State.m_HasBndL[i] || State.m_xc[i]>=State.m_scaledbndl[i],__FUNCTION__+": integrity error"))
return(false);
if(!CAp::Assert(!State.m_HasBndU[i] || State.m_xc[i]<=State.m_scaledbndu[i],__FUNCTION__+": integrity error"))
return(false);
State.m_samplegmbc.Set(j,i,State.m_samplegm.Get(j,i));
if(State.m_HasBndL[i] && State.m_HasBndU[i] && State.m_scaledbndl[i]==State.m_scaledbndu[i])
{
//--- I-th box constraint is of equality type (lower bound matches upper one).
//--- Simplest case, always active.
State.m_samplegmbc.Set(j,i,0.0);
continue;
}
if(State.m_HasBndL[i] && State.m_xc[i]==State.m_scaledbndl[i])
{
//--- We are at lower bound: activate/deactivate constraint depending on gradient at XC
if(State.m_samplegm.Get(0,i)>=0.0)
State.m_samplegmbc.Set(j,i,0.0);
continue;
}
if(State.m_HasBndU[i] && State.m_xc[i]==State.m_scaledbndu[i])
{
//--- We are at upper bound: activate/deactivate constraint depending on gradient at XC
if(State.m_samplegm.Get(0,i)<=0.0)
State.m_samplegmbc.Set(j,i,0.0);
continue;
}
}
}
if(dotracesample)
{
CAp::Trace("> gradient sample\n");
for(i=0; i<=cursamplesize-1; i++)
{
CAp::Trace("SampleGrad[] = ");
CApServ::TraceRowAutopRec(State.m_samplegmbc,i,0,n);
CAp::Trace("\n");
}
}
//--- Calculate diagonal Hessian.
//--- This Hessian serves two purposes:
//--- * first, it improves performance of gradient descent step
//--- * second, it improves condition number of QP subproblem
//--- solved to determine step
//--- The idea is that for each variable we check whether sample
//--- includes entries with alternating sign of gradient:
//--- * if gradients with different signs are present, Hessian
//--- component is set to M/R, where M is a maximum magnitude
//--- of corresponding gradient component, R is a sampling radius.
//--- Note that sign=0 and sign=1 are treated as different ones
//--- * if all gradients have same sign, Hessian component is
//--- set to M/R0, where R0 is initial sampling radius.
State.m_colmax.Fill(0.0);
State.m_signmin.Fill(1);
State.m_signmax.Fill(-1);
for(i=0; i<cursamplesize; i++)
{
for(j=0; j<n; j++)
{
v=State.m_samplegmbc.Get(i,j);
State.m_colmax.Set(j,MathMax(State.m_colmax[j],MathAbs(v)));
State.m_signmin.Set(j,MathMin(State.m_signmin[j],MathSign(v)));
State.m_signmax.Set(j,MathMax(State.m_signmax[j],MathSign(v)));
}
}
for(j=0; j<n; j++)
{
if(State.m_signmin[j]!=State.m_signmax[j])
{
//--- Alternating signs of gradient - step is proportional to current sampling radius
if(!CAp::Assert(State.m_colmax[j]!=0.0,__FUNCTION__+": integrity check failed (2975)"))
return(false);
if(!CAp::Assert(radius!=0.0,__FUNCTION__+": integrity check failed (8473)"))
return(false);
State.m_diagh.Set(j,State.m_colmax[j]/radius);
continue;
}
if(State.m_colmax[j]!=0.0)
{
//--- Non-alternating sign of gradient, but non-zero.
//--- Step is proportional to recommended step
if(!CAp::Assert(recommendedstep!=0.0,__FUNCTION__+": integrity check failed (3274)"))
return(false);
State.m_diagh.Set(j,State.m_colmax[j]/recommendedstep);
continue;
}
State.m_diagh.Set(j,1);
}
if(dodetailedtrace)
{
CAp::Trace("> diagonal quasi-Hessian\n");
CAp::Trace("H = ");
CApServ::TraceVectorAutopRec(State.m_diagh,0,n);
CAp::Trace("\n");
}
//--- PROJECTION PHASE
//--- We project zero vector on convex hull of gradient sample.
//--- If projection is small enough, we decrease radius and restart.
//--- Otherwise, this phase returns search direction in State.D.
//--- NOTE: because we use iterative solver, it may have trouble
//--- dealing with ill-conditioned problems. So we also employ
//--- second, backup test for stationarity - when too many
//--- subsequent backtracking searches resulted in short steps.
SolveQP(State.m_samplegmbc,State.m_diagh,cursamplesize,n,State.m_tmp0,State.m_dbgncholesky,State.m_nsqp);
State.m_d.Fill(0.0);
for(i=0; i<cursamplesize; i++)
{
v=State.m_tmp0[i];
for(i_=0; i_<n; i_++)
State.m_d.Add(i_,State.m_samplegmbc.Get(i,i_)*v);
}
v=0.0;
for(j=0; j<n; j++)
v=MathMax(v,MathAbs(State.m_d[j]/CApServ::Coalesce(State.m_colmax[j],1.0)));
if(dotrace)
CAp::Trace(StringFormat("> stationarity test:\n|proj(0)| = %.3E (projection of zero vector on convex hull of gradient sample)\n",v));
if(v<=State.m_agsstattold)
{
//--- Stationarity test succeeded.
//--- Decrease radius and restart.
//--- NOTE: we also clear ShortStepsCnt on restart
if(dotrace)
CAp::Trace("> stationarity test satisfied,decreasing radius\n");
radius*=State.m_agsraddecay;
shortstepscnt=0;
radiusdecays++;
State.m_repinneriterationscount++;
label=4;
break;
}
for(i=0; i<n; i++)
State.m_d.Mul(i,(-1.0)/State.m_diagh[i]);
//--- Perform backtracking line search.
//--- Update initial step length depending on search results.
//--- Here we assume that D is non-zero.
//--- NOTE: if AGSShortLimit subsequent line searches resulted
//--- in steps shorter than AGSStatTolStp, we decrease radius.
dhd=0;
for(i=0; i<n; i++)
dhd+=State.m_diagh[i]*CMath::Sqr(State.m_d[i]);
dnrminf=CAblasF::RMaxAbsV(n,State.m_d);
if(dotrace)
{
CAp::Trace(StringFormat("> search direction is ready:\n|D| = %.3E (inf-norm)\n(D,grad) = %.3E\n",dnrminf,CAblasF::RDotVR(n,State.m_d,State.m_samplegmbc,0)));
if(dodetailedtrace)
{
CAp::Trace("D = ");
CApServ::TraceVectorAutopRec(State.m_d,0,n);
CAp::Trace("\n");
}
}
if(!CAp::Assert(dnrminf>0.0,__FUNCTION__+": integrity error (2752)"))
return(false);
alpha=recommendedstep/dnrminf;
alphadecreased=false;
backtrackits=0;
if(fullsample)
maxbacktrackits=State.m_agsmaxbacktrack;
else
maxbacktrackits=State.m_agsmaxbacktracknonfull;
case 11:
//--- Prepare XN and evaluate merit function at XN
State.m_xn=State.m_xc.ToVector()+State.m_d*alpha;
COptServ::EnforceBoundaryConstraints(State.m_xn,State.m_scaledbndl,State.m_HasBndL,State.m_scaledbndu,State.m_HasBndU,n,0);
State.m_samplex.Row(maxsamplesize,State.m_xn);
State.m_x=State.m_xn;
ClearRequestFields(State);
State.m_needfij=true;
State.m_rstateags.stage=3;
label=-1;
break;
case 3:
State.m_needfij=false;
State.m_samplef.Set(maxsamplesize,State.m_meritf);
State.m_samplegm.Row(maxsamplesize,State.m_meritg);
//--- Check sufficient decrease condition
if(!CAp::Assert(dnrminf>0.0,__FUNCTION__+": integrity error (9642)"))
return(false);
if(State.m_samplef[maxsamplesize]<=(State.m_samplef[0]-alpha*State.m_agsdecrease*dhd))
{
label=12;
break;
}
//--- Decrease Alpha
alpha*=State.m_agsalphadecay;
alphadecreased=true;
//--- Update and check iterations counter.
backtrackits++;
if(backtrackits>=maxbacktrackits)
{
//--- Too many backtracking searches performed without success.
//--- Terminate iterations.
alpha=0.0;
alphadecreased=true;
State.m_xn=State.m_xc;
label=12;
break;
}
label=11;
break;
case 12:
if(dotrace)
CAp::Trace(StringFormat("> backtracking line search finished:\nstp = %.3E\n",alpha));
if((alpha*dnrminf)<=State.m_agsshortstpabs || (alpha*dnrminf)<=(State.m_agsshortstprel*radius) || MathAbs(State.m_samplef[0]-State.m_samplef[maxsamplesize])<=State.m_agsshortf)
shortstepscnt++;
else
shortstepscnt=0;
if(shortstepscnt>=State.m_agsshortlimit)
{
//--- Too many subsequent short steps.
//--- It may be possible that optimizer is unable to find out
//--- that we have to decrease radius because of ill-conditioned
//--- gradients.
//--- Decrease radius and restart.
if(dotrace)
CAp::Trace("> too many subsequent short steps,decreasing radius\n");
radius*=State.m_agsraddecay;
shortstepscnt=0;
radiusdecays++;
State.m_repinneriterationscount++;
label=4;
break;
}
if(!alphadecreased)
recommendedstep*=2.0;
if(alphadecreased && fullsample)
recommendedstep*=0.5;
//--- Next iteration
State.m_xc=State.m_xn;
State.m_repinneriterationscount++;
label=4;
break;
case 5:
//--- Convert back from scaled to unscaled representation
UnscalePointBC(State,State.m_xc);
result=false;
return(result);
}
}
//--- Saving State
result=true;
State.m_rstateags.ba[0]=b;
State.m_rstateags.ba[1]=alphadecreased;
State.m_rstateags.ba[2]=fullsample;
State.m_rstateags.ba[3]=dotrace;
State.m_rstateags.ba[4]=dodetailedtrace;
State.m_rstateags.ba[5]=dotracesample;
State.m_rstateags.ia.Set(0,n);
State.m_rstateags.ia.Set(1,nec);
State.m_rstateags.ia.Set(2,nic);
State.m_rstateags.ia.Set(3,ng);
State.m_rstateags.ia.Set(4,nh);
State.m_rstateags.ia.Set(5,i);
State.m_rstateags.ia.Set(6,j);
State.m_rstateags.ia.Set(7,k);
State.m_rstateags.ia.Set(8,radiusdecays);
State.m_rstateags.ia.Set(9,maxsamplesize);
State.m_rstateags.ia.Set(10,cursamplesize);
State.m_rstateags.ia.Set(11,shortstepscnt);
State.m_rstateags.ia.Set(12,backtrackits);
State.m_rstateags.ia.Set(13,maxbacktrackits);
State.m_rstateags.ra.Set(0,radius0);
State.m_rstateags.ra.Set(1,radius);
State.m_rstateags.ra.Set(2,alpha);
State.m_rstateags.ra.Set(3,recommendedstep);
State.m_rstateags.ra.Set(4,dhd);
State.m_rstateags.ra.Set(5,dnrminf);
State.m_rstateags.ra.Set(6,v);
State.m_rstateags.ra.Set(7,vv);
State.m_rstateags.ra.Set(8,v0);
State.m_rstateags.ra.Set(9,v1);
State.m_rstateags.ra.Set(10,currentf0);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function performs transformation of X from scaled |
//| coordinates to unscaled ones, paying special attention to box |
//| constraints: |
//| * points which were exactly at the boundary before scaling will|
//| be mapped to corresponding boundary after scaling |
//| * in any case, unscaled box constraints will be satisfied |
//+------------------------------------------------------------------+
void CMinNS::UnscalePointBC(CMinNSState &State,CRowDouble &x)
{
for(int i=0; i<State.m_n; i++)
{
if(State.m_HasBndL[i] && x[i]<=State.m_scaledbndl[i])
{
x.Set(i,State.m_bndl[i]);
continue;
}
if(State.m_HasBndU[i] && x[i]>=State.m_scaledbndu[i])
{
x.Set(i,State.m_bndu[i]);
continue;
}
x.Mul(i,State.m_s[i]);
if(State.m_HasBndL[i] && x[i]<=State.m_bndl[i])
x.Set(i,State.m_bndl[i]);
if(State.m_HasBndU[i] && x[i]>=State.m_bndu[i])
x.Set(i,State.m_bndu[i]);
}
}
//+------------------------------------------------------------------+
//| This function solves QP problem of the form |
//| [ ] |
//| min [0.5 * c'*(G*inv(H)*G')*c ] s.t. c[i] >= 0, SUM(c[i]) = 1.0 |
//| [ ] |
//| where G is stored in SampleG[] array, diagonal H is stored in |
//| DiagH[]. |
//| DbgNCholesky is incremented every time we perform Cholesky |
//| decomposition. |
//+------------------------------------------------------------------+
void CMinNS::SolveQP(CMatrixDouble &sampleg,CRowDouble &diagh,
int nsample,int nvars,CRowDouble &coeffs,
int &dbgncholesky,CMinNSQP &State)
{
//--- create variables
int n=nsample;
int i=0;
int j=0;
int k=0;
double v=0;
double vv=0;
int idx0=0;
int idx1=0;
int ncandbnd=0;
int innerits=0;
int outerits=0;
double dnrm=0;
double stp=0;
double stpmax=0;
int actidx=0;
double dtol=0;
bool kickneeded=false;
double kicklength=0;
double lambdav=0;
double maxdiag=0;
bool wasactivation=false;
bool werechanges=false;
int termcnt=0;
int i_=0;
//--- Allocate arrays, prepare data
coeffs=vector<double>::Full(n,1.0/(double)n);
State.m_xc=vector<double>::Full(n,1.0/(double)n);
CApServ::RVectorSetLengthAtLeast(State.m_xn,n);
CApServ::RVectorSetLengthAtLeast(State.m_x0,n);
CApServ::RVectorSetLengthAtLeast(State.m_gc,n);
CApServ::RVectorSetLengthAtLeast(State.m_d,n);
CApServ::RMatrixSetLengthAtLeast(State.m_uh,n,n);
CApServ::RMatrixSetLengthAtLeast(State.m_ch,n,n);
CApServ::RMatrixSetLengthAtLeast(State.m_rk,nsample,nvars);
CApServ::RVectorSetLengthAtLeast(State.m_invutc,n);
CApServ::RVectorSetLengthAtLeast(State.m_tmp0,n);
ArrayResize(State.m_tmpb,n);
for(i=0; i<nsample; i++)
{
for(j=0; j<nvars; j++)
State.m_rk.Set(i,j,sampleg.Get(i,j)/MathSqrt(diagh[j]));
}
CAblas::RMatrixSyrk(nsample,nvars,1.0,State.m_rk,0,0,0,0.0,State.m_uh,0,0,true);
maxdiag=(State.m_uh.Diag()+0).Max();
maxdiag=CApServ::Coalesce(maxdiag,1.0);
//--- Main cycle:
innerits=0;
outerits=0;
dtol=1.0E5*CMath::m_machineepsilon;
kicklength=CMath::m_machineepsilon;
lambdav=1.0E5*CMath::m_machineepsilon;
termcnt=0;
while(true)
{
//--- Save current point to X0
State.m_x0=State.m_xc;
//--- Calculate gradient at initial point, solve NNLS problem
//--- to determine descent direction D subject to constraints.
//--- In order to do so we solve following constrained
//--- minimization problem:
//--- ( )^2
//--- min ( SUM(lambda[i]*A[i]) + G )
//--- ( )
//--- Here:
//--- * G is a gradient (column vector)
//--- * A[i] is a column vector of I-th constraint
//--- * lambda[i] is a Lagrange multiplier corresponding to I-th constraint
//--- NOTE: all A[i] except for last one have only one element being set,
//--- so we rely on sparse capabilities of NNLS solver. However,
//--- in order to use these capabilities we have to reorder variables
//--- in such way that sparse ones come first.
//--- After finding lambda[] coefficients, we can find constrained descent
//--- direction by subtracting lambda[i]*A[i] from D=-G. We make use of the
//--- fact that first NCandBnd columns are just columns of identity matrix,
//--- so we can perform exact projection by explicitly setting elements of D
//--- to zeros.
QPCalculateGradFunc(sampleg,diagh,nsample,nvars,State.m_xc,State.m_gc,State.m_fc,State.m_tmp0);
CApServ::IVectorSetLengthAtLeast(State.m_tmpidx,n);
CApServ::RVectorSetLengthAtLeast(State.m_tmpd,n);
CApServ::RMatrixSetLengthAtLeast(State.m_tmpc2,n,1);
idx0=0;
ncandbnd=0;
for(i=0; i<n; i++)
if(State.m_xc[i]==0.0)
ncandbnd++;
idx1=ncandbnd;
for(i=0; i<n; i++)
{
if(State.m_xc[i]==0.0)
{
//--- Candidate for activation of boundary constraint,
//--- comes first.
//--- NOTE: multiplication by -1 is due to the fact that
//--- it is lower bound, and has specific direction
//--- of constraint gradient.
State.m_tmpidx.Set(idx0,i);
State.m_tmpd.Set(idx0,State.m_gc[i]);
State.m_tmpc2.Set(idx0,0,-1.0);
idx0++;
}
else
{
//--- We are far away from boundary.
State.m_tmpidx.Set(idx1,i);
State.m_tmpd.Set(idx1,State.m_gc[i]);
State.m_tmpc2.Set(idx1,0,-1.0);
idx1++;
}
}
if(!CAp::Assert(idx0==ncandbnd,__FUNCTION__+": integrity check failed (2346)"))
return;
if(!CAp::Assert(idx1==n,__FUNCTION__+": integrity check failed (4535)"))
return;
CSNNLS::SNNLSInit(n,1,n,State.m_nnls);
CSNNLS::SNNLSSetProblem(State.m_nnls,State.m_tmpc2,State.m_tmpd,ncandbnd,1,n);
CSNNLS::SNNLSDropNNC(State.m_nnls,ncandbnd);
CSNNLS::SNNLSSolve(State.m_nnls,State.m_tmplambdas);
State.m_d=State.m_gc*(-1.0)-State.m_tmplambdas[ncandbnd];
for(i=0; i<ncandbnd; i++)
{
if(State.m_tmplambdas[i]>0.0)
State.m_d.Set(State.m_tmpidx[i],0.0);
}
//--- Additional stage to "polish" D (improve situation
//--- with sum-to-one constraint and boundary constraints)
//--- and to perform additional integrity check.
//--- After this stage we are pretty sure that:
//--- * if x[i]=0.0, then d[i]>=0.0
//--- * if d[i]<0.0, then x[i]>0.0
for(i=0; i<n; i++)
{
if(State.m_xc[i]==0.0 && State.m_d[i]<0.0)
State.m_d.Set(i,0.0);
}
//--- Decide whether we need "kick" stage: special stage
//--- that moves us away from boundary constraints which are
//--- not strictly active (i.e. such constraints that x[i]=0.0 and d[i]>0).
//--- If we need kick stage, we make a kick - and restart iteration.
//--- If not, after this block we can rely on the fact that
//--- for all x[i]=0.0 we have d[i]=0.0
kickneeded=false;
for(i=0; i<n; i++)
{
if(State.m_xc[i]==0.0 && State.m_d[i]>0.0)
kickneeded=true;
}
if(kickneeded)
{
//--- Perform kick.
//--- Restart.
//--- Do not increase outer iterations counter.
v=0.0;
for(i=0; i<n; i++)
{
if(State.m_xc[i]==0.0 && State.m_d[i]>0.0)
State.m_xc.Set(i,kicklength);
v+=State.m_xc[i];
}
if(!CAp::Assert(v>0.0,__FUNCTION__+": integrity check failed (2572)"))
return;
State.m_xc/=v;
innerits++;
continue;
}
//--- Calculate Cholesky decomposition of constrained Hessian
//--- for Newton phase.
while(true)
{
for(i=0; i<n; i++)
{
//--- Diagonal element
if(State.m_xc[i]>0.0)
State.m_ch.Set(i,i,State.m_uh.Get(i,i)+lambdav*maxdiag);
else
State.m_ch.Set(i,i,1.0);
//--- Offdiagonal elements
for(j=i+1; j<n; j++)
{
if(State.m_xc[i]>0.0 && State.m_xc[j]>0.0)
State.m_ch.Set(i,j,State.m_uh.Get(i,j));
else
State.m_ch.Set(i,j,0.0);
}
}
dbgncholesky++;
if(!CTrFac::SPDMatrixCholeskyRec(State.m_ch,0,n,true,State.m_tmp0))
{
//--- Cholesky decomposition failed.
//--- Increase LambdaV and repeat iteration.
//--- Do not increase outer iterations counter.
lambdav*=10;
continue;
}
break;
}
//--- Newton phase
while(true)
{
//--- Calculate constrained (equality and sum-to-one) descent direction D.
//--- Here we use Sherman-Morrison update to calculate direction subject to
//--- sum-to-one constraint.
QPCalculateGradFunc(sampleg,diagh,nsample,nvars,State.m_xc,State.m_gc,State.m_fc,State.m_tmp0);
for(i=0; i<n; i++)
{
if(State.m_xc[i]>0.0)
{
State.m_invutc.Set(i,1.0);
State.m_d.Set(i,-State.m_gc[i]);
}
else
{
State.m_invutc.Set(i,0.0);
State.m_d.Set(i,0.0);
}
}
QPSolveUT(State.m_ch,n,State.m_invutc);
QPSolveUT(State.m_ch,n,State.m_d);
v=CAblasF::RDotV(n,State.m_invutc,State.m_d);
vv=CAblasF::RDotV2(n,State.m_invutc);
for(i=0; i<n; i++)
State.m_d.Add(i,-State.m_invutc[i]*v/vv);
QPSolveU(State.m_ch,n,State.m_d);
v=State.m_d.Sum();
k=0;
for(i=0; i<n; i++)
{
if(State.m_d[i]!=0.0)
k++;
}
if(k>0 && v>0.0)
{
vv=v/k;
for(i=0; i<n; i++)
{
if(State.m_d[i]!=0.0)
State.m_d.Add(i,-vv);
}
}
//--- Calculate length of D, maximum step and component which is
//--- activated by this step.
//--- Break if D is exactly zero. We do not break here if DNrm is
//--- small - this check is performed later. It is important to
//--- perform last step with nearly-zero D, it allows us to have
//--- extra-precision in solution which is often needed for convergence
//--- of AGS algorithm.
dnrm=State.m_d.Dot(State.m_d);
dnrm=MathSqrt(dnrm);
actidx=-1;
stpmax=1.0E50;
for(i=0; i<n; i++)
{
if(State.m_d[i]<0.0)
{
v=stpmax;
stpmax=CApServ::SafeMinPosRV(State.m_xc[i],-State.m_d[i],stpmax);
if(stpmax<v)
actidx=i;
}
}
if(dnrm==0.0)
break;
//--- Calculate trial function value at unconstrained full step.
//--- If trial value is greater or equal to FC, terminate iterations.
State.m_xn=State.m_xc+State.m_d+0;
QPCalculateFunc(sampleg,diagh,nsample,nvars,State.m_xn,State.m_fn,State.m_tmp0);
if(State.m_fn>=State.m_fc)
break;
//--- Perform step
//--- Update Hessian
//--- Update XC
//--- Break if:
//--- a) no constraint was activated
//--- b) norm of D is small enough
stp=MathMin(1.0,stpmax);
for(i=0; i<n; i++)
State.m_xn.Set(i,MathMax(State.m_xc[i]+stp*State.m_d[i],0.0));
if(stp==stpmax && actidx>=0)
State.m_xn.Set(actidx,0.0);
wasactivation=false;
for(i=0; i<n; i++)
{
State.m_tmpb[i]=(State.m_xn[i]==0.0 && State.m_xc[i]!=0.0);
wasactivation=wasactivation || State.m_tmpb[i];
}
State.m_xc=State.m_xn;
if(!wasactivation)
break;
if(dnrm<=dtol)
break;
CTrFac::SPDMatrixCholeskyUpdateFixBuf(State.m_ch,n,true,State.m_tmpb,State.m_tmp0);
}
//--- Compare status of boundary constraints - if nothing changed during
//--- last outer iteration, TermCnt is increased. Otherwise it is reset
//--- to zero.
//--- When TermCnt is large enough, we terminate algorithm.
werechanges=false;
for(i=0; i<n; i++)
{
werechanges=werechanges || MathSign(State.m_x0[i])!=MathSign(State.m_xc[i]);
}
if(!werechanges)
termcnt++;
else
termcnt=0;
if(termcnt>=2)
break;
//--- Increase number of outer iterations.
//--- Break if we performed too many.
outerits++;
if(outerits==10)
break;
}
//--- Store result
coeffs=State.m_xc;
}
//+------------------------------------------------------------------+
//| Function / gradient calculation for QP solver. |
//+------------------------------------------------------------------+
void CMinNS::QPCalculateGradFunc(CMatrixDouble &sampleg,CRowDouble &diagh,
int nsample,int nvars,CRowDouble &coeffs,
CRowDouble &g,double &f,CRowDouble &tmp)
{
//--- create variables
vector<double> c=coeffs.ToVector();
vector<double> d=diagh.ToVector();
matrix<double> s=sampleg.ToMatrix();
c.Resize(nsample);
d.Resize(nvars);
s.Resize(nsample,nvars);
f=0;
//--- Calculate GS*p
tmp=c.MatMul(s);
//--- Calculate F
f=0.5*(tmp.Pow(2.0)/d).Sum();
//--- Multiply by inverse Hessian
tmp/=d;
//--- Function gradient
g=tmp.MatMul(s.Transpose())+0;
}
//+------------------------------------------------------------------+
//| Function calculation for QP solver. |
//+------------------------------------------------------------------+
void CMinNS::QPCalculateFunc(CMatrixDouble &sampleg,CRowDouble &diagh,
int nsample,int nvars,CRowDouble &coeffs,
double &f,CRowDouble &tmp)
{
//--- create variables
vector<double> c=coeffs.ToVector();
vector<double> d=diagh.ToVector();
matrix<double> s=sampleg.ToMatrix();
c.Resize(nsample);
d.Resize(nvars);
s.Resize(nsample,nvars);
f=0;
//--- Calculate GS*p
tmp=c.MatMul(s);
//--- Calculate F
f=0.5*(tmp.Pow(2.0)/d).Sum();
}
//+------------------------------------------------------------------+
//| Triangular solver for QP solver. |
//+------------------------------------------------------------------+
void CMinNS::QPSolveU(CMatrixDouble &a,int n,CRowDouble &x)
{
double v=0;
//--- A^(-1)*X
for(int i=n-1; i>=0; i--)
{
v=x[i];
for(int j=i+1; j<n; j++)
v-=a.Get(i,j)*x[j];
x.Set(i,v/a.Get(i,i));
}
}
//+------------------------------------------------------------------+
//| Triangular solver for QP solver. |
//+------------------------------------------------------------------+
void CMinNS::QPSolveUT(CMatrixDouble &a,int n,CRowDouble &x)
{
double v=0;
//--- A^(-T)*X
for(int i=0; i<n; i++)
{
x.Set(i,x[i]/a.Get(i,i));
v=x[i];
for(int j=i+1; j<n; j++)
x.Add(j,-a.Get(i,j)*v);
}
}
//+------------------------------------------------------------------+
//| This object stores nonlinear optimizer State.m_ |
//| You should use functions provided by MinBC subpackage to work |
//| with this object |
//+------------------------------------------------------------------+
struct CMinBCState
{
int m_bufsize;
int m_maxits;
int m_mcstage;
int m_nfev;
int m_nmain;
int m_nonmonotoniccnt;
int m_prectype;
int m_repiterationscount;
int m_repnfev;
int m_repterminationtype;
int m_repvaridx;
int m_smoothnessguardlevel;
double m_curstpmax;
double m_diffstep;
double m_epsf;
double m_epsg;
double m_epsx;
double m_f;
double m_fbase;
double m_fc;
double m_fm1;
double m_fm2;
double m_fn;
double m_fp1;
double m_fp2;
double m_fp;
double m_gm1;
double m_gp1;
double m_lastscaledgoodstep;
double m_stp;
double m_stpmax;
double m_teststep;
double m_trimthreshold;
double m_xm1;
double m_xp1;
bool m_HasBndL[];
bool m_HasBndU[];
bool m_needf;
bool m_needfg;
bool m_userterminationneeded;
bool m_xrep;
bool m_xupdated;
RCommState m_rstate;
CSmoothnessMonitor m_smonitor;
CRowDouble m_bndl;
CRowDouble m_bndu;
CRowDouble m_bufrho;
CRowDouble m_buftheta;
CRowDouble m_cgc;
CRowDouble m_cgn;
CRowDouble m_d;
CRowDouble m_diagh;
CRowDouble m_g;
CRowDouble m_invs;
CRowDouble m_lastscaleused;
CRowDouble m_s;
CRowDouble m_tmp0;
CRowDouble m_tmpprec;
CRowDouble m_ugc;
CRowDouble m_ugn;
CRowDouble m_work;
CRowDouble m_x;
CRowDouble m_xc;
CRowDouble m_xn;
CRowDouble m_xp;
CRowDouble m_xstart;
CMatrixDouble m_bufsk;
CMatrixDouble m_bufyk;
CLinMinState m_lstate;
//--- constructor / destructor
CMinBCState(void);
~CMinBCState(void) {}
//---
void Copy(const CMinBCState &obj);
//--- overloading
void operator=(const CMinBCState &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CMinBCState::CMinBCState(void)
{
m_bufsize=0;
m_maxits=0;
m_mcstage=0;
m_nfev=0;
m_nmain=0;
m_nonmonotoniccnt=0;
m_prectype=0;
m_repiterationscount=0;
m_repnfev=0;
m_repterminationtype=0;
m_repvaridx=0;
m_smoothnessguardlevel=0;
m_curstpmax=0;
m_diffstep=0;
m_epsf=0;
m_epsg=0;
m_epsx=0;
m_f=0;
m_fbase=0;
m_fc=0;
m_fm1=0;
m_fm2=0;
m_fn=0;
m_fp1=0;
m_fp2=0;
m_fp=0;
m_gm1=0;
m_gp1=0;
m_lastscaledgoodstep=0;
m_stp=0;
m_stpmax=0;
m_teststep=0;
m_trimthreshold=0;
m_xm1=0;
m_xp1=0;
m_needf=false;
m_needfg=false;
m_userterminationneeded=false;
m_xrep=false;
m_xupdated=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CMinBCState::Copy(const CMinBCState &obj)
{
m_bufsize=obj.m_bufsize;
m_maxits=obj.m_maxits;
m_mcstage=obj.m_mcstage;
m_nfev=obj.m_nfev;
m_nmain=obj.m_nmain;
m_nonmonotoniccnt=obj.m_nonmonotoniccnt;
m_prectype=obj.m_prectype;
m_repiterationscount=obj.m_repiterationscount;
m_repnfev=obj.m_repnfev;
m_repterminationtype=obj.m_repterminationtype;
m_repvaridx=obj.m_repvaridx;
m_smoothnessguardlevel=obj.m_smoothnessguardlevel;
m_curstpmax=obj.m_curstpmax;
m_diffstep=obj.m_diffstep;
m_epsf=obj.m_epsf;
m_epsg=obj.m_epsg;
m_epsx=obj.m_epsx;
m_f=obj.m_f;
m_fbase=obj.m_fbase;
m_fc=obj.m_fc;
m_fm1=obj.m_fm1;
m_fm2=obj.m_fm2;
m_fn=obj.m_fn;
m_fp1=obj.m_fp1;
m_fp2=obj.m_fp2;
m_fp=obj.m_fp;
m_gm1=obj.m_gm1;
m_gp1=obj.m_gp1;
m_lastscaledgoodstep=obj.m_lastscaledgoodstep;
m_stp=obj.m_stp;
m_stpmax=obj.m_stpmax;
m_teststep=obj.m_teststep;
m_trimthreshold=obj.m_trimthreshold;
m_xm1=obj.m_xm1;
m_xp1=obj.m_xp1;
ArrayCopy(m_HasBndL,obj.m_HasBndL);
ArrayCopy(m_HasBndU,obj.m_HasBndU);
m_needf=obj.m_needf;
m_needfg=obj.m_needfg;
m_userterminationneeded=obj.m_userterminationneeded;
m_xrep=obj.m_xrep;
m_xupdated=obj.m_xupdated;
m_rstate=obj.m_rstate;
m_smonitor=obj.m_smonitor;
m_bndl=obj.m_bndl;
m_bndu=obj.m_bndu;
m_bufrho=obj.m_bufrho;
m_buftheta=obj.m_buftheta;
m_cgc=obj.m_cgc;
m_cgn=obj.m_cgn;
m_d=obj.m_d;
m_diagh=obj.m_diagh;
m_g=obj.m_g;
m_invs=obj.m_invs;
m_lastscaleused=obj.m_lastscaleused;
m_s=obj.m_s;
m_tmp0=obj.m_tmp0;
m_tmpprec=obj.m_tmpprec;
m_ugc=obj.m_ugc;
m_ugn=obj.m_ugn;
m_work=obj.m_work;
m_x=obj.m_x;
m_xc=obj.m_xc;
m_xn=obj.m_xn;
m_xp=obj.m_xp;
m_xstart=obj.m_xstart;
m_bufsk=obj.m_bufsk;
m_bufyk=obj.m_bufyk;
m_lstate=obj.m_lstate;
}
//+------------------------------------------------------------------+
//| This structure stores optimization report: |
//| * iterationscount number of iterations |
//| * nfev number of gradient evaluations |
//| * terminationtype termination type (see below) |
//| TERMINATION CODES |
//| Terminationtype field contains completion code, which can be: |
//| -8 internal integrity control detected infinite or NAN |
//| values in function/gradient. Abnormal termination |
//| signalled. |
//| -3 inconsistent constraints. |
//| 1 relative function improvement is no more than EpsF. |
//| 2 relative step is no more than EpsX. |
//| 4 gradient norm is no more than EpsG |
//| 5 MaxIts steps was taken |
//| 7 stopping conditions are too stringent, further |
//| improvement is impossible, X contains best point |
//| found so far. |
//| 8 terminated by user who called |
//| MinBCRequestTermination(). X contains point which was |
//| "current accepted" when termination request was |
//| submitted. |
//+------------------------------------------------------------------+
struct CMinBCReport
{
int m_iterationscount;
int m_nfev;
int m_terminationtype;
int m_varidx;
//--- constructor / destructor
CMinBCReport(void) { ZeroMemory(this); }
~CMinBCReport(void) {}
void Copy(const CMinBCReport &obj);
//--- overloading
void operator=(const CMinBCReport &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CMinBCReport::Copy(const CMinBCReport &obj)
{
m_iterationscount=obj.m_iterationscount;
m_nfev=obj.m_nfev;
m_terminationtype=obj.m_terminationtype;
m_varidx=obj.m_varidx;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CMinBC
{
public:
//--- constants
static const double m_gtol;
static const double m_maxnonmonotoniclen;
static const double m_initialdecay;
static const double m_mindecay;
static const double m_decaycorrection;
static void MinBCCreate(int n,CRowDouble &x,CMinBCState &State);
static void MinBCCreateF(int n,CRowDouble &x,double diffstep,CMinBCState &State);
static void MinBCSetBC(CMinBCState &State,CRowDouble &bndl,CRowDouble &bndu);
static void MinBCSetCond(CMinBCState &State,double epsg,double epsf,double epsx,int maxits);
static void MinBCSetScale(CMinBCState &State,CRowDouble &s);
static void MinBCSetPrecDefault(CMinBCState &State);
static void MinBCSetPrecDiag(CMinBCState &State,CRowDouble &d);
static void MinBCSetPrecScale(CMinBCState &State);
static void MinBCSetXRep(CMinBCState &State,bool needxrep);
static void MinBCSetStpMax(CMinBCState &State,double stpmax);
static bool MinBCIteration(CMinBCState &State);
static void MinBCOptGuardGradient(CMinBCState &State,double teststep);
static void MinBCOptGuardSmoothness(CMinBCState &State,int level);
static void MinBCOptGuardResults(CMinBCState &State,COptGuardReport &rep);
static void MinBCOptGuardNonC1Test0Results(CMinBCState &State,COptGuardNonC1Test0Report &strrep,COptGuardNonC1Test0Report &lngrep);
static void MinBCOptGuardNonC1Test1Results(CMinBCState &State,COptGuardNonC1Test1Report &strrep,COptGuardNonC1Test1Report &lngrep);
static void MinBCResults(CMinBCState &State,CRowDouble &x,CMinBCReport &rep);
static void MinBCResultsBuf(CMinBCState &State,CRowDouble &x,CMinBCReport &rep);
static void MinBCRestartFrom(CMinBCState &State,CRowDouble &x);
static void MinBCRequestTermination(CMinBCState &State);
protected:
static void ClearRequestFields(CMinBCState &State);
static void MinBCInitInternal(int n,CRowDouble &x,double diffstep,CMinBCState &State);
static void UpdateEstimateOfGoodStep(double &estimate,double newstep);
};
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
const double CMinBC::m_gtol=0.4;
const double CMinBC::m_maxnonmonotoniclen=1.0E-5;
const double CMinBC::m_initialdecay=0.5;
const double CMinBC::m_mindecay=0.1;
const double CMinBC::m_decaycorrection=0.8;
//+------------------------------------------------------------------+
//| BOX CONSTRAINED OPTIMIZATION WITH FAST ACTIVATION OF MULTIPLE BOX|
//| CONSTRAINTS |
//| DESCRIPTION: |
//| The subroutine minimizes function F(x) of N arguments subject to |
//| box constraints (with some of box constraints actually being |
//| equality ones). |
//| This optimizer uses algorithm similar to that of MinBLEIC |
//| (optimizer with general linear constraints), but presence of |
//| box-only constraints allows us to use faster constraint |
//| activation strategies. On large-scale problems, with multiple |
//| constraints active at the solution, this optimizer can be several|
//| times faster than BLEIC. |
//| REQUIREMENTS: |
//| * user must provide function value and gradient |
//| * starting point X0 must be feasible or not too far away from |
//| the feasible set |
//| * grad(f) must be Lipschitz continuous on a level set: |
//| L = { x : f(x) <= f(x0) } |
//| * function must be defined everywhere on the feasible set F |
//| USAGE: |
//| Constrained optimization if far more complex than the |
//| unconstrained one. Here we give very brief outline of the BC |
//| optimizer. We strongly recommend you to read examples in the |
//| ALGLIB Reference Manual and to read ALGLIB User Guide on |
//| optimization, which is available at |
//| http://www.alglib.net/optimization/ |
//| 1. User initializes algorithm State with MinBCCreate() call |
//| 2. USer adds box constraints by calling MinBCSetBC() function. |
//| 3. User sets stopping conditions with MinBCSetCond(). |
//| 4. User calls MinBCOptimize() function which takes algorithm |
//| State and pointer (delegate, etc.) to callback function which |
//| calculates F / G. |
//| 5. User calls MinBCResults() to get solution |
//| 6. Optionally user may call MinBCRestartFrom() to solve another |
//| problem with same N but another starting point. |
//| MinBCRestartFrom() allows to reuse already initialized |
//| structure. |
//| INPUT PARAMETERS: |
//| N - problem dimension, N > 0: |
//| * if given, only leading N elements of X are used |
//| * if not given, automatically determined from size |
//| ofX |
//| X - starting point, array[N]: |
//| * it is better to set X to a feasible point |
//| * but X can be infeasible, in which case algorithm |
//| will try to find feasible point first, using X as|
//| initial approximation. |
//| OUTPUT PARAMETERS: |
//| State - structure stores algorithm State |
//+------------------------------------------------------------------+
void CMinBC::MinBCCreate(int n,CRowDouble &x,CMinBCState &State)
{
CMatrixDouble c;
CRowInt ct;
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
MinBCInitInternal(n,x,0.0,State);
}
//+------------------------------------------------------------------+
//| The subroutine is finite difference variant of MinBCCreate(). It |
//| uses finite differences in order to differentiate target function|
//| Description below contains information which is specific to this|
//| function only. We recommend to read comments on MinBCCreate() in |
//| order to get more information about creation of BC optimizer. |
//| INPUT PARAMETERS: |
//| N - problem dimension, N > 0: |
//| * if given, only leading N elements of X are used |
//| * if not given, automatically determined from size |
//| of X |
//| X - starting point, array[0..N - 1]. |
//| DiffStep - differentiation step, > 0 |
//| OUTPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NOTES: |
//| 1. algorithm uses 4-point central formula for differentiation. |
//| 2. differentiation step along I-th axis is equal to |
//| DiffStep*S[I] where S[] is scaling vector which can be set |
//| by MinBCSetScale() call. |
//| 3. we recommend you to use moderate values of differentiation |
//| step. Too large step will result in too large truncation |
//| errors, while too small step will result in too large |
//| numerical errors. 1.0E-6 can be good value to start with. |
//| 4. Numerical differentiation is very inefficient - one gradient |
//| calculation needs 4*N function evaluations. This function will|
//| work for any N - either small(1...10), moderate(10...100) or |
//| large(100...). However, performance penalty will be too severe|
//| for any N's except for small ones. |
//| We should also say that code which relies on numerical |
//| differentiation is less robust and precise. CG needs exact |
//| gradient values. Imprecise gradient may slow down convergence,|
//| especially on highly nonlinear problems. |
//| Thus we recommend to use this function for fast prototyping on|
//| small-dimensional problems only, and to implement analytical |
//| gradient as soon as possible. |
//+------------------------------------------------------------------+
void CMinBC::MinBCCreateF(int n,CRowDouble &x,double diffstep,
CMinBCState &State)
{
//--- create variables
CMatrixDouble c;
CRowInt ct;
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
if(!CAp::Assert(MathIsValidNumber(diffstep),__FUNCTION__+": DiffStep is infinite or NaN!"))
return;
if(!CAp::Assert(diffstep>0.0,__FUNCTION__+": DiffStep is non-positive!"))
return;
MinBCInitInternal(n,x,diffstep,State);
}
//+------------------------------------------------------------------+
//| This function sets boundary constraints for BC optimizer. |
//| Boundary constraints are inactive by default (after initial |
//| creation). They are preserved after algorithm restart with |
//| MinBCRestartFrom(). |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| BndL - lower bounds, array[N]. If some (all) variables |
//| are unbounded, you may specify very small number|
//| or -INF. |
//| BndU - upper bounds, array[N]. If some (all) variables |
//| are unbounded, you may specify very large number|
//| or +INF. |
//| NOTE 1: it is possible to specify BndL[i] = BndU[i]. In this case|
//| I-th variable will be "frozen" at X[i] = BndL[i]=BndU[i].|
//| NOTE 2: this solver has following useful properties: |
//| * bound constraints are always satisfied exactly |
//| * function is evaluated only INSIDE area specified by |
//| bound constraints, even when numerical differentiation |
//| is used (algorithm adjusts nodes according to boundary |
//| constraints) |
//+------------------------------------------------------------------+
void CMinBC::MinBCSetBC(CMinBCState &State,CRowDouble &bndl,CRowDouble &bndu)
{
int n=State.m_nmain;
//--- check
if(!CAp::Assert(CAp::Len(bndl)>=n,__FUNCTION__+": Length(BndL)<N"))
return;
if(!CAp::Assert(CAp::Len(bndu)>=n,__FUNCTION__+": Length(BndU)<N"))
return;
for(int i=0; i<n; i++)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(bndl[i]) || IsNegInf(bndl[i]),__FUNCTION__+": BndL contains NAN or +INF"))
return;
if(!CAp::Assert(MathIsValidNumber(bndu[i]) || IsPosInf(bndu[i]),__FUNCTION__+": BndL contains NAN or -INF"))
return;
State.m_bndl.Set(i,bndl[i]);
State.m_bndu.Set(i,bndu[i]);
State.m_HasBndL[i]=MathIsValidNumber(bndl[i]);
State.m_HasBndU[i]=MathIsValidNumber(bndu[i]);
}
}
//+------------------------------------------------------------------+
//| This function sets stopping conditions for the optimizer. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| EpsG - >= 0. The subroutine finishes its work if the |
//| condition | v | < EpsG is satisfied, where: |
//| * | . | means Euclidian norm |
//| * v - scaled gradient vector, v[i] = g[i] * s[i]|
//| * g - gradient |
//| * s - scaling coefficients set by MinBCSetScale |
//| EpsF - >= 0. The subroutine finishes its work if on |
//| k+1-th iteration the condition |
//| | F(k + 1) - F(k) | <= EpsF * max{ | F(k) |, | F(k + 1) |, 1} |
//| is satisfied. |
//| EpsX - >= 0. The subroutine finishes its work if on |
//| k+1-th iteration the condition | v | <= EpsX is |
//| fulfilled, where: |
//| * | . | means Euclidian norm |
//| * v - scaled step vector, v[i] = dx[i] / s[i] |
//| * dx - step vector, dx = X(k + 1) - X(k) |
//| * s - scaling coefficients set by MinBCSetScale |
//| MaxIts - maximum number of iterations. If MaxIts = 0, the|
//| number of iterations is unlimited. |
//| Passing EpsG = 0, EpsF = 0 and EpsX = 0 and MaxIts = 0 |
//| (simultaneously) will lead to automatic stopping criterion |
//| selection. |
//| NOTE: when SetCond() called with non-zero MaxIts, BC solver may |
//| perform slightly more than MaxIts iterations. I.e., MaxIts |
//| sets non-strict limit on iterations count. |
//+------------------------------------------------------------------+
void CMinBC::MinBCSetCond(CMinBCState &State,double epsg,double epsf,
double epsx,int maxits)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(epsg),__FUNCTION__+": EpsG is not finite number"))
return;
if(!CAp::Assert(epsg>=0.0,__FUNCTION__+": negative EpsG"))
return;
if(!CAp::Assert(MathIsValidNumber(epsf),__FUNCTION__+": EpsF is not finite number"))
return;
if(!CAp::Assert(epsf>=0.0,__FUNCTION__+": negative EpsF"))
return;
if(!CAp::Assert(MathIsValidNumber(epsx),__FUNCTION__+": EpsX is not finite number"))
return;
if(!CAp::Assert(epsx>=0.0,__FUNCTION__+": negative EpsX"))
return;
if(!CAp::Assert(maxits>=0,__FUNCTION__+": negative MaxIts!"))
return;
if(epsg==0.0 && epsf==0.0 && epsx==0.0 && maxits==0)
epsx=1.0E-6;
State.m_epsg=epsg;
State.m_epsf=epsf;
State.m_epsx=epsx;
State.m_maxits=maxits;
}
//+------------------------------------------------------------------+
//| This function sets scaling coefficients for BC optimizer. |
//| ALGLIB optimizers use scaling matrices to test stopping |
//| conditions (step size and gradient are scaled before comparison |
//| with tolerances). Scale of the I-th variable is a translation |
//| invariant measure of: |
//| a) "how large" the variable is |
//| b) how large the step should be to make significant changes in |
//| the function |
//| Scaling is also used by finite difference variant of the |
//| optimizer - step along I-th axis is equal to DiffStep*S[I]. |
//| In most optimizers (and in the BC too) scaling is NOT a form of |
//| preconditioning. It just affects stopping conditions. You should |
//| set preconditioner by separate call to one of the MinBCSetPrec...|
//| functions. |
//| There is a special preconditioning mode, however, which uses |
//| scaling coefficients to form diagonal preconditioning matrix. You|
//| can turn this mode on, if you want. But you should understand |
//| that scaling is not the same thing as preconditioning - these are|
//| two different, although related forms of tuning solver. |
//| INPUT PARAMETERS: |
//| State - structure stores algorithm State |
//| S - array[N], non-zero scaling coefficients S[i] may|
//| be negative, sign doesn't matter. |
//+------------------------------------------------------------------+
void CMinBC::MinBCSetScale(CMinBCState &State,CRowDouble &s)
{
//--- check
if(!CAp::Assert(CAp::Len(s)>=State.m_nmain,__FUNCTION__+": Length(S)<N"))
return;
for(int i=0; i<State.m_nmain; i++)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(s[i]),__FUNCTION__+": S contains infinite or NAN elements"))
return;
if(!CAp::Assert(s[i]!=0.0,__FUNCTION__+": S contains zero elements"))
return;
State.m_s.Set(i,MathAbs(s[i]));
}
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: preconditioning is turned off|
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//+------------------------------------------------------------------+
void CMinBC::MinBCSetPrecDefault(CMinBCState &State)
{
State.m_prectype=0;
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: diagonal of approximate |
//| Hessian is used. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| D - diagonal of the approximate Hessian, |
//| array[0..N - 1], (if larger, only leading N |
//| elements are used). |
//| NOTE 1: D[i] should be positive. Exception will be thrown |
//| otherwise. |
//| NOTE 2: you should pass diagonal of approximate Hessian - NOT ITS|
//| INVERSE. |
//+------------------------------------------------------------------+
void CMinBC::MinBCSetPrecDiag(CMinBCState &State,CRowDouble &d)
{
//--- check
if(!CAp::Assert(CAp::Len(d)>=State.m_nmain,__FUNCTION__+": D is too short"))
return;
for(int i=0; i<State.m_nmain; i++)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(d[i]),__FUNCTION__+": D contains infinite or NAN elements"))
return;
if(!CAp::Assert(d[i]>0.0,__FUNCTION__+": D contains non-positive elements"))
return;
}
State.m_prectype=2;
State.m_diagh=d;
State.m_diagh.Resize(State.m_nmain);
}
//+------------------------------------------------------------------+
//| Modification of the preconditioner: scale - based diagonal |
//| preconditioning. |
//| This preconditioning mode can be useful when you don't have |
//| approximate diagonal of Hessian, but you know that your variables|
//| are badly scaled (for example, one variable is in [1, 10], and |
//| another in [1000, 100000]), and most part of the ill-conditioning|
//| comes from different scales of vars. |
//| In this case simple scale-based preconditioner, with |
//| H[i] = 1/(s[i]^2), can greatly improve convergence. |
//| IMPRTANT: you should set scale of your variables with |
//| MinBCSetScale() call (before or after |
//| MinBCSetPrecScale() call). Without knowledge of the |
//| scale of your variables scale-based preconditioner |
//| will be just unit matrix. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//+------------------------------------------------------------------+
void CMinBC::MinBCSetPrecScale(CMinBCState &State)
{
State.m_prectype=3;
}
//+------------------------------------------------------------------+
//| This function turns on / off reporting. |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| NeedXRep - whether iteration reports are needed or not |
//| If NeedXRep is True, algorithm will call rep() callback function |
//| if it is provided to MinBCOptimize(). |
//+------------------------------------------------------------------+
void CMinBC::MinBCSetXRep(CMinBCState &State,bool needxrep)
{
State.m_xrep=needxrep;
}
//+------------------------------------------------------------------+
//| This function sets maximum step length |
//| INPUT PARAMETERS: |
//| State - structure which stores algorithm State |
//| StpMax - maximum step length, >= 0. Set StpMax to 0.0, |
//| if you don't want to limit step length. |
//| Use this subroutine when you optimize target function which |
//| contains exp() or other fast growing functions, and optimization |
//| algorithm makes too large steps which lead to overflow. This |
//| function allows us to reject steps that are too large (and |
//| therefore expose us to the possible overflow) without actually |
//| calculating function value at the x + stp*d. |
//+------------------------------------------------------------------+
void CMinBC::MinBCSetStpMax(CMinBCState &State,double stpmax)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(stpmax),__FUNCTION__+": StpMax is not finite!"))
return;
if(!CAp::Assert(stpmax>=0.0,__FUNCTION__+": StpMax<0!"))
return;
State.m_stpmax=stpmax;
}
//+------------------------------------------------------------------+
//| NOTES: 1. This function has two different implementations: one |
//| which uses exact (analytical) user-supplied gradient, |
//| and one which uses function value only and numerically |
//| differentiates function in order to obtain gradient. |
//| Depending on the specific function used to create optimizer |
//| object (either MinBCCreate() for analytical gradient or |
//| MinBCCreateF() for numerical differentiation) you should choose |
//| appropriate variant of MinBCOptimize() - one which accepts |
//| function AND gradient or one which accepts function ONLY. |
//| Be careful to choose variant of MinBCOptimize() which corresponds|
//| to your optimization scheme! Table below lists different |
//| combinations of callback (function/gradient) passed to |
//| MinBCOptimize() and specific function used to create optimizer. |
//| | USER PASSED TO MinBCOptimize() |
//| CREATED WITH | function only | function and gradient |
//| ------------------------------------------------------------ |
//| MinBCCreateF() | works FAILS |
//| MinBCCreate() | FAILS works |
//| Here "FAIL" denotes inappropriate combinations of optimizer |
//| creation function and MinBCOptimize() version. Attemps to use |
//| such combination (for example, to create optimizer with |
//| MinBCCreateF() and to pass gradient information to MinCGOptimize)|
//| will lead to exception being thrown. Either you did not pass |
//| gradient when it WAS needed or you passed gradient when it was |
//| NOT needed. |
//+------------------------------------------------------------------+
bool CMinBC::MinBCIteration(CMinBCState &State)
{
//--- create variables
bool result=false;
int freezeidx=0;
double freezeval=0;
double scaleddnorm=0;
int n=0;
int m=0;
int i=0;
int j=0;
double v=0;
double vv=0;
double v0=0;
bool b=false;
int mcinfo=0;
int itidx=0;
double ginit=0;
double gdecay=0;
bool activationstatus=false;
double activationstep=0;
int i_=0;
int label=-1;
vector<double> temp_v;
//--- Reverse communication preparations
//--- This code initializes locals by:
//--- * random values determined during code
//--- generation - on first subroutine call
//--- * values from previous call - on subsequent calls
if(State.m_rstate.stage>=0)
{
freezeidx=State.m_rstate.ia[0];
n=State.m_rstate.ia[1];
m=State.m_rstate.ia[2];
i=State.m_rstate.ia[3];
j=State.m_rstate.ia[4];
mcinfo=State.m_rstate.ia[5];
itidx=State.m_rstate.ia[6];
b=State.m_rstate.ba[0];
activationstatus=State.m_rstate.ba[1];
freezeval=State.m_rstate.ra[0];
scaleddnorm=State.m_rstate.ra[1];
v=State.m_rstate.ra[2];
vv=State.m_rstate.ra[3];
v0=State.m_rstate.ra[4];
ginit=State.m_rstate.ra[5];
gdecay=State.m_rstate.ra[6];
activationstep=State.m_rstate.ra[7];
}
else
{
freezeidx=359;
n=-58;
m=-919;
i=-909;
j=81;
mcinfo=255;
itidx=74;
b=false;
activationstatus=true;
freezeval=205;
scaleddnorm=-838;
v=939;
vv=-526;
v0=763;
ginit=-541;
gdecay=-698;
activationstep=-900;
}
switch(State.m_rstate.stage)
{
case 0:
label=0;
break;
case 1:
label=1;
break;
case 2:
label=2;
break;
case 3:
label=3;
break;
case 4:
label=4;
break;
case 5:
label=5;
break;
case 6:
label=6;
break;
case 7:
label=7;
break;
case 8:
label=8;
break;
case 9:
label=9;
break;
case 10:
label=10;
break;
case 11:
label=11;
break;
case 12:
label=12;
break;
case 13:
label=13;
break;
case 14:
label=14;
break;
case 15:
label=15;
break;
case 16:
label=16;
break;
case 17:
label=17;
break;
case 18:
label=18;
break;
case 19:
label=19;
break;
case 20:
label=20;
break;
case 21:
label=21;
break;
case 22:
label=22;
break;
case 23:
label=23;
break;
case 24:
label=24;
break;
case 25:
label=25;
break;
case 26:
label=26;
break;
case 27:
label=27;
break;
case 28:
label=28;
break;
case 29:
label=29;
break;
default:
//--- Routine body
//--- Algorithm parameters:
//--- * M number of L-BFGS corrections.
//--- This coefficient remains fixed during iterations.
//--- * GDecay desired decrease of constrained gradient during L-BFGS iterations.
//--- This coefficient is decreased after each L-BFGS round until
//--- it reaches minimum decay.
m=MathMin(5,State.m_nmain);
gdecay=m_initialdecay;
//--- Init
n=State.m_nmain;
State.m_xc=State.m_xstart;
if(!COptServ::EnforceBoundaryConstraints(State.m_xc,State.m_bndl,State.m_HasBndL,State.m_bndu,State.m_HasBndU,n,0))
{
//--- Inconsistent constraints
State.m_repterminationtype=-3;
result=false;
return(result);
}
State.m_userterminationneeded=false;
State.m_repterminationtype=0;
State.m_repiterationscount=0;
State.m_repnfev=0;
State.m_repvaridx=-1;
State.m_bufyk=matrix<double>::Zeros(m+1,n);
State.m_bufsk=matrix<double>::Zeros(m+1,n);
State.m_bufrho=vector<double>::Zeros(m);
State.m_buftheta=vector<double>::Zeros(m);
State.m_tmp0=vector<double>::Zeros(n);
COptServ::SmoothnessMonitorInit(State.m_smonitor,State.m_s,n,1,State.m_smoothnessguardlevel>0);
State.m_lastscaleused=State.m_s;
State.m_invs=State.m_s.Pow(-1)+0;
//--- Fill TmpPrec with current preconditioner
State.m_tmpprec=vector<double>::Ones(n);
switch(State.m_prectype)
{
case 2:
State.m_tmpprec=State.m_diagh.Pow(-1)+0;
break;
case 3:
State.m_tmpprec=State.m_s.Pow(2)+0;
break;
}
//--- Check correctness of user-supplied gradient
ClearRequestFields(State);
if(!(State.m_diffstep==0.0 && State.m_teststep>0.0))
label=30;
else
label=32;
break;
}
//--- main loop
while(label>=0)
{
switch(label)
{
case 32:
if(!COptServ::SmoothnessMonitorCheckGradientATX0(State.m_smonitor,State.m_xc,State.m_s,State.m_bndl,State.m_bndu,true,State.m_teststep))
{
label=33;
break;
}
State.m_x=State.m_smonitor.m_x;
State.m_needfg=true;
State.m_rstate.stage=0;
label=-1;
break;
case 0:
State.m_needfg=false;
State.m_smonitor.m_fi.Set(0,State.m_f);
State.m_smonitor.m_j.Row(0,State.m_g);
label=32;
break;
case 33:
case 30:
//--- Main cycle of BC-PG algorithm
State.m_repterminationtype=0;
State.m_lastscaledgoodstep=0;
State.m_nonmonotoniccnt=(int)MathRound(1.5*n)+5;
State.m_x=State.m_xc;
ClearRequestFields(State);
if(State.m_diffstep!=0.0)
{
label=34;
break;
}
State.m_needfg=true;
State.m_rstate.stage=1;
label=-1;
break;
case 1:
State.m_needfg=false;
label=35;
break;
case 34:
State.m_needf=true;
State.m_rstate.stage=2;
label=-1;
break;
case 2:
State.m_needf=false;
case 35:
State.m_fc=State.m_f;
COptServ::TrimPrepare(State.m_f,State.m_trimthreshold);
State.m_repnfev++;
if(!State.m_xrep)
{
label=36;
break;
}
//--- Report current point
State.m_x=State.m_xc;
State.m_f=State.m_fc;
State.m_xupdated=true;
State.m_rstate.stage=3;
label=-1;
break;
case 3:
State.m_xupdated=false;
case 36:
if(State.m_userterminationneeded)
{
//--- User requested termination
State.m_repterminationtype=8;
result=false;
return(result);
}
case 38:
//--- Steepest descent phase
//--- (a) calculate unconstrained gradient
//--- (b) check F/G for NAN/INF, abnormally terminate algorithm if needed
//--- (c) perform one steepest descent step, activating only those constraints
//--- which prevent us from moving outside of box-constrained area
State.m_x=State.m_xc;
ClearRequestFields(State);
if(State.m_diffstep!=0.0)
{
label=40;
break;
}
//--- Analytic gradient
State.m_needfg=true;
State.m_rstate.stage=4;
label=-1;
break;
case 4:
State.m_needfg=false;
label=41;
break;
case 40:
//--- Numerical differentiation
State.m_needf=true;
State.m_rstate.stage=5;
label=-1;
break;
case 5:
State.m_fbase=State.m_f;
i=0;
case 42:
if(i>n-1)
{
label=44;
break;
}
v=State.m_x[i];
b=false;
if(State.m_HasBndL[i])
b=b || (v-State.m_diffstep*State.m_s[i])<State.m_bndl[i];
if(State.m_HasBndU[i])
b=b || (v+State.m_diffstep*State.m_s[i])>State.m_bndu[i];
if(b)
{
label=45;
break;
}
State.m_x.Set(i,v-State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=6;
label=-1;
break;
case 6:
State.m_fm2=State.m_f;
State.m_x.Set(i,v-0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=7;
label=-1;
break;
case 7:
State.m_fm1=State.m_f;
State.m_x.Set(i,v+0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=8;
label=-1;
break;
case 8:
State.m_fp1=State.m_f;
State.m_x.Set(i,v+State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=9;
label=-1;
break;
case 9:
State.m_fp2=State.m_f;
State.m_g.Set(i,(8*(State.m_fp1-State.m_fm1)-(State.m_fp2-State.m_fm2))/(6*State.m_diffstep*State.m_s[i]));
label=46;
break;
case 45:
State.m_xm1=v-State.m_diffstep*State.m_s[i];
State.m_xp1=v+State.m_diffstep*State.m_s[i];
if(State.m_HasBndL[i] && State.m_xm1<State.m_bndl[i])
State.m_xm1=State.m_bndl[i];
if(State.m_HasBndU[i] && State.m_xp1>State.m_bndu[i])
State.m_xp1=State.m_bndu[i];
State.m_x.Set(i,State.m_xm1);
State.m_rstate.stage=10;
label=-1;
break;
case 10:
State.m_fm1=State.m_f;
State.m_x.Set(i,State.m_xp1);
State.m_rstate.stage=11;
label=-1;
break;
case 11:
State.m_fp1=State.m_f;
if(State.m_xm1!=State.m_xp1)
State.m_g.Set(i,(State.m_fp1-State.m_fm1)/(State.m_xp1-State.m_xm1));
else
State.m_g.Set(i,0);
case 46:
State.m_x.Set(i,v);
i++;
label=42;
break;
case 44:
State.m_f=State.m_fbase;
State.m_needf=false;
case 41:
State.m_fc=State.m_f;
State.m_ugc=State.m_g;
State.m_cgc=State.m_g;
COptServ::ProjectGradientIntoBC(State.m_xc,State.m_cgc,State.m_bndl,State.m_HasBndL,State.m_bndu,State.m_HasBndU,n,0);
ginit=MathPow(State.m_cgc.ToVector()*State.m_s.ToVector(),2.0).Sum();
ginit=MathSqrt(ginit);
if(!MathIsValidNumber(ginit) || !MathIsValidNumber(State.m_fc))
{
//--- Abnormal termination - infinities in function/gradient
State.m_repterminationtype=-8;
result=false;
return(result);
}
if(State.m_userterminationneeded)
{
//--- User requested termination
State.m_repterminationtype=8;
result=false;
return(result);
}
if(ginit<=State.m_epsg)
{
//--- Gradient is small enough.
//--- Optimization is terminated
State.m_repterminationtype=4;
result=false;
return(result);
}
State.m_d=(State.m_tmpprec*State.m_cgc)*(-1.0);
scaleddnorm= MathPow(State.m_d.ToVector()/State.m_s.ToVector(),2.0).Sum();
scaleddnorm=MathSqrt(scaleddnorm);
//--- check
if(!CAp::Assert(scaleddnorm>0.0,__FUNCTION__+": integrity check failed"))
return(false);
if(State.m_lastscaledgoodstep>0.0)
State.m_stp=State.m_lastscaledgoodstep/scaleddnorm;
else
State.m_stp=1.0/scaleddnorm;
COptServ::CalculateStepBound(State.m_xc,State.m_d,1.0,State.m_bndl,State.m_HasBndL,State.m_bndu,State.m_HasBndU,n,0,freezeidx,freezeval,State.m_curstpmax);
activationstep=State.m_curstpmax;
if(freezeidx<0 || State.m_curstpmax>1.0E50)
State.m_curstpmax=1.0E50;
if(State.m_stpmax>0.0)
State.m_curstpmax=MathMin(State.m_curstpmax,State.m_stpmax/scaleddnorm);
State.m_xn=State.m_xc;
State.m_cgn=State.m_cgc;
State.m_ugn=State.m_ugc;
State.m_fn=State.m_fc;
State.m_mcstage=0;
COptServ::SmoothnessMonitorStartLineSearch1u(State.m_smonitor,State.m_s,State.m_invs,State.m_xn,State.m_fn,State.m_ugn);
CLinMin::MCSrch(n,State.m_xn,State.m_fn,State.m_cgn,State.m_d,State.m_stp,State.m_curstpmax,m_gtol,mcinfo,State.m_nfev,State.m_work,State.m_lstate,State.m_mcstage);
case 47:
if(State.m_mcstage==0)
{
label=48;
break;
}
//--- Copy XN to X, perform on-the-fly correction w.r.t box
//--- constraints (projection onto feasible set).
State.m_x=State.m_xn;
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i] && State.m_xn[i]<State.m_bndl[i])
State.m_x.Set(i,State.m_bndl[i]);
if(State.m_HasBndU[i] && State.m_xn[i]>State.m_bndu[i])
State.m_x.Set(i,State.m_bndu[i]);
}
//--- Gradient, either user-provided or numerical differentiation
ClearRequestFields(State);
if(State.m_diffstep!=0.0)
{
label=49;
break;
}
//--- Analytic gradient
State.m_needfg=true;
State.m_rstate.stage=12;
label=-1;
break;
case 12:
State.m_needfg=false;
State.m_repnfev=State.m_repnfev+1;
label=50;
break;
case 49:
//--- Numerical differentiation
State.m_needf=true;
State.m_rstate.stage=13;
label=-1;
break;
case 13:
State.m_fbase=State.m_f;
i=0;
case 51:
if(i>n-1)
{
label=53;
break;
}
v=State.m_x[i];
b=false;
if(State.m_HasBndL[i])
b=b || (v-State.m_diffstep*State.m_s[i])<State.m_bndl[i];
if(State.m_HasBndU[i])
b=b || (v+State.m_diffstep*State.m_s[i])>(State.m_bndu[i]);
if(b)
{
label=54;
break;
}
State.m_x.Set(i,v-State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=14;
label=-1;
break;
case 14:
State.m_fm2=State.m_f;
State.m_x.Set(i,v-0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=15;
label=-1;
break;
case 15:
State.m_fm1=State.m_f;
State.m_x.Set(i,v+0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=16;
label=-1;
break;
case 16:
State.m_fp1=State.m_f;
State.m_x.Set(i,v+State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=17;
label=-1;
break;
case 17:
State.m_fp2=State.m_f;
State.m_g.Set(i,(8*(State.m_fp1-State.m_fm1)-(State.m_fp2-State.m_fm2))/(6*State.m_diffstep*State.m_s[i]));
State.m_repnfev+=4;
label=55;
break;
case 54:
State.m_xm1=v-State.m_diffstep*State.m_s[i];
State.m_xp1=v+State.m_diffstep*State.m_s[i];
if(State.m_HasBndL[i] && State.m_xm1<State.m_bndl[i])
State.m_xm1=State.m_bndl[i];
if(State.m_HasBndU[i] && State.m_xp1>State.m_bndu[i])
State.m_xp1=State.m_bndu[i];
State.m_x.Set(i,State.m_xm1);
State.m_rstate.stage=18;
label=-1;
break;
case 18:
State.m_fm1=State.m_f;
State.m_x.Set(i,State.m_xp1);
State.m_rstate.stage=19;
label=-1;
break;
case 19:
State.m_fp1=State.m_f;
if(State.m_xm1!=State.m_xp1)
State.m_g.Set(i,(State.m_fp1-State.m_fm1)/(State.m_xp1-State.m_xm1));
else
State.m_g.Set(i,0);
State.m_repnfev+=2;
case 55:
State.m_x.Set(i,v);
i++;
label=51;
break;
case 53:
State.m_f=State.m_fbase;
State.m_needf=false;
case 50:
//--- Back to MCSRCH
COptServ::SmoothnessMonitorEnqueuePoint1u(State.m_smonitor,State.m_s,State.m_invs,State.m_d,State.m_stp,State.m_x,State.m_f,State.m_g);
COptServ::TrimFunction(State.m_f,State.m_g,n,State.m_trimthreshold);
State.m_fn=State.m_f;
State.m_cgn=State.m_g;
State.m_ugn=State.m_g;
for(i=0; i<n; i++)
{
if(State.m_d[i]==0.0)
State.m_cgn.Set(i,0);
}
CLinMin::MCSrch(n,State.m_xn,State.m_fn,State.m_cgn,State.m_d,State.m_stp,State.m_curstpmax,m_gtol,mcinfo,State.m_nfev,State.m_work,State.m_lstate,State.m_mcstage);
label=47;
break;
case 48:
COptServ::SmoothnessMonitorFinalizeLineSearch(State.m_smonitor);
v=State.m_fn;
for(i=0; i<n; i++)
v=0.1*v+State.m_ugn[i];
if(!MathIsValidNumber(v))
{
//--- Abnormal termination - infinities in function/gradient
State.m_repterminationtype=-8;
result=false;
return(result);
}
if(mcinfo!=1 && mcinfo!=5)
{
//--- We can not find step which decreases function value. We have
//--- two possibilities:
//--- (a) numerical properties of the function do not allow us to
//--- find good step.
//--- (b) we are close to activation of some constraint, and it is
//--- so close that step which activates it leads to change in
//--- target function which is smaller than numerical noise.
//--- Optimization algorithm must be able to handle case (b), because
//--- inability to handle it will cause failure when algorithm
//--- started very close to boundary of the feasible area.
//--- In order to correctly handle such cases we allow limited amount
//--- of small steps which increase function value.
if(freezeidx>=0 && (scaleddnorm*State.m_curstpmax)<=m_maxnonmonotoniclen && State.m_nonmonotoniccnt>0)
{
//--- We enforce non-monotonic step:
//--- * Stp := CurStpMax
//--- * MCINFO := 5
//--- * XN := XC+CurStpMax*D
//--- * non-monotonic counter is decreased
//--- NOTE: UGN/CGN are not updated because step is so short that we assume that
//--- GN is approximately equal to GC.
State.m_stp=State.m_curstpmax;
mcinfo=5;
v=State.m_curstpmax;
State.m_xn=State.m_xc.ToVector()+State.m_d.ToVector()*v;
State.m_nonmonotoniccnt--;
}
else
{
//--- Numerical properties of the function does not allow
//--- us to solve problem. Algorithm is terminated
State.m_repterminationtype=7;
result=false;
return(result);
}
}
if(State.m_userterminationneeded)
{
//--- User requested termination
State.m_repterminationtype=8;
result=false;
return(result);
}
//--- check
if(!CAp::Assert(mcinfo!=5 || (double)State.m_stp==(double)State.m_curstpmax,__FUNCTION__+": integrity check failed"))
return(false);
COptServ::PostProcessBoundedStep(State.m_xn,State.m_xc,State.m_bndl,State.m_HasBndL,State.m_bndu,State.m_HasBndU,n,0,freezeidx,freezeval,State.m_stp,activationstep);
State.m_fp=State.m_fc;
State.m_fc=State.m_fn;
State.m_xp=State.m_xc;
State.m_xc=State.m_xn;
State.m_cgc=State.m_cgn;
State.m_ugc=State.m_ugn;
if(!State.m_xrep)
{
label=56;
break;
}
State.m_x=State.m_xc;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=20;
label=-1;
break;
case 20:
State.m_xupdated=false;
case 56:
State.m_repiterationscount++;
if(mcinfo==1)
{
v=0;
for(i=0; i<n; i++)
v=v+CMath::Sqr((State.m_xc[i]-State.m_xp[i])/State.m_s[i]);
v=MathSqrt(v);
if(v<=State.m_epsx)
{
//--- Step is small enough
State.m_repterminationtype=2;
result=false;
return(result);
}
if(MathAbs(State.m_fp-State.m_fc)<=(State.m_epsf*MathMax(MathAbs(State.m_fc),MathMax(MathAbs(State.m_fp),1.0))))
{
//--- Function change is small enough
State.m_repterminationtype=1;
result=false;
return(result);
}
}
if(State.m_maxits>0 && State.m_repiterationscount>=State.m_maxits)
{
//--- Iteration counter exceeded limit
State.m_repterminationtype=5;
result=false;
return(result);
}
//--- LBFGS stage:
//--- * during LBFGS iterations we activate new constraints, but never
//--- deactivate already active ones.
//--- * we perform at most N iterations of LBFGS before re-evaluating
//--- active set and restarting LBFGS.
//--- About termination:
//--- * LBFGS iterations can be terminated because of two reasons:
//--- *"termination" - non-zero termination code in RepTerminationType,
//--- which means that optimization is done
//--- *"restart" - zero RepTerminationType, which means that we
//--- have to re-evaluate active set and resume LBFGS stage.
//---*one more option is "refresh" - to continue LBFGS iterations,
//--- but with all BFGS updates (Sk/Yk pairs) being dropped;
//--- it happens after changes in active set
ginit=0.0;
State.m_cgc=State.m_ugc;
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i] && State.m_xc[i]==State.m_bndl[i])
State.m_cgc.Set(i,0);
if(State.m_HasBndU[i] && State.m_xc[i]==State.m_bndu[i])
State.m_cgc.Set(i,0);
ginit+=CMath::Sqr(State.m_cgc[i]*State.m_s[i]);
}
ginit=MathSqrt(ginit);
State.m_bufsize=0;
itidx=0;
case 58:
if(itidx>n-1)
{
label=60;
break;
}
//--- At the beginning of each iteration:
//--- * XC stores current point
//--- * FC stores current function value
//--- * UGC stores current unconstrained gradient
//--- * CGC stores current constrained gradient
//--- * D stores constrained step direction (calculated at this block)
//--- 1. Calculate search direction D according to L-BFGS algorithm
//--- using constrained preconditioner to perform inner multiplication.
//--- 2. Evaluate scaled length of direction D; restart LBFGS if D is zero
//--- (it may be possible that we found minimum, but it is also possible
//--- that some constraints need deactivation)
//--- 3. If D is non-zero, try to use previous scaled step length as initial estimate for new step.
//--- 4. Calculate bound on step length.
State.m_work=State.m_cgc;
for(i=State.m_bufsize-1; i>=0; i--)
{
v=State.m_work.DotR(State.m_bufsk,i);
State.m_buftheta.Set(i,v);
vv=v*State.m_bufrho[i];
State.m_work-=State.m_bufyk[i]*vv;
}
State.m_work*=State.m_tmpprec;
for(i=0; i<State.m_bufsize; i++)
{
v=State.m_work.DotR(State.m_bufyk,i);
vv=State.m_bufrho[i]*(-v+State.m_buftheta[i]);
State.m_work+=State.m_bufsk[i]*vv;
}
State.m_d=State.m_work.ToVector()*(-1);
b=false;
for(i=0; i<n; i++)
{
b=b || ((State.m_HasBndL[i] && State.m_xc[i]==State.m_bndl[i]) && State.m_d[i]!=0.0);
b=b || ((State.m_HasBndU[i] && State.m_xc[i]==State.m_bndu[i]) && State.m_d[i]!=0.0);
}
//--- check
if(!CAp::Assert(!b,__FUNCTION__+": integrity check failed (q)"))
return(false);
scaleddnorm=0;
for(i=0; i<n; i++)
scaleddnorm=scaleddnorm+CMath::Sqr(State.m_d[i]/State.m_s[i]);
scaleddnorm=MathSqrt(scaleddnorm);
if(scaleddnorm==0.0)
{
//--- Search direction is zero.
//--- Skip back to steepest descent phase.
label=60;
break;
}
if(State.m_lastscaledgoodstep>0.0)
State.m_stp=State.m_lastscaledgoodstep/scaleddnorm;
else
State.m_stp=1.0/scaleddnorm;
State.m_curstpmax=1.0E50;
if(State.m_stpmax>0.0)
State.m_curstpmax=MathMin(State.m_curstpmax,State.m_stpmax/scaleddnorm);
//--- Minimize G(t) = F(CONSTRAIN(XC + t*D)), with t being scalar, XC and D being vectors.
State.m_xn=State.m_xc;
State.m_cgn=State.m_cgc;
State.m_ugn=State.m_ugc;
State.m_fn=State.m_fc;
State.m_mcstage=0;
COptServ::SmoothnessMonitorStartLineSearch1u(State.m_smonitor,State.m_s,State.m_invs,State.m_xn,State.m_fn,State.m_ugn);
CLinMin::MCSrch(n,State.m_xn,State.m_fn,State.m_cgn,State.m_d,State.m_stp,State.m_curstpmax,m_gtol,mcinfo,State.m_nfev,State.m_work,State.m_lstate,State.m_mcstage);
case 61:
if(State.m_mcstage==0)
{
label=62;
break;
}
//--- Copy XN to X, perform on-the-fly correction w.r.t box
//--- constraints (projection onto feasible set).
State.m_x=State.m_xn;
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i] && State.m_xn[i]<=State.m_bndl[i])
State.m_x.Set(i,State.m_bndl[i]);
if(State.m_HasBndU[i] && State.m_xn[i]>=State.m_bndu[i])
State.m_x.Set(i,State.m_bndu[i]);
}
//--- Gradient, either user-provided or numerical differentiation
ClearRequestFields(State);
if(State.m_diffstep!=0.0)
{
label=63;
break;
}
//--- Analytic gradient
State.m_needfg=true;
State.m_rstate.stage=21;
label=-1;
break;
case 21:
State.m_needfg=false;
State.m_repnfev ++;
label=64;
break;
case 63:
//--- Numerical differentiation
State.m_needf=true;
State.m_rstate.stage=22;
label=-1;
break;
case 22:
State.m_fbase=State.m_f;
i=0;
case 65:
if(i>n-1)
{
label=67;
break;
}
v=State.m_x[i];
b=false;
if(State.m_HasBndL[i])
b=b || (v-State.m_diffstep*State.m_s[i])<State.m_bndl[i];
if(State.m_HasBndU[i])
b=b || (v+State.m_diffstep*State.m_s[i])>State.m_bndu[i];
if(b)
{
label=68;
break;
}
State.m_x.Set(i,v-State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=23;
label=-1;
break;
case 23:
State.m_fm2=State.m_f;
State.m_x.Set(i,v-0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=24;
label=-1;
break;
case 24:
State.m_fm1=State.m_f;
State.m_x.Set(i,v+0.5*State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=25;
label=-1;
break;
case 25:
State.m_fp1=State.m_f;
State.m_x.Set(i,v+State.m_diffstep*State.m_s[i]);
State.m_rstate.stage=26;
label=-1;
break;
case 26:
State.m_fp2=State.m_f;
State.m_g.Set(i,(8*(State.m_fp1-State.m_fm1)-(State.m_fp2-State.m_fm2))/(6*State.m_diffstep*State.m_s[i]));
State.m_repnfev+=4;
label=69;
break;
case 68:
State.m_xm1=v-State.m_diffstep*State.m_s[i];
State.m_xp1=v+State.m_diffstep*State.m_s[i];
if(State.m_HasBndL[i] && State.m_xm1<State.m_bndl[i])
State.m_xm1=State.m_bndl[i];
if(State.m_HasBndU[i] && State.m_xp1>State.m_bndu[i])
State.m_xp1=State.m_bndu[i];
State.m_x.Set(i,State.m_xm1);
State.m_rstate.stage=27;
label=-1;
break;
case 27:
State.m_fm1=State.m_f;
State.m_x.Set(i,State.m_xp1);
State.m_rstate.stage=28;
label=-1;
break;
case 28:
State.m_fp1=State.m_f;
if(State.m_xm1!=State.m_xp1)
State.m_g.Set(i,(State.m_fp1-State.m_fm1)/(State.m_xp1-State.m_xm1));
else
State.m_g.Set(i,0);
State.m_repnfev+=2;
case 69:
State.m_x.Set(i,v);
i+=1;
label=65;
break;
case 67:
State.m_f=State.m_fbase;
State.m_needf=false;
case 64:
//--- Back to MCSRCH
COptServ::SmoothnessMonitorEnqueuePoint1u(State.m_smonitor,State.m_s,State.m_invs,State.m_d,State.m_stp,State.m_x,State.m_f,State.m_g);
COptServ::TrimFunction(State.m_f,State.m_g,n,State.m_trimthreshold);
State.m_fn=State.m_f;
State.m_ugn=State.m_g;
State.m_cgn=State.m_g;
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i] && State.m_xn[i]<=State.m_bndl[i])
State.m_cgn.Set(i,0);
if(State.m_HasBndU[i] && State.m_xn[i]>=State.m_bndu[i])
State.m_cgn.Set(i,0);
}
CLinMin::MCSrch(n,State.m_xn,State.m_fn,State.m_cgn,State.m_d,State.m_stp,State.m_curstpmax,m_gtol,mcinfo,State.m_nfev,State.m_work,State.m_lstate,State.m_mcstage);
label=61;
break;
case 62:
COptServ::SmoothnessMonitorFinalizeLineSearch(State.m_smonitor);
for(i=0; i<n; i++)
{
if(State.m_HasBndL[i] && State.m_xn[i]<=State.m_bndl[i])
State.m_xn.Set(i,State.m_bndl[i]);
if(State.m_HasBndU[i] && State.m_xn[i]>=State.m_bndu[i])
State.m_xn.Set(i,State.m_bndu[i]);
}
State.m_bufsk.Row(State.m_bufsize,State.m_xn.ToVector()-State.m_xc.ToVector());
State.m_bufyk.Row(State.m_bufsize,State.m_cgn.ToVector()-State.m_cgc.ToVector());
//--- Handle special situations:
//--- * check for presence of NAN/INF in function/gradient
//--- * handle failure of line search
v=State.m_fn;
for(i=0; i<n; i++)
v=0.1*v+State.m_ugn[i];
if(!MathIsValidNumber(v))
{
//--- Abnormal termination - infinities in function/gradient
State.m_repterminationtype=-8;
result=false;
return(result);
}
if(State.m_userterminationneeded)
{
//--- User requested termination
State.m_repterminationtype=8;
result=false;
return(result);
}
if(mcinfo!=1)
{
//--- Terminate LBFGS phase
label=60;
break;
}
//--- Current point is updated:
//--- * move XC/FC/GC to XP/FP/GP
//--- * move XN/FN/GN to XC/FC/GC
//--- * report current point and update iterations counter
//--- * push new pair SK/YK to LBFGS buffer
//--- * update length of the good step
activationstatus=false;
for(i=0; (i<n && !activationstatus); i++)
{
if(State.m_HasBndL[i] && State.m_xn[i]==State.m_bndl[i] && State.m_xn[i]!=State.m_xc[i])
activationstatus=true;
if(State.m_HasBndU[i] && State.m_xn[i]==State.m_bndu[i] && State.m_xn[i]!=State.m_xc[i])
activationstatus=true;
}
State.m_fp=State.m_fc;
State.m_fc=State.m_fn;
State.m_xp=State.m_xc;
State.m_xc=State.m_xn;
State.m_cgc=State.m_cgn;
State.m_ugc=State.m_ugn;
if(!State.m_xrep)
{
label=70;
break;
}
State.m_x=State.m_xc;
ClearRequestFields(State);
State.m_xupdated=true;
State.m_rstate.stage=29;
label=-1;
break;
case 29:
State.m_xupdated=false;
case 70:
State.m_repiterationscount=State.m_repiterationscount+1;
if(State.m_bufsize==m)
{
//--- Buffer is full, shift contents by one row
for(i=0; i<State.m_bufsize; i++)
{
State.m_bufsk.Row(i,State.m_bufsk,i+1);
State.m_bufyk.Row(i,State.m_bufyk,i+1);
}
for(i=0; i<State.m_bufsize-1; i++)
{
State.m_bufrho.Set(i,State.m_bufrho[i+1]);
State.m_buftheta.Set(i,State.m_buftheta[i+1]);
}
}
else
{
//--- Buffer is not full, increase buffer size by 1
State.m_bufsize++;
}
v=(State.m_bufyk[State.m_bufsize-1]*State.m_bufsk[State.m_bufsize-1]).Sum();
vv=(State.m_bufyk[State.m_bufsize-1]*State.m_bufyk[State.m_bufsize-1]).Sum();
if(v==0.0 || vv==0.0)
{
//--- Strange internal error in LBFGS - either YK=0
//--- (which should not have been) or (SK,YK)=0 (again,
//--- unexpected). It should not take place because
//--- MCINFO=1, which signals "good" step. But just
//--- to be sure we have special branch of code which
//--- restarts LBFGS
label=60;
break;
}
State.m_bufrho.Set(State.m_bufsize-1,1/v);
//--- check
if(!CAp::Assert(State.m_bufsize<=m,__FUNCTION__+": internal error"))
return(false);
temp_v=State.m_xc-State.m_xp;
v=MathPow(temp_v/State.m_s.ToVector(),2.0).Sum();
vv=temp_v.Dot(temp_v);
UpdateEstimateOfGoodStep(State.m_lastscaledgoodstep,MathSqrt(v));
//--- Check MaxIts-based stopping condition.
if(State.m_maxits>0 && State.m_repiterationscount>=State.m_maxits)
{
State.m_repterminationtype=5;
result=false;
return(result);
}
//--- Smooth reset (LBFGS memory model is refreshed) or hard restart:
//--- * LBFGS model is refreshed, if line search was performed with activation of constraints
//--- * algorithm is restarted if scaled gradient decreased below GDecay
if(activationstatus)
{
State.m_bufsize=0;
label=59;
break;
}
temp_v=State.m_cgc*State.m_s;
v=temp_v.Dot(temp_v);
if(MathSqrt(v)<(gdecay*ginit))
{
label=60;
break;
}
case 59:
itidx++;
label=58;
break;
case 60:
//--- Decrease decay coefficient. Subsequent L-BFGS stages will
//--- have more stringent stopping criteria.
gdecay=MathMax(gdecay*m_decaycorrection,m_mindecay);
label=38;
break;
case 39:
result=false;
return(result);
}
}
//--- Saving State
result=true;
State.m_rstate.ba[0]=b;
State.m_rstate.ba[1]=activationstatus;
State.m_rstate.ia.Set(0,freezeidx);
State.m_rstate.ia.Set(1,n);
State.m_rstate.ia.Set(2,m);
State.m_rstate.ia.Set(3,i);
State.m_rstate.ia.Set(4,j);
State.m_rstate.ia.Set(5,mcinfo);
State.m_rstate.ia.Set(6,itidx);
State.m_rstate.ra.Set(0,freezeval);
State.m_rstate.ra.Set(1,scaleddnorm);
State.m_rstate.ra.Set(2,v);
State.m_rstate.ra.Set(3,vv);
State.m_rstate.ra.Set(4,v0);
State.m_rstate.ra.Set(5,ginit);
State.m_rstate.ra.Set(6,gdecay);
State.m_rstate.ra.Set(7,activationstep);
//--- return result
return(result);
}
//+------------------------------------------------------------------+
//| This function activates / deactivates verification of the user - |
//| supplied analytic gradient. |
//| Upon activation of this option OptGuard integrity checker |
//| performs numerical differentiation of your target function at |
//| the initial point (note: future versions may also perform check |
//| at the final point) and compares numerical gradient with analytic|
//| one provided by you. |
//| If difference is too large, an error flag is set and optimization|
//| session continues. After optimization session is over, you can |
//| retrieve the report which stores both gradients and specific |
//| components highlighted as suspicious by the OptGuard. |
//| The primary OptGuard report can be retrieved with |
//| MinBCOptGuardResults(). |
//| IMPORTANT: gradient check is a high - overhead option which will |
//| cost you about 3*N additional function evaluations. In many cases|
//| it may cost as much as the rest of the optimization session. |
//| YOU SHOULD NOT USE IT IN THE PRODUCTION CODE UNLESS YOU WANT TO |
//| CHECK DERIVATIVES PROVIDED BY SOME THIRD PARTY. |
//| NOTE: unlike previous incarnation of the gradient checking code, |
//| OptGuard does NOT interrupt optimization even if it |
//| discovers bad gradient. |
//| INPUT PARAMETERS: |
//| State - structure used to store algorithm State |
//| TestStep - verification step used for numerical |
//| differentiation: |
//| * TestStep = 0 turns verification off |
//| * TestStep > 0 activates verification |
//| You should carefully choose TestStep. Value |
//| which is too large (so large that function |
//| behavior is non-cubic at this scale) will lead |
//| to false alarms. Too short step will result in |
//| rounding errors dominating numerical derivative.|
//| You may use different step for different parameters by means of |
//| setting scale with MinBCSetScale(). |
//| === EXPLANATION ================================================ |
//| In order to verify gradient algorithm performs following steps: |
//| * two trial steps are made to X[i] - TestStep * S[i] and |
//| X[i] + TestStep * S[i], where X[i] is i-th component of the |
//| initial point and S[i] is a scale of i-th parameter |
//| * F(X) is evaluated at these trial points |
//| * we perform one more evaluation in the middle point of the |
//| interval |
//| * we build cubic model using function values and derivatives |
//| at trial points and we compare its prediction with actual |
//| value in the middle point |
//+------------------------------------------------------------------+
void CMinBC::MinBCOptGuardGradient(CMinBCState &State,double teststep)
{
//--- check
if(!CAp::Assert(MathIsValidNumber(teststep),__FUNCTION__+": TestStep contains NaN or INF"))
return;
if(!CAp::Assert(teststep>=0.0,__FUNCTION__+": invalid argument TestStep(TestStep<0)"))
return;
State.m_teststep=teststep;
}
//+------------------------------------------------------------------+
//| This function activates / deactivates nonsmoothness monitoring |
//| option of the OptGuard integrity checker. Smoothness monitor |
//| silently observes solution process and tries to detect ill-posed |
//| problems, i.e. ones with: |
//| a) discontinuous target function(non - C0) |
//| b) nonsmooth target function(non - C1) |
//| Smoothness monitoring does NOT interrupt optimization even if it|
//| suspects that your problem is nonsmooth. It just sets |
//| corresponding flags in the OptGuard report which can be retrieved|
//| after optimization is over. |
//| Smoothness monitoring is a moderate overhead option which often |
//| adds less than 1 % to the optimizer running time. Thus, you can |
//| use it even for large scale problems. |
//| NOTE: OptGuard does NOT guarantee that it will always detect |
//| C0 / C1 continuity violations. |
//| First, minor errors are hard to catch - say, a 0.0001 difference|
//| in the model values at two sides of the gap may be due to |
//| discontinuity of the model - or simply because the model has |
//| changed. |
//| Second, C1 - violations are especially difficult to detect |
//| in a noninvasive way. The optimizer usually performs very |
//| short steps near the nonsmoothness, and differentiation usually|
//| introduces a lot of numerical noise. It is hard to tell |
//| whether some tiny discontinuity in the slope is due to real |
//| nonsmoothness or just due to numerical noise alone. |
//| Our top priority was to avoid false positives, so in some rare |
//| cases minor errors may went unnoticed(however, in most cases they|
//| can be spotted with restart from different initial point). |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| Level - monitoring level: |
//| * 0 - monitoring is disabled |
//| * 1 - noninvasive low - overhead monitoring; |
//| function values and / or gradients are recorded,|
//| but OptGuard does not try to perform additional |
//| evaluations in order to get more information |
//| about suspicious locations. |
//| === EXPLANATION ================================================ |
//| One major source of headache during optimization is the |
//| possibility of the coding errors in the target function / |
//| constraints (or their gradients). Such errors most often |
//| manifest themselves as discontinuity or nonsmoothness of the|
//| target / constraints. |
//| Another frequent situation is when you try to optimize something |
//| involving lots of min() and max() operations, i.e. nonsmooth |
//| target. Although not a coding error, it is nonsmoothness anyway-|
//| and smooth optimizers usually stop right after encountering |
//| nonsmoothness, well before reaching solution. |
//| OptGuard integrity checker helps you to catch such situations: |
//| it monitors function values / gradients being passed to the |
//| optimizer and tries to errors. Upon discovering suspicious pair |
//| of points it raises appropriate flag (and allows you to continue|
//| optimization). When optimization is done, you can study OptGuard |
//| result. |
//+------------------------------------------------------------------+
void CMinBC::MinBCOptGuardSmoothness(CMinBCState &State,int level)
{
//--- check
if(!CAp::Assert(level==0 || level==1,__FUNCTION__+": unexpected value of level parameter"))
return;
State.m_smoothnessguardlevel=level;
}
//+------------------------------------------------------------------+
//| Results of OptGuard integrity check, should be called after |
//| optimization session is over. |
//| === PRIMARY REPORT ============================================= |
//| OptGuard performs several checks which are intended to catch |
//| common errors in the implementation of nonlinear function / |
//| gradient: |
//| * incorrect analytic gradient |
//| * discontinuous(non - C0) target functions(constraints) |
//| * nonsmooth(non - C1) target functions(constraints) |
//| Each of these checks is activated with appropriate function: |
//| * MinBCOptGuardGradient() for gradient verification |
//| * MinBCOptGuardSmoothness() for C0 / C1 checks |
//| Following flags are set when these errors are suspected: |
//| * rep.badgradsuspected, and additionally: |
//| * rep.badgradvidx for specific variable (gradient element) |
//| suspected |
//| * rep.badgradxbase, a point where gradient is tested |
//| * rep.badgraduser, user - provided gradient(stored as 2D |
//| matrix with single row in order to make report structure |
//| compatible with more complex optimizers like MinNLC or |
//| MinLM) |
//| * rep.badgradnum, reference gradient obtained via |
//| numerical differentiation (stored as 2D matrix with single|
//| row in order to make report structure compatible with more|
//| complex optimizers like MinNLC or MinLM) |
//| * rep.nonc0suspected |
//| * rep.nonc1suspected |
//| === ADDITIONAL REPORTS / LOGS ================================== |
//| Several different tests are performed to catch C0 / C1 errors, |
//| you can find out specific test signaled error by looking to: |
//| * rep.nonc0test0positive, for non - C0 test #0 |
//| * rep.nonc1test0positive, for non - C1 test #0 |
//| * rep.nonc1test1positive, for non - C1 test #1 |
//| Additional information (including line search logs) can be |
//| obtained by means of: |
//| * MinBCOptGuardNonC1Test0Results() |
//| * MinBCOptGuardNonC1Test1Results() |
//| which return detailed error reports, specific points where |
//| discontinuities were found, and so on. |
//| ================================================================ |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| Rep - generic OptGuard report; more detailed reports |
//| can be retrieved with other functions. |
//| NOTE: false negatives (nonsmooth problems are not identified as |
//| nonsmooth ones) are possible although unlikely. |
//| The reason is that you need to make several evaluations |
//| around nonsmoothness in order to accumulate enough information |
//| about function curvature. Say, if you start right from the |
//| nonsmooth point, optimizer simply won't get enough data to |
//| understand what is going wrong before it terminates due to abrupt|
//| changes in the derivative. It is also possible that "unlucky"|
//| step will move us to the termination too quickly. |
//| Our current approach is to have less than 0.1 % false negatives|
//| in our test examples(measured with multiple restarts from |
//| random points), and to have exactly 0 % false positives. |
//+------------------------------------------------------------------+
void CMinBC::MinBCOptGuardResults(CMinBCState &State,COptGuardReport &rep)
{
COptServ::SmoothnessMonitorExportReport(State.m_smonitor,rep);
}
//+------------------------------------------------------------------+
//|Detailed results of the OptGuard integrity check for nonsmoothness|
//| test #0 |
//| Nonsmoothness (non - C1) test #0 studies function values (not |
//| gradient!) obtained during line searches and monitors behavior |
//| of the directional derivative estimate. |
//| This test is less powerful than test #1, but it does not depend|
//| on the gradient values and thus it is more robust against |
//| artifacts introduced by numerical differentiation. |
//| Two reports are returned: |
//| *a "strongest" one, corresponding to line search which |
//| had highest value of the nonsmoothness indicator |
//| *a "longest" one, corresponding to line search which had more |
//| function evaluations, and thus is more detailed |
//| In both cases following fields are returned: |
//| * positive - is TRUE when test flagged suspicious point; |
//| FALSE if test did not notice anything (in the|
//| latter cases fields below are empty). |
//| * x0[], d[] - arrays of length N which store initial point and |
//| direction for line search (d[] can be normalized,|
//| but does not have to) |
//| * stp[], f[] - arrays of length CNT which store step lengths |
//| and function values at these points; f[i] is |
//| evaluated in x0 + stp[i]*d. |
//| * stpidxa, stpidxb - we suspect that function violates C1 |
//| continuity between steps #stpidxa and #stpidxb |
//| (usually we have stpidxb = stpidxa + 3, with |
//| most likely position of the violation between |
//| stpidxa + 1 and stpidxa + 2. |
//| ================================================================ |
//| = SHORTLY SPEAKING: build a 2D plot of (stp, f) and look at it - |
//| = you will see where C1 continuity is violated.|
//| ================================================================ |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| strrep - C1 test #0 "strong" report |
//| lngrep - C1 test #0 "long" report |
//+------------------------------------------------------------------+
void CMinBC::MinBCOptGuardNonC1Test0Results(CMinBCState &State,
COptGuardNonC1Test0Report &strrep,
COptGuardNonC1Test0Report &lngrep)
{
COptGuardApi::SmoothnessMonitorExportC1Test0Report(State.m_smonitor.m_nonc1test0strrep,State.m_lastscaleused,strrep);
COptGuardApi::SmoothnessMonitorExportC1Test0Report(State.m_smonitor.m_nonc1test0lngrep,State.m_lastscaleused,lngrep);
}
//+------------------------------------------------------------------+
//| Detailed results of the OptGuard integrity check for |
//| nonsmoothness test #1 |
//| Nonsmoothness (non-C1) test #1 studies individual components of |
//| the gradient computed during line search. |
//| When precise analytic gradient is provided this test is more |
//| powerful than test #0 which works with function values and |
//| ignores user-provided gradient. However, test #0 becomes more |
//| powerful when numerical differentiation is employed (in such |
//| cases test #1 detects higher levels of numerical noise and |
//| becomes too conservative). |
//| This test also tells specific components of the gradient which |
//| violate C1 continuity, which makes it more informative than #0, |
//| which just tells that continuity is violated. |
//| Two reports are returned: |
//| *a "strongest" one, corresponding to line search which had |
//| highest value of the nonsmoothness indicator |
//| *a "longest" one, corresponding to line search which had more |
//| function evaluations, and thus is more detailed |
//| In both cases following fields are returned: |
//| * positive - is TRUE when test flagged suspicious point; |
//| FALSE if test did not notice anything (in the |
//| latter cases fields below are empty). |
//| * vidx - is an index of the variable in [0, N) with nonsmooth |
//| derivative |
//| * x0[], d[] - arrays of length N which store initial point and|
//| direction for line search(d[] can be normalized,|
//| but does not have to) |
//| * stp[], g[] - arrays of length CNT which store step lengths |
//| and gradient values at these points; g[i] is |
//| evaluated in x0 + stp[i]*d and contains vidx-th |
//| component of the gradient. |
//| * stpidxa, stpidxb - we suspect that function violates C1 |
//| continuity between steps #stpidxa and #stpidxb |
//| (usually we have stpidxb = stpidxa + 3, with |
//| most likely position of the violation between |
//| stpidxa + 1 and stpidxa + 2. |
//| ================================================================ |
//| = SHORTLY SPEAKING: build a 2D plot of (stp, f) and look at it - |
//| = you will see where C1 continuity is violated.|
//| ================================================================ |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| strrep - C1 test #1 "strong" report |
//| lngrep - C1 test #1 "long" report |
//+------------------------------------------------------------------+
void CMinBC::MinBCOptGuardNonC1Test1Results(CMinBCState &State,
COptGuardNonC1Test1Report &strrep,
COptGuardNonC1Test1Report &lngrep)
{
COptGuardApi::SmoothnessMonitorExportC1Test1Report(State.m_smonitor.m_nonc1test1strrep,State.m_lastscaleused,strrep);
COptGuardApi::SmoothnessMonitorExportC1Test1Report(State.m_smonitor.m_nonc1test1lngrep,State.m_lastscaleused,lngrep);
}
//+------------------------------------------------------------------+
//| BC results |
//| INPUT PARAMETERS: |
//| State - algorithm State |
//| OUTPUT PARAMETERS: |
//| X - array[0..N - 1], solution |
//| Rep - optimization report. You should check |
//| Rep.TerminationType in order to distinguish |
//| successful termination from unsuccessful one: |
//| * -8 internal integrity control detected infinite or |
//| NAN values in function / gradient. Abnormal |
//| termination signalled. |
//| * -3 inconsistent constraints. |
//| * 1 relative function improvement is no more than EpsF.|
//| * 2 scaled step is no more than EpsX. |
//| * 4 scaled gradient norm is no more than EpsG. |
//| * 5 MaxIts steps was taken |
//| * 8 terminated by user who called |
//| MinBCRequestTermination(). |
//| X contains point which was "current accepted" when termination |
//| request was submitted. More information about fields of this |
//| structure can be found in the comments on MinBCReport datatype. |
//+------------------------------------------------------------------+
void CMinBC::MinBCResults(CMinBCState &State,
CRowDouble &x,
CMinBCReport &rep)
{
x.Resize(0);
MinBCResultsBuf(State,x,rep);
}
//+------------------------------------------------------------------+
//| BC results |
//| Buffered implementation of MinBCResults() which uses pre - |
//| allocated buffer to store X[]. If buffer size is too small, it |
//| resizes buffer. It is intended to be used in the inner cycles of |
//| performance critical algorithms where array reallocation penalty |
//| is too large to be ignored. |
//+------------------------------------------------------------------+
void CMinBC::MinBCResultsBuf(CMinBCState &State,
CRowDouble &x,
CMinBCReport &rep)
{
//--- copy data
rep.m_iterationscount=State.m_repiterationscount;
rep.m_nfev=State.m_repnfev;
rep.m_varidx=State.m_repvaridx;
rep.m_terminationtype=State.m_repterminationtype;
if(State.m_repterminationtype>0)
x=State.m_xc;
else
x=vector<double>::Full(State.m_nmain,AL_NaN);
}
//+------------------------------------------------------------------+
//| This subroutine restarts algorithm from new point. |
//| All optimization parameters (including constraints) are left |
//| unchanged. |
//| This function allows to solve multiple optimization problems |
//| (which must have same number of dimensions) without object |
//| reallocation penalty. |
//| INPUT PARAMETERS: |
//| State - structure previously allocated with MinBCCreate |
//| call. |
//| X - new starting point. |
//+------------------------------------------------------------------+
void CMinBC::MinBCRestartFrom(CMinBCState &State,CRowDouble &x)
{
//--- create variables
int n=State.m_nmain;
//--- First, check for errors in the inputs
if(!CAp::Assert(CAp::Len(x)>=n,__FUNCTION__+": Length(X)<N"))
return;
if(!CAp::Assert(CApServ::IsFiniteVector(x,n),__FUNCTION__+": X contains infinite or NaN values!"))
return;
//--- Set XC
State.m_xstart=x;
//--- prepare RComm facilities
CAblasF::ISetAllocV(7,0,State.m_rstate.ia);
CAblasF::BSetAllocV(2,false,State.m_rstate.ba);
CAblasF::RSetAllocV(8,0,State.m_rstate.ra);
State.m_rstate.stage=-1;
ClearRequestFields(State);
}
//+------------------------------------------------------------------+
//| This subroutine submits request for termination of running |
//| optimizer. It should be called from user-supplied callback when |
//| user decides that it is time to "smoothly" terminate optimization|
//| process. As result, optimizer stops at point which was "current |
//| accepted" when termination request was submitted and returns |
//| error code 8 (successful termination). |
//| INPUT PARAMETERS: |
//| State - optimizer structure |
//| NOTE: after request for termination optimizer may perform several|
//| additional calls to user-supplied callbacks. It does NOT |
//| guarantee to stop immediately - it just guarantees that |
//| these additional calls will be discarded later. |
//| NOTE: calling this function on optimizer which is NOT running |
//| will have no effect. |
//| NOTE: multiple calls to this function are possible. First call is|
//| counted, subsequent calls are silently ignored. |
//+------------------------------------------------------------------+
void CMinBC::MinBCRequestTermination(CMinBCState &State)
{
State.m_userterminationneeded=true;
}
//+------------------------------------------------------------------+
//| Clears request fileds (to be sure that we don't forget to clear |
//| something) |
//+------------------------------------------------------------------+
void CMinBC::ClearRequestFields(CMinBCState &State)
{
State.m_needf=false;
State.m_needfg=false;
State.m_xupdated=false;
}
//+------------------------------------------------------------------+
//| Internal initialization subroutine. |
//+------------------------------------------------------------------+
void CMinBC::MinBCInitInternal(int n,CRowDouble &x,double diffstep,
CMinBCState &State)
{
//--- create variables
CMatrixDouble c;
CRowInt ct;
//--- Initialize
State.m_teststep=0;
State.m_smoothnessguardlevel=0;
COptServ::SmoothnessMonitorInit(State.m_smonitor,State.m_s,0,0,false);
State.m_nmain=n;
State.m_diffstep=diffstep;
CApServ::BVectorSetLengthAtLeast(State.m_HasBndL,n);
CApServ::BVectorSetLengthAtLeast(State.m_HasBndU,n);
State.m_bndl.Resize(n);
State.m_bndu.Resize(n);
State.m_xstart.Resize(n);
State.m_xc.Resize(n);
State.m_cgc.Resize(n);
State.m_ugc.Resize(n);
State.m_xn.Resize(n);
State.m_cgn.Resize(n);
State.m_ugn.Resize(n);
State.m_xp.Resize(n);
State.m_d.Resize(n);
State.m_s.Resize(n);
State.m_invs.Resize(n);
State.m_lastscaleused.Resize(n);
State.m_x.Resize(n);
State.m_g.Resize(n);
State.m_work.Resize(n);
State.m_bndl.Fill(AL_NEGINF);
ArrayInitialize(State.m_HasBndL,false);
State.m_bndu.Fill(AL_POSINF);
ArrayInitialize(State.m_HasBndU,false);
State.m_s.Fill(1.0);
State.m_invs.Fill(1.0);
State.m_lastscaleused.Fill(1.0);
MinBCSetCond(State,0.0,0.0,0.0,0);
MinBCSetXRep(State,false);
MinBCSetStpMax(State,0.0);
MinBCSetPrecDefault(State);
MinBCRestartFrom(State,x);
}
//+------------------------------------------------------------------+
//| This subroutine updates estimate of the good step length given: |
//| 1) previous estimate |
//| 2) new length of the good step |
//| It makes sure that estimate does not change too rapidly - ratio |
//| of new and old estimates will be at least 0.01, at most 100.0 |
//| In case previous estimate of good step is zero (no estimate), new|
//| estimate is used unconditionally. |
//+------------------------------------------------------------------+
void CMinBC::UpdateEstimateOfGoodStep(double &estimate,double newstep)
{
if(estimate==0.0)
{
estimate=newstep;
return;
}
if(newstep<(estimate*0.01))
{
estimate*=0.01;
return;
}
if(newstep>estimate*100)
{
estimate*=100;
return;
}
estimate=newstep;
}
//+------------------------------------------------------------------+
//| This is a test problem class intended for internal performance |
//| tests. |
//| Never use it directly in your projects. |
//+------------------------------------------------------------------+
struct CLPTestProblem
{
int m_m;
int m_n;
double m_targetf;
bool m_hasknowntarget;
CSparseMatrix m_a;
CRowDouble m_al;
CRowDouble m_au;
CRowDouble m_bndl;
CRowDouble m_bndu;
CRowDouble m_c;
CRowDouble m_s;
//--- constructor / destructor
CLPTestProblem(void);
~CLPTestProblem(void) {}
//---
void Copy(const CLPTestProblem &obj);
//--- overloading
void operator=(const CLPTestProblem &obj) { Copy(obj); }
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
CLPTestProblem::CLPTestProblem(void)
{
m_m=0;
m_n=0;
m_targetf=0;
m_hasknowntarget=false;
}
//+------------------------------------------------------------------+
//| Copy |
//+------------------------------------------------------------------+
void CLPTestProblem::Copy(const CLPTestProblem &obj)
{
m_m=obj.m_m;
m_n=obj.m_n;
m_targetf=obj.m_targetf;
m_hasknowntarget=obj.m_hasknowntarget;
m_a=obj.m_a;
m_al=obj.m_al;
m_au=obj.m_au;
m_bndl=obj.m_bndl;
m_bndu=obj.m_bndu;
m_c=obj.m_c;
m_s=obj.m_s;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class COPTS
{
public:
static void LPTestProblemCreate(int n,bool hasknowntarget,double targetf,CLPTestProblem &p);
static bool LPTestProblemHasKnownTarget(CLPTestProblem &p);
static double LPTestProblemGetTargetF(CLPTestProblem &p);
static int LPTestProblemGetN(CLPTestProblem &p);
static int LPTestProblemGetM(CLPTestProblem &p);
static void LPTestProblemSetScale(CLPTestProblem &p,CRowDouble &s);
static void LPTestProblemSetCost(CLPTestProblem &p,CRowDouble &c);
static void LPTestProblemSetBC(CLPTestProblem &p,CRowDouble &bndl,CRowDouble &bndu);
static void LPTestProblemSetLC2(CLPTestProblem &p,CSparseMatrix &a,CRowDouble &al,CRowDouble &au,int m);
static void LPTestProblemAlloc(CSerializer &s,CLPTestProblem &p);
static void LPTestProblemSerialize(CSerializer &s,CLPTestProblem &p);
static void LPTestProblemUnserialize(CSerializer &s,CLPTestProblem &p);
static void XDBGMinLPCreateFromTestProblem(CLPTestProblem &p,CMinLPState &State);
};
//+------------------------------------------------------------------+
//| Initialize test LP problem. |
//| This function is intended for internal use by ALGLIB. |
//+------------------------------------------------------------------+
void COPTS::LPTestProblemCreate(int n,bool hasknowntarget,
double targetf,CLPTestProblem &p)
{
//--- check
if(!CAp::Assert(n>=1,__FUNCTION__+": N<1"))
return;
p.m_n=n;
p.m_hasknowntarget=hasknowntarget;
if(hasknowntarget)
p.m_targetf=targetf;
else
p.m_targetf=AL_NaN;
p.m_s=vector<double>::Ones(n);
p.m_c=vector<double>::Zeros(n);
p.m_bndl=vector<double>::Zeros(n);
p.m_bndu=vector<double>::Zeros(n);
p.m_m=0;
p.m_al.Resize(0);
p.m_au.Resize(0);
}
//+------------------------------------------------------------------+
//| Query test problem info |
//| This function is intended for internal use by ALGLIB. |
//+------------------------------------------------------------------+
bool COPTS::LPTestProblemHasKnownTarget(CLPTestProblem &p)
{
return(p.m_hasknowntarget);
}
//+------------------------------------------------------------------+
//| Query test problem info |
//| This function is intended for internal use by ALGLIB. |
//+------------------------------------------------------------------+
double COPTS::LPTestProblemGetTargetF(CLPTestProblem &p)
{
return(p.m_targetf);
}
//+------------------------------------------------------------------+
//| Query test problem info |
//| This function is intended for internal use by ALGLIB. |
//+------------------------------------------------------------------+
int COPTS::LPTestProblemGetN(CLPTestProblem &p)
{
return(p.m_n);
}
//+------------------------------------------------------------------+
//| Query test problem info |
//| This function is intended for internal use by ALGLIB. |
//+------------------------------------------------------------------+
int COPTS::LPTestProblemGetM(CLPTestProblem &p)
{
return(p.m_m);
}
//+------------------------------------------------------------------+
//| Set scale for test LP problem |
//| This function is intended for internal use by ALGLIB. |
//+------------------------------------------------------------------+
void COPTS::LPTestProblemSetScale(CLPTestProblem &p,CRowDouble &s)
{
p.m_s=s;
p.m_s.Resize(p.m_n);
}
//+------------------------------------------------------------------+
//| Set cost for test LP problem |
//| This function is intended for internal use by ALGLIB. |
//+------------------------------------------------------------------+
void COPTS::LPTestProblemSetCost(CLPTestProblem &p,CRowDouble &c)
{
p.m_c=c;
p.m_c.Resize(p.m_n);
}
//+------------------------------------------------------------------+
//| Set box constraints for test LP problem |
//| This function is intended for internal use by ALGLIB. |
//+------------------------------------------------------------------+
void COPTS::LPTestProblemSetBC(CLPTestProblem &p,CRowDouble &bndl,
CRowDouble &bndu)
{
p.m_bndl=bndl;
p.m_bndl.Resize(p.m_n);
p.m_bndu=bndu;
p.m_bndu.Resize(p.m_n);
}
//+------------------------------------------------------------------+
//| Set box constraints for test LP problem |
//| This function is intended for internal use by ALGLIB. |
//+------------------------------------------------------------------+
void COPTS::LPTestProblemSetLC2(CLPTestProblem &p,CSparseMatrix &a,
CRowDouble &al,
CRowDouble &au,
int m)
{
//--- quick exit
if(m<=0)
{
p.m_m=0;
return;
}
//--- check
if(!CAp::Assert(CSparse::SparseGetNRows(a)==m,__FUNCTION__+": rows(A)<>M"))
return;
p.m_m=m;
CSparse::SparseCopyToCRS(a,p.m_a);
p.m_al=al;
p.m_au=au;
p.m_al.Resize(m);
p.m_au.Resize(m);
}
//+------------------------------------------------------------------+
//| Serializer: allocation |
//+------------------------------------------------------------------+
void COPTS::LPTestProblemAlloc(CSerializer &s,CLPTestProblem &p)
{
s.Alloc_Entry();
s.Alloc_Entry();
s.Alloc_Entry();
s.Alloc_Entry();
s.Alloc_Entry();
CApServ::AllocRealArray(s,p.m_s,p.m_n);
CApServ::AllocRealArray(s,p.m_c,p.m_n);
CApServ::AllocRealArray(s,p.m_bndl,p.m_n);
CApServ::AllocRealArray(s,p.m_bndu,p.m_n);
s.Alloc_Entry();
if(p.m_m>0)
{
CSparse::SparseAlloc(s,p.m_a);
CApServ::AllocRealArray(s,p.m_al,p.m_m);
CApServ::AllocRealArray(s,p.m_au,p.m_m);
}
s.Alloc_Entry();
}
//+------------------------------------------------------------------+
//| Serializer: serialization |
//+------------------------------------------------------------------+
void COPTS::LPTestProblemSerialize(CSerializer &s,CLPTestProblem &p)
{
s.Serialize_Int(CSCodes::GetLpTestSerializationCode());
s.Serialize_Int(0);
s.Serialize_Int(p.m_n);
s.Serialize_Bool(p.m_hasknowntarget);
s.Serialize_Double(p.m_targetf);
CApServ::SerializeRealArray(s,p.m_s,p.m_n);
CApServ::SerializeRealArray(s,p.m_c,p.m_n);
CApServ::SerializeRealArray(s,p.m_bndl,p.m_n);
CApServ::SerializeRealArray(s,p.m_bndu,p.m_n);
s.Serialize_Int(p.m_m);
if(p.m_m>0)
{
CSparse::SparseSerialize(s,p.m_a);
CApServ::SerializeRealArray(s,p.m_al,p.m_m);
CApServ::SerializeRealArray(s,p.m_au,p.m_m);
}
s.Serialize_Int(872);
}
//+------------------------------------------------------------------+
//| Serializer: unserialization |
//+------------------------------------------------------------------+
void COPTS::LPTestProblemUnserialize(CSerializer &s,CLPTestProblem &p)
{
int k=s.Unserialize_Int();
//--- check
if(!CAp::Assert(k==CSCodes::GetLpTestSerializationCode(),__FUNCTION__+": stream header corrupted"))
return;
k=s.Unserialize_Int();
//--- check
if(!CAp::Assert(k==0,__FUNCTION__+": stream header corrupted"))
return;
p.m_n=s.Unserialize_Int();
p.m_hasknowntarget=s.Unserialize_Bool();
p.m_targetf=s.Unserialize_Double();
CApServ::UnserializeRealArray(s,p.m_s);
CApServ::UnserializeRealArray(s,p.m_c);
CApServ::UnserializeRealArray(s,p.m_bndl);
CApServ::UnserializeRealArray(s,p.m_bndu);
p.m_m=s.Unserialize_Int();
if(p.m_m>0)
{
CSparse::SparseUnserialize(s,p.m_a);
CApServ::UnserializeRealArray(s,p.m_al);
CApServ::UnserializeRealArray(s,p.m_au);
}
k=s.Unserialize_Int();
//--- check
if(!CAp::Assert(k==872,__FUNCTION__+": end-of-stream marker not found"))
return;
}
//+------------------------------------------------------------------+
//| This is internal function intended to be used only by ALGLIB |
//| itself. Although for technical reasons it is made publicly |
//| available (and has its own manual entry), you should never call |
//| it. |
//+------------------------------------------------------------------+
void COPTS::XDBGMinLPCreateFromTestProblem(CLPTestProblem &p,
CMinLPState &State)
{
CMinLP::MinLPCreate(p.m_n,State);
CMinLP::MinLPSetScale(State,p.m_s);
CMinLP::MinLPSetCost(State,p.m_c);
CMinLP::MinLPSetBC(State,p.m_bndl,p.m_bndu);
CMinLP::MinLPSetLC2(State,p.m_a,p.m_al,p.m_au,p.m_m);
}
//+------------------------------------------------------------------+
Reference in New Issue View Git Blame Copy Permalink