I'm usig the TR solver with constrained bounds and I'm experiencing some problems. My domain and image dimensions are 2 and 170 respectively. I'm currently testing a basic problem to validate conformity the algorithm; my minimization problem has a solution (i.e. a global minimum).
I'm using dtrnlspbc_solve and djacobi with black box interface. I'm using 32 byte-aligned buffers, sequential library, MKL 10.3.9.300
My initial trust region size is 100.
1. Whenever the solver reaches one bound for a variable, it seems to stick to it;
2. The stop criterion number 6 is almost always the reason for stopping. Even if I set eps(5) = 1e-300, in the hope that another stop criterion is reached before eps(5). In ||F(x)||2 - ||F(x) - J(x)s||2 < eps(5), is it that the lhs can take on negative values and trigger an early stop ? (possible bug or typo in the documentation?) Could you please provide a clear interpretation of that parameter?
3. What is the meaning of eps(6) (The trial step precision)? Is it a really a stopping criterion?
4. After many tests, I ended up with eps vector set to ridiculously small values (1e-300), but I know it is more a desperate than a rational choice.
EDIT: I used a too small eps parameter for the djacobix function. By setting it to larger values I'm getting much better performances.
5. What is the recommended value of iter2, the number of iterations of trial step calculation?
My reference algorithm is the trust region reflective algorithm used by MATLAB's optimization toolbox (lsqnonlin), which seems far more performant.
While MATLAB always gets the exact solution (using reasonable initial seeds), the MKL TR solver converges to poor solutions.
I can not for now include a test case with my particular problem, since it has too many dependances with other parts of the application.
The surface of which the solver has to find the minimum is shown in the attached figure. The global minimum is at [270, 1234]. The datatips indicate convergence points (approx) of the solver for different initial seed points.
Since my first post, I'm using an analytic, exact jacobian function.
For example, the final point [880, 259] was seeded with [200 300], and no matter what the eps value is, I always end up with a stopping criterion of 6 for that particular seed (and others). Notice that at my last try, eps is very small! (dtrnlspbc_init won't let me put a 0).
I notice that if I restart the solver with the last solution found, I can finally reach the exact minimum. For example: