topic MKL 10.2 Update 1 Optimization Solver in IntelĀ® oneAPI Math Kernel Library & IntelĀ® Math Kernel Library
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775290#M949
I have a similar problem. We are using the optimizer from C# code to minimize an error function; we are using the approximate jacobi calculator as well (although we are likely to switch to an analyic formula soon). The outcome is non-deterministic; most of the time, the optimization works well and converges to a reasonable answer that is repeatable, while at random other times, the optimizer's step algorithm fills the "x" array with NaN. This always occurs when the "RCI_Request" variable is set to 1, indicating that a function evaluation is requested; we have a NaN check in the function evaluation that realizes this and throws an exception whenever the array has a NaN in it - incidentally, whenever it has one NaN, it is entirely filled with NaN. We have checked that the function and jacobian evaluations that we pass into MKL have no NaNs. It seems that the step that the routine takes is sometimes filled with NaNs, but when it is not filled with NaN it is repeatable. Why would this happen?<BR /><BR />It is difficult to repeat it; sequential calls with the same input will complete the optimization smoothly or have the NaN in a random fashion. Is there a threading problem?<BR /><BR />It happens on different machines.<BR /><BR />We've noticed that it usually happens more regualrly in Release mode than in Debug mode.<BR /><BR />All of this has led us to suspect it was related to memory management, like the CLR's garbage collector, by Release mode optimizations performed by the compiler, or by Interop marshalling problems. I am not very familiar with all of these topics. We are pinning all 3 arrays ("x", "fec", and "fjac") as is necessary to send the arrays to MKL as pointers. In trying to fix the issue, I used GCHandle.Alloc (followed by GCHandle.Free at the end of the optimization call) to pin the arrays in another way, and it still fills the x array with NaN occasionally. I've also tried making a deep copy of the x array before function evaluations, also to no avail.<BR />Thu, 16 Aug 2012 20:09:48 GMTmichaelkinnally2012-08-16T20:09:48ZMKL 10.2 Update 1 Optimization Solver
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775282#M941
Hi, <BR /><BR />I am trying to optimize an exponential multivariable function using the TR solver (dtrnlsp_solve). I have been going over it for a week and the solver fails miserably. All I get everytime is "NaN" for the coefficients. The same problem is optimized in C# NI libraries.<BR /><BR />The function is sometime like F(x) = a0 + sum(ai*exp(-x/ti)) (i=1 to 6)<BR />'ai' and 'ti' are my design variables. (Vector X)<BR /><BR />I understand it's difficult to gague without further details? Generally, any ideas of what's wrong? Is it normal for the solver to behave this way at times for certain equations? I have successfully used the solver for optimizing other equations.<BR /><BR />Also, I can't seem to find good examples for the central differences solver (DJACOBI_SOLVE). The one described in the manual is very limited and has a lot of errors. Can anyone point me to certain examples using the central differences solver?<BR /><BR /><BR /><BR />Thanks, <BR />Shriram<BR />--<BR />Mechanica Engineer - FEA Development<BR />Moore Nanotechnology Systems<BR />Swanzey, NH - 03431Thu, 22 Jul 2010 15:25:00 GMThttps://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775282#M941shriisall2010-07-22T15:25:00ZMKL 10.2 Update 1 Optimization Solver
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775283#M942
Hi Shriram,<BR /><BR />As I understood, F(x) is not differentiableif t = 0. Moreover, if t approaches 0 then the objective function is unbounded. I would recommend changing aninitial point or using solver for problems with boundary constraints (where X > 0).<BR /><BR />Please let me know how it works. <BR /><BR />Thank you!<BR />--NikitaFri, 23 Jul 2010 04:04:12 GMThttps://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775283#M942Nikita_S_Intel2010-07-23T04:04:12ZMKL 10.2 Update 1 Optimization Solver
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775284#M943
Hey Nikita, <BR /><BR />Thanks for the reply.<BR /><BR />I am not sure if the function should cause any problem. The same function is optimized very well using National Instruments curvefit library. That however uses the Leverberg-Marquardt algorithm and Intel uses the Trust-Region algorithm.<BR /><BR />- I am not sure what you mean by an unbounded function. As t nears zero, the exponential term nears 1 and the function returns a constant value. At t=0, the exponential terms are all equal to 1. The function values are still finite.<BR /><BR />- I was under the impression that the Trust-Region algorithm was a derivative free method. If t = 0, the function value is a constant and hence the derivative is zero.<BR /><BR />However, I tried assigning lower bounds and changing initial guesses, but with little or no success to the results. After certain iterations, the design variables and function values suddenly become "NaN". I can't understand this unexpected behavior.<BR /><BR />But, as you say, if I remove the data points which are equal to 0 or have very small values (like 1E-7, 1E-3) etc. then the solver does a much better job.<BR /><BR /><BR />Thanks again, <BR />ShriramFri, 23 Jul 2010 15:17:36 GMThttps://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775284#M943shriisall2010-07-23T15:17:36ZMKL 10.2 Update 1 Optimization Solver
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775285#M944
<P class="MsoNormal" style="margin: 0cm 0cm 10pt;"><SPAN style="font-size: small;"><SPAN lang="EN-US" style="mso-ansi-language: EN-US;"><SPAN style="font-family: Calibri;">Hi Shriram,<BR /><BR />Actually, TR solvers are not <SPAN style="color: #333333;">derivative free methods,<SPAN style="mso-spacerun: yes;"> </SPAN>users objective function F(x) = y f(x), where F(x): Rn -></SPAN></SPAN></SPAN><SPAN lang="EN-US" style="color: #333333; mso-ansi-language: EN-US;"><SPAN style="font-family: Calibri;">Rm, m>=nshould bea twice differentiable function in Rn. <P></P></SPAN></SPAN></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt;"><SPAN lang="EN-US" style="color: #333333; mso-ansi-language: EN-US;"><SPAN style="font-size: small;"><SPAN style="font-family: Calibri;">As of your function, it is unbounded, if t nears 0 and t < 0. <SPAN style="mso-spacerun: yes;"></SPAN><SPAN style="mso-spacerun: yes;"></SPAN>Something likes that<P></P></SPAN></SPAN></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt 35.4pt;"><SPAN lang="EN-US" style="line-height: 115%; font-family: " courier="" new="">F(x) = 1.935444e+44, t = -1.000000e-02<P></P></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt 35.4pt;"><SPAN lang="EN-US" style="line-height: 115%; font-family: " courier="" new="">F(x) = 1.295015e+49, t = -9.000000e-03<P></P></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt 35.4pt;"><SPAN lang="EN-US" style="line-height: 115%; font-family: " courier="" new="">F(x) = 1.393615e+55, t = -8.000000e-03<P></P></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt 35.4pt;"><SPAN lang="EN-US" style="line-height: 115%; font-family: " courier="" new="">F(x) = 7.932340e+62, t = -7.000000e-03<P></P></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt 35.4pt;"><SPAN lang="EN-US" style="line-height: 115%; font-family: " courier="" new="">F(x) = 1.736785e+73, t = -6.000000e-03<P></P></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt 35.4pt;"><SPAN lang="EN-US" style="line-height: 115%; font-family: " courier="" new="">F(x) = 5.202701e+87, t = -5.000000e-03<P></P></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt 35.4pt;"><SPAN lang="EN-US" style="line-height: 115%; font-family: " courier="" new="">F(x) = 2.697447e+109, t = -4.000000e-03<P></P></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt 35.4pt;"><SPAN lang="EN-US" style="line-height: 115%; font-family: " courier="" new="">F(x) = 4.189477e+145, t = -3.000000e-03<P></P></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt 35.4pt;"><SPAN lang="EN-US" style="line-height: 115%; font-family: " courier="" new="">F(x) = 1.010586e+218, t = -2.000000e-03<P></P></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt 35.4pt;"><SPAN lang="EN-US" style="line-height: 115%; font-family: " courier="" new="">F(x) = inf, t = -1.000000e-03<P></P></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt 35.4pt;"><SPAN lang="EN-US" style="line-height: 115%; font-family: " courier="" new="">F(x) = 1.200000e+00, t = 0.000000e+00<P></P></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt 35.4pt;"><SPAN lang="EN-US" style="line-height: 115%; font-family: " courier="" new="">F(x) = 1.200000e+00, t = 1.000000e-03<P></P></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt 35.4pt;"><SPAN lang="EN-US" style="line-height: 115%; font-family: " courier="" new="">F(x) = 1.200000e+00, t = 2.000000e-03</SPAN><SPAN lang="EN-US" style="color: #333333; mso-ansi-language: EN-US;"><P></P></SPAN></P><P class="MsoNormal" style="margin: 0cm 0cm 0pt;"><SPAN lang="EN-US" style="color: #333333; mso-ansi-language: EN-US;"><SPAN style="font-size: small;"><SPAN style="font-family: Calibri;">Did try to use solver for problems with boundary constraints [ T > 0] and use initial point [ T > 0 ]?<BR /><BR />Thank you!<BR />--Nikita</SPAN></SPAN></SPAN></P>Mon, 26 Jul 2010 03:37:44 GMThttps://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775285#M944Nikita_S_Intel2010-07-26T03:37:44ZMKL 10.2 Update 1 Optimization Solver
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775286#M945
Oh ok. The domain of my design variables is all positive, so I didn't think about the solver attempting -ve design values. <BR /><BR />I think I had tried with lower bounds greater than 0, and still had the same problems. However, I will still make sure once again as what you say makes good sense.<BR /><BR /><BR />Thanks Nikita!<BR />ShriramTue, 27 Jul 2010 16:49:20 GMThttps://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775286#M945shriisall2010-07-27T16:49:20ZMKL 10.2 Update 1 Optimization Solver
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775287#M946
Hey Nikita, <BR /><BR />I had a quick question. I can specify the lower and upper bounds and use the function 'dtrnlspbc_xxx' solver. <BR /><BR />However, I have no upper bound and I cannot set an arbitrary number. Is it possible to set a lower bound and have upper bound, unbounded? <BR /><BR />Thanks, <BR />ShriramTue, 27 Jul 2010 21:11:59 GMThttps://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775287#M946shriisall2010-07-27T21:11:59ZMKL 10.2 Update 1 Optimization Solver
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775288#M947
Hi Shriram,<DIV>Thedtrnlspbc_xxx solver is the solver with boundaries so you need to put some upper boundaries conditions. Nevertheless you could put some boundary estimation that certainly greater than exact solution.</DIV><DIV>With best regards,</DIV><DIV>Alexander Kalinkin</DIV>Wed, 28 Jul 2010 02:17:12 GMThttps://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775288#M947Alexander_K_Intel22010-07-28T02:17:12ZMKL 10.2 Update 1 Optimization Solver
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775289#M948
Hi Shriram,<BR /><BR />Or you can use max. value of double precision as upper bound. <BR /><BR />Thank you!<BR />--NikitaWed, 28 Jul 2010 03:52:35 GMThttps://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775289#M948Nikita_S_Intel2010-07-28T03:52:35ZMKL 10.2 Update 1 Optimization Solver
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775290#M949
I have a similar problem. We are using the optimizer from C# code to minimize an error function; we are using the approximate jacobi calculator as well (although we are likely to switch to an analyic formula soon). The outcome is non-deterministic; most of the time, the optimization works well and converges to a reasonable answer that is repeatable, while at random other times, the optimizer's step algorithm fills the "x" array with NaN. This always occurs when the "RCI_Request" variable is set to 1, indicating that a function evaluation is requested; we have a NaN check in the function evaluation that realizes this and throws an exception whenever the array has a NaN in it - incidentally, whenever it has one NaN, it is entirely filled with NaN. We have checked that the function and jacobian evaluations that we pass into MKL have no NaNs. It seems that the step that the routine takes is sometimes filled with NaNs, but when it is not filled with NaN it is repeatable. Why would this happen?<BR /><BR />It is difficult to repeat it; sequential calls with the same input will complete the optimization smoothly or have the NaN in a random fashion. Is there a threading problem?<BR /><BR />It happens on different machines.<BR /><BR />We've noticed that it usually happens more regualrly in Release mode than in Debug mode.<BR /><BR />All of this has led us to suspect it was related to memory management, like the CLR's garbage collector, by Release mode optimizations performed by the compiler, or by Interop marshalling problems. I am not very familiar with all of these topics. We are pinning all 3 arrays ("x", "fec", and "fjac") as is necessary to send the arrays to MKL as pointers. In trying to fix the issue, I used GCHandle.Alloc (followed by GCHandle.Free at the end of the optimization call) to pin the arrays in another way, and it still fills the x array with NaN occasionally. I've also tried making a deep copy of the x array before function evaluations, also to no avail.<BR />Thu, 16 Aug 2012 20:09:48 GMThttps://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/MKL-10-2-Update-1-Optimization-Solver/m-p/775290#M949michaelkinnally2012-08-16T20:09:48Z