I am using the trusted region algorithm (from MKL 10.3) to solve a non linear least square problem with boundary constraints. I followed the documentation and the examples that are provided to implement it in my code, and I find some stability issues with this method. Every time I launch my optimization, I get a (slightly) different results, with different termination criteria, and different number of iteration. It seems to me that the algorithm takes a different path for every trial that I make. As far as I know, the TR algorithm is determinist, and when starting from the same initial guess, it should always converge to the same solution. This is at least what I observe in the matlab implementation of this algorithm.
I precise that I am quite confident in my implementation because the residual that I obtain with MKL TR algorithm are consistant with those of matlab.
Does any one of you has an explanation for this puzzling behavior?
I suspect bugs such as uninitialized variables or array overruns in your program. Please post a "reproducer" program source. If feasible, see whether the same behavior occurs with MKL 11.0.5.
I finally found my mistake. The reason is that i was modifying one parameter in the additional data passed into the objective function, when the objective function was called. This explain the variability in the results of the TR problem. I corrected this and the results are identical when I call the optimizer several time with the same starting point.
However, I still observe a difference in the number of iteration and number of call function. Is it normal, or do you think it means that I miss an other bug?
I checked the initialization of the variable and array overruns, I think there is no problem from this side.