Trust Region Algorithm

[skyboating]

it is the first time i use trust region.
i used to use levenberg marquardt, which is most popular in scientific optimization.
And the interface seems to be different as the classical LM. i don't know if it is possible to include also LM algorithm with traditional interface?

[Nikita_S_Intel]

Quoting - skyboating

it is the first time i use trust region.
i used to use levenberg marquardt, which is most popular in scientific optimization.
And the interface seems to be different as the classical LM. i don't know if it is possible to include also LM algorithm with traditional interface?

hi skyboating,

I would like to ask you. Are you interested in LM algorithm? or are you interested in new interface? Could pls you provide an example of traditional interface?

Thank you!
--Nikita

[skyboating]

Quoting - Nikita Shustrov (Intel)

hi skyboating,

I would like to ask you. Are you interested in LM algorithm? or are you interested in new interface? Could pls you provide an example of traditional interface?

Thank you!
--Nikita

Hi, Nikita,
thank you very much for your kind reply.
i am currently doing the curve fitting of two compartment model kinetic modelling.
What i got the first result is that the LM has different results as the Trust Region (I am not sure if i modify my codes correctly. i am still checking it) and it is obvious that LM looks better than the Trust-Region from my first results.
I am not sure if this compariment is fair. I am still investigating my codes.
I used to use LevMar(http://www.ics.forth.gr/~lourakis/levmar/) for my optimization computation.
The interface is provide two pointer function, one to compute function, and the other to compute the jacobian.
Now when i use Trust-Region, the function is not the original function, but the residual directly, i need to pay more attention for the switch, i am not sure if it is necessary.
Thank you very much.
Kuangyu

[Nikita_S_Intel]

Hi Kuangyu,

The MKL TR solvers are RCI solvers. If you are interested in Black-Box solvers (like levmar) you can submit a features request.

"What i got the first result is that the LM has different results as the Trust Region (I am not sure if i modify my codes correctly. i am still checking it) and it is obvious that LM looks better than the Trust-Region from my first results."

Its interesting. Please let me know aboutresults of your experiments.

Thank you!
--Nikita

[skyboating]

Quoting - Nikita Shustrov (Intel)

Hi Kuangyu,

The MKL TR solvers are RCI solvers. If you are interested in Black-Box solvers (like levmar) you can submit a features request.

"What i got the first result is that the LM has different results as the Trust Region (I am not sure if i modify my codes correctly. i am still checking it) and it is obvious that LM looks better than the Trust-Region from my first results."

Its interesting. Please let me know aboutresults of your experiments.

Thank you!
--Nikita

Hi, Nikita,
i found that it is the problem of the jocobian computation.
when i used the finite difference "djacobi", it works well and get the same result as LM.
But when i use analytical computation of Jacobian fjac, the result is wrong.
in your reference manual, i couldn't find the detail about the fjac.
i don't know if it is as well the jacobian of the residual?
and i don't know if it is the following sequence? (n the parameter dimention, m the residual dimentsion
F0,0 F0,1 .... F0,n-1
F1,0 F1,1 .... F1,n-1
.....................
Fm-1,0 Fm-1,1 .... Fm-1,n-1
in a sequence of this form?
thank you very much!
Kuangyu

[skyboating]

Quoting - skyboating

Hi, Nikita,
i found that it is the problem of the jocobian computation.
when i used the finite difference "djacobi", it works well and get the same result as LM.
But when i use analytical computation of Jacobian fjac, the result is wrong.
in your reference manual, i couldn't find the detail about the fjac.
i don't know if it is as well the jacobian of the residual?
and i don't know if it is the following sequence? (n the parameter dimention, m the residual dimentsion
F0,0 F0,1 .... F0,n-1
F1,0 F1,1 .... F1,n-1
.....................
Fm-1,0 Fm-1,1 .... Fm-1,n-1
in a sequence of this form?
thank you very much!
Kuangyu

I have tested to change transpose the fjac sequence. it now get the same result as LM.
cheers!

[Nikita_S_Intel]

Quoting - skyboating

I have tested to change transpose the fjac sequence. it now get the same result as LM.
cheers!

Great! :)

--Nikita

[Thomas_B_3]

Quoting - skyboating

I have tested to change transpose the fjac sequence. it now get the same result as LM.
cheers!

Hi Kuangyu,

what speed-up (if any?) do you observeby using the analytically calculated Jacobi??

Best regards,
thomas