topic Re: FGMRES Using initial solution has worse performance than not using it in Intel® oneAPI Math Kernel Library

FGMRES Using initial solution has worse performance than not using it

DvdC — Fri, 01 Jul 2022 13:43:03 GMT

Hello. I have encountered some unexpected behavior when using the RCI FGMRES solver.

The solver is used in a c# environment. I am solving a system of 108 unknown, with the initial solution array x set to zero. The solution converges in 209 iterations to 1e-6 tolerance.

However, if i take the converged solution, and use it as initial solution, the solver will now take 1833 iterations (and converge to the same tolerance 1e-6).

I believe the solver is implemented correctly, as everything behaves as expected, except this issue with initial solution.

I have attached txt files for the system. Rhs, Matrix (as full matrix, row major) and Matrix (as CSR3 format) and initial solution (i.e. converged solution when initial solution is set to zero).

Re: FGMRES Using initial solution has worse performance than not using it

VidyalathaB_Intel — Mon, 04 Jul 2022 09:27:35 GMT

Hi David,

Thanks for reaching out to us.

Could you please provide us with the following details

sample reproducer code
steps you have followed to reproduce the issue
MKL version being used
OS details on which you are working

so that we can test the same from our end as well?

Additionally, we suggest you refer to the provided example code for FGMRES solver that comes with MKL installation.

Regards,

Vidya.

Re: FGMRES Using initial solution has worse performance than not using it

DvdC — Fri, 08 Jul 2022 11:21:22 GMT

Hi Vidya,

Thanks for the reply. I have attached a .zip file containing the code to reproduce the issue. This is a VisualStudio solution which targets .Net framework 4.7.2.

Program.cs contains all the code and input data, and the only dependency are the MKL .dll files. We are using MKL version 2020.0.2.1 and Windows 10 Pro Ver. 21H2.

The solver behaves as expected except for this issue with the initial solution. In fact when using the CG solver (which is implemented the very similarly to the FMGRES solver), this behaves as expected, and passing an initial solution does improve the convergence.

Please let me know if you need any additional info or clarification,

Kind Regards,

David

Re:FGMRES Using initial solution has worse performance than not using it

VidyalathaB_Intel — Fri, 08 Jul 2022 14:45:41 GMT

Hi David,

Thanks for sharing the project file.

Could you please share with us the sample reproducer with the CG solver as well?

Meanwhile, we suggest you to please try the latest version of oneMKL 2022.1 and confirm if you still see the same performance for the FGMRES solver.

Regards,

Vidya.

Re: FGMRES Using initial solution has worse performance than not using it

DvdC — Mon, 11 Jul 2022 14:16:12 GMT

Hi Vidya,

I have attached the solution which includes the CG solver as well. Also, we have tried MKL 2022.1 and the problem persists (actually has been made worse).

We got the following results.

FGMRES 2020.0.2.1:

Zero Solution 209 iterations

Initial Solution 1833

FGMRES 2022.1:

Zero Solution 209 iterations

Initial Solution 2808

CG 2020.0.2.1:

Zero Solution 3348 iterations

Initial Solution 105

Thanks,

David

Re:FGMRES Using initial solution has worse performance than not using it

VidyalathaB_Intel — Wed, 13 Jul 2022 08:55:59 GMT

Hi David,

Thanks for sharing the details.

We are able to reproduce the issue from our end with oneMKL 2022.1. We are working on it and we will get back to you soon.

Regards,

Vidya.

Re: Re:FGMRES Using initial solution has worse performance than not using it

Hsbsh — Fri, 07 Oct 2022 18:44:38 GMT

Hi, 2022.2 as part of 2022.3 oneAPI SDK no longer allows to reproduce. It says sparseBLASMatrixVectorMultiplyCSRFormat caused an exception.

Re: FGMRES Using initial solution has worse performance than not using it

Hsbsh — Sat, 24 Dec 2022 08:34:41 GMT

There is still a problem on 2023.0.

209 vs 2808 for FGMRES

And

209 vs 4275 for CG

Re: FGMRES Using initial solution has worse performance than not using it

Spencer_P_Intel — Mon, 26 Dec 2022 06:38:21 GMT

Hi Hsbsh,

I see that in RCIISSExternalInterface :: SolveUsingFGMRES() , they are setting the following options:

initializeFGMRESSolver(ref n, x, b, ref request, ipar, dpar, tmp);
ipar[4] = maxIter == 150 ? Math.Min(maxIter, n) : maxIter;            
ipar[7] = 1;   // do stopping test for the maximum number of iterations            
ipar[8] = 1;   // do residual stopping test            
ipar[9] = 0;   // do not request for the user defined stopping test            
ipar[11] = 1;  // do the check of the norm of the next generated vector automatically            
dpar[0] = tolerance;

which is good when the initial guess is "far" from the exact one, but I bet if they specified both relative and absolute tolerances

dpar[0] = tolerance;  // relative tolerance
dpar[1] = 1e-8;       // absolute tolerance

they might find that it converges in a more reasonable amount of time. When they only have relative tolerance and they give the exact answer, then they are checking "error <= tolerance * ||A*x - b|| " but for exact solution, ||A * x - b|| is already very small so we may be getting to requiring machine precision here for error and it may take a very long time (if ever) to achieve the required precisions... When we add in the absolute tolerance, " error <= abs_tolerance" , it should exit with success almost immediately when true solution is passed in...

You can verify the actual measurements using dpar[2] and dpar[4] to check the current residual and original residual.

Best,

Spencer

Re: FGMRES Using initial solution has worse performance than not using it

Hsbsh — Tue, 27 Dec 2022 22:11:57 GMT

What about the "the problem persists (actually has been made worse)."

Re: FGMRES Using initial solution has worse performance than not using it

Spencer_P_Intel — Tue, 27 Dec 2022 22:19:10 GMT

>What about the "the problem persists (actually has been made worse)."

I am not sure I understand. Can you please elaborate ?

Re: FGMRES Using initial solution has worse performance than not using it

Hsbsh — Tue, 27 Dec 2022 22:21:53 GMT

FGMRES 2022.1 is worse than even FGMRES 2020.0.2.1. Is that expected? I mean it is bad in both cases, so does not really matter, I suppose.

Re: FGMRES Using initial solution has worse performance than not using it

Spencer_P_Intel — Tue, 27 Dec 2022 23:15:30 GMT

The original post and concern was understood to be an issue in the application since it was using only relative tolerance for convergence and resolved by adding in an absolute tolerance for convergence. Do you have a different issue with fgmres or are you continuing the original post ?