FGMRES

[Amar_K_]

Hi!

I'm intending to use Intel mkl with my FORTRAN90 routines to particularly solve a linear system of equations. The coefficient matrix is large, unsymmetric and sparse (typical of Finite Element Method - FEM) and therefore I have selected the FGMRES subroutine from mkl.

I started with the example - fgmres_full_funct_f.f (attached)  from the examples folder of the mkl installation. The first problem was that the example worked only if the same system was used. Changing a number in the EXPECTED_SOLUTION resulted in the solver to crash (i.e. gave no errors/warnings and kept running forever). I figured it was the pre-conditioner that was messing things up. Instead, using the example fgmres_no_precon_f.f(attached) made the situation slightly better. I could have custom definitions for matrix (CSR format) and expected solution and have the solver to converge. However, there still are unresolved issues:

1a. I cannot have control over the maximum number of iterations. I understand that ipar(5) sets the maximum no. of iterations, but the solver still seems to use the default min(150,n) value instead.

1b. I don't understand why expected number of iterations must be an input from the user. Why/how is the user suppose to predict the solver performance. (in the example (size=5*5 => N = 5), expected number of iterations = N = 5)

2. I cannot solve matrices bigger than 150*150. The norm which is expected to be less than 1E-14 jumps to the order of 1! I was just wondering if this 150 (size of the matrix) was related to ipar(5) = min(150,n)?

3. Also noticed that reducing dpar(1)=relative_tolerance to values less than 1e-6 made the examples that worked fine with dpar(1) = 1e-6, to return an error code -1.

4. Similarly, having dpar(2)=absolute_tolerance=1e-6, makes the solver retun an error code -1.

5. Kindly correct me if I'm wrong. My understanding is that every call to fgmres results in just 1 iteration. I would have expected that every call to fgmes would result in a number of internal sub-iterations (until the tolerance value set through dpar(1) and dpar(2) is satisfied).

6. Lastly, I see no change in the behaviour of the solver when using different values for dpar(15)=number of non-restarted iterations. Perhaps the documentation in this regard is not clear or has a typo/s. The documentation says:

ipar(15) :specifies the number of the non-restarted FGMRES iterations. To run the restarted version of the FGMRES method, assign the number of  

            iterations to ipar(15)before the restart. The default value ismin(150,n), which means that by default the non-restarted version of FGMRES

            method is used. 

            My question: If the default is min(150,n), how is it non-restarted GMRES by default? Since the default value is 150 or n, a non-zero number                               gets assigned to ipar(15), in which case it becomes restarted GMRES?  

Apologies for the long list of questions.

Any help-comment-advise on all/any of the questions is welcome and highly appreciated.

Many Thanks

Amar

[Zhang_Z_Intel]

Please allow me some time to investigate the problem. Thanks for sending your test cases.

[Alexander_K_Intel2]

Hi Amar,

Sorry for delay in answering but it’s a real huge questioner J

First of all add following string in your example to make it work correct:

      DOUBLE PRECISION DNRM2

      EXTERNAL DNRM2

I can answer on question with matrix in your example but you also mentioned about matrix bigger size – could you send example with such kind of matrix to reproduce this problem too? About small matrix:

1a: In your example ipar(9) is set to 1 so FGMRES routine performs the residual stopping test that doesn’t achieve on smaller number of iteration. To verify it try to set ipar(9) to 0.

1b: FGMRES is an iterative method so on each iteration step some norm of residual become smaller. So for matrix of huge sizes (about millions) it’s not useful to set number of iteration to the size of matrix. But several iteration steps of FGMRES could achieve sufficiently good approximation of solution vector.

3-4. I can’t reproduce it on small matrices. Probably it’s could happened in situation when maximum number of iteration achieved but norm of residual still huge.

5. FGMRES is RCI solver so one needs to call it to multiply vector on stiffness matrix or preconditioner.  So on each internal or external iteration it’s need to call FGMRES once or several times.

6. By default ipar(15) is equal to ipar(5) so by default non restarted version of FGMRES used. To change it set ipar(15) < ipar(5)

With best regards,

Alexander Kalinkin

[C_J_X_]

Hi Alexander,

    I have a question about the iteration number we get from dfgmres_get. Each time we call it we can get an iteration number. If I set ipar(15) = 2, which means the solver will restart after each 2 iterations, in this case the iteration number we get from dfgmres_get is the inner iteration number, outer iteration number or total interation number? I ask this becasue I want to know how many mat-vec operations does the solver actually do so that I can count the FLOPS inside the solver. Whether inside one iteration there is a mat-vec multiplicaiton operation? Thanks!

C.J X

Alexander Kalinkin (Intel) wrote:

Hi Amar,

Sorry for delay in answering but it’s a real huge questioner J

First of all add following string in your example to make it work correct:

      DOUBLE PRECISION DNRM2

      EXTERNAL DNRM2

I can answer on question with matrix in your example but you also mentioned about matrix bigger size – could you send example with such kind of matrix to reproduce this problem too? About small matrix:

1a: In your example ipar(9) is set to 1 so FGMRES routine performs the residual stopping test that doesn’t achieve on smaller number of iteration. To verify it try to set ipar(9) to 0.

1b: FGMRES is an iterative method so on each iteration step some norm of residual become smaller. So for matrix of huge sizes (about millions) it’s not useful to set number of iteration to the size of matrix. But several iteration steps of FGMRES could achieve sufficiently good approximation of solution vector.

3-4. I can’t reproduce it on small matrices. Probably it’s could happened in situation when maximum number of iteration achieved but norm of residual still huge.

5. FGMRES is RCI solver so one needs to call it to multiply vector on stiffness matrix or preconditioner.  So on each internal or external iteration it’s need to call FGMRES once or several times.

6. By default ipar(15) is equal to ipar(5) so by default non restarted version of FGMRES used. To change it set ipar(15) < ipar(5)

With best regards,

Alexander Kalinkin