Intel® oneAPI Math Kernel Library
Ask questions and share information with other developers who use Intel® Math Kernel Library.
6956 Discussions

RCI ISS solver FGMRES working well in IPS XE 2020 Update 4 does not work in oneAPI 2021.2

jirina
New Contributor I
1,214 Views

Hello.

I have a problem with the FGMRES solver. While its worked well in IPS XE 2020 Update 4, the very same source code does not work in oneAPI 2021.2 (and also 2021.1), issuing following warning to the file "MKL_RCI_FGMRES_Log.txt" after calling "dfgmres_check", with the RCI_request returned -1001:

Intel MKL RCI FGMRES WARNING:
Parameters IPAR(1), IPAR(15) and IPAR(16) have incompatible values.
Some of these parameters were probably altered after DFGMRES init routine.

The same warning is repeated multiple times, with IPAR(16) changing to 17, 18, 19, 20, 21.

After calling "dfgmres_init", following values of ipar are modified from their respective default values:

ipar(2) = 1 ! Error/Warnings output: 6 (default) = screen, 0 = none, other = text files

ipar(5) = maxIterationsISS ! Maximum number of iterations (user-defined integer value)

ipar(10) = 0 ! user-defined stopping test not performed

ipar(11) = 1 ! RCI_request=3 asking for preconditioning will be generated

ipar(15) = min(restartIterationsISS,maxIterationsISS) ! both integer values are user-defined

In my test, I have always been using maxIterationsISS = restartIterationsISS so far, which means ipar(5) = ipar(15) and, if I understand documentation well, that by default the non-restarted version of FGMRES method is used.

In addition, there is one confusing thing which I have not noticed before:

To initialize ipar and dpar, I call "dfgmres_init". One of this subroutine's argument is "tmp" array whose size depends on ipar(15) that I would like to change later. Does it mean "tmp" does not need to be allocated at the time of the call to "dfgmres_init"? Does the initialization do anything with "tmp"?

0 Kudos
1 Solution
Kirill_V_Intel
Employee
1,163 Views

Hello!

Between 2020u4 and 2021.2 we have made a change for fgmres routines, aiming to straighten up the code logic. Let me explain what we did.

For 2020u4:
dfgmres_init does not initialize ipar[15]-ipar[20].
dfmgres_check always initialiazes ipar[15]-ipar[20]. 
So, opposite to what the name suggested, a call to dfgmres_check was not optional.

For 2021.2:
dfgmres_init initializes ipar[15]-ipar[20] based on the values of other ipar (in particular, size of tmp array defined in ipar[14]).
dfmgres_check checks if there is no inconsitency between values of ipar. If it finds an inconsistency for ipar[15]-ipar[20], it returns the new code -1001 and re-initializes the inconsistent values of ipar[15]-ipar[20] with values implied by other ipar values.

This allows the customer to change ipar values after dfgmres_init and get ipar[15]-ipar[20] corrected during the call dfgmres_check. And the call to dfgmres_check is now optional (but recommended) if the customer is sure about how to set ipar[15]-ipar[20].

We have updated our documentation, and modified examples to handle the new return code (-1001) which is not an error but merely a warning (we thought it would be a good idea to notify users that dfgmres_check has had an effective side-effect of changing the values of ipar).

So, it should work fine if you call dfgmres_init, then change, e.g. ipar[4] = ipar(5) and then call dfmres_check. The latter call will fix the values of ipar[15]-ipar[20] set during dfmres_init by the values which reflect your changes. Warning 1001 can be ignored (and if it bothers you, get disabled by setting lower verbosity through ipar[6]).

I hope this answers your question.

Best,
Kirill

 

View solution in original post

8 Replies
ArpitaP_Intel
Moderator
1,174 Views

Hi,


Can you please specify your OS version? Also, can you share a minimal reproducer code along with output/warning?


>> The same warning is repeated multiple times, with IPAR(16) changing to 17, 18, 19, 20, 21.


In particular, the routine checks the consistency of ipar(16)-ipar(21) and ipar(1), ipar(15). If the values do not agree, the routine emits a warning and modifies ipar(16)-ipar(21) to comply with the values of ipar(1), ipar(15).


For further reference you can follow the below link


https://software.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-fortran/top/sparse-solver-routines/iterative-sparse-solvers-based-on-reverse-communication-interface-rci-iss/rci-iss-routines/dfgmres-check.html


Regards,

Arpita


0 Kudos
Kirill_V_Intel
Employee
1,164 Views

Hello!

Between 2020u4 and 2021.2 we have made a change for fgmres routines, aiming to straighten up the code logic. Let me explain what we did.

For 2020u4:
dfgmres_init does not initialize ipar[15]-ipar[20].
dfmgres_check always initialiazes ipar[15]-ipar[20]. 
So, opposite to what the name suggested, a call to dfgmres_check was not optional.

For 2021.2:
dfgmres_init initializes ipar[15]-ipar[20] based on the values of other ipar (in particular, size of tmp array defined in ipar[14]).
dfmgres_check checks if there is no inconsitency between values of ipar. If it finds an inconsistency for ipar[15]-ipar[20], it returns the new code -1001 and re-initializes the inconsistent values of ipar[15]-ipar[20] with values implied by other ipar values.

This allows the customer to change ipar values after dfgmres_init and get ipar[15]-ipar[20] corrected during the call dfgmres_check. And the call to dfgmres_check is now optional (but recommended) if the customer is sure about how to set ipar[15]-ipar[20].

We have updated our documentation, and modified examples to handle the new return code (-1001) which is not an error but merely a warning (we thought it would be a good idea to notify users that dfgmres_check has had an effective side-effect of changing the values of ipar).

So, it should work fine if you call dfgmres_init, then change, e.g. ipar[4] = ipar(5) and then call dfmres_check. The latter call will fix the values of ipar[15]-ipar[20] set during dfmres_init by the values which reflect your changes. Warning 1001 can be ignored (and if it bothers you, get disabled by setting lower verbosity through ipar[6]).

I hope this answers your question.

Best,
Kirill

 

jirina
New Contributor I
1,150 Views

Hello Kirill,

Thank you very much for your explanation; it made it clear. I could see also in the sample code (which I have not checked before) that the warning code -1001 is processed as informational only, not a problem.

I understand how initialization and checking work now and I have the same functionality as before.

I would like to ask several additional questions regarding FGMRES interface description (https://software.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-fortran/top/sparse-solver-routines/iterative-sparse-solvers-based-on-reverse-communication-interface-rci-iss/fgmres-interface-description.html#fgmres-interface-description_RCI_ID_COMMONPARAMETERS

  1. The documentation states that "ipar(24:128) are reserved and not used in the current RCI FGMRES routines" (the same is for dpar(9:128)), but preconditioning seems to use ipar(31) and dpar(31:32) to set how zero diagonal elements are treated. I use ipar(31) = 1 and nonzero dpar(31:32) in my code. Is this OK? Calculation seems to run well.
  2. Regarding the tmp array, the documentation states: "You can define this array in the code as DOUBLE PRECISION tmp((2*ipar(15)+1)*n + ipar(15)*(ipar(15)+9)/2 + 1))  if you run only non-preconditioned FGMRES iterations." I have the tmp array defined in exactly the same way and I am running preconditioned FGMRES iterations. Should I expect any problems?

Jiri

0 Kudos
Kirill_V_Intel
Employee
1,130 Views

Hi again,

Answering your questions:

1) I don't see any evidence that ipar(31) and dpar(31:32) have any effect in fgmres. If you can, please share an example of how you're using it. I believe the mentioned parameters are simply ignored in the code of FGMRES (unlike other iterative solvers).

2) I think the formula in the NOTE which you cited is incorrect. For non-preconditioned FGMRES the memory requirements are smaller and less memory can be used but the formula in the note is the same as the full tmp size (which will be sufficient for both preconditoned and non-preconditioned version). We'll check what the formula should be for the non-preconditioned case but you must be safe anyway with tmp of that size. 

Best,
Kirill

0 Kudos
jirina
New Contributor I
1,120 Views

Hi Kirill,

Information that parameters ipar(31) and dpar(31:32) are used by the ILU0 preconditioner can be found here: https://software.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-fortran/top/sparse-solver-routines/preconditioners-based-on-incomplete-lu-factorization-technique/dcsrilu0.html

Maybe this information is outdated and they are simply ignored.

In my code, I set ipar(31) = 1, dpar(31) = 1.0d-16, dpar(32) = 1.0d-10. I have not tried keeping these values as initialized by dfmgres_init.

Best regards,

Jiri

0 Kudos
Kirill_V_Intel
Employee
1,117 Views

Hi,

Yes, sorry for not explaining it well. I meant that the parameters are ignored for fgmres code (which implements FGMRES method) specifically. They are used indeed by ILU0 preconditioner. 

Update regarding my second point about the tmp size. I got quite certain that the NOTE should say

"You can define this array in the code as double tmp[(ipar[14] + 1)*n + ipar[14]*(ipar[14] + 9)/2 + 1)] if you run only non-preconditioned FGMRES iterations."

(so, a factor of 2 should be removed from the first term with ipar[14]*n in comparison to the general formula for tmp size).

Best,
Kirill

 

0 Kudos
jirina
New Contributor I
1,108 Views

Hi,

Got it. Everything is clear now.

Thank you again for your kind and patient support.

Best regards,

Jiri

0 Kudos
ArpitaP_Intel
Moderator
1,090 Views

Hi,


Thanks for accepting the solution. Intel will no longer monitor this thread. Further discussions on this thread will be considered community only.


Regards,

Arpita


0 Kudos
Reply