Help on running GMRES with MKL in C++

[Dilshan_Sithira_Priy]

Hi,

I am new to MKL library. I tried to use to use it to solve a sparse system matrix using “GMRES” functions in MKL library.

I tried the steps in the given in examples which come with the MKL library ( “fgmres_st_criterion_c.c”). I was able to compile and run the example with ICC v19.0 ( I am using Windows 10 64bit with Intel Parallel studio 2019. I have a student license). The problem is when I ported the code into my C++ program, nothing seems to be working.

My code is written in C++ and I am using template classes. I am not sure whether this is possible or not. I have included my code until gmres solver is initialized. The initialization fails and the error code is not listed in the documentation (“RCI_request =-3689348814741910324” ). “ipar” variable has wrong values.

Additionally, I notice something strange. Let's say if I am to set “RCI_request =0” (which is not needed) before calling “dfgmres_init”. Then “dfgmres_init” will simply return RCI_request =0. But still, the “ipar” variable values are wrong. I notice the same behavior with the Conjugate Gradient method.

I have 16k elements in my problem but it will be much larger when the mesh is refined. ( n = 16000).

Thank you all in advance

Dilshan

template<class T>

void SPMV<T>::call_gmres_solver_mkl(const vector<T>& coefNeigbor, const vector<T>& RHS,

       vector<T>& phi, vector<T>& phiOld) {

if (sizeof(T) == sizeof(float)) {

              cout << "currently this program is written for double floatting point numbers only.\nplease update the source code and recompiler" << endl;

              cout << "Program will terminate now" << endl;

              cin.get();

              exit(0);

       }

       MKL_INT n = nFl;// number of unknows

       MKL_INT nEle = (nFl + (nFl*nface) - (nTtl - nFl));

       if (nEle <= 0) {

              cout << "The number of interfaces must be greater than zero. current value =" << n << endl;

              cout << "Program will exit now" << endl;

              cin.get();

              exit(0);

       }

MKL_INT RCI_request, itercount, expected_itercount = 8;//      

       double *A = (double*)malloc(nEle * sizeof(double));

       double *residual = (double*)malloc(n * sizeof(double));

       int id = 0;

       ia[0] = 0;

       int n0 = n * (2 * n + 1) + (n * (n + 9)) / 2 + 1;

       double *tmp = (double *)malloc(n0 * sizeof(double));

       double *rhs = (double *)malloc(n * sizeof(double));

       double *solution = (double *)malloc(n * sizeof(double));

MKL_INT ipar[128];

       double dpar[128];

// At first run phiOld = 0

for (int i = 0; i < n; i++) {

              solution = phiOld;

              rhs = RHS;

       }

dfgmres_init(&n, solution, rhs, &RCI_request, ipar, dpar, tmp);

if (RCI_request != 0) {

cout << "Some error occured during the gmres initialisation phase. Error code =" << RCI_request << endl;

       goto FAILED;

}

// rest of the code

}

[mecej4]

Decimal ‭-3689348814741910324 is 0xCCCCCCCCCCCCCCCC‬, so I suspect that your code is not calling the MKL routine(s) with the correct ABI. You seem to be passing integer arguments to MKL as 64-bit integers; was that your intention?

MKL is intended to be called from Fortran, C, and anything else that makes C-like calls. It is up to you to make sure that your C++ code calls MKL in a compatible manner.

If you want more help with this problem, please post the full source code, rather than snippets.

[Gennady_F_Intel]

and show how do you link this example

[Dilshan_Sithira_Priy]

Thank you for the reply.

As I mentioned, I am new to using MKL. there was no special requirement for using 64bit integer data types, I have simply followed the provided example files and the user manual ( Intel Math Kernel Library reference guide.pdf - 2019, page 1812). 

previously included code is a small section of large flow code and I have included the function containing gmres call, attaching the entire code is not possible therefore I have attached a relevant section of the source code ("gmres.h") and also the example ( "fgmres_st_criterion_c.c"  ) since it was mentioned.

It will be a great help if someone can provide some documentation or example how to use these sparse solvers in C++ environment. 

thank you

Dilshan

[mecej4 · Accepted solution]

Please provide the answer to Gennady's question. I suspect that you have changed MKL_INT without choosing the corresponding proper settings to use ILP64 libraries instead of LP64 libraries. You may find it useful to print (with printf() or cout) the value of sizeof(RCI_request) just before the call to dfgmres_init().

[Dilshan_Sithira_Priy]

thank you mecej4, for the tip. you are right, I was not using the correct ILP64 setting. Any guess why it was possible for me to run "fgmres_st_criterion_c.c" file without using the correct setting? by the way, previously I did not understand what Gennady asked. sorry for that. 

thank you very much for your time.

Dilshan