How to use Intel® MKL OOC PARDISO

[smile]

My current programming language is fortran, It is based on visual studio and Intel oneapi (https://www.intel.cn/content/www/cn/zh/developer/tools/oneapi/toolkits.html#gs.3pslfc), I now need to call the pardiso solver in the Intel® oneAPI Math Kernel Library in opeapi in my program to solve the large linear equations. Pardiso is a direct solver and solves large linear equations consumes a lot of memory. So I find the information in website

(https://www.intel.cn/content/www/cn/zh/developer/articles/technical/pardiso-tips.html).

in other words, by setting the parameter iparm(60), the default in-core mode (using memory to store information during equation solving process) can be switched to out-of-core mode (using hard disk to store information during equation solving process) to reduce memory consumption and hardware requirements.
By default, iparm(60)=0, that is, using in-core mode, according to the tips in the grave picture, I call pardiso(...) in the program. The command iparm(60)=2 was added before, but the memory consumption in the process of solving large equations was the same as in the in-core mode of iparm(60)=0, and the memory consumption did not decrease.
Then I click on the blue field of article in the picture above, Enter the  this page: https://www.intel.cn/content/www/cn/zh/developer/articles/training/how-to-use-ooc-pardiso.html.

According to the content in this page, I set iparm(59)=2 at this time, but the memory consumption in the process of solving large equations is the same as that in the in-core mode where iparm(60)=0, and the memory consumption is not reduced.
The content in the two levels of the page does not describe the parameters that need to be set to use out-of-core mode (using the hard disk to store information during the solving of the equation), and neither setting can enable out-of-core mode.

What parameters do I need to set to use out-of-core mode?

The last image with a dark background is the code I used to call pardiso.

[VarshaS_Intel]

Hi,

Thanks for posting in Intel Communities.

Could you please let us know the Visual Studio version and Intel MKL version being used by you?

Could you please try changing the iparm[59] value to 2 and let iparm[60] value be 0 (as it is already reserved to 0) and let us know if the issue still persists?  Please refer to the below link for more details regarding the iparm parameters. 

https://www.intel.com/content/www/us/en/docs/onemkl/developer-reference-c/2023-2/pardiso-iparm-parameter.html

>>and the memory consumption is not reduced.

Could you please get back to us regarding the insights, based on which you have confirmed the same memory consumption is used, as it helps us understand the issue better.

And also, could you please provide us with the complete sample reproducer code you are using to investigate at our end?

Thanks & Regards,

Varsha

[smile]

Thanks for your reply. There was a problem with my Intel account that prevented me from replying to your message. It was recently repaired.

Yes, I have tried changing only the iparm[59] value to 2 and iparm[60] value to 0. However, memory consumption didn't change. The Visual Studio version is 2017. The Intel MKL version is 2021.3. 

After modifying the parameters mentioned, I'm pretty sure the memory consumption hasn't changed. 

Here's my code.

subroutine solver_pardiso(mtype, n, length, a, ia, ja, nrhs, iparm, b, x)

implicit none

integer     maxfct, mnum, mtype, phase, n, length, nrhs, error, msglvl

integer(kind=8)           :: pt(64)

integer     iparm(64),perm(n)!, solver

integer     ia(n+1) !ia(9) 

integer     ja(length) !ja(20)

integer,external:: mkl_get_max_threads

real*8  dparm(64) 

complex*16  a(length) !a(20) 

complex*16  b(n) !b(8)

complex*16  x(n) !x(8)

integer i, idum

real*8  ddum !waltime1, waltime2, 

    x = 0.d0   !解出的向量x存于这个数组

    pt = 0     !// pointer initialization

    maxfct = 1 !大多数情况设1

    mnum = 1   !大多数情况设1

    !// mtype = -2: symmetric nonpositive definite matrix, mtype = 2: symmetric positive definite matrix

    !结构有限元常用：2 实对称正定矩阵;-2 实对称不定矩阵;流体计算可能会用到 11实数不对称矩阵

    perm = 0; !一般不用，给个虚参

    nrhs = 1  !方程右边b的列数

    iparm(1) = 0  !// iparm use default values

    iparm(3) = 4

    iparm(59) = 2

    !iparm(60) = 2

    !// iparm(3): parallel threads are obtained by MKL. In general, the number of cpus is the same as the number of threads

    msglvl =0 !设为1则在屏幕上显示计算统计信息

    error = 0  !错误代码，有错则输出一个代码，以便查错。

    phase = 13  !// LU decompose

    !一般设为13，analysis->numericalfactorization->solve->iterativerefinement

    call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

    !phase = 33  !// solve equations

    !call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

    !write ( *,'(a)' ) '>> The solution as follows ...... '

    open(98,file='x.txt')

    do i = 1, n

        write( 98,* ) i,x(i)

    end do

    write( *,* ) 'max memory:',max(iparm(15), iparm(16)+iparm(17))/1024.0/1024.0,'GB.'

    phase = -1 !释放内存

    call pardiso (pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error)

end subroutine solver_pardiso

[VarshaS_Intel]

Hi,

We have not heard back from you. Could you please provide us with an update on your issue?

Thanks & Regards,

Varsha

[smile]

Thanks for your reply. There was a problem with my Intel account that prevented me from replying to your message. It was recently repaired.

Yes, I have tried changing only the iparm[59] value to 2 and iparm[60] value to 0. However, memory consumption didn't change. The Visual Studio version is 2017. The Intel MKL version is 2021.3. 

After modifying the parameters mentioned, I'm pretty sure the memory consumption hasn't changed. 

[VarshaS_Intel]

Hi,

We have not heard back from you. Could you please provide us with an update on your issue?

Thanks & Regards,

Varsha

[smile]

Thanks for your reply. Sorry, I was unable to reply to your message because of some problems with my Intel account. Intel staff just repaired my account recently.

Yes, I have tried changing only the iparm[59] value to 2 and iparm[60] value to 0. However, memory consumption didn't change. The Visual Studio version is 2017. The Intel MKL version is 2021.3. 

After modifying the parameters mentioned, I'm pretty sure the memory consumption hasn't changed. 

Here's my code.

subroutine solver_pardiso(mtype, n, length, a, ia, ja, nrhs, iparm, b, x)

implicit none

integer     maxfct, mnum, mtype, phase, n, length, nrhs, error, msglvl

integer(kind=8)           :: pt(64)

integer     iparm(64),perm(n)!, solver

integer     ia(n+1) !ia(9) 

integer     ja(length) !ja(20)

integer,external:: mkl_get_max_threads

real*8  dparm(64) 

complex*16  a(length) !a(20) 

complex*16  b(n) !b(8)

complex*16  x(n) !x(8)

integer i, idum

real*8  ddum !waltime1, waltime2, 

    x = 0.d0   !解出的向量x存于这个数组

    pt = 0     !// pointer initialization

    maxfct = 1 !大多数情况设1

    mnum = 1   !大多数情况设1

    !// mtype = -2: symmetric nonpositive definite matrix, mtype = 2: symmetric positive definite matrix

    !结构有限元常用：2 实对称正定矩阵;-2 实对称不定矩阵;流体计算可能会用到 11实数不对称矩阵

    perm = 0; !一般不用，给个虚参

    nrhs = 1  !方程右边b的列数

    iparm(1) = 0  !// iparm use default values

    iparm(3) = 4

    iparm(59) = 2

    !iparm(60) = 2

    !// iparm(3): parallel threads are obtained by MKL. In general, the number of cpus is the same as the number of threads

    msglvl =0 !设为1则在屏幕上显示计算统计信息

    error = 0  !错误代码，有错则输出一个代码，以便查错。

    phase = 13  !// LU decompose

    !一般设为13，analysis->numericalfactorization->solve->iterativerefinement

    call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

    !phase = 33  !// solve equations

    !call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

    !write ( *,'(a)' ) '>> The solution as follows ...... '

    open(98,file='x.txt')

    do i = 1, n

        write( 98,* ) i,x(i)

    end do

    write( *,* ) 'max memory:',max(iparm(15), iparm(16)+iparm(17))/1024.0/1024.0,'GB.'

    phase = -1 !释放内存

    call pardiso (pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error)

end subroutine solver_pardiso

[smile]

My English is not very good, if I have described it clearly?

[smile]

Thanks for your reply. There was a problem with my Intel account that prevented me from replying to your message. It was recently repaired.

Yes, I have tried changing only the iparm[59] value to 2 and iparm[60] value to 0. However, memory consumption didn't change. The Visual Studio version is 2017. The Intel MKL version is 2021.3. 

After modifying the parameters mentioned, I'm pretty sure the memory consumption hasn't changed. 

Here's my code.

subroutine solver_pardiso(mtype, n, length, a, ia, ja, nrhs, iparm, b, x)

implicit none

integer     maxfct, mnum, mtype, phase, n, length, nrhs, error, msglvl

integer(kind=8)           :: pt(64)

integer     iparm(64),perm(n)!, solver

integer     ia(n+1) !ia(9) 

integer     ja(length) !ja(20)

integer,external:: mkl_get_max_threads

real*8  dparm(64) 

complex*16  a(length) !a(20) 

complex*16  b(n) !b(8)

complex*16  x(n) !x(8)

integer i, idum

real*8  ddum !waltime1, waltime2, 

    x = 0.d0   !解出的向量x存于这个数组

    pt = 0     !// pointer initialization

    maxfct = 1 !大多数情况设1

    mnum = 1   !大多数情况设1

    !// mtype = -2: symmetric nonpositive definite matrix, mtype = 2: symmetric positive definite matrix

    !结构有限元常用：2 实对称正定矩阵;-2 实对称不定矩阵;流体计算可能会用到 11实数不对称矩阵

    perm = 0; !一般不用，给个虚参

    nrhs = 1  !方程右边b的列数

    iparm(1) = 0  !// iparm use default values

    iparm(3) = 4

    iparm(59) = 2

    !iparm(60) = 2

    !// iparm(3): parallel threads are obtained by MKL. In general, the number of cpus is the same as the number of threads

    msglvl =0 !设为1则在屏幕上显示计算统计信息

    error = 0  !错误代码，有错则输出一个代码，以便查错。

    phase = 13  !// LU decompose

    !一般设为13，analysis->numericalfactorization->solve->iterativerefinement

    call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

    !phase = 33  !// solve equations

    !call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

    !write ( *,'(a)' ) '>> The solution as follows ...... '

    open(98,file='x.txt')

    do i = 1, n

        write( 98,* ) i,x(i)

    end do

    write( *,* ) 'max memory:',max(iparm(15), iparm(16)+iparm(17))/1024.0/1024.0,'GB.'

    phase = -1 !释放内存

    call pardiso (pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error)

end subroutine solver_pardiso

[smile]

Thanks for your reply. There was a problem with my Intel account that prevented me from replying to your message. It was recently repaired.

Yes, I have tried changing only the iparm[59] value to 2 and iparm[60] value to 0. However, memory consumption didn't change. The Visual Studio version is 2017. The Intel MKL version is 2021.3. 

After modifying the parameters mentioned, I'm pretty sure the memory consumption hasn't changed. 

Here's my code.

subroutine solver_pardiso(mtype, n, length, a, ia, ja, nrhs, iparm, b, x)

implicit none

integer     maxfct, mnum, mtype, phase, n, length, nrhs, error, msglvl

integer(kind=8)           :: pt(64)

integer     iparm(64),perm(n)!, solver

integer     ia(n+1) !ia(9) 

integer     ja(length) !ja(20)

integer,external:: mkl_get_max_threads

real*8  dparm(64) 

complex*16  a(length) !a(20) 

complex*16  b(n) !b(8)

complex*16  x(n) !x(8)

integer i, idum

real*8  ddum !waltime1, waltime2, 

    x = 0.d0   !解出的向量x存于这个数组

    pt = 0     !// pointer initialization

    maxfct = 1 !大多数情况设1

    mnum = 1   !大多数情况设1

    !// mtype = -2: symmetric nonpositive definite matrix, mtype = 2: symmetric positive definite matrix

    !结构有限元常用：2 实对称正定矩阵;-2 实对称不定矩阵;流体计算可能会用到 11实数不对称矩阵

    perm = 0; !一般不用，给个虚参

    nrhs = 1  !方程右边b的列数

    iparm(1) = 0  !// iparm use default values

    iparm(3) = 4

    iparm(59) = 2

    !iparm(60) = 2

    !// iparm(3): parallel threads are obtained by MKL. In general, the number of cpus is the same as the number of threads

    msglvl =0 !设为1则在屏幕上显示计算统计信息

    error = 0  !错误代码，有错则输出一个代码，以便查错。

    phase = 13  !// LU decompose

    !一般设为13，analysis->numericalfactorization->solve->iterativerefinement

    call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

    !phase = 33  !// solve equations

    !call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

    !write ( *,'(a)' ) '>> The solution as follows ...... '

    open(98,file='x.txt')

    do i = 1, n

        write( 98,* ) i,x(i)

    end do

    write( *,* ) 'max memory:',max(iparm(15), iparm(16)+iparm(17))/1024.0/1024.0,'GB.'

    phase = -1 !释放内存

    call pardiso (pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error)

end subroutine solver_pardiso

[smile]

Thanks for your reply. There was a problem with my Intel account that prevented me from replying to your message. It was recently repaired.

Yes, I have tried changing only the iparm[59] value to 2 and iparm[60] value to 0. However, memory consumption didn't change. The Visual Studio version is 2017. The Intel MKL version is 2021.3. 

After modifying the parameters mentioned, I'm pretty sure the memory consumption hasn't changed.