topic Re:How to use Intel® MKL OOC PARDISO in Intel® oneAPI Math Kernel Library

How to use Intel® MKL OOC PARDISO

smile — Mon, 31 Jul 2023 09:53:24 GMT

My current programming language is fortran, It is based on visual studio and Intel oneapi (https://www.intel.cn/content/www/cn/zh/developer/tools/oneapi/toolkits.html#gs.3pslfc), I now need to call the pardiso solver in the Intel® oneAPI Math Kernel Library in opeapi in my program to solve the large linear equations. Pardiso is a direct solver and solves large linear equations consumes a lot of memory. So I find the information in website

(https://www.intel.cn/content/www/cn/zh/developer/articles/technical/pardiso-tips.html).

in other words, by setting the parameter iparm(60), the default in-core mode (using memory to store information during equation solving process) can be switched to out-of-core mode (using hard disk to store information during equation solving process) to reduce memory consumption and hardware requirements.
By default, iparm(60)=0, that is, using in-core mode, according to the tips in the grave picture, I call pardiso(...) in the program. The command iparm(60)=2 was added before, but the memory consumption in the process of solving large equations was the same as in the in-core mode of iparm(60)=0, and the memory consumption did not decrease.
Then I click on the blue field of article in the picture above, Enter the this page: https://www.intel.cn/content/www/cn/zh/developer/articles/training/how-to-use-ooc-pardiso.html.

According to the content in this page, I set iparm(59)=2 at this time, but the memory consumption in the process of solving large equations is the same as that in the in-core mode where iparm(60)=0, and the memory consumption is not reduced.
The content in the two levels of the page does not describe the parameters that need to be set to use out-of-core mode (using the hard disk to store information during the solving of the equation), and neither setting can enable out-of-core mode.

What parameters do I need to set to use out-of-core mode?

The last image with a dark background is the code I used to call pardiso.

Re: How to use Intel® MKL OOC PARDISO

VarshaS_Intel — Wed, 02 Aug 2023 07:22:48 GMT

Hi,

Thanks for posting in Intel Communities.

Could you please let us know the Visual Studio version and Intel MKL version being used by you?

Could you please try changing the iparm[59] value to 2 and let iparm[60] value be 0 (as it is already reserved to 0) and let us know if the issue still persists? Please refer to the below link for more details regarding the iparm parameters.

https://www.intel.com/content/www/us/en/docs/onemkl/developer-reference-c/2023-2/pardiso-iparm-parameter.html

>>and the memory consumption is not reduced.

Could you please get back to us regarding the insights, based on which you have confirmed the same memory consumption is used, as it helps us understand the issue better.

And also, could you please provide us with the complete sample reproducer code you are using to investigate at our end?

Thanks & Regards,

Varsha

Re:How to use Intel® MKL OOC PARDISO

VarshaS_Intel — Mon, 07 Aug 2023 13:28:58 GMT

Hi,

We have not heard back from you. Could you please provide us with an update on your issue?

Thanks & Regards,

Varsha

Re:How to use Intel® MKL OOC PARDISO

VarshaS_Intel — Fri, 11 Aug 2023 08:26:50 GMT

Hi,

We have not heard back from you. Could you please provide us with an update on your issue?

Thanks & Regards,

Varsha

Re: Re:How to use Intel® MKL OOC PARDISO

smile — Mon, 04 Sep 2023 01:12:00 GMT

Thanks for your reply. Sorry, I was unable to reply to your message because of some problems with my Intel account. Intel staff just repaired my account recently.

Yes, I have tried changing only the iparm[59] value to 2 and iparm[60] value to 0. However, memory consumption didn't change. The Visual Studio version is 2017. The Intel MKL version is 2021.3.

After modifying the parameters mentioned, I'm pretty sure the memory consumption hasn't changed.

Here's my code.

subroutine solver_pardiso(mtype, n, length, a, ia, ja, nrhs, iparm, b, x)

implicit none

integer maxfct, mnum, mtype, phase, n, length, nrhs, error, msglvl

integer(kind=8) :: pt(64)

integer iparm(64),perm(n)!, solver

integer ia(n+1) !ia(9)

integer ja(length) !ja(20)

integer,external:: mkl_get_max_threads

real*8 dparm(64)

complex*16 a(length) !a(20)

complex*16 b(n) !b(8)

complex*16 x(n) !x(8)

integer i, idum

real*8 ddum !waltime1, waltime2,

x = 0.d0 !解出的向量x存于这个数组

pt = 0 !// pointer initialization

maxfct = 1 !大多数情况设1

mnum = 1 !大多数情况设1

!// mtype = -2: symmetric nonpositive definite matrix, mtype = 2: symmetric positive definite matrix

!结构有限元常用：2 实对称正定矩阵;-2 实对称不定矩阵;流体计算可能会用到 11实数不对称矩阵

perm = 0; !一般不用，给个虚参

nrhs = 1 !方程右边b的列数

iparm(1) = 0 !// iparm use default values

iparm(3) = 4

iparm(59) = 2

!iparm(60) = 2

!// iparm(3): parallel threads are obtained by MKL. In general, the number of cpus is the same as the number of threads

msglvl =0 !设为1则在屏幕上显示计算统计信息

error = 0 !错误代码，有错则输出一个代码，以便查错。

phase = 13 !// LU decompose

!一般设为13，analysis->numericalfactorization->solve->iterativerefinement

call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

!phase = 33 !// solve equations

!call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

!write ( *,'(a)' ) '>> The solution as follows ...... '

open(98,file='x.txt')

do i = 1, n

write( 98,* ) i,x(i)

end do

write( *,* ) 'max memory:',max(iparm(15), iparm(16)+iparm(17))/1024.0/1024.0,'GB.'

phase = -1 !释放内存

call pardiso (pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error)

end subroutine solver_pardiso

Re: Re:How to use Intel® MKL OOC PARDISO

smile — Mon, 04 Sep 2023 01:14:15 GMT

My English is not very good, if I have described it clearly?

Re: Re:How to use Intel® MKL OOC PARDISO

smile — Tue, 05 Sep 2023 02:21:30 GMT

Thanks for your reply. There was a problem with my Intel account that prevented me from replying to your message. It was recently repaired.

Yes, I have tried changing only the iparm[59] value to 2 and iparm[60] value to 0. However, memory consumption didn't change. The Visual Studio version is 2017. The Intel MKL version is 2021.3.

After modifying the parameters mentioned, I'm pretty sure the memory consumption hasn't changed.

Here's my code.

subroutine solver_pardiso(mtype, n, length, a, ia, ja, nrhs, iparm, b, x)

implicit none

integer maxfct, mnum, mtype, phase, n, length, nrhs, error, msglvl

integer(kind=8) :: pt(64)

integer iparm(64),perm(n)!, solver

integer ia(n+1) !ia(9)

integer ja(length) !ja(20)

integer,external:: mkl_get_max_threads

real*8 dparm(64)

complex*16 a(length) !a(20)

complex*16 b(n) !b(8)

complex*16 x(n) !x(8)

integer i, idum

real*8 ddum !waltime1, waltime2,

x = 0.d0 !解出的向量x存于这个数组

pt = 0 !// pointer initialization

maxfct = 1 !大多数情况设1

mnum = 1 !大多数情况设1

!// mtype = -2: symmetric nonpositive definite matrix, mtype = 2: symmetric positive definite matrix

!结构有限元常用：2 实对称正定矩阵;-2 实对称不定矩阵;流体计算可能会用到 11实数不对称矩阵

perm = 0; !一般不用，给个虚参

nrhs = 1 !方程右边b的列数

iparm(1) = 0 !// iparm use default values

iparm(3) = 4

iparm(59) = 2

!iparm(60) = 2

!// iparm(3): parallel threads are obtained by MKL. In general, the number of cpus is the same as the number of threads

msglvl =0 !设为1则在屏幕上显示计算统计信息

error = 0 !错误代码，有错则输出一个代码，以便查错。

phase = 13 !// LU decompose

!一般设为13，analysis->numericalfactorization->solve->iterativerefinement

call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

!phase = 33 !// solve equations

!call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

!write ( *,'(a)' ) '>> The solution as follows ...... '

open(98,file='x.txt')

do i = 1, n

write( 98,* ) i,x(i)

end do

write( *,* ) 'max memory:',max(iparm(15), iparm(16)+iparm(17))/1024.0/1024.0,'GB.'

phase = -1 !释放内存

call pardiso (pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error)

end subroutine solver_pardiso

Re: Re:How to use Intel® MKL OOC PARDISO

smile — Tue, 05 Sep 2023 02:22:20 GMT

Thanks for your reply. There was a problem with my Intel account that prevented me from replying to your message. It was recently repaired.

Yes, I have tried changing only the iparm[59] value to 2 and iparm[60] value to 0. However, memory consumption didn't change. The Visual Studio version is 2017. The Intel MKL version is 2021.3.

After modifying the parameters mentioned, I'm pretty sure the memory consumption hasn't changed.

Here's my code.

subroutine solver_pardiso(mtype, n, length, a, ia, ja, nrhs, iparm, b, x)

implicit none

integer maxfct, mnum, mtype, phase, n, length, nrhs, error, msglvl

integer(kind=8) :: pt(64)

integer iparm(64),perm(n)!, solver

integer ia(n+1) !ia(9)

integer ja(length) !ja(20)

integer,external:: mkl_get_max_threads

real*8 dparm(64)

complex*16 a(length) !a(20)

complex*16 b(n) !b(8)

complex*16 x(n) !x(8)

integer i, idum

real*8 ddum !waltime1, waltime2,

x = 0.d0 !解出的向量x存于这个数组

pt = 0 !// pointer initialization

maxfct = 1 !大多数情况设1

mnum = 1 !大多数情况设1

!// mtype = -2: symmetric nonpositive definite matrix, mtype = 2: symmetric positive definite matrix

!结构有限元常用：2 实对称正定矩阵;-2 实对称不定矩阵;流体计算可能会用到 11实数不对称矩阵

perm = 0; !一般不用，给个虚参

nrhs = 1 !方程右边b的列数

iparm(1) = 0 !// iparm use default values

iparm(3) = 4

iparm(59) = 2

!iparm(60) = 2

!// iparm(3): parallel threads are obtained by MKL. In general, the number of cpus is the same as the number of threads

msglvl =0 !设为1则在屏幕上显示计算统计信息

error = 0 !错误代码，有错则输出一个代码，以便查错。

phase = 13 !// LU decompose

!一般设为13，analysis->numericalfactorization->solve->iterativerefinement

call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

!phase = 33 !// solve equations

!call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

!write ( *,'(a)' ) '>> The solution as follows ...... '

open(98,file='x.txt')

do i = 1, n

write( 98,* ) i,x(i)

end do

write( *,* ) 'max memory:',max(iparm(15), iparm(16)+iparm(17))/1024.0/1024.0,'GB.'

phase = -1 !释放内存

call pardiso (pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error)

end subroutine solver_pardiso

Re: How to use Intel® MKL OOC PARDISO

smile — Tue, 05 Sep 2023 02:31:31 GMT

Thanks for your reply. There was a problem with my Intel account that prevented me from replying to your message. It was recently repaired.

Yes, I have tried changing only the iparm[59] value to 2 and iparm[60] value to 0. However, memory consumption didn't change. The Visual Studio version is 2017. The Intel MKL version is 2021.3.

After modifying the parameters mentioned, I'm pretty sure the memory consumption hasn't changed.

Here's my code.

subroutine solver_pardiso(mtype, n, length, a, ia, ja, nrhs, iparm, b, x)

implicit none

integer maxfct, mnum, mtype, phase, n, length, nrhs, error, msglvl

integer(kind=8) :: pt(64)

integer iparm(64),perm(n)!, solver

integer ia(n+1) !ia(9)

integer ja(length) !ja(20)

integer,external:: mkl_get_max_threads

real*8 dparm(64)

complex*16 a(length) !a(20)

complex*16 b(n) !b(8)

complex*16 x(n) !x(8)

integer i, idum

real*8 ddum !waltime1, waltime2,

x = 0.d0 !解出的向量x存于这个数组

pt = 0 !// pointer initialization

maxfct = 1 !大多数情况设1

mnum = 1 !大多数情况设1

!// mtype = -2: symmetric nonpositive definite matrix, mtype = 2: symmetric positive definite matrix

!结构有限元常用：2 实对称正定矩阵;-2 实对称不定矩阵;流体计算可能会用到 11实数不对称矩阵

perm = 0; !一般不用，给个虚参

nrhs = 1 !方程右边b的列数

iparm(1) = 0 !// iparm use default values

iparm(3) = 4

iparm(59) = 2

!iparm(60) = 2

!// iparm(3): parallel threads are obtained by MKL. In general, the number of cpus is the same as the number of threads

msglvl =0 !设为1则在屏幕上显示计算统计信息

error = 0 !错误代码，有错则输出一个代码，以便查错。

phase = 13 !// LU decompose

!一般设为13，analysis->numericalfactorization->solve->iterativerefinement

call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

!phase = 33 !// solve equations

!call pardiso( pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error )

!write ( *,'(a)' ) '>> The solution as follows ...... '

open(98,file='x.txt')

do i = 1, n

write( 98,* ) i,x(i)

end do

write( *,* ) 'max memory:',max(iparm(15), iparm(16)+iparm(17))/1024.0/1024.0,'GB.'

phase = -1 !释放内存

call pardiso (pt, maxfct, mnum, mtype, phase, n, a, ia, ja, perm, nrhs, iparm, msglvl, b, x, error)

end subroutine solver_pardiso

Re: Re:How to use Intel® MKL OOC PARDISO

smile — Tue, 05 Sep 2023 02:31:55 GMT

Thanks for your reply. There was a problem with my Intel account that prevented me from replying to your message. It was recently repaired.

Yes, I have tried changing only the iparm[59] value to 2 and iparm[60] value to 0. However, memory consumption didn't change. The Visual Studio version is 2017. The Intel MKL version is 2021.3.

After modifying the parameters mentioned, I'm pretty sure the memory consumption hasn't changed.

Re: Re:How to use Intel® MKL OOC PARDISO

smile — Tue, 05 Sep 2023 02:33:37 GMT

Thanks for your reply. There was a problem with my Intel account that prevented me from replying to your message. It was recently repaired.

Yes, I have tried changing only the iparm[59] value to 2 and iparm[60] value to 0. However, memory consumption didn't change. The Visual Studio version is 2017. The Intel MKL version is 2021.3.

After modifying the parameters mentioned, I'm pretty sure the memory consumption hasn't changed.