topic After check and correct, I in Intel® oneAPI Math Kernel Library

How to achieve Parallel back substitution when using Pardiso?

wen_qiang_z_ — Sat, 01 Dec 2018 00:56:03 GMT

I want to speed up the procedure of back substitution, but I did not figure out what should I do.

I used to use OpenMp simply, but the reason differs.

!$OMP parallel do
       do j = 1,sub(i)%Nob
           call Inver(par,pa(j)%Righthand,pa(j)%solt,2)
       end do

!$OMP end parallel do

where par is pardiso_par_t type as follows

type pardiso_par_t
       type(mkl_pardiso_handle),dimension(64) :: pt
   integer :: maxfct,mnum,mtype,phase,n,nrhs,error,msglvl
   integer,dimension(64) :: iparm
   integer,allocatable,dimension(:) :: ia,ja
   complex(kind=8),allocatable,dimension(:) :: a
   integer,dimension(1) :: idum
   complex(kind=8),dimension(1) :: ddum
   end type pardiso_par_t

and subroutine INVER is

subroutine Inver(left,right,solution,info)
   use MKL_PARDISO
   use MeshDefMod
   implicit none
!------------------------------------------
   type(pardiso_par_t),intent(inout) :: left

!righthand
complex(kind=8),allocatable,dimension(:),intent(inout) :: right

!solution
   complex(kind=8),allocatable,dimension(:),intent(inout) :: solution
   integer(kind=4),intent(in) :: info
!------------------------------------------
   if(info == 1) then
!Reordering and Symbolic Factorization,this step also allocates
!all memory that is necessary for the factorization
       left%phase = 11 !only reordering and symbolic factorization
       call pardiso (left%pt, left%maxfct, left%mnum, left%mtype, left%phase, &
           left%n, left%a, left%ia, left%ja,left%idum, left%nrhs, left%iparm,&
           left%msglvl, left%ddum, left%ddum, left%error)
       write(*,*) 'Reordering completed...'
       if(left%error /= 0) then
           write(*,*) 'The following ERROR was detected :',left%error
           !stop 1
       end if
       write(*,*) 'Number of Nonzeros in factors = ', left%iparm(18)
       write(*,*) 'Number of factorization MFLOPS = ',left%iparm(19)
!Factorization
       left%phase = 22 !only factorizatin
       call pardiso (left%pt, left%maxfct, left%mnum, left%mtype, left%phase,&
           left%n, left%a, left%ia, left%ja,left%idum, left%nrhs, left%iparm,&
           left%msglvl, left%ddum, left%ddum, left%error)
       write(*,*) 'Factorization completed...'
       if(left%error /= 0) then
           write(*,*) 'The following ERROR was detected :',left%error
           stop
       end if
   else if(info == 2) then
       !left%phase = 33 !only factorization
       !left%iparm(8) =0 !max number of iterative refinement steps
       call pardiso (left%pt, left%maxfct, left%mnum, left%mtype, left%phase,&
           left%n, left%a, left%ia, left%ja,left%idum, left%nrhs, left%iparm,&
           left%msglvl, right, solution, left%error)
       !write(*,*) 'Solve completed...'
   else if(info == 3) then
       !Termination and release of memory
       left%phase = -1
       call pardiso (left%pt, left%maxfct, left%mnum, left%mtype, left%phase,&
           left%n, left%a, left%ia, left%ja,left%idum, left%nrhs, left%iparm,&
           left%msglvl, left%ddum, left%ddum, left%error)
   end if
   end subroutine Inver

And my pardiso control

wen_qiang_z_ — Sat, 01 Dec 2018 01:00:43 GMT

And my pardiso control parameters are

iparm(1) = 1

iparm(2) = 3

iparm(11) = 1

iparm(25) = 2

After check and correct, I

wen_qiang_z_ — Sat, 01 Dec 2018 08:55:48 GMT

After check and correct, I only have one question to ask:

Parallel Direct Factorization is running on 1 OpenMP

It should be Parallel Direct Factorization is running on 8 OpenMP I guess if I want to accelerate the backup substitution procedure, So what iparm should I set?

Hello,

Kirill_V_Intel — Tue, 04 Dec 2018 07:55:14 GMT

Hello,

If you want to have OpenMP-based parallel back substitution, you can just set the number of OpenMP threads (e.g., from the environment variables MKL_NUM_THREADS and OMP_NUM_THREADS) or by calling mkl_set_num_threads, or other options listed at https://software.intel.com/en-us/mkl-macos-developer-guide-techniques-to-set-the-number-of-threads, and call pardiso, you don't need to do an OpenMP do-loop.

Another thing is that you need to build and link your program appropriately. For that have a look at the MKL Link Line Advisor at https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/ where you can find correct options for OpenMP on your OS.

I hope you'll find this useful.

Quote:Voronin, Kirill (Intel)

wen_qiang_z_ — Thu, 06 Dec 2018 00:54:31 GMT

Voronin, Kirill (Intel) wrote:
Hello,
If you want to have OpenMP-based parallel back substitution, you can just set the number of OpenMP threads (e.g., from the environment variables MKL_NUM_THREADS and OMP_NUM_THREADS) or by calling mkl_set_num_threads, or other options listed at https://software.intel.com/en-us/mkl-macos-developer-guide-techniques-to..., and call pardiso, you don't need to do an OpenMP do-loop.
Another thing is that you need to build and link your program appropriately. For that have a look at the MKL Link Line Advisor at https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/ where you can find correct options for OpenMP on your OS.
I hope you'll find this useful.

Thanks a lot for your help!