How to achieve Parallel back substitution when using Pardiso?

[wen_qiang_z_]

I want to speed up the procedure of back substitution, but I did not figure out what should I do.

I used to use OpenMp simply, but the reason differs.

!$OMP parallel do
        do j = 1,sub(i)%Nob    
            call Inver(par,pa(j)%Righthand,pa(j)%solt,2)
        end do

 !$OMP end parallel do

where par is pardiso_par_t type as follows

type pardiso_par_t
        type(mkl_pardiso_handle),dimension(64) :: pt
    integer :: maxfct,mnum,mtype,phase,n,nrhs,error,msglvl
    integer,dimension(64) :: iparm
    integer,allocatable,dimension(:) :: ia,ja
    complex(kind=8),allocatable,dimension(:) :: a
    integer,dimension(1) :: idum
    complex(kind=8),dimension(1) :: ddum
    end type pardiso_par_t

and subroutine INVER is

subroutine Inver(left,right,solution,info)
    use MKL_PARDISO
    use MeshDefMod
    implicit none
!------------------------------------------
    type(pardiso_par_t),intent(inout) :: left

!righthand
    complex(kind=8),allocatable,dimension(:),intent(inout) :: right

!solution
    complex(kind=8),allocatable,dimension(:),intent(inout) :: solution
    integer(kind=4),intent(in) :: info
!------------------------------------------
    if(info == 1) then
!Reordering and Symbolic Factorization,this step also allocates
!all memory that is necessary for the factorization
        left%phase = 11  !only reordering and symbolic factorization
        call pardiso (left%pt, left%maxfct, left%mnum, left%mtype, left%phase, &
            left%n, left%a, left%ia, left%ja,left%idum, left%nrhs, left%iparm,&
            left%msglvl, left%ddum, left%ddum, left%error)
        write(*,*) 'Reordering completed...'
        if(left%error /= 0) then
            write(*,*) 'The following ERROR was detected :',left%error
            !stop 1
        end if
        write(*,*) 'Number of Nonzeros in factors = ', left%iparm(18)
        write(*,*) 'Number of factorization MFLOPS = ',left%iparm(19)
!Factorization
        left%phase = 22  !only factorizatin
        call pardiso (left%pt, left%maxfct, left%mnum, left%mtype, left%phase,&
            left%n, left%a, left%ia, left%ja,left%idum, left%nrhs, left%iparm,&
            left%msglvl, left%ddum, left%ddum, left%error)
        write(*,*) 'Factorization completed...'
        if(left%error /= 0) then
            write(*,*) 'The following ERROR was detected :',left%error
            stop 
        end if
    else if(info == 2) then
        !left%phase = 33 !only factorization
        !left%iparm(8) =0 !max number of iterative refinement steps
        call pardiso (left%pt, left%maxfct, left%mnum, left%mtype, left%phase,&
            left%n, left%a, left%ia, left%ja,left%idum, left%nrhs, left%iparm,&
            left%msglvl, right, solution, left%error)
        !write(*,*) 'Solve completed...'
    else if(info == 3) then
        !Termination and release of memory
        left%phase = -1
        call pardiso (left%pt, left%maxfct, left%mnum, left%mtype, left%phase,&
            left%n, left%a, left%ia, left%ja,left%idum, left%nrhs, left%iparm,&
            left%msglvl, left%ddum, left%ddum, left%error)
    end if
    end subroutine Inver

[wen_qiang_z_]

And my pardiso control parameters are

iparm(1) = 1

iparm(2) = 3

iparm(11) = 1

iparm(25) = 2

[wen_qiang_z_]

After check and correct, I only have one question to ask:

Parallel Direct Factorization is running on 1 OpenMP

It should be Parallel Direct Factorization is running on 8 OpenMP I guess if I want to accelerate the backup substitution procedure, So what iparm should I set?

[Kirill_V_Intel · Accepted solution]

Hello,

If you want to have OpenMP-based parallel back substitution, you can just set the number of OpenMP threads (e.g., from the environment variables MKL_NUM_THREADS and OMP_NUM_THREADS) or by calling mkl_set_num_threads, or other options listed at https://software.intel.com/en-us/mkl-macos-developer-guide-techniques-to-set-the-number-of-threads, and call pardiso, you don't need to do an OpenMP do-loop.

Another thing is that you need to build and link your program appropriately. For that have a look at the MKL Link Line Advisor at https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/ where you can find correct options for OpenMP on your OS.

I hope you'll find this useful.

[wen_qiang_z_]

Voronin, Kirill (Intel) wrote:

Hello,

If you want to have OpenMP-based parallel back substitution, you can just set the number of OpenMP threads (e.g., from the environment variables MKL_NUM_THREADS and OMP_NUM_THREADS) or by calling mkl_set_num_threads, or other options listed at https://software.intel.com/en-us/mkl-macos-developer-guide-techniques-to..., and call pardiso, you don't need to do an OpenMP do-loop.

Another thing is that you need to build and link your program appropriately. For that have a look at the MKL Link Line Advisor at https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/ where you can find correct options for OpenMP on your OS.

I hope you'll find this useful.

Thanks a lot for your help!