- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

I have been reading many posts here about trying to let PARDISO run in more than processor. However, I still cant get a proper idea on how this can be done.

What I have done is:

call omp_set_num_threads(4).

iparm(1) = 1 ! no solver default

iparm(2) = 3 ! fill-in reordering from METIS

iparm(3) = 4 ! numbers of processors, value of MKL_NUM_THREADS

iparm(4) = 0 ! no iterative-direct algorithm

iparm(5) = 0 ! no user fill-in reducing permutation

iparm(6) = 0 ! =0 solution on the first n components of x

iparm(7) = 0 ! not in use

iparm(8) = 20 ! numbers of iterative refinement steps

iparm(9) = 0 ! not in use

iparm(10) = 13 ! perturb the pivot elements with 1E-13

iparm(11) = 1 ! use nonsymmetric permutation and scaling MPS

iparm(12) = 0 ! not in use

iparm(13) = 1 ! not in use

iparm(14) = 0 ! Output: number of perturbed pivots

iparm(15) = 0 ! not in use

iparm(16) = 0 ! not in use

iparm(17) = 0 ! not in use

iparm(18) = -1 ! Output: number of nonzeros in the factor LU

iparm(19) = -1 ! Output: Mflops for LU factorization

iparm(20) = 0 ! Output: Numbers of CG Iterations

iparm(27) = 1 !Matrix checker on

iparm(60) = 1

During the run, it always say that the number of procesors use is 1.

(Direct factorization using #processors) 1

Does this mean that only one processor is running?

The machine that I am using is Intel Xeon W55803.2GHz. There are 4 physical processors. The funny thing is when i check the task manager, all 4 processors are running to 100% during factorization.

Thank you.

EH

Link Copied

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

There are 113762 equations. The matrix is sparse and the number of nonzeros are 11741784

The PARDISO is running in the in-core mode

Do I need to specify anything for mkl_num_threads and omp_num_threads?

Thank you.

Best wishes,

EH

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Sorry because I am not very familiar with the programming jargons. I have here the entire subroutine where pardiso is called. Would this help? If not, could you tell me what linkline of a project is?

Thank you and sorry for the inconvenience.

!------------------------------------------------------------------------------------!

SUBROUTINE PARDISO_SOLVER(Nc,n,A,ia,ja,Bin,Z)

! SUBROUTINE PARDISO_SOLVER(Nc,n,Ain,iin,jin,Bin,Z)

! THIS SUBROUTINE CALLS THE SOLVER PARADISO TO SOLVE THE SYSTEM...

! OF LINEAR ALGEBRAIC EQUATIONS

!------------------------------------------------------------------------------------!

IMPLICIT NONE

! Internal solver memory pointer for 64-bit architectures

! INTEGER*8 pt(64)

! Internal solver memory pointer for 32-bit architectures

! INTEGER*4 pt(64)

! This is OK in both cases

! DOUBLE PRECISION, ALLOCATABLE :: A(:)

! INTEGER, ALLOCATABLE :: ia(:), ja(:)

INTEGER*4 pt(64)

INTEGER maxfct, mnum, mtype, phase, n, nrhs, error, msglvl, Nc, i, idum

INTEGER iparm(64)

INTEGER ja(Nc), ia(n+1)

DOUBLE PRECISION A(Nc), Bin(n), Z(n)

DOUBLE PRECISION waltime1, waltime2, ddum

INTEGER mkl_get_max_threads

INTEGER MKL_NUM_THREADS,OMP_NUM_THREADS

EXTERNAL mkl_get_max_threads

EXTERNAL omp_num_threads

MKL_NUM_THREADS=2

call OMP_SET_NUM_THREADS(4)

! Set up PARDISO control parameter

DO i=1,64

iparm(i) = 0

END DO

iparm(1) = 1 ! no solver default

iparm(2) = 3 ! fill-in reordering from METIS

iparm(3) = 4 ! numbers of processors, value of MKL_NUM_THREADS

iparm(4) = 0 ! no iterative-direct algorithm

iparm(5) = 0 ! no user fill-in reducing permutation

iparm(6) = 0 ! =0 solution on the first n components of x

iparm(7) = 0 ! not in use

iparm(8) = 20 ! numbers of iterative refinement steps

iparm(9) = 0 ! not in use

iparm(10) = 13 ! perturb the pivot elements with 1E-13

iparm(11) = 1 ! use nonsymmetric permutation and scaling MPS

iparm(12) = 0 ! not in use

iparm(13) = 1 ! not in use

iparm(14) = 0 ! Output: number of perturbed pivots

iparm(15) = 0 ! not in use

iparm(16) = 0 ! not in use

iparm(17) = 0 ! not in use

iparm(18) = -1 ! Output: number of nonzeros in the factor LU

iparm(19) = -1 ! Output: Mflops for LU factorization

iparm(20) = 0 ! Output: Numbers of CG Iterations

iparm(27) = 1 !Matrix checker on

iparm(60) = 1

error = 0 ! initialize error flag

msglvl = 1 ! print statistical information

mtype = 11 ! real unsymmetric

maxfct=1

mnum=1

nrhs=1

! Initialize the internal solver memory pointer. This is only...

! necessary for the FIRST call of the PARDISO solver.

DO i=1,64

pt(i) = 0

END DO

! Reordering and Symbolic Factorization, This step also allocates...

! all memory that is necessary for the factorization

phase = 11 ! only reordering and symbolic factorization

CALL pardiso(pt,maxfct,mnum,mtype,phase,n,A,ia,ja,idum,nrhs,iparm,msglvl,ddum,ddum,error)

WRITE(*,*) 'Reordering completed ... '

IF (error .NE. 0) THEN

WRITE(*,*) 'The following ERROR was detected: ', error

PAUSE

STOP

END IF

WRITE(*,*) 'Number of nonzeros in factors = ',iparm(18)

WRITE(*,*) 'Number of factorization MFLOPS = ',iparm(19)

! Factorization.

phase = 22 ! only factorization

CALL pardiso(pt,maxfct,mnum,mtype,phase,n,A,ia,ja,idum,nrhs,iparm,msglvl,ddum,ddum,error)

WRITE(*,*) 'Factorization completed ... '

IF (error .NE. 0) THEN

WRITE(*,*) 'The following ERROR was detected: ', error

PAUSE

STOP

ENDIF

! Back substitution and iterative refinement

iparm(8) = 20 ! max numbers of iterative refinement steps

phase = 33 ! only factorization

CALL pardiso(pt,maxfct,mnum,mtype,phase,n,A,ia,ja,idum,nrhs,iparm,msglvl,Bin,Z,error)

WRITE(*,*) 'Solve completed ... '

! Termination and release of memory

phase = -1 ! release internal memory

CALL pardiso(pt,maxfct,mnum,mtype,phase,n,ddum,idum,idum,idum,nrhs,iparm,msglvl,ddum,ddum,error)

! DEALLOCATE(A,ia,ja)

RETURN

END

Thank you.

Best wishes,

EH

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

I did not put in any mkl library names. Instead, under Properties -> Fortran -> Libraries:Include Math Kernel Library option, I chose the Parallel (/Qmkl:parallel) options. Is this not the correct way of doing it?

Regarding the mkl_get_max_threads(), the printed value was 4. However the Parallel Direct Factorization with #provessors still say 1.

Thanks.

EH

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Project -> Your Project Properties... -> Configuration Properties -> Linker -> Command line

and send us the content of this text field?

Regards,

Konstantin

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

The content in the Command line reads:

/OUT:"x64\Debug\Test1.exe" /INCREMENTAL:NO /NOLOGO /LIBPATH:"C:\Program Files\Intel\Compiler\11.1\048\mkl\ia64\lib" /MANIFEST /MANIFESTFILE:"D:\USERS\EHOOI\Fortran\3D\Natural_Convection\Test3-steady\x64\Debug\Test1.exe.intermediate.manifest" /DEBUG /PDB:"D:\USERS\EHOOI\Fortran\3D\Natural_Convection\Test3-steady\x64\Debug\Test1.pdb" /SUBSYSTEM:CONSOLE /IMPLIB:"D:\USERS\EHOOI\Fortran\3D\Natural_Convection\Test3-steady\x64\Debug\Test1.lib"

Thank you.

EH

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Thank you. I will check it out. By the way, if all four CPU in the Task Manager show 100% computation during the running of PARDISO (only fortran is running), then is this an indicator that it is using 4 processors in parallel although in the Command Window it says 1 processor in the Direct Factorization?

Thank you.

EH

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

Hmm, it's strange that you useIntel Xeon W5580, but it seems like you are linking with IA-64 architecture (Itanium-2):

/OUT:"x64\Debug\Test1.exe" /INCREMENTAL:NO /NOLOGO /LIBPATH:"C:\Program Files\Intel\Compiler\11.1\048\mkl\

**ia64**\lib"

Also you can easily understand is your code parallelized or not running your program in 2 modes: with mkl_set_num_threads(1); and mkl_set_num_threads(4); set prior to calling PARDISO andcompare the timings reported.

Regards,

Konstantin

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

When I let mkl_set_num_threads(4), during PARDISO all 4 CPUs in Task manager were running at 100%. However, when I change it to mkl_set_num_threads(1), only 1CPU is running. I guess it is running in parallel, although I dont know why the CommandWindow says only 1 processor is running for the Direct Factorization.

Is it wrong to link with the IA-64 architecture?

Thanks.

BW,

EH

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page