- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello, I'm currently trying to get pardiso to work with multi threading and I'm wondering if it is a linking issue or something else. I have tried some "easy" fixes that didn't work, then I tried the link advisor and get an error when linking.
Question: How do I get pardiso to work with multiple cores?
Background:
When calling pardiso I use the following iparm
iparm= 0
iparm(1) = 1! !0=solver default
iparm(2) = 2 ! !2
iparm(3) = 0 !reserved, set to zero
iparm(4) = 0 ! Preconditioned CGS/CG.
iparm(5) = 0 !
iparm(6) = 0 !
iparm(7) = 0 !
iparm(8) = 9 ! Iterative refinement step.
iparm(9) = 0 ! reserved, set to zero
iparm(10) = 13
iparm(11) = 1
iparm(12) = 0
iparm(13) = 0
iparm(14) = 0
iparm(15) = 0
iparm(16) = 0
iparm(17) = 0
iparm(18) = -1
iparm(19) = -1
iparm(20) = 0
The problem is "large" with
#equations: 71574 #non-zeros in A: 3815460 non-zeros in A (%): 0.074479
The simplest fix
call mkl_set_dynamic(0) ! disable adjustment of the number of threads call mkl_set_num_threads(4) call omp_set_num_threads(4)
This is done right before I call pardiso and has no effect on the number of cores used, the next step was to check the MKL linking, I then tried to use the link advisor, https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/ but I get and error.
With the advisor,I choose the "Composer XE 2011" in the link advisor since that is where my MKL roots are in the makefile(please correct me if this is a wrong assumption),
MKLroot=/software/intel/parallel_studio/composer_xe_2011_sp1.7.256/mkl/lib/intel64
My OS is then Linux, the compiler is iFort/intel fortran, linking is dynamic(unsure of this, trying static yields same result/error), the interface is LP64.
For threading layer Im uncertain of which option to select, sequential or OpenMP. In the makefile there is "sequential" in LLIBS, but there are sections of the code that are already multithreading with OpenMP, only do-loops and no explicit MKL routines.
LLIBS = -L/software/matlab/R2011a/bin/glnxa64 -leng -lmat -lmx -lut -Wl,--start-group ${MKLroot}/libmkl_intel_lp64.a ${MKLroot}/libmkl_sequential.a ${MKLroot}/libmkl_core.a -Wl,--end-group -lpthread
Lastly I'm advised to do the following linking and add to my compiler
-lpthread -lm -openmp -mkl=parallel
Adding the compiler options work but when linking I get the following message/error
$ ifort -lpthread -lmifort: warning #10315: specifying -lm before files may supercede the Intel(R) math library and affect performance /software/intel/parallel_studio/composer_xe_2011_sp1.7.256/compiler/lib/intel64/for_main.o: In function `main': /export/users/nbtester/efi2linux_nightly/branch-12_1/20111012_000000/libdev/frtl/src/libfor/for_main.c:(.text+0x38): undefined reference to `MAIN__'
I assume this means that the linking is incomplete, how can I complete it? And hopefully get pardiso to work on more then 1 core
--
Sorry if this post is messy, this is my first time attempting to link anything. If any clarification is needed please ask, any help is much appreciated
/Viktor
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Viktor,
I can't try your command line, so just comment
It seems the line F90FLAGS is not expected. please use your original one. or
remove "$(OMP_LIB) -lpthread -lm -L
"/software/intel/parallel_studio/composer_xe_2011_sp1.7.256/mkl/../compiler/lib/intel64"
-liomp5 -lmkl_core
"
they are not needed.
The line
LLIBS = -L/software/matlab/R2011a/bin/glnxa64 -leng -lmat -lmx -lut -Wl,--start-group ${MKLroot}/libmkl_intel_lp64.a ${MKLroot}/libmkl_sequential.a ${MKLroot}/libmkl_core.a -Wl,--end-group -lpthread -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_core.a ${MKLROOT}/lib/intel64/libmkl_intel_thread.a -Wl,--end-group -lpthread -lm
Please remove the below part. (it used sequential mkl, will cause mkl run in 1 thread. )
-Wl,--start-group ${MKLroot}/libmkl_intel_lp64.a ${MKLroot}/libmkl_sequential.a ${MKLroot}/libmkl_core.a -Wl,--end-group -lpthread.
the thread MKL should be enough, the correct line is like
-L/software/matlab/R2011a/bin/glnxa64 -leng -lmat -lmx -lut -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_intel_thread.a ${MKLROOT}/lib/intel64/libmkl_core.a -Wl,--end-group -liomp5 -lpthread -lm
Best Regards,
Ying
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Viktor,
It seems you have complex environment including Matlab and but there are sections of the code that are already multithreading with OpenMP, only do-loops.
Let's simplify the problem first.
Do you have the input data ready? I may suggest you to modify the MKL pardiso sample code pardiso_sym_f.f or pardiso_unsym_f.f , which in
/software/intel/parallel_studio/composer_xe_2011_sp1.7.256/ |
/mkl/examples/solver/source
1) copy the solver folder to one folder you can write
2) modify pardiso_unsym_f.f (if you are working on fortran )
3) then build it , ite
make libem64 function=pardiso_unsym_f
You will see the exact link option and exe.
4) run the exe to see if the multiple core was used.
5) Then reconsider your real makefile, also please use the same order, for example, -lphread, -lm is supposed at the end of link line etc.
Best Regards,
Ying
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello and thank you for your answer,
I don't have ready input data but I could copy the sample code and make the example work on multiple cores.
I then looked at the sample makefile and copied the links and flags the parts that were missing from my own makefile,
CF90=ifort F77FLAGS= -O3 F90FLAGS = ${F77FLAGS} -I${SOLIDroot} -openmp -m64 -w -fpp -I$(MKLROOT)/../.. $(OMP_LIB) -lpthread -lm -L"/software/intel/parallel_studio/composer_xe_2011_sp1.7.256/mkl/../compiler/lib/intel64" -liomp5 -lmkl_core SOLIDroot=../Modules OMP_LIB=-L"/software/intel/parallel_studio/composer_xe_2011_sp1.7.256/mkl/../compiler/lib/intel64" -liomp5 MKLroot=/software/intel/parallel_studio/composer_xe_2011_sp1.7.256/mkl/lib/intel64 LLIBS = -L/software/matlab/R2011a/bin/glnxa64 -leng -lmat -lmx -lut -Wl,--start-group ${MKLroot}/libmkl_intel_lp64.a ${MKLroot}/libmkl_sequential.a ${MKLroot}/libmkl_core.a -Wl,--end-group -lpthread -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_core.a ${MKLROOT}/lib/intel64/libmkl_intel_thread.a -Wl,--end-group -lpthread -lm LD_LIBRARY_PATH=/software/matlab/R2011a/bin/glnxa64
I can compile the code but I get the following error:
./run_tl_J2iso.x: symbol lookup error: /software/intel/parallel_studio/composer_xe_2011_sp1.7.256/mkl/lib/intel64/libmkl_avx.so: undefined symbol: mkl_serv_check_ptr
I never had this error before, so it must be from the additional linking, tried to eliminate different parts of the linking and
found that the problem is caused by:
-L"/software/intel/parallel_studio/composer_xe_2011_sp1.7.256/mkl/../compiler/lib/intel64" -liomp5 -lmkl_core
Is the linking done correctly? I quick search showed that this error can occur when different versions of openmp are used by fortran and matlab(and I use matlab in my program) Previously the compiler looked like this
CF90=ifort F77FLAGS= -O3 F90FLAGS = ${F77FLAGS} -I${SOLIDroot} -openmp #linking done here SOLIDroot=../Modules #extra path here MKLroot=/software/intel/parallel_studio/composer_xe_2011_sp1.7.256/mkl/lib/intel64 LLIBS = -L/software/matlab/R2011a/bin/glnxa64 -leng -lmat -lmx -lut -Wl,--start-group ${MKLroot}/libmkl_intel_lp64.a ${MKLroot}/libmkl_sequential.a ${MKLroot}/libmkl_core.a -Wl,--end-group -lpthread LD_LIBRARY_PATH=/software/matlab/R2011a/bin/glnxa64 If any clarification is needed please just ask and any help is much appreciated
/Viktor
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Viktor,
I can't try your command line, so just comment
It seems the line F90FLAGS is not expected. please use your original one. or
remove "$(OMP_LIB) -lpthread -lm -L
"/software/intel/parallel_studio/composer_xe_2011_sp1.7.256/mkl/../compiler/lib/intel64"
-liomp5 -lmkl_core
"
they are not needed.
The line
LLIBS = -L/software/matlab/R2011a/bin/glnxa64 -leng -lmat -lmx -lut -Wl,--start-group ${MKLroot}/libmkl_intel_lp64.a ${MKLroot}/libmkl_sequential.a ${MKLroot}/libmkl_core.a -Wl,--end-group -lpthread -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_core.a ${MKLROOT}/lib/intel64/libmkl_intel_thread.a -Wl,--end-group -lpthread -lm
Please remove the below part. (it used sequential mkl, will cause mkl run in 1 thread. )
-Wl,--start-group ${MKLroot}/libmkl_intel_lp64.a ${MKLroot}/libmkl_sequential.a ${MKLroot}/libmkl_core.a -Wl,--end-group -lpthread.
the thread MKL should be enough, the correct line is like
-L/software/matlab/R2011a/bin/glnxa64 -leng -lmat -lmx -lut -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_intel_thread.a ${MKLROOT}/lib/intel64/libmkl_core.a -Wl,--end-group -liomp5 -lpthread -lm
Best Regards,
Ying
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello, removing that line work.
Thank you so much for helping and and giving advice.
With sincere thanks,
Viktor
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page