topic DFT interface error when call 1D TT in parallel loop in Intel® oneAPI Math Kernel Library

DFT interface error when call 1D TT in parallel loop

Chang_Lei — Thu, 04 Nov 2010 09:03:39 GMT

As the title, detailed codesare as follows:
"
use mkl_dfti
use mkl_trig_transforms
!define variables
double precision ff(1:Nz+1,1:Nr)
integer ir, ipar(128),tt_type
double precision dpar(3*nz/2+1)
type(dfti_descriptor), pointer :: handle
.............

!init the variables
tt_type=2
ff(Nz+1,:)=0.d0
.........
CALL D_INIT_TRIG_TRANSFORM(nz,tt_type,ipar,dpar,ir)
CALL D_COMMIT_TRIG_TRANSFORM(ff(:,1),handle,ipar,dpar,ir)
!parallel loop call TT transform routines
!$omp parallel do private(i,handle,ipar,dpar,ir)
do i=1,nr
CALL D_BACKWARD_TRIG_TRANSFORM(ff(:,i),handle,ipar,dpar,ir)
if (ir.ne.0) then
write(*,*) "TT transform error number:",ir
stop
end if
enddo
.....
CALL FREE_TRIG_TRANSFORM(handle,ipar,ir) "

When compile with -openmp option and run the program, DFT interface error is found because it gives " "TT transform error number: -1000". But if not compile with -openmp option, it runs ok.
If I put the commit routine in the parallel do region, it also give error information, the error number is -100.
The omp_threads number ipar(9) is not altered.
I have tried to declare ffprivate variable in OpenMP clause, but the same problem happened.
Anyone can give help is greatly appreciated.
Changlei

DFT interface error when call 1D TT in parallel loop

Alexander_K_Intel2 — Fri, 05 Nov 2010 04:09:17 GMT

Hi,

Firstly, if you sethandle,ipar,dpar,ir as private and not initialized them for each thread TT library wouldn't work correctly, so change them on shared. And secondary, set ipar[9] as maximum number of threads before commit step (more detailed could be found in Intel MKL Reference Manual, chapter 13)

With best regards,

Alexander Kalinkin

DFT interface error when call 1D TT in parallel loop

Chang_Lei — Fri, 05 Nov 2010 07:57:02 GMT

Hi Alexander,
I followed your suggestions and changed handle,ipar,dpar,ir as shared variables, set maximumthreads numbers,then no DFT interface error is found.But there are still something incorrect. That is,transform results ofsome specific columnsof ff are not right.

For example, if total number of columns of ff Nr=128 and the maximunnumber of thread ipar(9)is set to 4. Run the program with command "env OMP_NUM_THREADS=4 ./a.out". The transform results for i=1,33,65,97and randomly for i=2, i=34 or other columns are not correct. It seems that it's the problem with the first transform in the threads.

Should I call commit for each thread? If so, how to? Need I to declarehandle,ipar,dpar,ir the as private?
Best wishes!
Chang lei

DFT interface error when call 1D TT in parallel loop

Alexander_K_Intel2 — Fri, 05 Nov 2010 10:25:52 GMT

It is a really strange situation, I'll try to reproduce it and wrote you about results. Nevertheless, I was a bit wrong in previous answer, sethandle,ipar,dparshared and ir firstprivate. If you really want to call commit on each thread then you need to set handle,ipar,dparprivate and also call init and free on each thread.

With best regards,

Alexander Kalinkin

DFT interface error when call 1D TT in parallel loop

Alexander_K_Intel2 — Sat, 06 Nov 2010 19:20:08 GMT

Hi,

I can't reproduce your issue, could you send me testcase which I can compiler and execute and provide information about your processor?

With best regards,

Alexander Kalinkin

DFT interface error when call 1D TT in parallel loop

Chang_Lei — Sun, 07 Nov 2010 06:25:18 GMT

Hi, see the attachment.
It's a simple test program new wrote, and it's about parallel calling TT staggered cosine routines. You can compile it with correct link to MKL library.
My environment is Intel Fortran Compiler 10.1, CMKL 9.0, the CPU is (XeonE5450/5430)*2, andjob manager isCHESS on Linux.
For the test program, I found the error was not the same as I'm found before. For example, if ipar(9) is set to 8, if I submit my job with 1 or 2 or 3 threads, it runs ok and the results is correct. But if 4 or more thread are used, the results is not correct. But I supposed to have maximum of 8 threads to use for my CPU.
Can you give a correct example to parallel call TT routines?
Best regards!
Changlei

DFT interface error when call 1D TT in parallel loop

Alexander_K_Intel2 — Sun, 07 Nov 2010 07:20:04 GMT

Hi,

Thanks a lot for your testcase! Everything is OK but parameter used for maximum number of threads is ipar[9] in C and ipar(10) in fortran, so change line 21 in your test from ipar(9) to ipar(10). With this change your test work correctly. Could you check it?

With best regards,

Alexander Kalinkin

DFT interface error when call 1D TT in parallel loop

Chang_Lei — Sun, 07 Nov 2010 14:27:58 GMT

Hi,
I have checked it. You are correct! In fortran the first element of array has 1 asdefault index, while in C, it's zero.
So thanks for your helpful suggestions in persistence.
Best regards.
Changlei

DFT interface error when call 1D TT in parallel loop

Alexander_K_Intel2 — Sun, 07 Nov 2010 14:37:03 GMT

You are welcome!

With best regards,

Alexander Kalinkin