mkl cluster fft function - linking problem

Raemon · ‎08-21-2009

Hello there:

I am a newbie and I have the following question.
I've set up a cluster and installed MPICH2 after installed the Intel fortran compiler 11.1 on ubuntu.
I have written a simple fourier transformation code which calls mkl fft fiunctions. It is tested on a single core machine and it works right.
Now, in order to use the cluster, I modify my code to call mkl cluster fft functions.
The problem is, I have no idea how to use MPI to parallel the fft part of my code. I included "mkl_cdft.f" and "mpif.h" in my code but I don't know what's the next step. I think I have to add MPI part in my code. Also I have to learn how to link intel compiler with MPI. I've searched for some tutorials but I still have no idea.

I am appreciate if anyone give me some advices. Thanks a lot!
p.s. here is my code: http://raemoen.googlepages.com/fft.f

TimP · ‎08-21-2009

This is not the forum to attempt to replace the documentation which comes with your MPI. Unfortunately, MPICH2 for linux isn't object compatible with Intel MPI, so you would have to watch your step about mixing it with Intel Cluster tools components. It seems, after you study MPI, some of these questions (those which don't relate to MKL) would fit better on the Intel HPC forum.

Todd_R_Intel · ‎08-21-2009

Intel MKL does support MPICH2. Take a look at the Intel MKL User's Guide (online here). Chapter 9 has a lot of information about setting up your program to use Intel MKL's cluster functions and MPI. It's no replacement for your MPI documentation, but it may get you started or give you some hints.

-Todd

Raemon · ‎08-21-2009

Thanks for responding.

Now I meet a simple linking problem. I use the MKL library inside intel fortran ver. 11.1 046. I've tried a simple "hello world" code to make sure the installation works fine (MPICH2+intel fortran compiler). But when I compile the code which contains fft cluster function some errors emerges

/tmp/ifortv5uTHa.o: In function `MAIN__':
/tmp/ifortv5uTHa.o: In function `mkl_cdft_mp_dfticreatedescriptordm1_s_':
test.f:(.text+0x106): undefined reference to `DftiCreateDescriptorDM1_fortran'
/tmp/ifortv5uTHa.o: In function `mkl_cdft_mp_dfticreatedescriptordm1_d_':
test.f:(.text+0x136): undefined reference to `DftiCreateDescriptorDM1_fortran'
/tmp/ifortv5uTHa.o: In function `mkl_cdft_mp_dfticreatedescriptordmn_s_':
test.f:(.text+0x164): undefined reference to `DftiCreateDescriptorDMn_fortran'
/tmp/ifortv5uTHa.o: In function `mkl_cdft_mp_dfticreatedescriptordmn_d_':
test.f:(.text+0x194): undefined reference to `DftiCreateDescriptorDMn_fortran'

Here is my code:

[plain]	INCLUDE "mkl_cdft.f"
!	INCLUDE "mpif.h"
!	PROGRAM clusterfft_test
	REAL*8 hfr,dx,pi
	INTEGER,PARAMETER::n=2**14
	REAL*8,DIMENSION(n)::x
	COMPLEX*16,DIMENSION(n)::trans
	INTEGER i
!	CALL omp_set_num_threads(1)
!	CALL mkl_set_num_threads(1)
	hfr=50.d0
	dx=(2.d0*hlf)/real(n-1)
	pi=dacos(-1.d0)
c	CREAT THE POSITION AXIS
	do i=1,n
	x(i)=-hfr+dx*real(i-1)
	enddo
c	INPUT
	do i=1,n
	trans(i)=pi**(-0.5)*dexp(-0.5d0*(x(i)**2))
	enddo
c	PERFORM THE FOURIER TRANSFORMATION
	CALL f(n,trans,dx)
	END
c
	SUBROUTINE f(n,trans,dx)
	USE MKL_CDFT
!	USE MKL_DFTI
c==============================================================
c	THIS SUBROUTINE PERFORMS THE FOURIER TRANSFORMATION
c==============================================================
	INTEGER,INTENT(IN)::n
	REAL*8,INTENT(IN)::dx
	COMPLEX*16,DIMENSION(n)::trans
	TYPE(DFTI_DESCRIPTOR),POINTER::My_Desc1_Handle
	INTEGER i,Status,tmp
c	PERFORM THE TRANSFORMATION
	Status = DftiCreateDescriptorDM(My_Desc1_Handle,DFTI_DOUBLE,DFTI_COMPLEX,1,n)
	Status = DftiCommitDescriptorDM(My_Desc1_Handle)
	Status = DftiComputeBackwardDM(My_Desc1_Handle,trans)
	Status = DftiFreeDescriptorDM(My_Desc1_Handle)
	END SUBROUTINE f[/plain]

The cluster fft function is called without using MPI temporarily. However, I've tried several ways to link but still encounting errors.

My linking file is like this
ifort -O3 -132 -Vaxlib -o $1 $1.f -L/opt/intel/Compiler/11.1/046/mkl/lib/32/ -lmkl_intel -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -openmp
it works well when I use a non-cluster PC.

Also when linking with MPI the error messages are not the same but have lots of errors like these:

mkl_cdft.f(223): remark #5082: Directive ignored - Syntax error, found ':' when expecting one of: :: ,
!DEC$ ATTRIBUTES C, DECORATE, ALIAS:'DftiComputeBackwardDMo_fortran' :: DftiComputeBackwardDMoc
-------------------------------------------------------------------------------------^
mkl_cdft.f(230): remark #5082: Directive ignored - Syntax error, found ':' when expecting one of: :: ,
!DEC$ ATTRIBUTES C, DECORATE, ALIAS:'DftiComputeBackwardDMo_fortran' :: DftiComputeBackwardDMoz
-------------------------------------------------------------------------------------^
mkl_cdft.f(236): remark #5082: Directive ignored - Syntax error, found ':' when expecting one of: :: ,
!DEC$ ATTRIBUTES C, DECORATE, ALIAS:'DftiComputeBackwardDMi_fortran' :: DftiComputeBackwardDMic
-------------------------------------------------------------------------------------^
mkl_cdft.f(242): remark #5082: Directive ignored - Syntax error, found ':' when expecting one of: :: ,
!DEC$ ATTRIBUTES C, DECORATE, ALIAS:'DftiComputeBackwardDMi_fortran' :: DftiComputeBackwardDMiz
......
......

In addition, I've read this topic: http://software.intel.com/en-us/forums/showthread.php?t=64038

My problem is just like him. However, I use a 32-bit machine temporarily and the so called "libmkl_intel_lp64.a" cannot be found in my mkl library. I really have no idea. Please tell me if you have any advice. Thanks a lot!

TimP · ‎08-21-2009

The lp64 library is present only in 64-bit versions of MKL, as there is no option for 64-bit integers in the 32-bit version. It looks like you require a dft library specification. Does the link advisor tool in the right column of the forum page help?

Raemon · ‎08-21-2009

Quoting - tim18

The lp64 library is present only in 64-bit versions of MKL, as there is no option for 64-bit integers in the 32-bit version. It looks like you require a dft library specification. Does the link advisor tool in the right column of the forum page help?

Thank you. I didn't notice the linker adviser.It is a useful tool!
However, errors still pop up.

Here is another test code (errors occur once I include "mkl_cdft.f"):

[plain]	INCLUDE "/home/raemoen/Desktop/mkl_cdft.f"
	INCLUDE "/home/raemoen/Desktop/mpif.h"
!	program EXERCISE
	call MPI_Init(ierr)
	write(*,*)"Hello world!"
	call MPI_Finalize(ierr)
	END[/plain]

From the linking advisor tool my link is like this:

MKLPATH="/opt/intel/Compiler/11.1/046/mkl/lib/32"
mpif90 test.f $MKLPATH/libmkl_solver.a -Wl,--start-group $MKLPATH/libmkl_intel.a $MKLPATH/libmkl_cdft_core.a $MKLPATH/libmkl_intel_thread.a $MKLPATH/libmkl_core.a $MKLPATH/libmkl_blacs_intelmpi.a -Wl,--end-group -openmp -lpthread

Here are the error messages:

[plain]/home/raemoen/Desktop/mkl_dfti.f(230): error #5082: Syntax error, found END-OF-STATEMENT when expecting one of: ) ,
	     FUNCTION dfti_create_descriptor_1d(desc, precision, domain, dim, length)
-------------------------------------------------------------------------------------^
/home/raemoen/Desktop/mkl_dfti.f(245): error #5082: Syntax error, found END-OF-STATEMENT when expecting one of: ) ,
	     FUNCTION dfti_create_descriptor_highd(desc, precision, domain, dim,length)
---------------------------------------------------------------------------------------^
/home/raemoen/Desktop/mkl_dfti.f(550): error #5082: Syntax error, found END-OF-STATEMENT when expecting one of: ) ,
	     FUNCTION dfti_compute_forward_ssss(desc,sSrcRe,sSrcIm,sDstRe,sDstIm)
---------------------------------------------------------------------------------^
/home/raemoen/Desktop/mkl_dfti.f(634): error #5082: Syntax error, found END-OF-STATEMENT when expecting one of: ) ,
	     FUNCTION dfti_compute_forward_dddd(desc,dSrcRe,dSrcIm,dDstRe,dDstIm)
---------------------------------------------------------------------------------^
/home/raemoen/Desktop/mkl_dfti.f(722): error #5082: Syntax error, found END-OF-STATEMENT when expecting one of: ) ,
	     FUNCTION dfti_compute_backward_ssss(desc,sSrcRe,sSrcIm,sDstRe,sDstIm)
----------------------------------------------------------------------------------^
/home/raemoen/Desktop/mkl_dfti.f(806): error #5082: Syntax error, found END-OF-STATEMENT when expecting one of: ) ,
	     FUNCTION dfti_compute_backward_dddd(desc,dSrcRe,dSrcIm,dDstRe,dDstIm)
----------------------------------------------------------------------------------^
/home/raemoen/Desktop/mkl_dfti.f(242): error #6451: A dummy argument name is required in this context.   [LENGTH]
	       INTEGER, INTENT(IN) :: dim, length
-------------------------------------------^
/home/raemoen/Desktop/mkl_dfti.f(257): error #6420: This array name is invalid in this context.   [LENGTH]
	       INTEGER, INTENT(IN), DIMENSION(*) :: length
...
...
[/plain]

I think there might have problems when adding the "mkl_cdft.f". I even didn't write USE MKL_CDFT here!

Raemon · ‎08-22-2009

Hello there!

I tried another 64-bit PC(centOS +intel fortran ver11.0 083 + MPICH2).

After some tests I found that my problem is: My code (which will call non-cluster fft functions) cannot be compiled using ifort after MPICH2 installed.

It is strange because that code can be compiled successfully using ifort before MPICH2 is installed.

Here are the error messages:

[plain]/tmp/ifort8dT8vH.o: In function `MAIN__':
schroprop.f:(.text+0x691): relocation truncated to fit: R_X86_64_PC32 against `var$158.0.7'
schroprop.f:(.text+0x698): relocation truncated to fit: R_X86_64_PC32 against `f_$LENGTH.0.7'
schroprop.f:(.text+0x69f): relocation truncated to fit: R_X86_64_PC32 against `f_$LENGTH.0.7'
schroprop.f:(.text+0x8aa): relocation truncated to fit: R_X86_64_PC32 against `var$188.0.6'
schroprop.f:(.text+0xccf): relocation truncated to fit: R_X86_64_PC32 against `var$188.0.6'
schroprop.f:(.text+0xdf0): relocation truncated to fit: R_X86_64_32S against `.bss'
schroprop.f:(.text+0xdf8): relocation truncated to fit: R_X86_64_32S against `.bss'
schroprop.f:(.text+0xe00): relocation truncated to fit: R_X86_64_32S against `.bss'
schroprop.f:(.text+0xe08): relocation truncated to fit: R_X86_64_32S against `.bss'
schroprop.f:(.text+0xe7c): relocation truncated to fit: R_X86_64_32S against `.bss'
schroprop.f:(.text+0xe88): additional relocation overflows omitted from the output
[/plain]

and the linking

[plain]ifort -O3 -132 -Vaxlib -o $1 $1.f -L/opt/intel/Compiler/11.0/083/mkl/lib/em64t -lmkl_em64t  -lguide -lpthread
[/plain]

I think I have to solve this before moving my code to call the cluster fft function. Any idea?

Raemon · ‎08-23-2009

Hello all:

After I turn my code into f90 style, I can successfully include "mkl_cdft.f90".
I think the previous compiling errors are due to the compiler think my code is fortran 77 type.

however, now, the non-cluster fft function can be called but the cluster fft function cannot.

error messages are:

[plain]a.f90(21): error #6284: There is no matching specific function for this generic function reference. [DFTICREATEDESCRIPTORDM]
Status = DftiCreateDescriptorDM(My_Desc1_Handle,DFTI_DOUBLE,DFTI_COMPLEX,1,n)
---------^
a.f90(22): error #6284: There is no matching specific function for this generic function reference. [DFTICOMMITDESCRIPTORDM]
Status = DftiCommitDescriptorDM(My_Desc1_Handle)
---------^
a.f90(23): error #6284: There is no matching specific function for this generic function reference. [DFTICOMPUTEBACKWARDDM]
Status = DftiComputeBackwardDM(My_Desc1_Handle,trans)
---------^
a.f90(24): error #6284: There is no matching specific function for this generic function reference. [DFTIFREEDESCRIPTORDM]
Status = DftiFreeDescriptorDM(My_Desc1_Handle)[/plain]

I think the compiler didn't recognize cluster fft function due to the non-proper linking.

Again, I've tried several ways to link. But still in vain.

Here are some link tries:

[plain]MKLPATH="/opt/intel/mkl/10.2.1.017/lib/32"
mpif90 a.f90 $MKLPATH/libmkl_cdft_core.a $MKLPATH/libmkl_blacs_intelmpi.a $MKLPATH/libmkl_intel.a $MKLPATH/libmkl_intel_thread.a $MKLPATH/libmkl_core.a -liomp5 -lpthread[/plain]

[plain]MKLPATH="/opt/intel/mkl/10.2.1.017/lib/32"
mpif90 a.f90 $MKLPATH/libmkl_solver_sequential.a -Wl,--start-group $MKLPATH/libmkl_intel.a $MKLPATH/libmkl_cdft_core.a $MKLPATH/libmkl_sequential.a $MKLPATH/libmkl_core.a $MKLPATH/libmkl_blacs_intelmpi.a -Wl,--end-group -lpthread[/plain]

I've tried all day long...Please help.

Raemon · ‎08-24-2009

Hi all.

This problem is solved. I have a typo of the descriptor name when using the cluster fft function.
The linking advisor is useful. Thanks to everyone.