Help on mixed programming Fortran and C with intel fortran and openmpi

[james_uta]

Hi, we have linux version intel fortran with openmpi on our
server. Now I have a package coded with C and fortran
languages, for example, as follows, how can I compile and
link them togather with intel fortran and openmpi? thanks.

Best Regards,

James


main.f
integer ncsub
write(*,*)'From main file'
call csub(ncsub)
stop
end

file1.c
int csub_(int ncsub)
{
printf("from csub, ncsub=%d\n",ncsub);
int ii=5;
fsub_(&ii);
return(1);
}

file2.f
subroutine fsub(ii)
integer ii
write(*,*)' From fsub'
write(*,*)' ii=',ii
return
end

[Ron_Green]

Quoting - james_uta

Hi, we have linux version intel fortran with openmpi on our
server. Now I have a package coded with C and fortran
languages, for example, as follows, how can I compile and
link them togather with intel fortran and openmpi? thanks.

Best Regards,

James


main.f
integer ncsub
write(*,*)'From main file'
call csub(ncsub)
stop
end

file1.c
int csub_(int ncsub)
{
printf("from csub, ncsub=%dn",ncsub);
int ii=5;
fsub_(&ii);
return(1);
}

file2.f
subroutine fsub(ii)
integer ii
write(*,*)' From fsub'
write(*,*)' ii=',ii
return
end

Assuming you built OpenMPI from source using the Intel Fortran compiler:
(see http://software.intel.com/en-us/articles/performance-tools-for-software-developers-building-open-mpi-with-the-intel-compilers/ this doc is a little dated, but it still is applicable)

then mpif77 and mpif90 wrappers will invoke the Intel Fortran compiler, and mpicc mpicxx will invoke gcc and g++.

then it's trivial:

mpicc -c file1.c
mpif77 -c file2.f
mpif77 -o myprog main.f file1.o file2.o

ron

[james_uta]

Quoting - Ronald Green (Intel)

Assuming you built OpenMPI from source using the Intel Fortran compiler:
(see http://software.intel.com/en-us/articles/performance-tools-for-software-developers-building-open-mpi-with-the-intel-compilers/ this doc is a little dated, but it still is applicable)

then mpif77 and mpif90 wrappers will invoke the Intel Fortran compiler, and mpicc mpicxx will invoke gcc and g++.

then it's trivial:

mpicc -c file1.c
mpif77 -c file2.f
mpif77 -o myprog main.f file1.o file2.o

ron

I have intel fortran compiler ifort and openmpi compiler mpicc, could you tell me how to compile it? thanks.

[Ron_Green]

Quoting - james_uta

I have intel fortran compiler ifort and openmpi compiler mpicc, could you tell me how to compile it? thanks.

mpicc -c file1.c
ifort -c -I file2.f
ifort -o myprog.exe -I main.f file1.o file2.o -L -lmpi

I haven't check this, so you may need to change the -lmpi to the name of the mpi library (this -lmpi assumes that libmpi.a exists in your lib dir)

For more help, go to open-mpi.org.

ron

[james_uta]

Appreciate.

[james_uta]

Quoting - Ronald W. Green (Intel)

mpicc -c file1.c
ifort -c -I file2.f
ifort -o myprog.exe -I main.f file1.o file2.o -L -lmpi

I haven't check this, so you may need to change the -lmpi to the name of the mpi library (this -lmpi assumes that libmpi.a exists in your lib dir)

For more help, go to open-mpi.org.

ron

Hi Ron, it works now, appreciate your help.
ButI need to do itat two steps as follows:
1)make libtmp.a from file1.c and file2.f first,
2)link the libtmp.a to main.f next, appreciate.
Could you please show me how to do it? thanks in advance.

James

[mrentropy1]

To make a static (.a) library,

$ ar -cvrs lib.a obj1.o obj2.o obj3.o ......

To link to static lib:

$ ifort (blah blah blah) -L -l

By convention libraries are named "libXXX.a" where XXX is the name of the library that is passed with the -l flag, e.g. -lXXX.

Oh by the way substitute mpi version of ifort etc (which are really just wrappers) as appropriate.....

Does that help, or did you already know that and it wasn't any help at all - or was I wrong and it didn't even work? :)

mrentropy

[james_uta]

Quoting - mrentropy

To make a static (.a) library,

$ ar -cvrs lib.a obj1.o obj2.o obj3.o ......

To link to static lib:

$ ifort (blah blah blah) -L -l

By convention libraries are named "libXXX.a" where XXX is the name of the library that is passed with the -l flag, e.g. -lXXX.

Oh by the way substitute mpi version of ifort etc (which are really just wrappers) as appropriate.....

Does that help, or did you already know that and it wasn't any help at all - or was I wrong and it didn't even work? :)

mrentropy

Appreciate Mrentropy, by following your suggestion I did the following way:

$ ar -cvrs libtmp.a file1.o file2.o
a - file1.o
a - file2.o

it is OK, then I did

$ ifort -o my.ex main.f -L./libtmp.a -I./
/tmp/ifort3tgOKM.o(.text+0x53): In function `MAIN__':
: undefined reference to `csub_'
[wenjun@petros new_test]$ ifort -o my.ex main.o -L./libtmp.a -I./
IPO Warning: can not find "main.o"
ld: main.o: No such file: No such file or directory

it is NOT OK, then I did another way
$ ifort -I/opt/openmpi/include -c main.f
$ifort -I/opt/openmpi/include -o my.ex main.o -I./ -L./libtmp.a -L/opt/openmpi/lib/libmpi.a
main.o(.text+0x53): In function `MAIN__':
: undefined reference to `csub_'

Please help me to fix it, appreciate.

James

[Kevin_D_Intel]

You didn't follow all of mrentropy's (Peter's) advice. The options are -l (lower-case "l" to name the library), and -L (upper-case "L" to specify the path to it), so -L wasused incorrectly.

Try these steps:

mpicc -c file1.c
ifort -c -I file2.f
ar -cvrs libtmp.a file1.o file2.o
ifort -o my.ex main.f -L./ -ltmp

[james_uta]

Quoting - Kevin Davis (Intel)

You didn't follow all of mrentropy's (Peter's) advice. The options are -l (lower-case "l" to name the library), and -L (upper-case "L" to specify the path to it), so -L wasused incorrectly.

Try these steps:

mpicc -c file1.c
ifort -c -I file2.f
ar -cvrs libtmp.a file1.o file2.o
ifort -o my.ex main.f -L./ -ltmp

Appreciate your great help, it works now.
But these are onlysimplified case for what I am doing now. When Iapplied this way to my real package. The lots of error information. Please help me to fix it, appreciate.

James

The lib makefile is
================================================

CC = /opt/openmpi/bin/mpicc
FC = ifort
AR = ar
FFLAGS = -O -I/opt/openmpi/include
CFLAGS = -O

OBJ_BLAS = blas1.o blas2.o blas3.o xerbla.o lsame.o
OBJ_TIME = md_timer_mpi.o
OBJ_MPI = md_wrap_mpi_c.o
OBJ_AZ = az_bilu.o az_cg.o az_cgs.o
az_cgstab.o az_check.o az_comm.o
az_converge.o az_dd_overlap.o az_dgemv2.o
az_dgemv3.o az_domain_decomp.o az_fortran_wrap.o
az_scaling.o az_flop_cnt.o az_gmres.o
az_gmresr.o az_ilu_util.o az_ilut.o
az_interface.o az_lu_y12.o az_matrix_util.o
az_matvec_mult.o az_old_matvec_mult.o az_pad_utils.o
az_poly.o az_precond.o az_qmrcgs.o
az_rilu.o az_solve.o az_sort.o
az_subdomain_solver.o az_tools.o az_util.o
az_icc.o az_fix_pt.o

OBJ_Y12M = y12m.o y12mae.o y12maf.o
y12mbe.o y12mbf.o y12mce.o
y12mcf.o y12mde.o y12mdf.o
y12mfe.o y12mge.o y12mhe.o
y12cck.o

OBJ_LA = la_dgetrf.o la_dgetri.o la_ilaenv.o
la_dgetf2.o la_dlaswp.o la_dtrtri.o
la_dtrti2.o la_dgetrs.o la_dgeco.o
la_dgedi.o la_dgefa.o la_dpotf2.o
la_dpotrf.o la_dlamch.o la_dlaic1.o
la_ieeeck.o

OBJ_F = $(OBJ_LA) $(OBJ_Y12M) $(OBJ_BLAS) az_reorder.o
OBJ_C = $(OBJ_AZ) $(OBJ_TIME) $(OBJ_MPI)
OBJ = $(OBJ_C) $(OBJ_F)

%.o:%.f
$(FC) $(FFLAGS) -c $<
%.o:%.c
$(CC) $(CFLAGS) -c $<

aztec: $(OBJ)

================================================
Application makefile is
================================================

OBJ = az_tutorial_with_MPI.o
CC = /opt/openmpi/bin/mpicc
FC = ifort

LDFLAGS = -L../lib -laztec -lm
CFLAGS = -O -c
FFLAGS = -O -c -I/opt/openmpi/include

all: sample

sample: $(OBJ)
ifort -I/opt/openmpi/include $(OBJ) -o sample $(LDFLAGS)
clean:
@echo "cleaning ..."
/bin/rm -f $(OBJ)

================================================
the link errors are
================================================
ifort -O -c -I/opt/openmpi/include -c -o az_tutorial_with_MPI.o az_tutorial_with_MPI.f
ifort -I/opt/openmpi/include az_tutorial_with_MPI.o -o sample -L../lib -laztec -lm
az_tutorial_with_MPI.o(.text+0x20): In function `MAIN__':
: undefined reference to `mpi_init_'
az_tutorial_with_MPI.o(.text+0x1bf): In function `MAIN__':
: undefined reference to `mpi_finalize_'
../lib/libaztec.a(az_interface.o)(.text+0x7): In function `AZ_second':
: undefined reference to `MPI_WTIME'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0xe): In function `get_parallel_info':
: undefined reference to `ompi_mpi_comm_world'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x13): In function `get_parallel_info':
: undefined reference to `MPI_Comm_size'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x1c): In function `get_parallel_info':
: undefined reference to `ompi_mpi_comm_world'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x21): In function `get_parallel_info':
: undefined reference to `MPI_Comm_rank'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x75): In function `md_read':
: undefined reference to `ompi_mpi_comm_world'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x7e): In function `md_read':
: undefined reference to `ompi_mpi_byte'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x8a): In function `md_read':
: undefined reference to `MPI_Recv'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x9b): In function `md_read':
: undefined reference to `ompi_mpi_comm_world'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0xa4): In function `md_read':
: undefined reference to `ompi_mpi_byte'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0xad): In function `md_read':

../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x3b8): In function `md_mpi_write':
: undefined reference to `ompi_mpi_byte'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x3c4): In function `md_mpi_write':
: undefined reference to `MPI_Send'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x3d5): In function `md_mpi_write':
: undefined reference to `ompi_mpi_byte'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x3de): In function `md_mpi_write':
: undefined reference to `MPI_Send'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x3f8): In function `md_mpi_wait':
: undefined reference to `MPI_Wait'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x42b): In function `md_mpi_wait':
: undefined reference to `ompi_mpi_byte'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x434): In function `md_mpi_wait':
: undefined reference to `MPI_Get_count'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x46e): In function `md_mpi_iwrite':
: undefined reference to `ompi_mpi_byte'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x47a): In function `md_mpi_iwrite':
: undefined reference to `MPI_Isend'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x48c): In function `md_mpi_iwrite':
: undefined reference to `ompi_mpi_byte'
../lib/libaztec.a(md_wrap_mpi_c.o)(.text+0x495): In function `md_mpi_iwrite':
: undefined reference to `MPI_Isend'
make: *** [sample] Error 1

================================================

[TimP]

If you have built openmpi for ifort, you can use mpif90 to link, then the mpi libraries would be searched automatically without naming them or the library path. When using ifort rather than mpif90, you must specify the mpi libraries. Maybe something like -lmpif was meant where you have -lm, which is redundant.

[james_uta]

Quoting - tim18

If you have built openmpi for ifort, you can use mpif90 to link, then the mpi libraries would be searched automatically without naming them or the library path. When using ifort rather than mpif90, you must specify the mpi libraries. Maybe something like -lmpif was meant where you have -lm, which is redundant.

Thanks, I try it mpif90 on our server, it shows
$ mpif90
--------------------------------------------------------------------------
Unfortunately, this installation of Open MPI was not compiled with
Fortran 90 support. As such, the mpif90 compiler is non-functional.
--------------------------------------------------------------------------
So we don't have it.


I replaced -lm with -lmpif in application make file on both ways as follows
1)
$ make
ifort -I/opt/openmpi/include az_tutorial_with_MPI.o -o sample -L../lib -laztec -L/opt/openmpi/lib -lmpif
IPO link: can not find -lmpif
ifort: error: problem during multi-file optimization compilation (code 1)
2)
$ make
ifort -I/opt/openmpi/include az_tutorial_with_MPI.o -o sample -L../lib -laztec -lmpif
IPO link: can not find -lmpif
ifort: error: problem during multi-file optimization compilation (code 1)

Please help me to fix it, appreciate.

[TimP]

You will need the ifort compiled openmpi libraries. The setup for ifort with openmpi should have been explained in the URL mentioned above, and information on openmpi web site should be relevant as well.

[james_uta]

Quoting - tim18

You will need the ifort compiled openmpi libraries. The setup for ifort with openmpi should have been explained in the URL mentioned above, and information on openmpi web site should be relevant as well.

I agree. But I need the help on the specific step to fix it.

For the simple example at the beginning, they include all the compilation options that are needed for this package, but it can not be applied for this package.

Youcansee that the parameters undifined in theerrorlink information are belong to openmpi includehead file, theyare already included in thelink option inthe application makefile. And those parameters were used in avery simple one-sentence wrap function. I don't knowwhat else I can do to letthe code find them out.

Please help me if you any one has some suggestion on it, thanks in advance.

James

[TimP]

If you want a pre-built MPI with ifort libraries, I don't think you have any choice other than Intel MPI.
If you tried the steps presented in the URL given above to build openmpi with ifort, and had problems, you could ask about it here, on the Intel HPC forum, or on the openmpi help forum.

[Ron_Green]

-lmpi is not being found. you need a -L option to help it along:

-L/opt/openmpi/lib -lmpi

[james_uta]

Quoting - tim18

If you want a pre-built MPI with ifort libraries, I don't think you have any choice other than Intel MPI.
If you tried the steps presented in the URL given above to build openmpi with ifort, and had problems, you could ask about it here, on the Intel HPC forum, or on the openmpi help forum.

You are right, it didn't have the Intel MPI library for library compilation.

I thought openmpi library would be enough due to 3-file simple casedidn't needit. Could you please tell me how to add it specifically into library makefile? appreciate.

[james_uta]

Quoting - Ronald W. Green (Intel)

-lmpi is not being found. you need a -L option to help it along:

-L/opt/openmpi/lib -lmpi

Appreciate Ron, I changed in the application makefile. Compilation is OK now as follows

$ make
ifort -I/opt/openmpi/include az_tutorial_with_MPI.o -o sample -L../lib -laztec -L/opt/openmpi/lib -lmpi

But when I ran it, it showed
$ ./sample
./sample: symbol lookup error: /opt/lam/gnu/lib/libmpi.so.0: undefined symbol: lam_ssi_base_module_compare

I didn't use the lam parallel library in compilation options, but it showed the running code need it and couldn't find it!
Please help me to fix it, appreciate.

[TimP]

You must set your LD_LIBRARY_PATH on all nodes so that you get the same libmpi shared library at run time which you used when building the application. e.g.
export LD_LIBRARY_PATH=/opt/openmpi/lib:$LD_LIBRARY_PATH
According to the foregoing, I still expect trouble, if you have built with ifort but use a library built with a different Fortran.

[james_uta]

Quoting - tim18

You must set your LD_LIBRARY_PATH on all nodes so that you get the same libmpi shared library at run time which you used when building the application. e.g.
export LD_LIBRARY_PATH=/opt/openmpi/lib:$LD_LIBRARY_PATH
According to the foregoing, I still expect trouble, if you have built with ifort but use a library built with a different Fortran.

You are right. Could you please tell me where I should set this path and how I can add it into Makefile? thanks.

[TimP]

Quoting - james_uta

You are right. Could you please tell me where I should set this path and how I can add it into Makefile? thanks.

You set LD_LIBRARY_PATH in your run time environment, not the build Makefile.