- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In the makefile for MUMPS,I need to specify the various BLACS and SCALAPACK libraries... which should I be using for MVAPICH-0.9.9? I only see options for Intel MPI, OpenMPI and SGI's MPT.
Thanks in advance for any help you can provide.
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In the makefile for MUMPS,I need to specify the various BLACS and SCALAPACK libraries... which should I be using for MVAPICH-0.9.9?
Thanks in advance for any help you can provide.
In the mumps makefile just put the path for scalapack, blacs, lapack, blas, metis etc. that are installed on your cluster and thats pretty much it.
You can test what works by using a trivial test program given at scalapacks home page. If that works with your mpi then it should work with mumps too.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
#
# This file is part of MUMPS 4.9.2, built on Thu Nov 5 07:05:08 UTC 2009
#
#Begin orderings
# NOTE that PORD is distributed within MUMPS by default. If you would like to
# use other orderings, you need to obtain the corresponding package and modify
# the variables below accordingly.
# For example, to have Metis available within MUMPS:
# 1/ download Metis and compile it
# 2/ uncomment (suppress # in first column) lines
# starting with LMETISDIR, LMETIS
# 3/ add -Dmetis in line ORDERINGSF
# ORDERINGSF = -Dpord -Dmetis
# 4/ Compile and install MUMPS
# make clean; make (to clean up previous installation)
#
# Metis/ParMetis and SCOTCH/PT-SCOTCH (ver 5.1 and later) orderings are now available for MUMPS.
#
#SCOTCHDIR = ${HOME}/scotch_5.1_esmumps
#ISCOTCH = -I$(SCOTCHDIR)/include
# You have to choose one among the following two lines depending on
# the type of analysis you want to perform. If you want to perform only
# sequential analysis choose the first (remember to add -Dscotch in the ORDERINGSF
# variable below); for both parallel and sequential analysis choose the second
# line (remember to add -Dptscotch in the ORDERINGSF variable below)
#LSCOTCH = -L$(SCOTCHDIR)/lib -lesmumps -lscotch -lscotcherr
#LSCOTCH = -L$(SCOTCHDIR)/lib -lptesmumps -lptscotch -lptscotcherr
LPORDDIR = $(topdir)/PORD/lib/
IPORD = -I$(topdir)/PORD/include/
LPORD = -L$(LPORDDIR) -lpord
#LMETISDIR = /local/metis/
#IMETIS = # Metis doesn't need include files (Fortran interface avail.)
# You have to choose one among the following two lines depending on
# the type of analysis you want to perform. If you want to perform only
# sequential analysis choose the first (remember to add -Dmetis in the ORDERINGSF
# variable below); for both parallel and sequential analysis choose the second
# line (remember to add -Dparmetis in the ORDERINGSF variable below)
#LMETIS = -L$(LMETISDIR) -lmetis
#LMETIS = -L$(LMETISDIR) -lparmetis -lmetis
# The following variables will be used in the compilation process.
# Please note that -Dptscotch and -Dparmetis imply -Dscotch and -Dmetis respectively.
#ORDERINGSF = -Dscotch -Dmetis -Dpord -Dptscotch -Dparmetis
ORDERINGSF = -Dpord
ORDERINGSC = $(ORDERINGSF)
LORDERINGS = $(LMETIS) $(LPORD) $(LSCOTCH)
IORDERINGSF = $(ISCOTCH)
IORDERINGSC = $(IMETIS) $(IPORD) $(ISCOTCH)
#End orderings
########################################################################
################################################################################
MKLPATH = /opt/intel/Compiler/11.0/083/mkl/lib/em64t
PLAT =
RM = /bin/rm -f
CC = mpiicc
FC = ifort
FL = mpiifort
AR = ar vr
#RANLIB = ranlib
RANLIB = echo
SCALAP = -L$(MKLPATH) -lmkl_scalapack_lp64 $(MKLPATH)/libmkl_solver_lp64.a -Wl,--start-group -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_intelmpi_lp64 -Wl,--end-group -openmp -lpthread
INCPAR = -I/home/opt/intel/impi/4.0.0.025/include64
LIBPAR = $(SCALAP) -L/home/opt/intel/impi/4.0.0.025/lib64 -lmpi
INCSEQ = -I$(topdir)/libseq
LIBSEQ = -L$(topdir)/libseq -lmpiseq
LIBBLAS = -L$(MKLPATH) $(MKLPATH)/libmkl_solver_lp64.a -Wl,--start-group -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_intelmpi_lp64 -Wl,--end-group -openmp -lpthread
LIBOTHERS = -lpthread
#Preprocessor defs for calling Fortran from C (-DAdd_ or -DAdd__ or -DUPPER)
CDEFS = -DAdd_
#Begin Optimized options
OPTF = -O -Dintel_ -DALLOW_NON_INIT -nofor_main
OPTL = -O -nofor_main
OPTC = -O
#End Optimized options
INC = $(INCPAR)
LIB = $(LIBPAR)
LIBSEQNEEDED =
Best regards,
Roberta
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks! It is very helpful. I am trying to install mumps with the MKL library. The "LIBBLAS" is not correct, as shown at the end.
ifort: command line warning #10006: ignoring unknown option '-/global/software/intel2012/composer_xe_2011_sp1.11.339/mkl/lib/intel64/libmkl_solver_lp64.a'
Any idea to fix it? The cluster has Xeon X5650 2.66GHz.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Apparently, you didn't succeed in setting MKLPATH so that the Makefile could use it.
Did you source the mklvars script first?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you, Tim. There is a typo in the path. It is running now!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Problem on another cluster with Xeon X5650 2.66GHz processors with more processors. I tried different IntelMPI and MKL using module load. It seems that only MKL 10.2.6 works with Intel 11.1. But this combination does not finally fit my main program (don't know why). So I tried to use MKL and compilers from the same package of Intel14/composer_xe_2013_sp1.2.144 with IntelMPI 4.1.1.036/intel64. Makefile and output files are attached.
Anybody can help again? I had similar mistake for quite a while.
mpif90 -O -DINTEL_ -DALLOW_NON_INIT -nofor_main -I/global/software/intel14/impi/4.1.1.036/intel64/include -I. -I/home/MyName/parallel/MUMPS/include -c dsimpletest.F -o dsimpletest.o
mpif90 -o dsimpletest -O -nofor_main dsimpletest.o /home/MyName/parallel/MUMPS/lib/libdmumps.a /home/MyName/parallel/MUMPS/lib/libmumps_common.a -L/home/MyName/parallel/parmetis -lparmetis -lmetis -L/home/MyName/parallel/MUMPS/PORD/lib/ -lpord -L/global/software/intel14/mkl/lib/intel64 /global/software/intel14/mkl/lib/intel64/libmkl_blas95_lp64.a /global/software/intel14/mkl/lib/intel64/libmkl_lapack95_lp64.a /global/software/intel14/mkl/lib/intel64/libmkl_scalapack_lp64.a -Wl,--start-group /global/software/intel14/mkl/lib/intel64/libmkl_cdft_core.a /global/software/intel14/mkl/lib/intel64/libmkl_intel_lp64.a /global/software/intel14/mkl/lib/intel64/libmkl_core.a /global/software/intel14/mkl/lib/intel64/libmkl_intel_thread.a -Wl,--end-group /global/software/intel14/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a -lpthread -lm -openmp -I/global/software/intel14/mkl/include/intel64/lp64 -I/global/software/intel14/mkl/include -L/global/software/intel14/impi/4.1.1.036/intel64/lib -lmpi -L/global/software/intel14/mkl/lib/intel64 /global/software/intel14/mkl/lib/intel64/libmkl_blas95_lp64.a /global/software/intel14/mkl/lib/intel64/libmkl_lapack95_lp64.a -Wl,--start-group /global/software/intel14/mkl/lib/intel64/libmkl_intel_lp64.a /global/software/intel14/mkl/lib/intel64/libmkl_core.a /global/software/intel14/mkl/lib/intel64/libmkl_intel_thread.a -Wl,--end-group -lpthread -lm -openmp -I/global/software/intel14/mkl/include/intel64/lp64 -I/global/software/intel14/mkl/include -lpthread
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
make[1]: *** [dsimpletest] Error 1
make[1]: Leaving directory `/g01/home/MyName/parallel/MUMPS/examples'
make: *** [dexamples] Error 2
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
On the face of it, your collection of .o files doesn't include a successful compilation of a C main() (which your nofor_main option tells the linker script to expect).
Your gcc libraries are rather old to be using with up to date MKL. I don't know whether that's a problem. I suppose Red Hat 5.5 should still be fully supported on Westmere CPUs.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you, Tim.
Does Intel compiler rely on default gcc? It does not seem that I can change gcc after I load Intel compiler and MKL. The Red Hat should have no problem, as it is being used on several clusters here. And it works with an older MKL (with Intel 11).
I attempted to load gcc-4.4.4 and set MPI /MKl path to Intel's directory. It does not work at all.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Intel tools which support Red Hat 6.1 should have no difficulty with gcc 4.4 being on PATH and LD_LIBRARY_PATH.
I don't know whether the configuration of mvapich or your application might be affected by so large a change in gcc version.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you very much, Tim.
I don't know how to set LD_LIBRARY_PATH for gcc 4.4 after I load Intel14 and MKL. It takes me lots of time to make small progress, as I am doing that in trial-and-error. Just set something in Makefile like:
LD_LIBRARY_PATH=usr/lib/gcc/x86_64-redhat-linux6E/4.4.6 ?
I know this one does not work. I have to try something else. Would you please give more instruction?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Setting up an Intel compiler by sourcing the compilervars script adds required entries to LD_LIBRARY_PATH. In order to change your gcc paths afterwards, you would add the new gcc paths to the beginning of PATH and LD_LIBRARY_PATH.
I find it a little difficult to believe that anyone would run a 32-bit cluster OS as I haven't seen one for 8 years. I don't know the installation path of your gcc 4.4, so I'll give as an example supposing that gcc 64-bit were installed with prefix set as /usr/local/gcc4.4:
export PATH=/usr/local/gcc4.4/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/gcc4.4/lib64:$LD_LIBRARY_PATH
I'd reiterate that the main problem is that your link line expects your application to use a C main(). If there is a Fortran main program (normally, one headed up by PROGRAM .....) you would remove the nofor_main options. If it is actually a C main(), you would check that this compiled successfully to a .o file included in your list.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page