Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.
Announcements
Welcome to the Intel Community. If you get an answer you like, please mark it as an Accepted Solution to help others. Thank you!
26732 Discussions

problem of MPI and Openmp parallel program running on different HPC platform

Yanyang_C_
Beginner
103 Views

Dear all,  

I have a parallel fortran code, write by hybrid of mpi and openmp, using MPI between nodes and openmp between cores in one node

 it run OK on HPC-A platform, the compile code is :

$ mpif90 -O5 -openmp program.f -o a.out

and the job submit script is (totally 216 nodes were used, 18 nodes x 12 core per node)

#!/bin/csh
#$ -ac P=2 
#$ -ac n=216
#$ -ac T=12
#$ -jc K
#$ -cwd
source ${TMP}/hybrid-mpi-$JOB_ID/env.csh
mpiexec_mpt -np $TOTAL_PROCS omplace -nt $OMP_NUM_THREADS ./a.out >! Out_a

When I transplant to HPC-B platform, compiled with 

mpif90 -O5 -mcmodel=large -openmp program.f -o a.out

and the jobsubmit script is (totally 160 nodes were used, 8 nodes x 20 cores per node)

#!/bin/bash
#SBATCH -J test1
#SBATCH -p ct
#SBATCH -N 8
#SBATCH --exclusive
#SBATCH --ntasks-per-node=1
#SBATCH -o a.o14_%j.out
#SBATCH -e a.o14_%j.err
#SBATCH --no-requeue
#SBATCH -A MBM_g1
#SBATCH --qos=MBMcns
source ~/.bash_profile
cd /home/MBMhpc-b/
export OMP_NUM_THREADS=20
mpiexec.hydra -bootstrap slurm ./a.out

it is much slower on HPC_B than on HPC_A, it seems that script on HPC_B only invoke one core in a node, actually there 20 cores in one node. How can I invoke 20 cores in one node in script?  

Thanks-Young

0 Kudos
1 Reply
Yanyang_C_
Beginner
103 Views

I added follow script following the other #SBATCHs, it still seems cannot invoke 20 cores in a node

#SBATCH --cpus-per-task=16

in fortran code the openmp was used by

!$omp parallel 

....

Reply