Dear all,
I have a parallel fortran code, write by hybrid of mpi and openmp, using MPI between nodes and openmp between cores in one node
it run OK on HPC-A platform, the compile code is :
$ mpif90 -O5 -openmp program.f -o a.out
and the job submit script is (totally 216 nodes were used, 18 nodes x 12 core per node)
#!/bin/csh
#$ -ac P=2
#$ -ac n=216
#$ -ac T=12
#$ -jc K
#$ -cwd
source ${TMP}/hybrid-mpi-$JOB_ID/env.csh
mpiexec_mpt -np $TOTAL_PROCS omplace -nt $OMP_NUM_THREADS ./a.out >! Out_a
When I transplant to HPC-B platform, compiled with
mpif90 -O5 -mcmodel=large -openmp program.f -o a.out
and the jobsubmit script is (totally 160 nodes were used, 8 nodes x 20 cores per node)
#!/bin/bash
#SBATCH -J test1
#SBATCH -p ct
#SBATCH -N 8
#SBATCH --exclusive
#SBATCH --ntasks-per-node=1
#SBATCH -o a.o14_%j.out
#SBATCH -e a.o14_%j.err
#SBATCH --no-requeue
#SBATCH -A MBM_g1
#SBATCH --qos=MBMcns
source ~/.bash_profile
cd /home/MBMhpc-b/
export OMP_NUM_THREADS=20
mpiexec.hydra -bootstrap slurm ./a.out
it is much slower on HPC_B than on HPC_A, it seems that script on HPC_B only invoke one core in a node, actually there 20 cores in one node. How can I invoke 20 cores in one node in script?
Thanks-Young
