Link Copied
Dear Vidya
Thanks for answering
Find attached the executed experiment (file zgemm.F90).
This is what I have done in the Xeon Gold 6240:
- Compilation:
$> source /mnt/beegfs2018/app/intel-oneapi-2021/compiler/2021.4.0/env/vars.sh
$> source /mnt/beegfs2018/app/intel-oneapi-2021/mkl/2021.4.0/env/vars.sh$> make
ifort -O3 -qopenmp -I/mnt/beegfs2018/app/intel-oneapi-2021/mkl/2021.4.0/include/ -I/mnt/beegfs2018/app/intel-oneapi-2021/mkl/2021.4.0/include/intel64/lp64 -o zgemm_ex zgemm.F90 -L/mnt/beegfs2018/app/intel-oneapi-2021/mkl/2021.4.0/lib/intel64/ -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread - Execution:
#!/bin/bash
#SBATCH --job-name=zgemm
#SBATCH --time=0-12:00:00
#SBATCH --partition=cpu36memory384
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=36
#SBATCH --exclusive
cd ${SLURM_SUBMIT_DIR}
source /mnt/beegfs2018/app/intel-oneapi-2021/compiler/2021.4.0/env/vars.sh
source /mnt/beegfs2018/app/intel-oneapi-2021/mkl/2021.4.0/env/vars.sh
M=1281 N=1281 K=38400 ./zgemm_ex &> out.6240.txt - Results file: times.6240.txt
There are small differences for the Xeon Platinum 8160:
- Compilation:
$> module load intel/2020.1 mkl/2021.4
$> make
ifort -O3 -qopenmp -I/apps/INTEL/oneapi/2021.4/mkl/2021.4.0//include/ -I/apps/INTEL/oneapi/2021.4/mkl/2021.4.0//include/intel64/lp64 -o zgemm_ex zgemm.F90 -L/apps/INTEL/oneapi/2021.4/mkl/2021.4.0//lib/intel64/ -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread - Execution:
#!/bin/bash
#SBATCH --job-name=zgemm
#SBATCH --output=par.out
#SBATCH --error=par.err
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=48
#SBATCH --time=00:30:00
#SBATCH --qos=debug
#SBATCH --exclusive
module load intel/2020.1 mkl/2021.4
M=1281 N=1281 K=38400 ./zgemm_ex &> out.8160.txt" - Results file: times.8160.txt
The speedup computation is independent in every machine:
SpeedUp( X threads ) = Time( 1 thread )/Time( X threads )
So, a better scalability doesn't means a lower execution time. In fact, if we use the same amount of threads, we always get a better times in the Xeon Gold 6240:
If we recall the scalability graph, the behaviour of the Xeon Platinum 8160 looks better.
Looking at these two graphics, I have a pair of questions:
- Why is zgemm going much more faster in the sequential case. The frequency difference does not explain so much difference.
- Why do we have a better speedup in the 8160? I assume that the experiment in the 6240 stress more the cache hierarchy and the main memory.
Please, let me know if do you need more information.
Cheers
Hi Vidya
You may find the all the timings in the attached files in my previous post (Included the time for one thread). In your experiment the time with one thread is equal to the sequential time. what did you do to get the sequential time?
These are the times of my experiments with one thread:
- Gold 6240: 4.7412997500000005
- Platinum 8160: 8.15284851
This plots your timings together with mines (logarithmic scale at your right):
 |
 |
And the speedup plot:
What I can see is that the speedup behaviour of your experiments are similar to what I have found in the Platinum 8160. In the other hand, Gold 6240 is the fastest processor when the number of threads are under 10, but it have problems when the number of threads is greater.
Regards
Rogeli
Dear Vidya
> Could you please try running the code with a larger work size and let us know the results?
> I'm attaching the results that i got on Intel(R) Xeon(R) Gold 6128 CPU @ 3.40GHz
> CPU model.
I have repeated your experiment (M=10000, N=10000 and K=10000) and I get very similar results:
> Activating larger number of cores needs more power, but power is limited by TDP (Thermal Design Power). Core frequency will be
> reduced by increasing number of active cores.
It makes sense, but is the first time that I read about this kind of behaviour. Is this behaviour documented anywhere? Do we have the same kind of limitation in the "Platinium family"? Is there any way to check the current frequency of the CPU ?
Regards
Rogeli
