Intel Fortran in Visual Studio not multithreading

[rudydiaz]

I have a computer with dual socket AMD Epyc Rom 7552, with 48 cpus per socket for a total of 96 cpu. Windows system sees the 96 cpus and says I have access to 192 threads. But when I run a FORTRAN code in Visual Studio windows 11, and then check the resource monitor, it only sees 96 cpus not the 192 threads. PLUS, the code is running only as as fast as it runs on another computer with 16 cores and 32 threads.

The manufacturer of the computer claims it must be a problem with the FORTRAN since the Windows system sees all the threads available.  Any idea of what I can do?

[jimdempseyatthecove]

You can supply the .exe to the mpiexec. I used a .bat file as a program placeholder.

Both batch files taking 2x of one batch file indicates both processes thread teams on the same logical processors.

mpiexec will (should) setup a process available affinity set (different set per process), any OMP_NUM_THREADS(n) should be selected from this system subset of logical processors (unless you explicitly use ...proc_set...)

Remove any system environment variables related to affinity/procset/places

You can set the number of threads for all processes launched by mpiexec, either by a system environment variable .OR. the mpeixec global option:

   -genv OMP_NUM_THREADS n

.OR.

within the application via omp_set_num_threads(n)

.OR. using the num_threads(n) clause on the !$omp parallel... directive.

Also, add as a global option KMP_AFFINITY=verbose

Note, use the Intel supplied mpiexec

Each process will display what was available to it when launched.

Jim Dempsey

[rudydiaz]

When I enter mpiexec at the command prompt, it tells me the Usage and all the parameters, and it ends with this:

I assume that means I am already using the Intel mpiexec.

I then type at the command prompt >mpiexec -genv OMP_NUM_THREADS 8 -n 1 Console52.exe : -n 1 Console52q.exe

Where Console52.exe and Console52q.exe are the two identical executables. I already set KMP_AFFINITY=verbose earlier.

What I do not understand is what I must insert into this mpiexec line  (local environment variables? and if so, which ones? and if so, what would the command line look like then?) that will force the first one to use a set of cpus exclusively and the second one to use a different set of cpus exclusively. (So that each one only takes X seconds instead of 2X.)

I still have SMT off so that the resource monitor shows 48 cpu per socket. 

I cannot find an an example of doing this anywhere I search on the web. And the one time I find mention of using affinity to exclusively use certain nodes, it does not tell me how.

Am I missing something trivial?

[jimdempseyatthecove]

mpiexec does not have an option to provide local environment variable as opposed to global environment variables (at least not listed by mpiexec -help). To provide local environment variables, you can provide a batch shell for the ranks as needed. IOW the batch shell sets the environment variable then launches the application.

The first test is to .NOT. set affinities and run the two processes and to verify that each process (rank) runs on different cores.

mpiexec is designed to constrict each rank (via different process affinity masks) to a different set of logical processors.

Your concern was that mpiexec did not do this. IOW run a simple test to verify this via KMP_AFFINITY=verbose

Afterwards, iif need be, you can try to manipulate affinities if you so desire.

Jim Dempsey

[rudydiaz · Accepted solution]

OK.

We might as well close this thread.

Thanks again for your help.