Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.
28384 Discussions

please help me parallel computing !!

sdj77
New Contributor I
1,100 Views

Dear, experts

I use fortran 77 and oneAPI compiler.

My computer consists of dual cpu and each has 20 cores. I tried to proceed with parallel calculation using all 40 cores, but only 20 cores were used. I don't know why... 

Plz help me experts:)제목 없음.png

0 Kudos
15 Replies
jimdempseyatthecove
Honored Contributor III
1,061 Views

Using environment variable:

    KMP_AFFINITY=verbose

May provide some insight as to what is going on.

Your display is showing 32 of the logical processors in use out of the 80 logical processors (2xCPU each with 20 cores, 2 HW threads per core).

One limiting factor would be OpenMP environment variable settings affecting thread count and/or affinity pinning. The KMP_AFFINITY=verbose setting may shed some light on incorrect OpenMP environment variable setting

 

*** Additional note. If your system is running Windows (your screen shot seems to indicate this), then when the Windows system has more than 64 hardware threads (yours has 80), the Windows O/S partitions the available hardware threads into "Processor Groups", each group with less than or equal to 64 hardware threads, and the group packing may or may not be balanced. One might expect for your system to have 2 Processor groups, one for each CPU (20 cores with 40 HW threads), but this may not necessarily be the case. Potentially, (from your screenshot) you could have 3 processor groups: two with 16 cores/32 HW threads, and one with 8 cores/16 HW threads. Again KMP_AFFINITY=verbose setting may shed some light on this. Due to how Windows handles affinity with a bitmask 32-bit or 64 -bit depending on application, the 32 logical processor selection could also be explained by your application is 32-bits.

 

Newer versions of (or different OpenMP considerations) may permit the Windows OpenMP application to utilize more than one "Processor Group".

Jim Dempsey

0 Kudos
sdj77
New Contributor I
1,033 Views
     I applied environment variable setting, but still, only 40 threads operated.
 
      program test
      use omp_lib
      parameter (n=50) 
      integer i, j, k, n, t
      real z(n,n,n), x(n,n,n), time, start_t, end_t
      start_t = omp_get_wtime()
      call omp_set_num_threads(80)
      t=omp_get_max_threads()
      !$omp parallel do
      do i= 1, n
          do j = 1, n
              do k = 1, n
                  x(i,j,k) = i**2 + j**2 + k**2
              call saxpy(z,a,x,y,n)
              end do
          call saxpy(z,a,x,y,n)
          end do
      call saxpy(z,a,x,y,n)
  end do
      !$omp end parallel do
      write (*,*) "number of threads =", t
      write(*,*) x(n,n,n)
      end_t = omp_get_wtime()
      result= end_t-start_t
      write (*,200) "run time:",result
200   format (a,F6.2)     
      write (*,*) ""
      stop
      end
    
      subroutine saxpy(z,a,x,y,n)
      integer i, j, n
      real z(n,n,n), a, x(n,n,n), y
      !$omp parallel do
      do i= 1, n
          do j= 1, n
              do k= 1,n 
                  z(i,j,k) = a * x(i,j,k)+y
              end do
          end do
      end do
      !$omp end parallel do
      return
      end
캡처.PNG
0 Kudos
jimdempseyatthecove
Honored Contributor III
1,008 Views

Did the KMP_AFFINITY=verbose indicate more than one Processor Group?

It should also show number of threads and thread binding (if any).

Jim Dempsey

0 Kudos
sdj77
New Contributor I
980 Views

How to indicate more than one Processor Group with " KMP_AFFINITY=verbose" ?

I applied "KMP_HW_SUBSET=40c,2t OMP_NUM_THREADS=80 KMP_AFFINITY=compact"

But still, it use 16 cores.

0 Kudos
jimdempseyatthecove
Honored Contributor III
935 Views

>>I applied "KMP_HW_SUBSET=40c,2t OMP_NUM_THREADS=80 KMP_AFFINITY=compact"

Is what you specify what you want. Which is not necessarily what you get.

Whereas " KMP_AFFINITY=verbose" tells you the topology (what is available) as well as what you got from what you requested.

Please post the console output from the " KMP_AFFINITY=verbose".

Jim Dempsey

0 Kudos
sdj77
New Contributor I
738 Views

There is "KMP_AFFINITY=verbose". I don't know why only one socket appears.

 

OMP: Info #155: KMP_AFFINITY: Initial OS proc set respected: 0-31
OMP: Info #217: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #157: KMP_AFFINITY: 32 available OS procs
OMP: Info #158: KMP_AFFINITY: Uniform topology
OMP: Info #288: KMP_AFFINITY: topology layer "LL cache" is equivalent to "socket".
OMP: Info #288: KMP_AFFINITY: topology layer "L3 cache" is equivalent to "socket".
OMP: Info #288: KMP_AFFINITY: topology layer "L2 cache" is equivalent to "core".
OMP: Info #288: KMP_AFFINITY: topology layer "L1 cache" is equivalent to "core".
OMP: Info #192: KMP_AFFINITY: 1 socket x 16 cores/socket x 2 threads/core (16 total cores)
OMP: Info #219: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #172: KMP_AFFINITY: OS proc 0 maps to socket 0 core 0 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 1 maps to socket 0 core 0 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 2 maps to socket 0 core 1 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 3 maps to socket 0 core 1 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 4 maps to socket 0 core 2 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 5 maps to socket 0 core 2 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 6 maps to socket 0 core 3 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 7 maps to socket 0 core 3 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 8 maps to socket 0 core 5 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 9 maps to socket 0 core 5 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 10 maps to socket 0 core 6 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 11 maps to socket 0 core 6 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 12 maps to socket 0 core 9 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 13 maps to socket 0 core 9 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 14 maps to socket 0 core 10 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 15 maps to socket 0 core 10 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 16 maps to socket 0 core 12 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 17 maps to socket 0 core 12 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 18 maps to socket 0 core 13 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 19 maps to socket 0 core 13 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 20 maps to socket 0 core 16 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 21 maps to socket 0 core 16 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 22 maps to socket 0 core 17 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 23 maps to socket 0 core 17 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 24 maps to socket 0 core 18 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 25 maps to socket 0 core 18 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 26 maps to socket 0 core 19 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 27 maps to socket 0 core 19 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 28 maps to socket 0 core 20 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 29 maps to socket 0 core 20 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 30 maps to socket 0 core 21 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 31 maps to socket 0 core 21 thread 1
OMP: Info #255: KMP_AFFINITY: pid 7916 tid 14548 thread 0 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 7916 tid 16824 thread 1 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 7916 tid 16816 thread 6 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 7916 tid 4864 thread 3 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 7916 tid 9732 thread 4 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 7916 tid 14028 thread 29 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 7916 tid 16176 thread 38 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 7916 tid 15028 thread 46 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 7916 tid 11852 thread 8 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 7916 tid 14292 thread 62 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 7916 tid 15060 thread 10 bound to OS proc set 0-31
------------------------------------

0 Kudos
sdj77
New Contributor I
886 Views

There is output from the "KMP_AFFINITY=verbose". I don't know why only one socket appears.

Also, I turned on "multi-processor compilation" function.
 
OMP: Info #155: KMP_AFFINITY: Initial OS proc set respected: 0-31
OMP: Info #217: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #157: KMP_AFFINITY: 32 available OS procs
OMP: Info #158: KMP_AFFINITY: Uniform topology
OMP: Info #288: KMP_AFFINITY: topology layer "LL cache" is equivalent to "socket".
OMP: Info #288: KMP_AFFINITY: topology layer "L3 cache" is equivalent to "socket".
OMP: Info #288: KMP_AFFINITY: topology layer "L2 cache" is equivalent to "core".
OMP: Info #288: KMP_AFFINITY: topology layer "L1 cache" is equivalent to "core".
OMP: Info #192: KMP_AFFINITY: 1 socket x 16 cores/socket x 2 threads/core (16 total cores)
OMP: Info #219: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #172: KMP_AFFINITY: OS proc 0 maps to socket 0 core 0 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 1 maps to socket 0 core 0 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 2 maps to socket 0 core 1 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 3 maps to socket 0 core 1 thread 1
0 Kudos
sdj77
New Contributor I
738 Views
There is output from the "KMP_AFFINITY=verbose". I don't know why only one socket appears.

 

OMP: Info #155: KMP_AFFINITY: Initial OS proc set respected: 0-31
OMP: Info #217: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #157: KMP_AFFINITY: 32 available OS procs
OMP: Info #158: KMP_AFFINITY: Uniform topology
OMP: Info #288: KMP_AFFINITY: topology layer "LL cache" is equivalent to "socket".
OMP: Info #288: KMP_AFFINITY: topology layer "L3 cache" is equivalent to "socket".
OMP: Info #288: KMP_AFFINITY: topology layer "L2 cache" is equivalent to "core".
OMP: Info #288: KMP_AFFINITY: topology layer "L1 cache" is equivalent to "core".
OMP: Info #192: KMP_AFFINITY: 1 socket x 16 cores/socket x 2 threads/core (16 total cores)
OMP: Info #219: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #172: KMP_AFFINITY: OS proc 0 maps to socket 1 core 0 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 1 maps to socket 1 core 0 thread 1

OMP: Info #172: KMP_AFFINITY: OS proc 2 maps to socket 1 core 1 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 3 maps to socket 1 core 1 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 4 maps to socket 1 core 3 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 5 maps to socket 1 core 3 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 6 maps to socket 1 core 5 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 7 maps to socket 1 core 5 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 8 maps to socket 1 core 6 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 9 maps to socket 1 core 6 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 10 maps to socket 1 core 8 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 11 maps to socket 1 core 8 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 12 maps to socket 1 core 10 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 13 maps to socket 1 core 10 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 14 maps to socket 1 core 12 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 15 maps to socket 1 core 12 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 16 maps to socket 1 core 13 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 17 maps to socket 1 core 13 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 18 maps to socket 1 core 16 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 19 maps to socket 1 core 16 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 20 maps to socket 1 core 17 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 21 maps to socket 1 core 17 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 22 maps to socket 1 core 18 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 23 maps to socket 1 core 18 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 24 maps to socket 1 core 19 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 25 maps to socket 1 core 19 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 26 maps to socket 1 core 20 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 27 maps to socket 1 core 20 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 28 maps to socket 1 core 21 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 29 maps to socket 1 core 21 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 30 maps to socket 1 core 25 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 31 maps to socket 1 core 25 thread 1
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16504 thread 0 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 9244 thread 1 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 12556 thread 3 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 15760 thread 2 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 13612 thread 10 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 10244 thread 5 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 4824 thread 21 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16040 thread 26 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 14664 thread 8 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 14456 thread 36 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 12264 thread 4 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 10592 thread 46 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 10852 thread 12 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 10700 thread 13 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16896 thread 62 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 10464 thread 68 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 13148 thread 16 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 15596 thread 17 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 7524 thread 18 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 13204 thread 19 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 13648 thread 20 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 4364 thread 22 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 17324 thread 6 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 9308 thread 23 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16512 thread 24 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16940 thread 25 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 17392 thread 7 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 14644 thread 27 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 12440 thread 28 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 15252 thread 29 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16544 thread 30 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16536 thread 31 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 13292 thread 32 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 13344 thread 33 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 15120 thread 34 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 12444 thread 35 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 2152 thread 9 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 7336 thread 37 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 15500 thread 38 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 12540 thread 39 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 12564 thread 40 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 15112 thread 41 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 13216 thread 42 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 13760 thread 43 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 13228 thread 44 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 15900 thread 45 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 15476 thread 47 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 15428 thread 11 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 12688 thread 48 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 14640 thread 49 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16620 thread 50 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 2512 thread 51 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16988 thread 52 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 14568 thread 53 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 14684 thread 54 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16904 thread 55 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 7076 thread 56 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 14448 thread 57 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 4480 thread 58 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 13328 thread 59 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 1684 thread 60 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16560 thread 61 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 7320 thread 63 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 14968 thread 14 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 17312 thread 64 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 7560 thread 65 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16564 thread 66 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16848 thread 67 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 15492 thread 15 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 13672 thread 69 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16692 thread 70 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16068 thread 72 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16008 thread 71 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 17260 thread 73 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 10572 thread 74 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 11260 thread 75 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 15936 thread 76 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16248 thread 77 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 12184 thread 78 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 14680 tid 16180 thread 79 bound to OS proc set 0-31

0 Kudos
jimdempseyatthecove
Honored Contributor III
868 Views

From your first post screen shot you should see 2 sockets or at least 1 socket x 20 cores x 2 threads/core.

The verbose should have shown the entire topology, including the Windows (compatible) "Processor Groups".

Your OMP: Info listing is incomplete. Please send the complete listing (you can strike out any sensitive information).

I would have expected (on Windows) 2 Processor Groups (one for each socket) with 20 cores x 2 threads/core. Each group .le. 64 logical processors.

Note, if this is on a server running virtualization, then the virtualization setup may be presenting your process access to subset of the complete system resources such as 4 virtual systems (16 cores, 16 cores, 4 cores, 4 cores) your process using a virtual 1 socket x 16 core subset of your system. I (we) are interpreting your posts without complete information..

 

Try building and running this C++ program: https://github.com/tpn/cpuid-topo (untested by me).

 

Jim Dempsey

0 Kudos
sdj77
New Contributor I
738 Views

First, I approximately understood about virtualization.

I don't know how to use C++ program, so I confirmed my CPU through CPU-Z program.

I tried a bench mark(force the load) and total 80 threads operated....

And there is the complete listing. I don't know why it doesn't present "Processor Groups".

 

OMP: Info #155: KMP_AFFINITY: Initial OS proc set respected: 0-31
OMP: Info #217: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #157: KMP_AFFINITY: 32 available OS procs
OMP: Info #158: KMP_AFFINITY: Uniform topology
OMP: Info #288: KMP_AFFINITY: topology layer "LL cache" is equivalent to "socket".
OMP: Info #288: KMP_AFFINITY: topology layer "L3 cache" is equivalent to "socket".
OMP: Info #288: KMP_AFFINITY: topology layer "L2 cache" is equivalent to "core".
OMP: Info #288: KMP_AFFINITY: topology layer "L1 cache" is equivalent to "core".
OMP: Info #192: KMP_AFFINITY: 1 socket x 16 cores/socket x 2 threads/core (16 total cores)
OMP: Info #219: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #172: KMP_AFFINITY: OS proc 0 maps to socket 1 core 0 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 1 maps to socket 1 core 0 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 2 maps to socket 1 core 1 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 3 maps to socket 1 core 1 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 4 maps to socket 1 core 3 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 5 maps to socket 1 core 3 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 6 maps to socket 1 core 5 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 7 maps to socket 1 core 5 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 8 maps to socket 1 core 6 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 9 maps to socket 1 core 6 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 10 maps to socket 1 core 8 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 11 maps to socket 1 core 8 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 12 maps to socket 1 core 10 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 13 maps to socket 1 core 10 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 14 maps to socket 1 core 12 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 15 maps to socket 1 core 12 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 16 maps to socket 1 core 13 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 17 maps to socket 1 core 13 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 18 maps to socket 1 core 16 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 19 maps to socket 1 core 16 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 20 maps to socket 1 core 17 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 21 maps to socket 1 core 17 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 22 maps to socket 1 core 18 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 23 maps to socket 1 core 18 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 24 maps to socket 1 core 19 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 25 maps to socket 1 core 19 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 26 maps to socket 1 core 20 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 27 maps to socket 1 core 20 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 28 maps to socket 1 core 21 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 29 maps to socket 1 core 21 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 30 maps to socket 1 core 25 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 31 maps to socket 1 core 25 thread 1
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 19788 thread 0 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 18988 thread 1 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 4100 thread 2 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 14952 thread 3 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 18032 thread 4 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 15232 thread 75 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 20256 thread 6 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 13788 thread 7 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 19832 thread 8 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 17288 thread 9 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 15928 thread 10 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 9728 thread 11 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 15620 thread 12 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 18480 thread 13 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 19592 thread 14 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 832 thread 16 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 7152 thread 15 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 14612 thread 17 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 14156 thread 18 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 18804 thread 19 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 9540 thread 20 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 15208 thread 22 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 1720 thread 21 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 18644 thread 23 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 3488 thread 24 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 14808 thread 25 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 12844 thread 26 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 19464 thread 27 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 12916 thread 28 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 15108 thread 29 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 9788 thread 30 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 14128 thread 31 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 10220 thread 32 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 18408 thread 33 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 7896 thread 34 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 3328 thread 35 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 7264 thread 36 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 16020 thread 37 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 16328 thread 38 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 8404 thread 39 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 3660 thread 40 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 13464 thread 41 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 16660 thread 42 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 14976 thread 43 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 4316 thread 44 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 3452 thread 45 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 15588 thread 46 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 16052 thread 47 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 19748 thread 48 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 14692 thread 49 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 16704 thread 50 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 14528 thread 51 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 17500 thread 52 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 15348 thread 53 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 17152 thread 54 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 20352 thread 55 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 18808 thread 56 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 11252 thread 57 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 18904 thread 58 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 8476 thread 59 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 14648 thread 60 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 17264 thread 61 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 19508 thread 62 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 3528 thread 63 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 20172 thread 64 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 17524 thread 65 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 18852 thread 66 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 13824 thread 67 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 8792 thread 68 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 12720 thread 69 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 2160 thread 70 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 15304 thread 71 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 12552 thread 72 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 16520 thread 73 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 15080 thread 74 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 14860 thread 5 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 4596 thread 76 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 15636 thread 77 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 4108 thread 78 bound to OS proc set 0-31
OMP: Info #255: KMP_AFFINITY: pid 19920 tid 9452 thread 79 bound to OS proc set 0-31

 

Socket 1,2 is presented at next picture and there is total 40 core.

sdj77_1-1640756975969.png12처.PNG

0 Kudos
sdj77
New Contributor I
840 Views

First, I approximately understood about virtualization.

I don't know how to use C++ program, so I confirmed my CPU through CPU-Z program.

I tried a bench mark(force the load) and total 80 threads operated....

And there is the complete listing. I don't know why it doesn't present "Processor Groups".

 

OMP: Info #155: KMP_AFFINITY: Initial OS proc set respected: 0-31
OMP: Info #217: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #157: KMP_AFFINITY: 32 available OS procs
OMP: Info #158: KMP_AFFINITY: Uniform topology
OMP: Info #288: KMP_AFFINITY: topology layer "LL cache" is equivalent to "socket".
OMP: Info #288: KMP_AFFINITY: topology layer "L3 cache" is equivalent to "socket".
OMP: Info #288: KMP_AFFINITY: topology layer "L2 cache" is equivalent to "core".
OMP: Info #288: KMP_AFFINITY: topology layer "L1 cache" is equivalent to "core".
OMP: Info #192: KMP_AFFINITY: 1 socket x 16 cores/socket x 2 threads/core (16 total cores)
OMP: Info #219: KMP_AFFINITY: OS proc to physical thread map:
OMP: Info #172: KMP_AFFINITY: OS proc 0 maps to socket 0 core 0 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 1 maps to socket 0 core 0 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 2 maps to socket 0 core 1 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 3 maps to socket 0 core 1 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 4 maps to socket 0 core 2 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 5 maps to socket 0 core 2 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 6 maps to socket 0 core 3 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 7 maps to socket 0 core 3 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 8 maps to socket 0 core 5 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 9 maps to socket 0 core 5 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 10 maps to socket 0 core 6 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 11 maps to socket 0 core 6 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 12 maps to socket 0 core 9 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 13 maps to socket 0 core 9 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 14 maps to socket 0 core 10 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 15 maps to socket 0 core 10 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 16 maps to socket 0 core 12 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 17 maps to socket 0 core 12 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 18 maps to socket 0 core 13 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 19 maps to socket 0 core 13 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 20 maps to socket 0 core 16 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 21 maps to socket 0 core 16 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 22 maps to socket 0 core 17 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 23 maps to socket 0 core 17 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 24 maps to socket 0 core 18 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 25 maps to socket 0 core 18 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 26 maps to socket 0 core 19 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 27 maps to socket 0 core 19 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 28 maps to socket 0 core 20 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 29 maps to socket 0 core 20 thread 1
OMP: Info #172: KMP_AFFINITY: OS proc 30 maps to socket 0 core 21 thread 0
OMP: Info #172: KMP_AFFINITY: OS proc 31 maps to socket 0 core 21 thread 1
OMP: Info #255: KMP_AFFINITY: pid 18912 tid 20144 thread 0 bound to OS proc set 0-31

 

Socket 1,2 is presented at next picture and there is total 40 core.캡처.PNG12처.PNG

0 Kudos
jimdempseyatthecove
Honored Contributor III
813 Views

If you are building your test application using the oneAPI Fortran compiler (ifx) then try setting up a build and run environment for Intel Fortran Compiler Classic (ifort). Potentially there may be an OpenMP implementation difference between versions. It is important that the runtime environment is selected for the same version of Fortran build. (IOW correct OpenMP .DLL is used)

Try:

     KMP_HW_SUBSET=

     OMP_NUM_THREADS=

     KMP_AFFINITY=verbose

     (and unset any/all other KMP/OMP/MPI environment variables)

 

This should run with default setting (all HW threads). See what your program reports and what verbose report shows.

Jim Dempsey

 

 

 

0 Kudos
sdj77
New Contributor I
795 Views

Thanks Jim.

So far, I've been debugging at 32 bits. Debugging to 64 bits solved the problem, and it was a simple problem. I learned a lot from your help.

 

Thank you.

0 Kudos
jimdempseyatthecove
Honored Contributor III
785 Views

>>32-bits

verbose should have shown you the multiple processor groups...

however, on Windows, 32-bit app, the maximum size of a group is 32 logical processors.

Does verbose show multiple processor groups when compiled as x64?

(try both ifort and ifx)

This would be good information for the forum readers.

 

Jim Dempsey

0 Kudos
sdj77
New Contributor I
763 Views

I compared both ifort and ifx. From x64 to 20 cores of 2 sockets, a total of 40 cores appeared at verbose, and ifort was faster than ifx. Ifx had no multi-threading option in the attribute. 

In x32, a total of 16 cores, one socket, appeared in verbose, and because there was no ifx compilation option, it could only run as ifort.

0 Kudos
Reply