Intel® oneAPI Base Toolkit
Support for core tools and libraries to build and deploy high-performance data-centric applications
318 Discussions

SYCL does not report NUMA nodes (on Intel OpenCL)

Michoumichmich
Beginner
642 Views

Hello, 

 

If I create a sycl::device in an application, on the CPU, after compiling it with `spir64` or `spir64_x86_64` and try to split it into sub-devices that corresponds to the NUMA nodes with:

d.create_sub_devices<sycl::info::partition_property::partition_by_affinity_domain>(sycl::info::partition_affinity_domain::numa);

 I get only a single device. However I'm running that on a server with two Intel Xeon and the NUMA nodes are recognised properly by other applications and "lscpu". I remember using that around June and it worked on the open source oneAPI. Now it does not work anywhere. Did you deprecate some CPUs from your OpenCL runtime ? 

 

Many thanks in advance, 

Michel

 

SYCL-LS:  [opencl:0] CPU : Intel(R) OpenCL 3.0 [2021.13.11.0.23_160000]

 

0 Kudos
5 Replies
SantoshY_Intel
Moderator
596 Views

Hi,

 

Thank you for posting in Intel Communities.

 

Could you please provide the below details to investigate your issue from our end?

  1. Name of the Operating system and its version.
  2. Details of the two Intel Xeon processors you are using on the server.
  3. A sample reproducer code and the steps to reproduce the issue.
  4. Provide us the screenshot of "lscpu" command.
    lscpu​

>>"I remember using that around June and it worked on the open source oneAPI."

Could you please provide us with the screenshots or logs of expected output if available?

 

>>"Now it does not work anywhere"

Could you please confirm whether you are using the latest Intel oneAPI 2022.1?

 

Thanks & Regards,

Santosh

Michoumichmich
Beginner
580 Views

Hi, 

Thanks for your answer. Here's a reproducer:

#include <CL/sycl.hpp>
#include <iostream>

int main() {
auto devices = sycl::device::get_devices();
for (auto &dev : devices) {
if (dev.is_cpu()) {
try {
auto numa_nodes = dev.create_sub_devices<
sycl::info::partition_property::partition_by_affinity_domain>(
sycl::info::partition_affinity_domain::numa);
for (auto &numa_node : numa_nodes) {
std::cout << "Found 'NUMA sub-device': "
<< dev.get_info<sycl::info::device::name>() << std::endl;
}
} catch (...) {
std::cout << dev.get_info<sycl::info::device::name>() << std::endl;
}
}
}
}

I am running on RHEL8 and two Intel XeonE5- x2670-v2 and the latest version of the Intel oneAPI toolkit. I don't have access to that system right now. I have tried on several other systems however, and it doesn't work either. 

Here's an output of lscpu:

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 48 bits virtual
CPU(s):                          288
On-line CPU(s) list:             0-287
Thread(s) per core:              4
Core(s) per socket:              72
Socket(s):                       1
NUMA node(s):                    4
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           133
Model name:                      Intel(R) Genuine Intel(R) CPU 0000 @ 1.40GHz
Stepping:                        0
CPU MHz:                         1492.048
CPU max MHz:                     1500.0000
CPU min MHz:                     1000.0000
BogoMIPS:                        2793.49
Virtualization:                  VT-x
L1d cache:                       2.3 MiB
L1i cache:                       2.3 MiB
L2 cache:                        36 MiB
NUMA node0 CPU(s):               0-17,72-89,144-161,216-233
NUMA node1 CPU(s):               18-35,90-107,162-179,234-251
NUMA node2 CPU(s):               36-53,108-125,180-197,252-269
NUMA node3 CPU(s):               54-71,126-143,198-215,270-287
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Vulnerable: Clear CPU buffers attempted, no microcode; SMT mitigated
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Spec store bypass: Not affected
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full generic retpoline, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs its
                                 rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes
                                 save avx f16c rdrand lahf_lm abm 3dnowprefetch ring3mwait cpuid_fault epb pti ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms avx512f                               
rdseed adx avx512pf avx512er avx512cd xsaveopt dtherm ida arat pln pts avx512_vpopcntdq avx512_4vnniw avx512_4fmaps

 

 Best regards, 

 Michel

SantoshY_Intel
Moderator
553 Views

Hi,


Thanks for providing the sample reproducer code.


We are able to reproduce your issue from our end. We redirected your issue to the concerned development team. They are looking into your issue and will get back to you soon with an update.



Thanks & Regards,

Santosh


SantoshY_Intel
Moderator
329 Views

Hi,


Thank you for your patience. The issue raised by you will be fixed in the next release. We are closing this thread. This thread will no longer be monitored by Intel. If you need further assistance, please post a new question.


Thanks & Regards,

Santosh


Michoumichmich
Beginner
324 Views

Thanks for the good news!

Best regards, 

Michel 

Reply