USM Allocations with more than 4G

[marcelld]

Hi,

I want to learn about how Intel USM works, because it looks similar to Nvidia UVM. I tried to run some code on my Intel ARC A770 in Ubuntu 26.04, but it seems like I am doing something wrong or I misunderstood the concept. Whenever I try to allocate more than 4G using USM it fails with CL_OUT_OF_RESOURCES. Allocating more than 4G without using USM succeeds. I am using the flasg "-ze-intel-greater-than-4GB-buffer-required" as option to build the kernel. Furthermore, I set the 23rd bit in the clBuffer options when using no USM. Allocating memory via clSharedMemAllocINTEL does not have an option to set the 23rd bit.

I have also tried the USM example code 300_smemhelloworld from https://github.com/bashbaug/SimpleOpenCLSamples/ and this also fails when increasing the buffer size over 4G.

Isn't it supposed to work also for larger buffers? As I understood the idea, the GPU can pagefault using shared memory and the driver will load pages into GPU memory. This looks like a solution for memory oversubscription to me. But when only buffers smaller than 4G are supported, this does not make any sense.

Best regards
Marcel

[marcelld]

I posted the wrong error code. It's CL_INVALID_BUFFER_SIZE instead of CL_OUT_OF_RESOURCES.

[Ben_A_Intel · Accepted solution]

Oh, even easier, looks like I have a sample in a branch that shows how to do this:

https://github.com/bashbaug/SimpleOpenCLSamples/compare/main...relaxed-allocations

For the USM path, the sample uses device USM rather than shared USM, but it should be straightforward to switch it to shared USM if you'd like.

One word of caution: shared USM is intended to support automatic migration between the host and device, but it doesn't necessarily imply page faults and demand paging while the kernel is executing, therefore it's not necessarily a solution for memory oversubscription.

[marcelld]

Thank you so much!
I was not aware of CL_MEM_FLAGS in cl_mem_properties_intel. It seems like this is not documented here: https://registry.khronos.org/OpenCL/extensions/intel/cl_intel_unified_shared_memory.html#cl_mem_properties_intel

When I set the 23rd bit, the shared memory allocation succeeds.

USM does indeed not help with memory oversubscription. Does intel provide a solution for this? Or is using host memory the only way?

[Ben_A_Intel]

Hi, you may want to have a look at the RelaxAllocationLimits control in the OpenCL intercept layer to see how to do allocations greater than 4GB on GPUs like your A770:

https://github.com/intel/opencl-intercept-layer/blob/main/docs/controls.md#relaxallocationlimits-bool

Specifically, you'll need to do two things:

1. When you make your allocation, you'll want to add a CL_MEM_FLAGS property with bit 23 set.  This will allow the >4GB allocation to succeed:

https://github.com/intel/opencl-intercept-layer/blob/12690fa58601cf228ac3e476274460b389e3ca7e/intercept/src/intercept.cpp#L9337

2. When you compile a program that uses these >4GB allocations, you'll want to include the build option "-cl-intel-greater-than-4GB-buffer-required".  This will instruct the compiler to generate code that supports larger allocations, otherwise you may observe issues accessing the upper parts of your allocation:

https://github.com/intel/opencl-intercept-layer/blob/12690fa58601cf228ac3e476274460b389e3ca7e/intercept/src/intercept.h#L3088

With both of these change you should be able to allocate and use USM allocations greater than 4GB.