sycl failed malloc_device on GPU takes 20 seconds

JakubH — Sun, 10 Nov 2024 12:55:11 GMT

Hello,

In SYCL/DPCPP, if the function `sycl::malloc_device` fails (because there is not enough memory available), it takes 20 seconds to return.

I created an example code (see below). There, I make two allocations. If the second allocation exceeds the total memory capacity, it takes 20 seconds to return (see sample outputs below) and returns a nullptr.

I am OK with it returning nullptr, that is reasonable, when there is not enough memory. **But why does it take so long?**

Almost exaclty 20 seconds. The amount of free/occupied/total memory capacity does not make any difference, using PVC1100 or PVC1550 makes no difference, using different intel toolkit versions (2024.0.2 -- 2025.0.0) makes no difference. Even the presence of `aspect::ext_intel_free_memory` makes no difference.

I was testing this with a PVC1550 on a Tiber devcloud instance, and with a PVC1100 on the training nodes in the Tiber devcloud.

What is going on? I would appreciate any help with this.

Jakub

--------

compile with: `icpx -fsycl -qopenmp source.cpp -o program.x`

run: `./program.x <GiB_first_alloc> <GiB_second_alloc>`

run: `./program.x 60 4`

example source code:

```

#include <cstdio>

#include <cstdlib>

#include <omp.h>

#include <sycl/sycl.hpp>

int main(int argc, char ** argv)

{

if(argc <= 2)

{

fprintf(stderr, "not enough arguments\n");

return 1;

}

size_t size1GiB = atoi(argv[1]);

size_t size2GiB = atoi(argv[2]);

size_t size1 = size1GiB << 30;

size_t size2 = size2GiB << 30;

void * ptr1 = nullptr;

void * ptr2 = nullptr;

sycl::device dev(sycl::gpu_selector_v);

printf("Device:\n");

printf(" Name: %s\n", dev.get_info<sycl::info::device::name>().c_str());

printf(" Platform: %s\n", dev.get_info<sycl::info::device::platform>().get_info<sycl::info::platform::name>().c_str());

printf(" Global memory: %lu MiB\n", dev.get_info<sycl::info::device::global_mem_size>() >> 20);

printf(" Free memory aspect: %s\n", dev.has(sycl::aspect::ext_intel_free_memory) ? "YES" : "NO");

sycl::context ctx(dev);

printf("\n");

printf("Allocating %zu MiB\n", size1 >> 20);

double start1 = omp_get_wtime();

ptr1 = sycl::malloc_device(size1, dev, ctx);

double stop1 = omp_get_wtime();

printf(" ptr1: %p\n", ptr1);

printf(" time: %10.3f ms\n", (stop1 - start1) * 1000.0);

printf("\n");

printf("Allocating %zu MiB\n", size2 >> 20);

double start2 = omp_get_wtime();

ptr2 = sycl::malloc_device(size2, dev, ctx);

double stop2 = omp_get_wtime();

printf(" ptr2: %p\n", ptr2);

printf(" time: %10.3f ms\n", (stop2 - start2) * 1000.0);

printf("\n");

sycl::free(ptr1, ctx);

sycl::free(ptr2, ctx);

printf("The end\n");

return 0;

}

```

output from PVC1550 Tiber devcloud instance with intel toolkit 2025.0.0, using FLAT mode for the 2-stack gpu:

```

$ ./program.x 60 5
Device:
Name: Intel(R) Data Center GPU Max 1550
Platform: Intel(R) oneAPI Unified Runtime over Level-Zero
Global memory: 65536 MiB
Free memory aspect: NO

Allocating 61440 MiB
ptr1: 0xff00000000200000
time: 21.875 ms

Allocating 5120 MiB
ptr2: (nil)
time: 20050.431 ms

The end

```

(using COMPOSITE mode and running with `./program.x 120 10`, the second allocation still takes 20 seconds)

output from PVC1100 on the Tiber training nodes through jupyterlab, intel toolkit 2024.2.1:

```

$ ./program.x 45 5
Device:
Name: Intel(R) Data Center GPU Max 1100
Platform: Intel(R) Level-Zero
Global memory: 49152 MiB
Free memory aspect: YES

Allocating 46080 MiB
ptr1: 0xff00e00000200000
time: 0.219 ms

Allocating 5120 MiB
ptr2: (nil)
time: 20042.670 ms

The end

```

edit:

Re: sycl failed malloc_device on GPU takes 20 seconds

JakubH — Sat, 13 Sep 2025 09:02:02 GMT

I report that with version 2025.2.1 on Aurora, the behavior is still the same.

This is no longer an issue for me, but it would be nice if it was resolved.

Re: sycl failed malloc_device on GPU takes 20 seconds

JakubH — Sat, 13 Sep 2025 10:32:02 GMT

correction: so I do still need this resolved, this is still an issue for me.

I need some way of allocating "the rest of the memory on the device".

Querrying free memory does not work, so I attempt to allocate the full memory capacity, and if it fails, lower it by some amount and attempt again in a loop, until it succeeds. Because a failed malloc_device takes so long, this approach is unusable.

With this and the linked issue, I have no way of allocating the rest of the free memory, apart from some manual bookkeeping and educated guesses about oneAPI internal allocations.

topic Re: sycl failed malloc_device on GPU takes 20 seconds in GPU Compute Software

sycl failed malloc_device on GPU takes 20 seconds

Re: sycl failed malloc_device on GPU takes 20 seconds

Re: sycl failed malloc_device on GPU takes 20 seconds