Any way to get past 2GB allocation limit?

[gbraytx]

We are needing to do some very large FFT calls that require overall memory allocation greater than 2GB for the data buffer containing the complex double values.

All of the IPP allocation routines take "int"s for the sizes (we are using the em64t version so 64 bit pointers should be the default I would think). This caps the allocation for a single buffer at 2GB. We are using a 64 bit Linux OS with lots of memory, so allocation beyond 2GB would not be a problem.

Any plans to change this in the near future or is there a way to get around it?

Thanks!

[greg_heberle]

gbraytx - We even tried the following to set a FFTspec with our own memory to get around it which worked for allocation but when performing the FFT it gave bad results.

int pOrder = 27;
IppsFFTSpec_C_64fc * pSpec;
int pFlag = IPP_FFT_DIV_INV_BY_N;
IppHintAlgorithm pHint = ippAlgHintNone;

IppStatus status;
Ipp8u * pSpecBuf;
Ipp8u * pSpecWorkBuf;
Ipp8u * pSpecFFTWorkBuf;

int specSize;
int specBufSize;
int specWorkBufSize;

status = ippsFFTGetSize_C_64fc(order, pFlag, pHint, &specSize, &specBufSize, &specWorkBufSize);

// SUPERHACK to get around IPPs negative returned size
// this would only work for order 27. Anything larger and
// it is an invalid assumption
unsigned int specSizeU = (unsigned int) specSize;

// Another hack to alloc larger memory beyond 2GB
pSpecBuf = (Ipp8u*) ippsMalloc_16u(specSizeU / 2);
pSpecWorkBuf = ippsMalloc_8u(specBufSize);

status = ippsFFTInit_C_64fc(pSpec, pOrder, pFlag, pHint, pSpecBuf, pSpecWorkBuf);

// pSpecBuf and pSpec point to the same memory which is fine.

return status;
// The following fails for order=27+
// return (ippsFFTInitAlloc_C_64fc(pSpec, pOrder, pFlag, pHint));

[Vladimir_Dudnik]

Hello,

In the current IPP version there is limitation on amount of memory which can be allocated with IPP functions (you probably notice the type of size parameter in ippMalloc function).

If you consider this as an importantfunctionality please submit your feature request to Intel Premier Support.

Regards,
Vladimir