FFT consumption

[audrius]

Why DftiCommitDescriptor allocates memory for FFT even DFTI_PLACEMENT = DFTI_INPLACE? Its annoying, because when working with data arrays of 1GB or more, there might NOT be another 1GB RAM available for FFT. Is there a possible way to force it towork withmy allocated arrayessentiallyin place?

Thanks,

Audrius Zaukevicius

[Dmitry_B_Intel]

Quoting - audrius

Why DftiCommitDescriptor allocates memory for FFT even DFTI_PLACEMENT = DFTI_INPLACE? Its annoying, because when working with data arrays of 1GB or more, there might NOT be another 1GB RAM available for FFT. Is there a possible way to force it towork withmy allocated arrayessentiallyin place?

Thanks,

Audrius Zaukevicius

Audrius,

DftiCommitDescriptor allocatesand computes trigonometric tables, and thismay take considerable amount of storage, even for in-place transforms. Storage for performing stages of FFT, such as transposition, may also be required, depending on parameters of the transform, such as strides, butit is mainly allocated byDftiCompute functions.The compute fuctionsmay fail if the memory is used up by that time, even if DftiCommitDescriptor succeeded. Anyway, consider submitting your issue at http://premier.intel.com.A small example to reproduce the problem will be helpful.

Thanks,
Dima

[audrius]

Dear Dima,
I did some more research on this issue, and I found that actually "DftiCommitDescriptor" allocates memory of about the size of arrays that length is twice FFT dimension length. So if I would like to compute 3D FFT with dimension lengths: m = 1024, n = 1024, k = 64, I will need 2*SizeOf(double)* (m + n + k) = 33 Kb of Extra memory. That's OK. BUT if I would like to compute 1D FFT with dimension length equal to 1024*1024*64 (note, that the size of my x_in array is the same as in 3D case = 1GB), I will need 2*SizeOf(double)*1024*1024*64 = 1GB of Extra memory. And that's the problem! Uncomment "//lengths[2] = m*n*k;" and you will see how much memory it allocates. So any comments on this? Is there a possible way to reduce such consumption of "DftiCommitDescriptor"? Bellow is my trivial codeof C++.NET console app (I couldn't login at https://premier.intel.com):

// 88.cpp : main project file. VS2008

#include "stdafx.h"
#include
#include "mkl_dfti.h"
#include

using namespace System;

int main(array<:STRING> ^args)
{
Console::WriteLine(L"Hello World");

DFTI_DESCRIPTOR_HANDLE Desc_Handle = 0;
array ^lengths;
int Status, i1, m, n, k;

m = 1024;
n = 1024;
k = 64;

i1 = 3;

lengths = gcnew array(i1);

lengths[0] = m;
lengths[1] = n;
lengths[2] = k;
//lengths[2] = m*n*k;

Console::ReadKey();
Status = DftiCreateDescriptor(&Desc_Handle, DFTI_DOUBLE, DFTI_COMPLEX, i1, lengths);
printf("DftiCreateDescriptor error_message = %s n", DftiErrorMessage(Status));
Console::ReadKey();

Status = DftiCommitDescriptor( Desc_Handle );
printf("DftiCommitDescriptor error_message = %s n", DftiErrorMessage(Status));
Console::ReadKey();
DftiFreeDescriptor(&Desc_Handle);
printf("DftiFreeDescriptor error_message = %s n", DftiErrorMessage(Status));
Console::ReadKey();
return 0;

}

Additionaly:

1. I included these libs Linker->Input->Additional dependencies: mkl_intel_c.lib, mkl_core.lib, libiomp5md.lib, mkl_intel_thread.lib.
2. I copied libiomp5md.dll to my project dir, as it somehow couldn't find it at ...MKL10.0.5.025ia32bin.

Thanks for help,
Audrius

[Dmitry_B_Intel]

Dear Audrius,

Thank you very much for the example and the code. I understand the problem.The memory is consumedfor the trigonometric tables and it takes almost the same size as the data itself. Unfortunately,the problemcannot be worked around currently.

Regards,
Dima

[audrius]

Dear Dima,

thank you for clearing things up for me.

Best regards,

Audrius Zaukevicius