topic Re: IPP DFT plans - data only? in Intel® Integrated Performance Primitives

IPP DFT plans - data only?

pdotb — Fri, 19 Jan 2007 17:25:09 GMT

We want to generate some DFT plans and then use them

a) across many processes on a single node

b) across multiple parallel machines

I understand the need to keep the plans 'opaque' to allow the need to add additional information in future releases, but are plans data only, or do they contain say memory pointers that make them valid in only a single virtual memoy space.

If I want to pass them around (or hold them in a central database, I also need to know the size of the plan and (ideally) want to be able to use a buffer of my choosing that I can put in shared memory.

Assuming plans don't use memory pointers (say to other plans), is it possible to

a) get an estimated size for a plan

b) pass in my own allocated buffer for IPP to use to store the plan. I can only find an API that assumes the IPP allocates and passes back a pointer so I have no knowledge of buffer size and no control over where in memory it is.

c) If I knew size and the structure was data only, I could then copy to my own memory area mapped into shared memory. Perhaps this is possible even if I can't specify the initial plan buffer.

Plans take between 10 and 100x the runtime of the actual DFT in the area I'm looking at. That makes on the fly plan creation impossible. My main goal is to share one set of plans across processes on one node and for this to work when processes are actually in different VM address spaces. (the stretch to multi-node is less of an issue)

Re: IPP DFT plans - data only?

Vladimir_Dudnik — Sat, 20 Jan 2007 07:46:54 GMT

I'm very sorry, but I did not understand what you are talking about. What are the plans? What is DFT? Do you mean discrete cosine transform? What area it comes from?

Vladimir

Re: IPP DFT plans - data only?

pdotb — Sun, 21 Jan 2007 02:16:16 GMT

Many thanks for taking the trouble to look at this. Sorry my last message wasn't totally clear.

The DFT refers to discrete fourier transform. Specifically I'm interested in ippsDFTFwd_CtoC(...) and friends. For each transform length, theyrequire a separate 'one-off' preparatory call to ippsDFTInitAlloc() to prepare a 'plan of action' from thedefinition of the transform being requested. It would pre-calculate coefficients, decide how to break the problem down into efficient sub-kernels if possible etc. etc. and store these as a DFTSpec.

I used the term 'plan' but now realise thats perhaps slightly esoteric if you aren't familiar with some of FFTW's (www.fftw.org) terminology regarding terms like 'plan' and 'wisdom'. It is merely that set of information pre-calculated in InitAlloc. Ipp refers to it as a DFTSpec. Sorry for the confusion.

InitAlloc currently allocates an area behind the scenes and passes back a pointer to that area with no details of the length and content of that init buffer ('opaque').

IppStatus ippsDFTInitAlloc_C_32f(IppsDFTSpec_C_32f**

pDFTSpec, int length, int flag, IppHintAlgorithm hint);

The preparation time actually takes much longer than the DFT call itself so when you care about throughput with lots of different possible input lengths, you try to prepare allDFTSpecs (for different vector lengths) ahead of time (using a combination of zero-padding and a prepared set of pre-planned DFT lengths).

Ideally I want to prepareDFTSpecs once and share them across many processes. Because different processes *may* have different virtual memory address ranges, I would need some guarantees about the contents of a DFTSpec in order to share them:

a)Are DFTspecs actually just data or contain memory pointers

b)If a) is OK - data only... Is there a way for me to either

1) force IPP's original allocation into preallocated shared memory (an IPP general configuration setting to tell IPP where to go for scratchpad memory?)

2) or I could copy the returned DFTSpec into my shared memory (which requires length and alignment constraints to be known)

Many thanks for any advice.

Re: IPP DFT plans - data only?

Vladimir_Dudnik — Mon, 29 Jan 2007 07:00:04 GMT

Hi,

according our expert we have no such possibility for DFT function in ipps library. For FFT functions there is GetBufSize-user alloc-Init functions, but for DFT we have only combined function InitAlloc. You may submit this as a feature request to be considered for the future versions of IPP.

Regards,
Vladimir