Intel MKL Cluster FFT Topologies

[JR]

I have some code which distributes a 3-dimensional array according to a certain Cartesian topology. I wish to Fourier transform this data. However, according to the MKL Cluster FFT documentation, there is only one supported topology which decomposes only the first dimension among all the available processes.

Now that means I have to either collect all the data into a global array inside one process and then redistribute the data according to the supported topology, or try to invent some way of parallelly Fourier transforming a domain which has been decomposed in a Cartesian topology, if that is possible at all.

Please advice. What is the cleanest or most efficient way of doing this? Thank you.

[ShanmukhS_Intel]

Hi,

Thank you for posting on Intel Communities.

Regarding distribution of data between process, as mentioned above, Order of FFT dimensions is the same as the order of array dimensions in the programming language.

For example, a 3-dimensional FFT with Lengths=(m,n,l) can be computed over an array Ar[m][n][l].

Regarding further the computation of a three-dimensional transform, please refer to below link,

https://www.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-c/top/fourier-transform-functions/cluster-fft-functions/distributing-data-among-processes.html

Best Regards,

Shanmukh.SS

[JR]

Dear Shanmukh.SS,

Thank you for your response. But I am afraid you have misunderstood my question. I am well aware of the documentation you linked. In the said documentation, you will notice that only one MPI topology is discussed, namely where the first dimension is distributed among all the processes. My problem is that I have a huge amount of data already distributed according to a different topology, namely a Cartesian one. This means one of two things: either it is the case that the cluster FFT functions support arbitrary MPI topologies (which includes Cartesian topology) and I am unaware of how to make use of this facility, or that I have to change the topology of my domain decomposition and redistribute the data before computing the Fourier transforms.

I wish to know which of the two cases it is, and if it is the latter (which is a tedious task), I would appreciate any advice on how to do it most efficiently. I hope this clears things up. Thank you.

Best regards,
J.R.

[ShanmukhS_Intel]

Hi JR,

Thank you for elaborating the details and apologies for difference in understanding.

Could you please share us a sample reproducer, so that it would help us in debugging the issue in a better way.

Best Regards,

Shanmukh.SS

[JR · Accepted solution]

I am sorry but this isn't a bug resulting in some unexpected behaviour that needs debugging. The question is simple. Does the current implementation of Intel MKL FFT functions support topologies other than the single one shown in "Distributing Data among Processes"? If I had to guess, the answer is probably not (otherwise, you would written about it in the documentation or the answers here). That means I have to redistribute the data, which I have done for my use-case. Nothing more can be done to help it, I suppose. Thank you for trying to help.

[ShanmukhS_Intel]

Hi,

Glad to know that your issue is resolved. Thanks for sharing the solution with us. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Best Regards,

Shanmukh.SS