Hi there,
I have a pretty basic/conceptual question with respect to the usage of oneMKL in combination with DPC++ and Visual Studio 2017.
I'm running the example code (see below) on the 16 cores of the Intel Core i9-9880H CPU @ 2.30 GHz of my Dell Precision 7540 and hoped that oneMKL would use as many cores are possible to calculate 10 complex-valued FFTs each (12.5 Mio points). At this moment it looks like I'm only using a single core given the fact that it takes 10 times longer to calculate these 10 FFTs than a single FFT.
Here's the output of the program:
Here are my settings in Visual Studio 2017 (enabling the use of oneTBB doesn't help)
Here's the output when running the VTune Profiler
Apparently I'm overlooking something essential to make sure the program takes full advantage of the 16 cores in my CPU. Please let me know what and where I can find more conceptual information about fine-tuning things with respect to optimal parallel usage of the CPU/GPU device (mainly from oneMKL, possibly in combination with oneTBB) using Visual Studio 2017.
Thanks and regards,
Frans
------------------------------------
#include <mkl.h>
#include <CL/sycl.hpp>
#include <iostream>
#include <string>
#include <oneapi/mkl/dfti.hpp>
#include <oneapi/mkl/rng.hpp>
#include <complex>
#include <chrono>
int main(int argc, char** argv)
{
try
{
// Probably not 100% idiot-proof ... using 25 Mio points on CPU by default and 4 parallel complex-valued FFTs
unsigned int nrOfPoints = (argc < 2) ? 25000000U : std::stoi(argv[1]);
std::string selector = (argc < 3) ? "cpu" : argv[2];
unsigned int nrOfParallelFFTs = (argc < 4) ? 4U : std::stoi(argv[3]);
sycl::queue Q;
if (selector == "cpu")
Q = sycl::queue(sycl::cpu_selector{});
else if (selector == "gpu")
Q = sycl::queue(sycl::gpu_selector{});
else if (selector == "host")
Q = sycl::queue(sycl::host_selector{});
else
{
std::cout << "Please use: " << argv[0] << " <nrOfPoints:25Mio> <selector cpu|gpu|host> <nrOfParallelFFTs:4>" << std::endl;
return EXIT_FAILURE;
}
auto sycl_device = Q.get_device();
auto sycl_context = Q.get_context();
// Get more specific info about the device
std::cout << "Running on: " << sycl_device.get_info<sycl::info::device::name>() << std::endl;
std::cout << " Available number of parallel compute units (a.k.a. cores): " << sycl_device.get_info<sycl::info::device::max_compute_units>() << std::endl;
// Use fixed seed in combination with random data
std::uint32_t seed = 0;
oneapi::mkl::rng::mcg31m1 pseudoRndGen(Q, seed); // Initialize the pseudo-random generator.
// Uniform distribution only supports floats and doubles (e.g. not std::complex<float>)
// Use reinterpret_cast<float*> to fill an array of complex<float> values
oneapi::mkl::rng::uniform<float, oneapi::mkl::rng::uniform_method::standard> uniformDistribution(-1, 1);
// Complex-valued IQ data.
auto iqData = sycl::malloc_shared<std::complex<float>>(nrOfPoints, sycl_device, sycl_context);
// Interpret as float values such that we can use the random generator
oneapi::mkl::rng::generate(uniformDistribution, pseudoRndGen, 2 * nrOfPoints, reinterpret_cast<float*>(iqData)).wait();
// Keeping track of 1st value to compensate for current scaling impact
// when combining forward and backward FFT.
auto iqData1 = iqData[0];
std::cout << "iqData before in-place FFT: " << iqData[0] << " .. " << iqData[1] << " .. " << std::endl;
oneapi::mkl::dft::descriptor<oneapi::mkl::dft::precision::SINGLE, oneapi::mkl::dft::domain::COMPLEX> fftDescriptorIQ(nrOfPoints);
// Don't forget to commit the FFT descriptor to the queue.
fftDescriptorIQ.commit(Q);
auto startTime = std::chrono::system_clock::now();
oneapi::mkl::dft::compute_forward< oneapi::mkl::dft::descriptor<oneapi::mkl::dft::precision::SINGLE, oneapi::mkl::dft::domain::COMPLEX>, std::complex<float>>(fftDescriptorIQ, iqData).wait();
auto stopTime = std::chrono::system_clock::now();
std::cout << "iqData after in-place forward FFT: " << iqData[0] << " .. " << iqData[1] << " .. " << std::endl;
std::cout << "Elapsed time (ms) for " << nrOfPoints << " points (complex, in-place): " << std::chrono::duration_cast<std::chrono::milliseconds>(stopTime - startTime).count() << std::endl;
startTime = std::chrono::system_clock::now();
oneapi::mkl::dft::compute_backward< oneapi::mkl::dft::descriptor<oneapi::mkl::dft::precision::SINGLE, oneapi::mkl::dft::domain::COMPLEX>, std::complex<float>>(fftDescriptorIQ, iqData).wait();
stopTime = std::chrono::system_clock::now();
std::cout << "iqData after in-place backward FFT: " << iqData[0] << " .. " << iqData[1] << " .. " << std::endl;
std::cout << "Elapsed time (ms) for " << nrOfPoints << " points (complex, in-place): " << std::chrono::duration_cast<std::chrono::milliseconds>(stopTime - startTime).count() << std::endl;
// PROPER SOLUTION: make sure we perform proper scaling in forward and backward FFT
auto scaleFactor = abs(iqData[0]) / abs(iqData1);
std::cout << "iqData after in-place backward FFT and scaling: " << iqData[0] / scaleFactor << " .. " << iqData[1] / scaleFactor << " .. " << std::endl;
sycl::free(iqData, sycl_context);
// +++ PARALLEL FFTS +++
// How to allocated an array of USM? Safe to use new?
std::complex<float> **iqDataMC = new std::complex<float> *[nrOfParallelFFTs];
sycl::event *eventsMC = new sycl::event[nrOfParallelFFTs];
std::cout << "Creating " << nrOfParallelFFTs << " sets of " << nrOfPoints << " random IQ data." << std::endl;
// Can we do this too in parallel?
int i;
for (i = 0; i < nrOfParallelFFTs; i++)
{
iqDataMC[i] = sycl::malloc_shared<std::complex<float>>(nrOfPoints, sycl_device, sycl_context);
oneapi::mkl::rng::generate(uniformDistribution, pseudoRndGen, 2 * nrOfPoints, reinterpret_cast<float*>(iqDataMC[i])).wait();
}
std::cout << "Done creating " << nrOfParallelFFTs << " sets of " << nrOfPoints << " random complex IQ data." << std::endl;
std::cout << "Performing " << nrOfParallelFFTs << " forward complex single-precision FFTs." << std::endl;
startTime = std::chrono::system_clock::now();
// Need for a parallel_for loop ?
// Based on timing we clearly don't have any speed improvement ... time multiplied by # FFTS ...
// Do we need to commit different FFT descriptors to different queues?
// Given the fact that a queue is linked to a device, one would expect this device to properly deal
// with parallelism ...
for (i = 0; i < nrOfParallelFFTs; i++)
eventsMC[i] = oneapi::mkl::dft::compute_forward<oneapi::mkl::dft::descriptor<oneapi::mkl::dft::precision::SINGLE, oneapi::mkl::dft::domain::COMPLEX>, std::complex<float>>(fftDescriptorIQ, iqDataMC[i]);
// Wait for all events to indicate the calculation is done ...
for (i = 0; i < nrOfParallelFFTs; i++)
eventsMC[i].wait();
stopTime = std::chrono::system_clock::now();
std::cout << "Done performing " << nrOfParallelFFTs << " forward complex single-precision FFTs." << std::endl;
std::cout << "Elapsed time (ms) for " << nrOfParallelFFTs << " forward complex single-precision FFTs: " << std::chrono::duration_cast<std::chrono::milliseconds>(stopTime - startTime).count() << std::endl;
for (i = 0; i < nrOfParallelFFTs; i++)
sycl::free(iqDataMC[i], sycl_context);
return EXIT_SUCCESS;
}
catch (sycl::exception& e)
{
std::cout << "SYCL exception: " << e.what() << std::endl;
}
}
Link Copied
Hi,
Thanks for reaching out to us.
Could you please try running your code on Intel oneAPI command prompt by following the below steps?
> set MKL_DOMAIN_NUM_THREADS="MKL_DOMAIN_ALL=1, MKL_DOMAIN_FFT=4"
> set KMP_AFFINITY=granularity=fine,compact,1,0
(Please refer to the following link for more details regarding improving performance with the threading
> dpcpp -DMKL_ILP64 -I"%MKLROOT%\include" file.cpp -c /EHsc
> dpcpp file.obj -fsycl-device-code-split=per_kernel mkl_sycl.lib mkl_intel_ilp64.lib mkl_intel_thread.lib mkl_core.lib libiomp5md.lib sycl.lib OpenCL.lib
> file.exe 12500000 cpu 10
When we tried the above steps from our end, we could see that it is taking less time when compared to yours. Please refer to the screenshot for the output.
>>At this moment it looks like I'm only using a single core given the fact that .....
We are working on this issue, we will get back to you soon.
Regards,
Vidya.
Thanks Vidya.
Your compile/link settings speed up the actual execution as you can see below.
Can you please explain why these settings don't show up when using https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl-link-line-advisor.html ?
Calculating 10 FFTs still takes roughly 10 times calculating a single FFT. Hence it looks like I'm still only using a single core.
Also, I question the relevance of
> set MKL_DOMAIN_NUM_THREADS="MKL_DOMAIN_ALL=1, MKL_DOMAIN_FFT=4"
because it is linked to OpenMP, while the documentation suggests OpenMP is only used in case of the C/Fortran API.
As such I repeated your proposal without setting these variables and I obtain the same performance.
Something else that bothers me is that the main speed improvement is due to the static linking instead of dynamic linking.
If I link dynamically using
> dpcpp HelloMKLwithDPCPP.obj -fsycl-device-code-split=per_kernel mkl_sycl_dll.lib mkl_intel_ilp64_dll.lib mkl_intel_thread_dll.lib mkl_core_dll.lib libiomp5md.lib sycl.lib OpenCL.lib
the speed is roughly back to what it was (about 8 times slower).
Also replacing the mkl_intel_thread.lib by mkl_tbb_thread.lib (static linking) results in a similar negative speed impact (factor 8).
Can you please explain the observed speed impact (dynamic versus static and mkl_tbb_thread versus mkl_intel_thread) ?
Looking forward to your feedback.
Thanks and regards,
Frans
Hey Vidya,
here's the verbose output when not setting the variables (before and after set MKL_VERBOSE=1)
Here's the verbose output when setting the variables (before and after set MKL_VERBOSE=1)
Here's what I get when I use the linker advisor tool
I also ran vtune (MKL_VERBOSE disabled and without setting the variables)
> vtune -collect threading HelloMKLwithDPCPP 12500000 cpu 10
Here's part of the report
Here's another portion of the output which I don't understand at this moment
Open questions:
- With respect to the link advisor tool: I don't see libiomp5md.lib showing up while it shows up at the top function with spin and overhead time. Why?
- Why doesn't the link advisor tool indicate that I should use mkl_intel_thread which is about 8 times faster?
- Where can I find information explaining me that I should use mkl_intel_thread instead of TBB? Speed is key to our application.
- Why is the dynamically linked executable 8 times slower than the statically linked one (both using mkl_intel_thread)?
- Based on the MKL verbose output, it looks like I'm using 8 threads. Correct?
- Where can I find information to interpret the MKL verbose output?
- Running VTune using the oneAPI Tools Command Window gives a different result and reports thread oversubscription. How can I use the MKL verbose output and the VTune report to speed up the application? What are the key take-aways from both reports?
Thanks and best wishes for 2022,
Frans
Hi Frans,
>>With respect to the link advisor tool: I don't see libiomp5md.lib showing up while it shows up at the top function with spin and overhead time. Why?
The possible reason might be is, mkl_intel_thread option is included in your linking command, hence the analysis shows up with libiomp5md.lib in the output from vtune.
If you link it against using TBB(as how the link line suggests you) you will get output similar to the below screenshot (no occurrence of libiomp5md.lib)
>>Based on the MKL verbose output, it looks like I'm using 8 threads. Correct?
Yes, you are correct. It is utilizing 8 threads.
>>Where can I find information to interpret the MKL verbose output?
Please refer to the below link which gives the information contained in a call description line for Verbose which might help you in interpreting the output of MKL_VERBOSE
We are working on your issue internally and will address your remaining questions shortly.
Regards,
Vidya.
