topic Comparing FFT Performance MKL11 with 1 thread and 4 threads in Intel® oneAPI Math Kernel Library

Comparing FFT Performance MKL11 with 1 thread and 4 threads

Marian_L_ — Mon, 18 Feb 2013 09:29:03 GMT

Hi All,

I'm evaluating the performance (this time not MKL6 vs MKL11) of MKL11 with 1 thread versus 4 threads.

The 4 thread version seems to be slower. Furthermore, the 4 thread implementation has a huge number of outliners. Does anyone have any explanations, why?

Below the source (float and double are similar), I shortened it for better overview.

Main function:

int _tmain(int argc, _TCHAR* argv[])
{
   int threads = 4; //or 1
   mkl_set_num_threads(threads);

   SetPriorityClass(GetCurrentProcess(), HIGH_PRIORITY_CLASS ); // Set a process priority to 'High'

   TEST FUNCTION HERE

   SetPriorityClass( GetCurrentProcess(), NORMAL_PRIORITY_CLASS ); // Restore the process priority to 'Norma'l
}

TEST FUNCTION

DFTI_DESCRIPTOR_HANDLE hand;
   cxdTimeLoops.alloc(loops);

   //    FLOAT
   k=0;
   for (exp=exp_start;exp<=exp_stop;exp++)
   {
      Nfft = (unsigned int) pow(2.0,exp);

      myRndNumber = 1; //seed

      for (i=0;i<Nfft;i++) //get pseudo random signal
      {
         myRndNumber    = NextRand32(myRndNumber);
         cxfTimesig = ((float) myRndNumber / UINT_MAX)*2-1;
         cxfTimeaxis = ((float) i + 1.0) / fs;
      }
      hand   = 0;
      status = DftiCreateDescriptor(&hand, DFTI_SINGLE, DFTI_REAL, 1, Nfft);
      status = DftiSetValue(hand, DFTI_PLACEMENT, DFTI_NOT_INPLACE);
      status = DftiCommitDescriptor(hand);

      for (i=0;i<loops;i++)
      {
         hpfcTimer.Start(); //start timer for single execution
         status = DftiComputeForward(hand, cxfTimesig.ptr(), cxfFreqsig.ptr());
         cxdTimeLoops = hpfcTimer.Time();
      }
      DftiFreeDescriptor(&hand);

      dTimeMax = 0;
      dTimeMin = cxdTimeLoops[0];
      dTimeAvg = 0;
      for (i=0;i<loops;i++)
      {
         dTimeAvg += cxdTimeLoops;
         dTimeMax = max(cxdTimeLoops,dTimeMax);
         dTimeMin = min(cxdTimeLoops,dTimeMin);
      }
      dTimeAvg /= (double) loops;
      k++;
   }

   //    DOUBLE
   k=0;
   for (exp=exp_start;exp<=exp_stop;exp++)
   {
      Nfft = (unsigned int) pow(2.0,exp);
      cxdFreqsig.alloc(Nfft);
      cxdTimesig.alloc(Nfft);
      cxdTimeaxis.alloc(Nfft);

      myRndNumber = 1; //seed
      for (i=0;i<Nfft;i++)      //get pseudo random signal
      {
         myRndNumber    = NextRand32(myRndNumber);
         cxdTimesig = ((double) myRndNumber / UINT_MAX)*2-1;
         cxdTimeaxis = ((double) i + 1.0) / fs;
      }

      hand   = 0;
      status = DftiCreateDescriptor(&hand, DFTI_DOUBLE, DFTI_REAL, 1, Nfft);
      status = DftiSetValue(hand, DFTI_PLACEMENT, DFTI_NOT_INPLACE);
      status = DftiCommitDescriptor(hand);

      for (i=0;i<loops;i++)
      {
         hpfcTimer.Start(); //start timer for single execution
         status = DftiComputeForward(hand, cxdTimesig.ptr(), cxdFreqsig.ptr());
         cxdTimeLoops = hpfcTimer.Time();
      }
      DftiFreeDescriptor(&hand);

      dTimeMax = 0;
      dTimeMin = cxdTimeLoops[0];
      dTimeAvg = 0;
      for (i=0;i<loops;i++)
      {
         dTimeAvg += cxdTimeLoops;
         dTimeMax = max(cxdTimeLoops,dTimeMax);
         dTimeMin = min(cxdTimeLoops,dTimeMin);
      }
      dTimeAvg /= (double) loops;
      k++;
   }
}

dTimeAvg is plottet versus Nfft for float and double. I'm attaching the individual plots with min/max for visualizing the outliners.

Thanks, Marian

Dear Customer,

Sridevi_A_Intel — Tue, 19 Feb 2013 23:56:00 GMT

Marian,

can you please give me your machine/processor specifications? Do you observe this in every machine?

- Sridevi

>>...

SergeyKostrov — Wed, 20 Feb 2013 04:48:00 GMT

>>... >>int _tmain(int argc, _TCHAR* argv[]) >>{ >> int threads = 4; //or 1 >> mkl_set_num_threads( threads ); >> >> SetPriorityClass( GetCurrentProcess(), HIGH_PRIORITY_CLASS ); >>... >>The 4 thread version seems to be slower. You've changed a priority of the main thread and I don't think it will raise priorities of OpenMP threads. Then, when MKL creates OpenMP threads they could have Normal priorities and because of this all these four threads could be preempted more often by the thread with the higher priority, that is your main application thread. Also, I'm not sure that this is the only problem in your test case. If Nfft is too big than cache related issues could negatively affect performance.

Does MKL library have a

Bernard — Wed, 20 Feb 2013 05:28:25 GMT

Does MKL library have a function which raises OpenMP thread's priority?

>>...Does MKL library have a

SergeyKostrov — Wed, 20 Feb 2013 06:01:50 GMT

>>...Does MKL library have a function which raises OpenMP thread's priority? I don't know if MKL has it but I know that OpenMP specification 3.1 ( July 2011 ) does not have any functions to change priorities of OpenMP threads.

@Sridevi: It's an Inten Core

Marian_L_ — Wed, 20 Feb 2013 06:19:41 GMT

@Sridevi: It's an Inten Core i5-2500@3.30 GHz, 8 GB RAM. I've only checked this on this computer.

@Sergey: Would you wrap the priority raising around the for loop like this:

//Set to high priority

for (i=0;i<loops;i++)
      {
         hpfcTimer.Start(); //start timer for single execution
         status = DftiComputeForward(hand, cxfTimesig.ptr(), cxfFreqsig.ptr());
         cxdTimeLoops = hpfcTimer.Time();
      }

//Set to normal priority

Hi,

barragan_villanueva_ — Wed, 20 Feb 2013 07:27:50 GMT

Hi,

You measure FFT performance on powers of two sizes in a range [exp_start, exp_stop] for real-to-complex.

What they are? You know, small sizes have always sequential implementation.

To check that MKL implementation is threaded please set environment KMP_AFFINITY=compact,verbose and be sure you linked with MKL threaded libraries.

>>> I know that OpenMP

Bernard — Wed, 20 Feb 2013 08:32:54 GMT

>>> I know that OpenMP specification 3.1 ( July 2011 ) does not have any functions to change priorities of OpenMP threads.>>>

So the priorities of OpenMP thread is hardcoded to be Normal.

Marian,

SergeyKostrov — Wed, 20 Feb 2013 13:08:08 GMT

Marian, >>SetPriorityClass( GetCurrentProcess(), HIGH_PRIORITY_CLASS ); Did you try to comment that?

Hi,

barragan_villanueva_ — Thu, 21 Feb 2013 05:55:00 GMT

Hi,

Setting high-priority may not help and used just for getting performance stability on overloaded machine.

However, you can initialize OMP-threads in your program easily with required priority before using MKL functions:

#pragma omp parallel

{

SetThreadPriority( GetCurrentThread(),THREAD_PRIORITY_HIGHEST);

}

so that MKL will use these theads doing parallel FFTs.

BTW, how many real CPUs are on your machine. And what about HT (hyper-threading)?

@Victor:

Marian_L_ — Thu, 21 Feb 2013 13:21:50 GMT

@Victor:

What they are? You know, small sizes have always sequential implementation.

The FFT length is written in the absissa, so the exponent runs from 10 to 20.

BTW, how many real CPUs are on your machine. And what about HT (hyper-threading)?

Intel Core i5-2500@3.30 GHz, according to this it has no hyper threading

SetThreadPriority( GetCurrentThread(),THREAD_PRIORITY_HIGHEST);

Sadly, that does not change the performance significantly.

@Sergey:

>>SetPriorityClass( GetCurrentProcess(), HIGH_PRIORITY_CLASS );

Did you try to comment that?

Yes, but it sets the priority of the whole process, not threads. So the placement should be irrelevant, correct?

>>...Yes, but it sets the

SergeyKostrov — Thu, 21 Feb 2013 13:40:00 GMT

>>...Yes, but it sets the priority of the whole process, not threads. So the placement should be irrelevant, >>correct? Yes, and this is what I've seen in my tests in 2012. Marian, I'll try to investigate it again and then post my new results ( I can't tell when it will be done ). Also, I actually have that task on my list for a long time and it's a right time to look into that.

Sergey, thanks for your help.

Marian_L_ — Thu, 21 Feb 2013 14:37:44 GMT

Sergey, thanks for your help. If you need some of my testing code or gnuplot export functions, let me know.

@marian

Bernard — Sun, 24 Feb 2013 09:46:55 GMT

@marian

Before running your tests can you measure overall sytem load?I would suggest to do it with xperf tool.There is possibility that your threads are preempted by code which is running at high IRQL(like driver's routines).As it was suggested in other post by Sergey it is recommended to disable some of the unneded windows services and even disable some of the hardware loke network cards.

>>... If you need some of my

SergeyKostrov — Sun, 24 Feb 2013 20:23:37 GMT

>>... If you need some of my testing code or gnuplot export functions, let me know... Yes, it would be nice to look at it. Thanks, Marian.

@ iliyapolak: Thanks for your

Marian_L_ — Mon, 25 Feb 2013 10:49:21 GMT

@ iliyapolak: Thanks for your comment. If the performance is not good with my system, it is very likely the same on a customer's computer. I agree it could be some driver or something else. But I can't go into details of the root cause here, because I don't know what is on other systems, that I cannot control.

@Sergey, I'll send you a pm.

>>...There is possibility

SergeyKostrov — Tue, 26 Feb 2013 00:49:31 GMT

>>...There is possibility that your threads are preempted by code which is running at high IRQL (like driver's routines).... Many software developers simply do not care what IRQL, or a driver routing, etc, are in essence. If a software developer has a problem he / she could ask a question on a forum and a solution / proposal / hands-on-R&D, test, etc is / are expected ( ideally ). I see that in Marian's case a real practical investigation is really needed what is going on with MKL v11. It is clear that performance has degraded compared to older versions of MKL. Best regards, Sergey

>>>I see that in Marian's

Bernard — Tue, 26 Feb 2013 05:34:00 GMT

>>>I see that in Marian's case a real practical investigation is really needed what is going on with MKL v11. It is clear that performance has degraded compared to older versions of MKL.>>>

As far as it concerns performance of some program/software at the beginning of the investigation you cannot exclude anything.I completely agree with you that any software developer should not care about the IRQL's and driver's routines,but in the case of software performance problems everything must be taken into account even system load.

@Marian

Bernard — Tue, 26 Feb 2013 05:46:14 GMT

@Marian

I was not talking about the other system.Testing your code on another machine is important to understand the root cause.Regarding the problem I believe that sometimes you software performance can degrade because of interference from the OS (it's services) and the other code maybe more priviledged one.

Marian,

Evgueni_P_Intel — Tue, 26 Feb 2013 12:03:30 GMT

Marian,

The best time observed by the benchmark scales (decreases with the number of threads) but this best time is dominated by instability of measurement.

Here are some tips to stabilize measurements.

Pin threads to CPU cores using the KMP_AFFINITY environment varibale or the Windows API for thread affinity
Ensure the benchmark single-threaded; if your use-case is multi-threadedm you may want to look through http://software.intel.com/en-us/articles/different-parallelization-techniques-and-intel-mkl-fft
Prevent the cache warm-up time from dominating your performance measurement -- either increase the value of the loops variable in your code, or exclude from measurement the first call to DftiComputeForward for each Nfft.

Please let us know if the above tips help.

Thanks,

Evgueni.