Speed Up with MIC/ in Parallel

[Bryar_S_]

My question is not specific to MIC developers, it for all parallel developers in general. 

By increasing data sampling (n) in the program, I get better speed up. Also the sequential  is better than parallel version with small (n). 

I know this is normal in parallel programming, but would you tell me why ? or give me some reasons to convince me.

[Sunny_G_Intel · Accepted solution]

Hello Bryar,

Can you please elaborate what kind of application and dataset you are referring to. I am assuming by increasing data sampling you mean increasing the size of data set. Correct?

If yes, your observation about correlation between data set v/s number of parallel threads/processes is correct for lot of HPC applications (especially embarrassingly parallel applications). As per my knowledge one of the reason this kind of behavior is observed is because there is always some finite amount of overhead involved in setting up and maintaining multiple threads/processes. Hence in case of small datasets this overhead subsides the benefit of parallel processing as a result you see that for small datasets sequential programs (without thread creation/maintainance overhead) runs faster. And as you increase the size of dataset, for any embarrassingly parallel application the benefits of parallel processing increases and hence the speedup increases. Besides parallel thread creation and maintenance there are lot of other things happening as well like communication, synchronization etc between threads. 

I hope this helps. 

[Bryar_S_]

Thanks for explanation, I suppose using Monte Carlo Simulation and some Pseudo random number. By increasing data sampling, I mean increasing data set.