Stochastic Optimization Boosting and Parallelization in Python

[Alexei_N_]

Hello,

I am looking for a way to increase performance of stochastic optimization in Python.
My SW: i am using basin-hoping method from scipy package on pure python 2.7.6 distribution.
My HW:

Through all solutions i have found IntelPython distribution and pyDAAL library.
My questions:

1). Which of above/other techniques and infrastructure enhancements/updates could boost my optimization
sequence with respect to Parallelization?

2). Would pyDAAL or IntelPython distribution vectorize my code existing implementation based on scipy?

3). Which is the best package/library for optimization boosting if i would write new code ?

Many thanks, 
Alexei Nomazov,
Algorithm Developer.

[Sergey_M_Intel2]

Hi Alexei,

As far I understand the basin-hopin algorithm you cannot expect much performance boost out-of-the-box by switching to Intel Python. pyDAAL won't be much useful too for solving global optimization problem. Sorry about that. We have not yet optimized Basin-Hoping in SciPy in Intel Python. You can try and see but my understanding of the Basin-Hopin suggests that you won't see big performance boost as of now.

If you consider rewriting Basin-Hopin from scratch then I'm sure there are opportunities to boost performance. The first and foremost, there is random_intel package which is a proxy for numpy.random which implements vector random number generators. These are 2-60x faster than respective ones from regular numpy.Since Basin-Hopin is based on random permutations I (apriori) expect benefit from the right use of random_intel. Basin-Hopin is also good for parallelization (it's embarrassingly parallel problem). I expect significant performance boost after appropriate parallelization.

Eventually we will optimize basin-hopin. it's currently lower priority relative to some other optimization tasks. If you can help us to better understand the role of Basin-Hopin performance to your project it may help us to prioritize this optimization work higher.

Regards,

Sergey Maidanov

Engineering Manager for Intel Distribution for Python