- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
The following link ( https://software.intel.com/en-us/articles/intelr-distribution-for-python-2017-update-2 ) states that the Intel Python Distribution 2017 Update 2 uses the DAAL as a backend for K-means clustering in Scikit-Learn.
Using the following code, on Linux, does not seem to use the DAAL as reported by vTune. It still runs the original Scikit-Learn code written in Cython. Is there anything to do to enable the DAAL?
import os from time import time import numpy as np from sklearn.cluster import KMeans dim = 784 nb_points = 60000 nb_clusters = 10 nb_iterations = 20 nb_threads = 1 os.environ['OMP_NUM_THREADS'] = '28' points = np.random.rand(nb_points, dim) estimator = KMeans(init = "random", verbose = 1, max_iter = nb_iterations, algorithm = "full", precompute_distances = False, tol = 0.0, n_clusters = nb_clusters, n_init = 1) print("Before computing") time_begin = time() estimator.fit(points) time_end = time() print("After computing") print((time_end - time_begin) / nb_iterations)
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Velvia,
Check the following comment related to KMeans and DAAL: https://software.intel.com/en-us/forums/intel-distribution-for-python/topic/731351#comment-1903917
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Velvia,
I think that the entire thread will be useful because in this thread it was suggested to disable daal optimizations. So, probably the same problem applies to the issue you are reporting. I haven't tested your code in my configs. However, I remember the last comments of the thread I mention: https://software.intel.com/en-us/forums/intel-distribution-for-python/topic/731351
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Velvia,
Can you give me the output of your 'conda list', 'conda info', and your system specs?
Thanks,
-David
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi David.
Sure. Here it is:
[fayard@grisbouille Digits]$ conda info Current conda install: platform : linux-64 conda version : 4.2.12 conda is private : False conda-env version : 4.2.12 conda-build version : not installed python version : 3.5.2.final.0 requests version : 2.11.1 root environment : /opt/intel/intelpython3 (writable) default environment : /opt/intel/intelpython3 envs directories : /opt/intel/intelpython3/envs package cache : /opt/intel/intelpython3/pkgs channel URLs : https://conda.anaconda.org/intel/linux-64 https://conda.anaconda.org/intel/noarch https://repo.continuum.io/pkgs/free/linux-64 https://repo.continuum.io/pkgs/free/noarch https://repo.continuum.io/pkgs/pro/linux-64 https://repo.continuum.io/pkgs/pro/noarch config file : /opt/intel/intelpython3/.condarc offline mode : False [fayard@grisbouille Digits]$ conda list # packages in environment at /opt/intel/intelpython3: # backports 1.0 py35_intel_6 [intel] intel boost 1.61.0 py35_intel_3 [intel] intel caffe 1.0.0 py35_intel_10 [intel] intel conda 4.2.12 py35_intel_0 [intel] intel conda-env 2.6.0 0 intel cycler 0.10.0 py35_intel_5 [intel] intel cython 0.24 py35_intel_16 [intel] intel dbus 1.10.10 0 decorator 4.0.10 py35_intel_5 [intel] intel distarray 0.6.0 py35_intel_4 [intel] intel entrypoints 0.2.2 py35_intel_1 [intel] intel expat 2.1.0 0 fontconfig 2.11.1 intel_7 [intel] intel freetype 2.5.5 intel_9 [intel] intel get_terminal_size 1.0.0 py35_intel_5 [intel] intel gflags 2.1.2 intel_3 [intel] intel glib 2.50.2 1 glog 0.3.4 intel_2 [intel] intel gst-plugins-base 1.8.0 0 gstreamer 1.8.0 0 h5py 2.6.0 np111py35_intel_1 [intel] intel hdf5 1.8.17 intel_0 [intel] intel icc_rt 16.0.3 intel_5 [intel] intel icu 54.1 intel_2 [intel] intel impi_rt 2017.0.1 intel_2 [intel] intel intelpython 2017.0.2 1 intel ipykernel 4.5.0 py35_intel_4 [intel] intel ipyparallel 5.2.0 py35_intel_1 [intel] intel ipython 4.2.0 py35_intel_8 [intel] intel ipython_genutils 0.1.0 py35_intel_5 [intel] intel ipywidgets 5.2.2 py35_intel_1 [intel] intel jbig 2.1 0 jinja2 2.8 py35_intel_5 [intel] intel jpeg 8d intel_3 [intel] intel jsonschema 2.5.1 py35_intel_5 [intel] intel jupyter 1.0.0 py35_intel_5 [intel] intel jupyter_client 4.4.0 py35_intel_2 [intel] intel jupyter_console 5.0.0 py35_intel_5 [intel] intel jupyter_core 4.2.0 py35_intel_5 [intel] intel keras 2.0.2 py35_0 leveldb 1.18 intel_2 [intel] intel libffi 3.2.1 1 libgcc 5.2.0 0 libiconv 1.14 0 libpng 1.6.28 intel_10 [intel] intel libsodium 1.0.10 intel_6 [intel] intel libtiff 4.0.7 intel_8 [intel] intel libxcb 1.12 1 libxml2 2.9.4 intel_10 [intel] intel llvmlite 0.15.0 py35_intel_0 [intel] intel lmdb 0.9.18 intel_2 [intel] intel lmdb 0.92 <pip> markupsafe 0.23 py35_intel_6 [intel] intel matplotlib 1.5.3 py35_intel_3 [intel] intel mistune 0.7.3 py35_intel_1 [intel] intel mkl 2017.0.2 intel_1 [intel] intel mkl-dnn 0.3dev intel_0 [intel] intel mock 2.0.0 py35_0 mpi4py 2.0.0 py35_intel_9 [intel] intel mpmath 0.19 py35_intel_5 [intel] intel nbconvert 4.2.0 py35_intel_5 [intel] intel nbformat 4.1.0 py35_intel_1 [intel] intel networkx 1.11 py35_intel_0 [intel] intel nose 1.3.7 py35_intel_12 [intel] intel notebook 4.2.3 py35_intel_7 [intel] intel numba 0.30.1 py35_intel_11 [intel] intel numexpr 2.6.1 np111py35_intel_11 [intel] intel numpy 1.11.2 py35_intel_51 [intel] intel olefile 0.44 py35_0 openssl 1.0.2k intel_3 [intel] intel pandas 0.19.2 py35_intel_1 [intel] intel path.py 8.2.1 py35_intel_7 [intel] intel pbr 1.10.0 py35_0 pcre 8.39 1 pexpect 4.0.1 py35_intel_5 [intel] intel pickleshare 0.7.4 py35_intel_1 [intel] intel pillow 3.4.2 py35_intel_4 [intel] intel pip 8.1.2 py35_intel_5 [intel] intel prompt_toolkit 1.0.3 py35_intel_5 [intel] intel protobuf 3.0.0 py35_intel_1 [intel] intel ptyprocess 0.5.1 py35_intel_5 [intel] intel pycosat 0.6.1 py35_intel_6 [intel] intel pydaal 2017.0.2.20170126 py35_intel_0 [intel] intel pygments 2.1.3 py35_intel_6 [intel] intel pyparsing 2.1.4 py35_intel_5 [intel] intel pytables 3.2.3.1 py35_intel_8 [intel] intel python 3.5.2 intel_27 [intel] intel python-dateutil 2.5.3 py35_intel_5 [intel] intel python-gflags 3.1.0 py35_intel_1 [intel] intel python-leveldb 0.193 py35_intel_0 [intel] intel pytz 2016.6.1 py35_intel_5 [intel] intel pyyaml 3.12 py35_intel_1 [intel] intel pyzmq 15.4.0 py35_intel_3 [intel] intel qt 5.6.2 0 readline 6.2 2 requests 2.11.1 py35_intel_1 [intel] intel ruamel_yaml 0.11.14 py35_intel_1 [intel] intel scikit-image 0.12.3 np111py35_intel_2 [intel] intel scikit-learn 0.18.1 py35_intel_4 [intel] intel scipy 0.18.1 np111py35_intel_10 [intel] intel setuptools 23.0.0 py35_intel_4 [intel] intel simplegeneric 0.8.1 py35_intel_5 [intel] intel six 1.10.0 py35_intel_7 [intel] intel snappy 1.1.3 intel_2 [intel] intel sqlite 3.13.0 intel_13 [intel] intel sympy 1.0 py35_intel_5 [intel] intel tbb 2017.0.4 py35_intel_0 [intel] intel tcl 8.6.4 intel_16 [intel] intel tensorflow 0.10.0rc0 np111py35_0 terminado 0.6 py35_intel_6 [intel] intel theano 1.0 py35_intel_0 [intel] intel tk 8.6.4 intel_18 [intel] intel tornado 4.4.1 py35_intel_3 [intel] intel traitlets 4.2.2 py35_intel_3 [intel] intel wcwidth 0.1.7 py35_intel_5 [intel] intel werkzeug 0.12.1 py35_0 wheel 0.29.0 py35_intel_5 [intel] intel widgetsnbextension 1.2.6 py35_0 intel xz 5.2.2 intel_15 [intel] intel yaml 0.1.6 intel_10 [intel] intel zeromq 4.1.4 intel_5 [intel] intel zlib 1.2.11 intel_0 [intel] intel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Velvia,
Thanks for your reply. I've asked engineering for further clarification on the DAAL functionality under scikit-learn, which will hopefully shed some light as to why your example is reverting back to normal Cython.
Thanks,
David
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Velvia,
There were some updates on our channel. You can try to update scikit-learn with
conda update -c intel scikit-learn
Denis.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
We've released Update 3, so you can try with it too.
Denis.
![](/skins/images/D3C0B914909A6564BBB97F4AD1ED1973/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page