- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
The following link ( https://software.intel.com/en-us/articles/intelr-distribution-for-python-2017-update-2 ) states that the Intel Python Distribution 2017 Update 2 uses the DAAL as a backend for K-means clustering in Scikit-Learn.
Using the following code, on Linux, does not seem to use the DAAL as reported by vTune. It still runs the original Scikit-Learn code written in Cython. Is there anything to do to enable the DAAL?
import os
from time import time
import numpy as np
from sklearn.cluster import KMeans
dim = 784
nb_points = 60000
nb_clusters = 10
nb_iterations = 20
nb_threads = 1
os.environ['OMP_NUM_THREADS'] = '28'
points = np.random.rand(nb_points, dim)
estimator = KMeans(init = "random", verbose = 1, max_iter = nb_iterations,
algorithm = "full", precompute_distances = False, tol = 0.0,
n_clusters = nb_clusters, n_init = 1)
print("Before computing")
time_begin = time()
estimator.fit(points)
time_end = time()
print("After computing")
print((time_end - time_begin) / nb_iterations)
Link Copied
7 Replies
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Velvia,
Check the following comment related to KMeans and DAAL: https://software.intel.com/en-us/forums/intel-distribution-for-python/topic/731351#comment-1903917
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Velvia,
I think that the entire thread will be useful because in this thread it was suggested to disable daal optimizations. So, probably the same problem applies to the issue you are reporting. I haven't tested your code in my configs. However, I remember the last comments of the thread I mention: https://software.intel.com/en-us/forums/intel-distribution-for-python/topic/731351
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Velvia,
Can you give me the output of your 'conda list', 'conda info', and your system specs?
Thanks,
-David
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi David.
Sure. Here it is:
[fayard@grisbouille Digits]$ conda info
Current conda install:
platform : linux-64
conda version : 4.2.12
conda is private : False
conda-env version : 4.2.12
conda-build version : not installed
python version : 3.5.2.final.0
requests version : 2.11.1
root environment : /opt/intel/intelpython3 (writable)
default environment : /opt/intel/intelpython3
envs directories : /opt/intel/intelpython3/envs
package cache : /opt/intel/intelpython3/pkgs
channel URLs : https://conda.anaconda.org/intel/linux-64
https://conda.anaconda.org/intel/noarch
https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
config file : /opt/intel/intelpython3/.condarc
offline mode : False
[fayard@grisbouille Digits]$ conda list
# packages in environment at /opt/intel/intelpython3:
#
backports 1.0 py35_intel_6 [intel] intel
boost 1.61.0 py35_intel_3 [intel] intel
caffe 1.0.0 py35_intel_10 [intel] intel
conda 4.2.12 py35_intel_0 [intel] intel
conda-env 2.6.0 0 intel
cycler 0.10.0 py35_intel_5 [intel] intel
cython 0.24 py35_intel_16 [intel] intel
dbus 1.10.10 0
decorator 4.0.10 py35_intel_5 [intel] intel
distarray 0.6.0 py35_intel_4 [intel] intel
entrypoints 0.2.2 py35_intel_1 [intel] intel
expat 2.1.0 0
fontconfig 2.11.1 intel_7 [intel] intel
freetype 2.5.5 intel_9 [intel] intel
get_terminal_size 1.0.0 py35_intel_5 [intel] intel
gflags 2.1.2 intel_3 [intel] intel
glib 2.50.2 1
glog 0.3.4 intel_2 [intel] intel
gst-plugins-base 1.8.0 0
gstreamer 1.8.0 0
h5py 2.6.0 np111py35_intel_1 [intel] intel
hdf5 1.8.17 intel_0 [intel] intel
icc_rt 16.0.3 intel_5 [intel] intel
icu 54.1 intel_2 [intel] intel
impi_rt 2017.0.1 intel_2 [intel] intel
intelpython 2017.0.2 1 intel
ipykernel 4.5.0 py35_intel_4 [intel] intel
ipyparallel 5.2.0 py35_intel_1 [intel] intel
ipython 4.2.0 py35_intel_8 [intel] intel
ipython_genutils 0.1.0 py35_intel_5 [intel] intel
ipywidgets 5.2.2 py35_intel_1 [intel] intel
jbig 2.1 0
jinja2 2.8 py35_intel_5 [intel] intel
jpeg 8d intel_3 [intel] intel
jsonschema 2.5.1 py35_intel_5 [intel] intel
jupyter 1.0.0 py35_intel_5 [intel] intel
jupyter_client 4.4.0 py35_intel_2 [intel] intel
jupyter_console 5.0.0 py35_intel_5 [intel] intel
jupyter_core 4.2.0 py35_intel_5 [intel] intel
keras 2.0.2 py35_0
leveldb 1.18 intel_2 [intel] intel
libffi 3.2.1 1
libgcc 5.2.0 0
libiconv 1.14 0
libpng 1.6.28 intel_10 [intel] intel
libsodium 1.0.10 intel_6 [intel] intel
libtiff 4.0.7 intel_8 [intel] intel
libxcb 1.12 1
libxml2 2.9.4 intel_10 [intel] intel
llvmlite 0.15.0 py35_intel_0 [intel] intel
lmdb 0.9.18 intel_2 [intel] intel
lmdb 0.92 <pip>
markupsafe 0.23 py35_intel_6 [intel] intel
matplotlib 1.5.3 py35_intel_3 [intel] intel
mistune 0.7.3 py35_intel_1 [intel] intel
mkl 2017.0.2 intel_1 [intel] intel
mkl-dnn 0.3dev intel_0 [intel] intel
mock 2.0.0 py35_0
mpi4py 2.0.0 py35_intel_9 [intel] intel
mpmath 0.19 py35_intel_5 [intel] intel
nbconvert 4.2.0 py35_intel_5 [intel] intel
nbformat 4.1.0 py35_intel_1 [intel] intel
networkx 1.11 py35_intel_0 [intel] intel
nose 1.3.7 py35_intel_12 [intel] intel
notebook 4.2.3 py35_intel_7 [intel] intel
numba 0.30.1 py35_intel_11 [intel] intel
numexpr 2.6.1 np111py35_intel_11 [intel] intel
numpy 1.11.2 py35_intel_51 [intel] intel
olefile 0.44 py35_0
openssl 1.0.2k intel_3 [intel] intel
pandas 0.19.2 py35_intel_1 [intel] intel
path.py 8.2.1 py35_intel_7 [intel] intel
pbr 1.10.0 py35_0
pcre 8.39 1
pexpect 4.0.1 py35_intel_5 [intel] intel
pickleshare 0.7.4 py35_intel_1 [intel] intel
pillow 3.4.2 py35_intel_4 [intel] intel
pip 8.1.2 py35_intel_5 [intel] intel
prompt_toolkit 1.0.3 py35_intel_5 [intel] intel
protobuf 3.0.0 py35_intel_1 [intel] intel
ptyprocess 0.5.1 py35_intel_5 [intel] intel
pycosat 0.6.1 py35_intel_6 [intel] intel
pydaal 2017.0.2.20170126 py35_intel_0 [intel] intel
pygments 2.1.3 py35_intel_6 [intel] intel
pyparsing 2.1.4 py35_intel_5 [intel] intel
pytables 3.2.3.1 py35_intel_8 [intel] intel
python 3.5.2 intel_27 [intel] intel
python-dateutil 2.5.3 py35_intel_5 [intel] intel
python-gflags 3.1.0 py35_intel_1 [intel] intel
python-leveldb 0.193 py35_intel_0 [intel] intel
pytz 2016.6.1 py35_intel_5 [intel] intel
pyyaml 3.12 py35_intel_1 [intel] intel
pyzmq 15.4.0 py35_intel_3 [intel] intel
qt 5.6.2 0
readline 6.2 2
requests 2.11.1 py35_intel_1 [intel] intel
ruamel_yaml 0.11.14 py35_intel_1 [intel] intel
scikit-image 0.12.3 np111py35_intel_2 [intel] intel
scikit-learn 0.18.1 py35_intel_4 [intel] intel
scipy 0.18.1 np111py35_intel_10 [intel] intel
setuptools 23.0.0 py35_intel_4 [intel] intel
simplegeneric 0.8.1 py35_intel_5 [intel] intel
six 1.10.0 py35_intel_7 [intel] intel
snappy 1.1.3 intel_2 [intel] intel
sqlite 3.13.0 intel_13 [intel] intel
sympy 1.0 py35_intel_5 [intel] intel
tbb 2017.0.4 py35_intel_0 [intel] intel
tcl 8.6.4 intel_16 [intel] intel
tensorflow 0.10.0rc0 np111py35_0
terminado 0.6 py35_intel_6 [intel] intel
theano 1.0 py35_intel_0 [intel] intel
tk 8.6.4 intel_18 [intel] intel
tornado 4.4.1 py35_intel_3 [intel] intel
traitlets 4.2.2 py35_intel_3 [intel] intel
wcwidth 0.1.7 py35_intel_5 [intel] intel
werkzeug 0.12.1 py35_0
wheel 0.29.0 py35_intel_5 [intel] intel
widgetsnbextension 1.2.6 py35_0 intel
xz 5.2.2 intel_15 [intel] intel
yaml 0.1.6 intel_10 [intel] intel
zeromq 4.1.4 intel_5 [intel] intel
zlib 1.2.11 intel_0 [intel] intel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Velvia,
Thanks for your reply. I've asked engineering for further clarification on the DAAL functionality under scikit-learn, which will hopefully shed some light as to why your example is reverting back to normal Cython.
Thanks,
David
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Velvia,
There were some updates on our channel. You can try to update scikit-learn with
conda update -c intel scikit-learn
Denis.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
We've released Update 3, so you can try with it too.
Denis.
Reply
Topic Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page