topic Denis, in Intel® Distribution for Python*

scikit-learn test errors under Intel Python

sergio_r_ — Mon, 17 Apr 2017 08:19:07 GMT

Testing scikit-learn via the Intel Python gives many errors, shown below. The test can be run using:

nosetests -v sklearn

What this errors means in terms of using this package under Intel Python?

Regards,

Sergio
Enhance your #MachineLearning and #BigData skills via #Python #SciPy
1) https://www.packtpub.com/big-data-and-business-intelligence/numerical-and-sci
entific-computing-scipy-video
2) https://www.packtpub.com/big-data-and-business-intelligence/learning-scipy-nu
merical-and-scientific-computing-second-edition

======================================================================
ERROR: /home/intel/intelpython3/lib/python3.5/site-packages/sklearn/tests/test_c
ommon.py.test_non_meta_estimators:check_fit2d_predict1d(LinearRegression)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/intel/intelpython3/lib/python3.5/site-packages/nose/case.py", line
198, in runTest
    self.test(*self.arg)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/utils/testi
ng.py", line 830, in __call__
    return self.check(*args, **kwargs)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/utils/estim
ator_checks.py", line 479, in check_fit2d_predict1d
    getattr(estimator, method), X[0])
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/utils/testi
ng.py", line 186, in assert_warns
    result = func(*args, **kw)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/daal4sklear
n/linear.py", line 156, in predict
    return daal_predict(self, X)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/daal4sklear
n/linear.py", line 56, in daal_predict
    predictionResult = algorithm.compute()
File "/home/intel/intelpython3/lib/python3.5/site-packages/daal/algorithms/lin
ear_regression/prediction.py", line 253, in compute
    return _prediction9.Batch_Float64DefaultDense_compute(self)
SystemError: Number of columns in numeric table is incorrect
Details:
Argument name: beta

======================================================================
ERROR: /home/intel/intelpython3/lib/python3.5/site-packages/sklearn/tests/test_c
ommon.py.test_non_meta_estimators:check_fit2d_predict1d(RANSACRegressor)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/intel/intelpython3/lib/python3.5/site-packages/nose/case.py", line
198, in runTest
    self.test(*self.arg)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/utils/testi
ng.py", line 830, in __call__
    return self.check(*args, **kwargs)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/utils/estim
ator_checks.py", line 479, in check_fit2d_predict1d
    getattr(estimator, method), X[0])
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/utils/testi
ng.py", line 186, in assert_warns
    result = func(*args, **kw)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/linear_mode
l/ransac.py", line 428, in predict
    return self.estimator_.predict(X)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/daal4sklear
n/linear.py", line 156, in predict
    return daal_predict(self, X)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/daal4sklear
n/linear.py", line 56, in daal_predict
    predictionResult = algorithm.compute()
File "/home/intel/intelpython3/lib/python3.5/site-packages/daal/algorithms/lin
ear_regression/prediction.py", line 253, in compute
    return _prediction9.Batch_Float64DefaultDense_compute(self)
SystemError: Number of columns in numeric table is incorrect
Details:
Argument name: beta

======================================================================
FAIL: sklearn.cluster.tests.test_bicluster.test_project_and_cluster
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/intel/intelpython3/lib/python3.5/site-packages/nose/case.py", line
198, in runTest
    self.test(*self.arg)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/cluster/tes
ts/test_bicluster.py", line 207, in test_project_and_cluster
    assert_array_equal(labels, [0, 0, 1, 1])
File "/home/intel/intelpython3/lib/python3.5/site-packages/numpy/testing/utils
.py", line 813, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
File "/home/intel/intelpython3/lib/python3.5/site-packages/numpy/testing/utils
.py", line 739, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Arrays are not equal

(mismatch 100.0%)
x: array([1, 1, 0, 0], dtype=int32)
y: array([0, 0, 1, 1])

======================================================================
FAIL: sklearn.cluster.tests.test_k_means.test_k_means_non_collapsed
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/intel/intelpython3/lib/python3.5/site-packages/nose/case.py", line
198, in runTest
self.test(*self.arg)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/cluster/tes
ts/test_k_means.py", line 552, in test_k_means_non_collapsed
assert_equal(len(np.unique(km.labels_)), 3)
AssertionError: 1 != 3

======================================================================
FAIL: sklearn.linear_model.tests.test_ridge.test_dense_sparse(<function _test_ri
dge_loo at 0x7fb59bd21840>,)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/intel/intelpython3/lib/python3.5/site-packages/nose/case.py", line
198, in runTest
    self.test(*self.arg)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/linear_mode
l/tests/test_ridge.py", line 454, in check_dense_sparse
    ret_dense = test_func(DENSE_FILTER)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/linear_mode
l/tests/test_ridge.py", line 329, in _test_ridge_loo
    assert_almost_equal(errors, errors2)
File "/home/intel/intelpython3/lib/python3.5/site-packages/numpy/testing/utils
.py", line 523, in assert_almost_equal
    return assert_array_almost_equal(actual, desired, decimal, err_msg)
File "/home/intel/intelpython3/lib/python3.5/site-packages/numpy/testing/utils
.py", line 918, in assert_array_almost_equal
    precision=decimal)
File "/home/intel/intelpython3/lib/python3.5/site-packages/numpy/testing/utils
.py", line 739, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 7 decimals

(mismatch 100.0%)
x: array([ 1.8295168e+04,   5.6704414e+02,   1.1782287e+03,   5.2810053e+03,
         3.8443835e+02,   8.0439421e+03,   2.3982367e+03,   3.3362619e+03,
         1.1783452e+01,   5.1091672e+03,   1.3139571e+04,   1.5552388e+01,...
y: array([ 1.8505354e+04,   6.0452467e+02,   1.1257434e+03,   5.1693552e+03,
         4.1556914e+02,   8.1834151e+03,   2.4742189e+03,   3.4259267e+03,
         1.7722638e+01,   4.9988370e+03,   1.3318625e+04,   1.0045693e+01,...

======================================================================
FAIL: sklearn.manifold.tests.test_spectral_embedding.test_pipeline_spectral_clus
tering
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/intel/intelpython3/lib/python3.5/site-packages/nose/case.py", line
198, in runTest
    self.test(*self.arg)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/manifold/te
sts/test_spectral_embedding.py", line 198, in test_pipeline_spectral_clustering
    true_labels), 1.0, 2)
File "/home/intel/intelpython3/lib/python3.5/site-packages/numpy/testing/utils
.py", line 918, in assert_array_almost_equal
    precision=decimal)
File "/home/intel/intelpython3/lib/python3.5/site-packages/numpy/testing/utils
.py", line 739, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 2 decimals

(mismatch 100.0%)
x: array(0.5866188259941206)
y: array(1.0)

======================================================================
FAIL: sklearn.mixture.tests.test_dpgmm.test_class_weights
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/intel/intelpython3/lib/python3.5/site-packages/nose/case.py", line
198, in runTest
    self.test(*self.arg)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/utils/testi
ng.py", line 355, in wrapper
    return fn(*args, **kwargs)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/mixture/tes
ts/test_dpgmm.py", line 41, in test_class_weights
    assert_array_less(dpgmm.weights_[~active], .05)
File "/home/intel/intelpython3/lib/python3.5/site-packages/numpy/testing/utils
.py", line 990, in assert_array_less
    header='Arrays are not less-ordered')
File "/home/intel/intelpython3/lib/python3.5/site-packages/numpy/testing/utils
.py", line 739, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Arrays are not less-ordered

(mismatch 14.285714285714292%)
x: array([ 0.227048, 0.035354, 0.034998, 0.03491 , 0.034875, 0.034874,
0.034887])
y: array(0.05)

======================================================================
FAIL: sklearn.model_selection.tests.test_validation.test_cross_val_score_with_sc
ore_func_regression
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/intel/intelpython3/lib/python3.5/site-packages/nose/case.py", line
198, in runTest
    self.test(*self.arg)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/model_selec
tion/tests/test_validation.py", line 418, in test_cross_val_score_with_score_fun
c_regression
    assert_array_almost_equal(neg_mse_scores, expected_neg_mse, 2)
File "/home/intel/intelpython3/lib/python3.5/site-packages/numpy/testing/utils
.py", line 918, in assert_array_almost_equal
    precision=decimal)
File "/home/intel/intelpython3/lib/python3.5/site-packages/numpy/testing/utils
.py", line 739, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 2 decimals

(mismatch 100.0%)
x: array([ -760.52, -514.85, -265.83, -275.24, -1647. ])
y: array([ -763.07, -553.16, -274.38, -273.26, -1681.99])

======================================================================
FAIL: /home/intel/intelpython3/lib/python3.5/site-packages/sklearn/tests/test_co
mmon.py.test_non_meta_estimators:check_estimators_nan_inf(LinearRegression)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/intel/intelpython3/lib/python3.5/site-packages/nose/case.py", line
198, in runTest
    self.test(*self.arg)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/utils/testi
ng.py", line 830, in __call__
    return self.check(*args, **kwargs)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/utils/estim
ator_checks.py", line 840, in check_estimators_nan_inf
    raise AssertionError(error_string_predict, Estimator)
AssertionError: ("Estimator doesn't check for NaN and inf in predict.", <class '
sklearn.linear_model.base.LinearRegression'>)

======================================================================
FAIL: /home/intel/intelpython3/lib/python3.5/site-packages/sklearn/tests/test_co
mmon.py.test_non_meta_estimators:check_estimators_nan_inf(RANSACRegressor)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/intel/intelpython3/lib/python3.5/site-packages/nose/case.py", line
198, in runTest
    self.test(*self.arg)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/utils/testi
ng.py", line 830, in __call__
    return self.check(*args, **kwargs)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/utils/estim
ator_checks.py", line 840, in check_estimators_nan_inf
    raise AssertionError(error_string_predict, Estimator)
AssertionError: ("Estimator doesn't check for NaN and inf in predict.", <class '
sklearn.linear_model.ransac.RANSACRegressor'>)

======================================================================
FAIL: sklearn.tests.test_cross_validation.test_cross_val_score_with_score_func_r
egression
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/intel/intelpython3/lib/python3.5/site-packages/nose/case.py", line
198, in runTest
    self.test(*self.arg)
File "/home/intel/intelpython3/lib/python3.5/site-packages/sklearn/tests/test_
cross_validation.py", line 918, in test_cross_val_score_with_score_func_regressi
on
    assert_array_almost_equal(neg_mse_scores, expected_neg_mse, 2)
File "/home/intel/intelpython3/lib/python3.5/site-packages/numpy/testing/utils
.py", line 918, in assert_array_almost_equal
    precision=decimal)
File "/home/intel/intelpython3/lib/python3.5/site-packages/numpy/testing/utils
.py", line 739, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Arrays are not almost equal to 2 decimals

(mismatch 100.0%)
x: array([ -760.52, -514.85, -265.83, -275.24, -1647. ])
y: array([ -763.07, -553.16, -274.38, -273.26, -1681.99])

----------------------------------------------------------------------
Ran 7166 tests in 203.755s

FAILED (SKIP=14, errors=6, failures=9)

Sergio,

gaston-hillar — Tue, 18 Apr 2017 03:04:56 GMT

Sergio,

This thread is a duplicate or provides additional information to this thread: https://software.intel.com/en-us/forums/intel-distribution-for-python/topic/731344

Hi Sergio,

DavidLiu — Wed, 19 Apr 2017 20:36:13 GMT

Hi Sergio,

We have seen the issue and are looking into it.

Thanks,

David

Quote:gaston-hillar wrote:

sergio_r_ — Wed, 19 Apr 2017 21:44:48 GMT

gaston-hillar wrote:

Sergio,

This thread is a duplicate or provides additional information to this thread: https://software.intel.com/en-us/forums/intel-distribution-for-python/topic/731344

Jumping to conclusions, Hillar, is not a polite way to request info.

I am in this forum because of an invitation to test the software. I am reporting errors as I find them and if we are using the same
Intel python release it is suppose to behave the same in any system.

I'm glad you profited from my post and learnt that in your system the errors are also happening.

As David mention in the next post, Intel's python gurus are looking into the reported issue. This is my major intention in this forum.

Sergio
Enhance your #MachineLearning and #BigData skills via #Python #SciPy
1) https://www.packtpub.com/big-data-and-business-intelligence/numerical-and-scientific-computing-scipy-video
2) https://www.packtpub.com/big-data-and-business-intelligence/learning-scipy-numerical-and-scientific-computing-second-edition

Sergio,

gaston-hillar — Wed, 19 Apr 2017 23:19:18 GMT

Sergio,

It's great to know the issue is being investigating.

Regarding some of the comments you include in your post, I cannot understand what you meant.

"Jumping to conclusions, Hillar, is not a polite way to request info."

I just indicated the thread was duplicated. Please, help me understand your comment, in case it is important. If it isn't important, don't worry at all.

David,

gaston-hillar — Thu, 20 Apr 2017 04:43:50 GMT

David,

In case it helps, I've provided the results of running the tests on a different configuration in the other thread that was opened (it was the first one i saw): https://software.intel.com/en-us/forums/intel-distribution-for-python/topic/731344

Quote:gaston-hillar wrote:

sergio_r_ — Thu, 20 Apr 2017 11:15:35 GMT

gaston-hillar wrote:

Sergio,

It's great to know the issue is being investigating.

Regarding some of the comments you include in your post, I cannot understand what you meant.

"Jumping to conclusions, Hillar, is not a polite way to request info."

I just indicated the thread was duplicated. Please, help me understand your comment, in case it is important. If it isn't important, don't worry at all.

In the (duplicated by error) previous post [ https://software.intel.com/en-us/forums/intel-distribution-for-python/topic/731344 ], Gaston, you wrongly mentioned I was spaming.

Sergio
Enhance your #MachineLearning and #BigData skills via #Python #SciPy
1) https://www.packtpub.com/big-data-and-business-intelligence/numerical-and-scientific-computing-scipy-video
2) https://www.packtpub.com/big-data-and-business-intelligence/learning-scipy-numerical-and-scientific-computing-second-edition

Sergio,

gaston-hillar — Thu, 20 Apr 2017 16:27:11 GMT

Sergio,

Got it... You are including the following three lines after your name with links to products in which you are involved:

Enhance your #MachineLearning and #BigData skills via #Python #SciPy
1) https://www.packtpub.com/big-data-and-business-intelligence/numerical-an.
2) https://www.packtpub.com/big-data-and-business-intelligence/learning-sci.

However, in the other post, you included those lines as the output for the tests. Hence, the output was confusing. I was confused.

You can include the information in your profile details at Intel Developer Zone. This way, we don't see the continuous promotion of your products mixed with the text of your posts. You are not in twitter, you are in a forum. I know that probably some marketing team member suggested you to SPAM in forums. Please, avoid doing so, it will only hurt your reputation.

However, don't worry, I'm not a moderator in the forum. So, it is just my opinion.

You are not polite when you spam in forums. Rule #1 in any professional forum: avoid SPAM.

When you SPAM you hurt reputation for others that work with the same publisher.

BTW, this is the last comment I will write about this specific issue.

Hi, thank you for your

Denis_N_Intel — Mon, 24 Apr 2017 18:45:53 GMT

Hi, thank you for your interest to our optimizations for sklearn in Intel Distribution for Python.
We are aware of these failures. These failures are caused by our optimizations. There are 3 groups of issues.
The first one - absence of some functionality - like failures with pickling or with processing of special values (like inf and nan). These failures arise from use of DAAL library whose state we can't serialize at this moment and which doesn't take special values into consideration.
The second one - differences in implementations which produces slightly different results. It means for example that order of centroids can be different for K-Means algorithm or values you get can slightly differ from those expected in tests. We are working on improving such tests.
The third - real minor bugs which was introduced by new implementations. We are working on fixing them. (i.e. fail in fastica test). This is going to be fixed in the forthcoming update.
Thank you again for helping us to improve the quality of our product. If you would rather disable use of DAAL in sklearn due to these failures, please let us know, and I will provide you will steps of doing so.
Denis Nagorny.

Hi Denis,

gaston-hillar — Tue, 25 Apr 2017 05:04:04 GMT

Hi Denis,

In my case, I was very interested in this issue because I'm working with K-means. It would be great if you can provide the steps to disable use of DAAL and to enable it again.

In addition, I think it would be a good idea to add a sticky with known issues in this forum on in the release notes. Intel does this for other products in which there are known issues that developers should be aware of. The fact that some tests fail can generate some uncertainty. If these issues are included in a known issues documentation, it will help developers to know that they can expect these specific tests to fail and know what they mean.

BTW, it is great to know that there is always a timely response in all the threads that have been started so far in this forum. :)

HI Gastón,

Denis_N_Intel — Tue, 25 Apr 2017 13:14:01 GMT

HI Gastón,

There is daal4sklearn module in our sklearn. If you want to disable our K-means implementation you can:

from sklearn.daal4sklearn import dispatcher
dispatcher.disable('KMeans')

accordingly there is enable method to return it back.

Denis.

Hi Denis,

gaston-hillar — Tue, 25 Apr 2017 16:22:36 GMT

Hi Denis,

Cool. Thanks for the details.

Quote:Denis Nagorny (Intel)

sergio_r_ — Thu, 27 Apr 2017 01:42:29 GMT

Denis Nagorny (Intel) wrote:

HI Gastón,

There is daal4sklearn module in our sklearn. If you want to disable our K-means implementation you can:
from sklearn.daal4sklearn import dispatcher
dispatcher.disable('KMeans')
accordingly there is enable method to return it back.

Denis.

Thanks Gastón for asking and Denis for answering. Nice to know the errors were traced down and a fix is already in place.

Sergio
Enhance your #MachineLearning and #BigData skills via #Python #SciPy
1) https://www.packtpub.com/big-data-and-business-intelligence/numerical-and-scientific-computing-scipy-video
2) https://www.packtpub.com/big-data-and-business-intelligence/learning-scipy-numerical-and-scientific-computing-second-edition

@Denis,

gaston-hillar — Thu, 27 Apr 2017 04:52:17 GMT

@Denis,

I've executed the following two lines in Intel Distribution for Python 2017 Update 2 (Python 3.5.2) on Windows and on macOS. I receive errors in both cases.

from sklearn.daal4sklearn import dispatcher
dispatcher.disable('KMeans')

I receive the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\IntelPython35\lib\site-packages\sklearn\daal4sklearn\dispatcher.py", line 56, in disable
    do_unpatch(name)
  File "C:\IntelPython35\lib\site-packages\sklearn\daal4sklearn\dispatcher.py", line 43, in do_unpatch
    raise ValueError("Has no patch for: " + name)
ValueError: Has no patch for: KMeans

Do I have to install any additional package to enable this feature?

@Sergio,

gaston-hillar — Thu, 27 Apr 2017 04:53:20 GMT

@Sergio,

It's great to know my additional questions provided you valuable information. :)

The solution doesn't work on my configurations and I'll wait for additional comments from Denis.

Hi Gastón,

Denis_N_Intel — Thu, 27 Apr 2017 15:26:56 GMT

Hi Gastón,

Original fail log indicates that topic starter has kmeans optimizations enabled.

It seems that KMean can be still unavailable in default 2017 U2, but it definitely should be in forthcoming U3.

To disable all daal's optimizations in sklearn you can use disable()/enable() methods without any arguments.

Denis.

Denis,

gaston-hillar — Thu, 27 Apr 2017 19:17:32 GMT

Denis,

Got it. Thanks for the super quick response.

Looking forward to U3.

Quote:gaston-hillar wrote:

sergio_r_ — Wed, 03 May 2017 10:23:33 GMT

gaston-hillar wrote:

@Denis,

I've executed the following two lines in Intel Distribution for Python 2017 Update 2 (Python 3.5.2) on Windows and on macOS. I receive errors in both cases.
from sklearn.daal4sklearn import dispatcher
dispatcher.disable('KMeans')
I receive the following error:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\IntelPython35\lib\site-packages\sklearn\daal4sklearn\dispatcher.py", line 56, in disable
    do_unpatch(name)
  File "C:\IntelPython35\lib\site-packages\sklearn\daal4sklearn\dispatcher.py", line 43, in do_unpatch
    raise ValueError("Has no patch for: " + name)
ValueError: Has no patch for: KMeans
Do I have to install any additional package to enable this feature?

Hi Gaston and Denis: the above function calls seems to work in my setup:

$ python
Python 3.5.2 |Intel Corporation| (default, Mar 27 2017, 10:34:52)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution

>>> from sklearn.daal4sklearn import dispatcher
>>> dispatcher.disable('KMeans')
>>> dispatcher.enable('KMeans')
>>> import sklearn
>>> sklearn.__version__
'0.18.1'

Salut,

Sergio
Enhance your #MachineLearning and #BigData skills via #Python #SciPy
1) https://www.packtpub.com/big-data-and-business-intelligence/numerical-and-scientific-computing-scipy-video
2) https://www.packtpub.com/big-data-and-business-intelligence/learning-scipy-numerical-and-scientific-computing-second-edition

@Sergio,

gaston-hillar — Wed, 03 May 2017 14:32:46 GMT

@Sergio,

Thanks for sharing the results in your configuration. I'll check on my Linux-based configuration.

Hi Gastón,

Denis_N_Intel — Wed, 10 May 2017 12:00:13 GMT

Hi Gastón,

It seems that kmeans could be already available on our channel.

You can try to use 'conda update -c intel scikit-learn'

Denis.