Community
cancel
Showing results for 
Search instead for 
Did you mean: 
RL5
Beginner
229 Views

Interoperability

Hi,

I am using two machines to develop my work. In my laptop I am running IPD 3.5.2 with virtualenv on ArchLinux (self-compiled), and I have access to a Windows 7 Pro workstation where I am running IPD 3.5.2 with Anaconda. In both I have pip showing Scipy at version 0.18.1.

However, I have trained a RandomForestClassifier on the linux system, saved the data normalizer (with pickle) and model (using joblib), and when I try to run it on the windows machine it issues the warning:

UserWarning: Trying to unpickle estimator MinMaxScaler from version pre-0.18 when using version 0.18.1. This might lead to breaking code or invalid results. Use at your own risk.

Trying to unpickle estimator DecisionTreeClassifier from version pre-0.18 when using version 0.18.1. This might lead to breaking code or invalid results. Use at your own risk.

Going on at "my own risk" after loading the data normalizer, I try to load the model (same warning), and it fails immediately with a "KeyError: 'max_depth'", which is still a valid argument in RandomForestClassifier v0.18 documentation.

Am I doing something wrong? Should I not expect the saved models to be usable across different OS even if the python dist is the same? Is this a glitch because I am not using Anaconda (or self-compiling) in both systems?

0 Kudos
4 Replies
229 Views

Hi Rui,

I suspect this is an environment issue. There may be some conflict with virtualenv and the conda package manager which comes bundled with Intel Python.  Can you send me the output of the following commands from the ArchLinux machine please?

$ which python
$ which pip
$ which conda
$ conda list
$ python -c 'import scipy; print(scipy.__version__)'

Chris

RL5
Beginner
229 Views

Sure. On the ArchLinux machine I am not using conda. I compiled everything myself, and I am using virtualenv. I include the output from pip list and I see already that the problem probably lies in the scikit-learn version. In the windows machine it is 0.18.1 as opposed to 0.17.1 in the output bellow. I will downgrade scikit-learn on the windows machine and re-test.

(intelpython) [rmlopes@baltra modelRF2000alldata]$ which python
/home/rmlopes/dev/intelpython/bin/python
(intelpython) [rmlopes@baltra modelRF2000alldata]$ which pip
/home/rmlopes/dev/intelpython/bin/pip

(intelpython) [rmlopes@baltra modelRF2000alldata]$ pip list --format=columns
Package                            Version                                         
---------------------------------- ------------------------------------------------
appdirs                            1.4.0                                           
backports.shutil-get-terminal-size 1.0.0                                           
beautifulsoup4                     4.5.3                                           
click                              6.7                                             
conda                              4.1.11                                          
conda-env                          2.5.0a0                                         
cycler                             0.10.0                                          
Cython                             0.24                                            
decorator                          4.0.10                                          
distarray                          0.6.0                                           
geocoder                           1.20.1                                          
h5py                               2.6.0                                           
ipykernel                          4.5.0                                           
ipyparallel                        5.2.0                                           
ipython                            4.2.0                                           
ipython-genutils                   0.1.0                                           
ipywidgets                         5.2.2                                           
Jinja2                             2.8                                             
joblib                             0.10.3                                          
jsonschema                         2.5.1                                           
jupyter                            1.0.0                                           
jupyter-client                     4.4.0                                           
jupyter-console                    5.0.0                                           
jupyter-core                       4.2.0                                           
Keras                              1.2.2                                           
llvmlite                           0.12.0.dev0+0.gbb25ba4.dirty                    
MarkupSafe                         0.23                                            
matplotlib                         1.5.3                                           
mistune                            0.7.3                                           
mpi4py                             2.0.0                                           
mpmath                             0.19                                            
nbconvert                          4.2.0                                           
nbformat                           4.1.0                                           
neupy                              0.4.0.dev0                                      
nose                               1.3.7                                           
notebook                           4.2.3                                           
numba                              0.29.0.dev0+0.gfe99fbc.dirty                    
numexpr                            2.6.1                                           
numpy                              1.11.2                                          
obspy                              1.0.1.post0+211.g3a1c40c359.dirty.rmlopes.master
packaging                          16.8                                            
pandas                             0.19.0                                          
path.py                            0.0.0                                           
pexpect                            4.0.1                                           
pickleshare                        0.7.4                                           
pip                                9.0.1                                           
prettytable                        0.7.2                                           
prompt-toolkit                     1.0.3                                           
protobuf                           3.2.0                                           
ptyprocess                         0.5.1                                           
pycosat                            0.6.1                                           
pydaal                             2017.0.1.20161006                               
pygeocoder                         1.2.5                                           
Pygments                           2.1.3                                           
pyparsing                          2.1.10                                          
python-dateutil                    2.5.3                                           
pytz                               2016.6.1                                        
PyYAML                             3.12                                            
pyzmq                              15.4.0                                          
ratelim                            0.1.6                                           
requests                           2.11.1                                          
ruamel.yaml                        0.13.14                                         
scikit-learn                       0.17.1                                          
scipy                              0.18.1                                          
setuptools                         34.2.0                                          
simplegeneric                      0.8.1                                           
six                                1.10.0                                          
sklearn                            0.0                                             
sympy                              1.0                                             
tables                             3.2.3.1                                         
TBB                                0.1                                             
tensorflow                         1.0.0                                           
terminado                          0.6                                             
Theano                             0.8.2                                           
tornado                            4.4.1                                           
traitlets                          4.2.2                                           
virtualenv                         15.1.0                                          
wcwidth                            0.1.7                                           
wheel                              0.29.0                                          
widgetsnbextension                 1.2.6      

(intelpython) [rmlopes@baltra modelRF2000alldata]$ python -c 'import scipy; print(scipy.__version__)'
0.18.1

 

229 Views

Hi Rui,

Please let me know if that resolves your issue.

Chris

RL5
Beginner
229 Views

Thanks for the input Chris. Downgrading to scikit-learn 0.17.1 on the windows machine apparently solved the issue. The model was loaded with success (and without warnings), and it is running without problems so far.

Reply