autosklearn install crashes

[JohnWestlund]

I've installed autosklearn locally using:

pip install auto-sklearn

But I get the following error whenever I try and use the fit function:

import autosklearn
import autosklearn.classification

#define the model
automl = autosklearn.classification.AutoSklearnClassifier(time_left_for_this_task=1200, n_jobs=1)

#train the model
automl.fit(features_train, target_train)

This is the error:

[ERROR] [2022-12-05 14:38:09,724:Client-AutoML(1):63847a06-74ed-11ed-b04d-5dc3c51ac30b] (' Dummy prediction failed with run state StatusType.CRASHED and additional output: {\'traceback\': \'Traceback (most recent call last):\\n  File "/home/u157922/.local/lib/python3.9/site-packages/autosklearn/evaluation/__init__.py", line 397, in run\\n    obj(**obj_kwargs)\\n  File "/home/u157922/.local/lib/python3.9/site-packages/pynisher/limit_function_call.py", line 302, in __call__\\n    tmp_dir.cleanup()\\n  File "/opt/intel/inteloneapi/intelpython/latest/lib/python3.9/tempfile.py", line 834, in cleanup\\n    self._rmtree(self.name)\\n  File "/opt/intel/inteloneapi/intelpython/latest/lib/python3.9/tempfile.py", line 816, in _rmtree\\n    _shutil.rmtree(name, onerror=onerror)\\n  File "/opt/intel/inteloneapi/intelpython/latest/lib/python3.9/shutil.py", line 740, in rmtree\\n    onerror(os.rmdir, path, sys.exc_info())\\n  File "/opt/intel/inteloneapi/intelpython/latest/lib/python3.9/shutil.py", line 738, in rmtree\\n    os.rmdir(path)\\nOSError: [Errno 39] Directory not empty: \\\'/home/u157922/tmp/tmpl7sxjn6a\\\'\\n\', \'error\': "OSError(39, \'Directory not empty\')"}.',)
[ERROR] [2022-12-05 14:38:09,724:Client-AutoML(1):63847a06-74ed-11ed-b04d-5dc3c51ac30b] (' Dummy prediction failed with run state StatusType.CRASHED and additional output: {\'traceback\': \'Traceback (most recent call last):\\n  File "/home/u157922/.local/lib/python3.9/site-packages/autosklearn/evaluation/__init__.py", line 397, in run\\n    obj(**obj_kwargs)\\n  File "/home/u157922/.local/lib/python3.9/site-packages/pynisher/limit_function_call.py", line 302, in __call__\\n    tmp_dir.cleanup()\\n  File "/opt/intel/inteloneapi/intelpython/latest/lib/python3.9/tempfile.py", line 834, in cleanup\\n    self._rmtree(self.name)\\n  File "/opt/intel/inteloneapi/intelpython/latest/lib/python3.9/tempfile.py", line 816, in _rmtree\\n    _shutil.rmtree(name, onerror=onerror)\\n  File "/opt/intel/inteloneapi/intelpython/latest/lib/python3.9/shutil.py", line 740, in rmtree\\n    onerror(os.rmdir, path, sys.exc_info())\\n  File "/opt/intel/inteloneapi/intelpython/latest/lib/python3.9/shutil.py", line 738, in rmtree\\n    os.rmdir(path)\\nOSError: [Errno 39] Directory not empty: \\\'/home/u157922/tmp/tmpl7sxjn6a\\\'\\n\', \'error\': "OSError(39, \'Directory not empty\')"}.',)
Traceback (most recent call last):
  File "/home/u157922/.local/lib/python3.9/site-packages/autosklearn/automl.py", line 765, in fit
    self._do_dummy_prediction()
  File "/home/u157922/.local/lib/python3.9/site-packages/autosklearn/automl.py", line 489, in _do_dummy_prediction
    raise ValueError(msg)
ValueError: (' Dummy prediction failed with run state StatusType.CRASHED and additional output: {\'traceback\': \'Traceback (most recent call last):\\n  File "/home/u157922/.local/lib/python3.9/site-packages/autosklearn/evaluation/__init__.py", line 397, in run\\n    obj(**obj_kwargs)\\n  File "/home/u157922/.local/lib/python3.9/site-packages/pynisher/limit_function_call.py", line 302, in __call__\\n    tmp_dir.cleanup()\\n  File "/opt/intel/inteloneapi/intelpython/latest/lib/python3.9/tempfile.py", line 834, in cleanup\\n    self._rmtree(self.name)\\n  File "/opt/intel/inteloneapi/intelpython/latest/lib/python3.9/tempfile.py", line 816, in _rmtree\\n    _shutil.rmtree(name, onerror=onerror)\\n  File "/opt/intel/inteloneapi/intelpython/latest/lib/python3.9/shutil.py", line 740, in rmtree\\n    onerror(os.rmdir, path, sys.exc_info())\\n  File "/opt/intel/inteloneapi/intelpython/latest/lib/python3.9/shutil.py", line 738, in rmtree\\n    os.rmdir(path)\\nOSError: [Errno 39] Directory not empty: \\\'/home/u157922/tmp/tmpl7sxjn6a\\\'\\n\', \'error\': "OSError(39, \'Directory not empty\')"}.',) 

 I've tried clearing out the 'tmp' directory -- but that does not resolve or change the error.

[JaideepK_Intel]

Hi,

Thank you for posting in Intel Communities.

We were able to reproduce your issue from our end, we are working on this internally and get back to you with an update.

Thanks,

Jaideep

[JohnWestlund]

Thanks Jaideep. I appreciate the quick response.

[JohnWestlund]

Looks like something was modified in my environment because auto-sklearn was working today. I then tried to add PipelineProfiler:

pip install PipelineProfiler

And now auto-sklearn is broken again.

Ideally, both would be available by default. But at a minimum, I'd like to understand the steps to install and use them successfully.

[JohnWestlund]

@JaideepK_Intel Hope all is well. Curious if there is an update on this issue. Thanks!!

[JaideepK_Intel]

Hi,

Sorry for the delay. We are going to contact you privately via mail. Have a great day ahead.

Regards,

Jaideep

[JaideepK_Intel]

Hi,

Good day to you,

We are going to contact you privately.

Thanks,

Jaideep

[JaideepK_Intel]

Hi,

Good day to you.

As your issue is resolved can we close this thread?

Thanks,

Jaideep

[JohnWestlund]

Hi @JaideepK_Intel ,

Can we get the resolution posted on the forum for other people to see?

Best,

John

[JaideepK_Intel]

Hi,

We are working with the Dev team, Soon we will update with an additional comment and add the notebook file.

Thanks,

Jaideep

[JaideepK_Intel · Accepted solution]

Hi,

I hope you are doing well.

Attaching workaround notebook for your reference. If this resolves your issue, make sure to accept this as a solution. This would help others with similar issue. Thank you!

Thanks,

Jaideep

[JaideepK_Intel]

Hi,

Glad to know that your issue is resolved. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Thanks,

Jaideep