"ImportError: cannot import name moduleTNC" when importing scipy.signal

[Tim_G_1]

Hi,

I installed Conda and the Intel Python distribution today. I am trying to use it with an existing Python script that uses SciPy, and in particular the SciPy 'signal' package.

However, the script crashes on the step of importing scipy.signal:

Traceback (most recent call last):
  File "P:\proj\Design\Electronic\Software\pc_serial_test\arbtest.py",
line 20, in <module>
    import scipy.signal
  File "C:\Python27\lib\site-packages\scipy\signal\__init__.py", line 281, in <m
odule>
    from .cont2discrete import *
  File "C:\Python27\lib\site-packages\scipy\signal\cont2discrete.py", line 12, i
n <module>
    from .ltisys import tf2ss, ss2tf, zpk2ss, ss2zpk
  File "C:\Python27\lib\site-packages\scipy\signal\ltisys.py", line 40, in <modu
le>
    from .filter_design import tf2zpk, zpk2tf, normalize, freqs
  File "C:\Python27\lib\site-packages\scipy\signal\filter_design.py", line 14, i
n <module>
    from scipy import special, optimize
  File "C:\Python27\lib\site-packages\scipy\optimize\__init__.py", line 222, in
<module>
    from ._minimize import *
  File "C:\Python27\lib\site-packages\scipy\optimize\_minimize.py", line 31, in
<module>
    from .tnc import _minimize_tnc
  File "C:\Python27\lib\site-packages\scipy\optimize\tnc.py", line 37, in <modul
e>
    from scipy.optimize import moduleTNC, approx_fprime
ImportError: cannot import name moduleTNC

(idp) P:\HAPTAC_II.5.2\Design\Electronic\Software\pc_serial_test>arbtest.py
Traceback (most recent call last):
  File "P:\HAPTAC_II.5.2\Design\Electronic\Software\pc_serial_test\arbtest.py",
line 20, in <module>
    import scipy.signal
  File "C:\Python27\lib\site-packages\scipy\signal\__init__.py", line 281, in <m
odule>
    from .cont2discrete import *
  File "C:\Python27\lib\site-packages\scipy\signal\cont2discrete.py", line 12, i
n <module>
    from .ltisys import tf2ss, ss2tf, zpk2ss, ss2zpk
  File "C:\Python27\lib\site-packages\scipy\signal\ltisys.py", line 40, in <modu
le>
    from .filter_design import tf2zpk, zpk2tf, normalize, freqs
  File "C:\Python27\lib\site-packages\scipy\signal\filter_design.py", line 14, i
n <module>
    from scipy import special, optimize
  File "C:\Python27\lib\site-packages\scipy\optimize\__init__.py", line 222, in
<module>
    from ._minimize import *
  File "C:\Python27\lib\site-packages\scipy\optimize\_minimize.py", line 31, in
<module>
    from .tnc import _minimize_tnc
  File "C:\Python27\lib\site-packages\scipy\optimize\tnc.py", line 37, in <modul
e>
    from scipy.optimize import moduleTNC, approx_fprime
ImportError: cannot import name moduleTNC

In addition, the respective Conda environment ("idp" per the installation example) was activated before running the script, and the console shows:

(idp) P:\proj\Design\Electronic\Software\pc_serial_test>arbtest.py

However, the tracebacks show paths e.g. C:\Python2.7\..., even though Conda is installed under C:\Users\me\AppData - is this normal?

Other notes:

Conda and Intel Python were installed according to the instructions at https://software.intel.com/en-us/articles/using-intel-distribution-for-python-with-anaconda for Python 2, using miniconda and the core distribution. Conda was newly installed and updated (conda: 4.3.11-py27_0 --> 4.3.17-py27_0) before following the remaining steps.

"conda list" reports the following packages and versions currently installed:

# packages in environment at C:\Users\me\AppData\Local\Continuum\Miniconda2

\envs\idp:
#
icc_rt                    13.1.5                  intel_5  [intel]  intel
intelpython               2017.0.2                      0    intel
intelpython2_core         2017.0.2                      0    intel
mkl                       2017.0.2                intel_1  [intel]  intel
numpy                     1.11.2            py27_intel_51  [intel]  intel
openssl                   1.0.2k              vc9_intel_3  [vc9 intel]  intel
pip                       8.1.2              py27_intel_5  [intel]  intel
python                    2.7.12                 intel_20  [intel]  intel
pyyaml                    3.12               py27_intel_0  [intel]  intel
scipy                     0.18.1          np111py27_intel_10  [intel]  intel
setuptools                23.0.0             py27_intel_4  [intel]  intel
tbb                       2017.0.4           py27_intel_0  [intel]  intel
tcl                       8.6.4              vc9_intel_16  [vc9 intel]  intel
tk                        8.6.4              vc9_intel_18  [vc9 intel]  intel
vc                        9.0                           1    intel
vs2008_runtime            9.00.30729.1            intel_3  [intel]  intel
wheel                     0.29.0             py27_intel_5  [intel]  intel
zlib                      1.2.11              vc9_intel_0  [vc9 intel]  intel

Tried forced reinstall of scipy (conda install -f scipy), no effect.

The file 'moduleTNC.pyd' is present at C:\Users\me\AppData\Local\Continuum\Miniconda2\pkgs\scipy-0.18.1-np111py27_intel_10\Lib\site-packages\scipy\optimize and with the capitalization shown.

Any suggestions?

[Rohit_J_Intel · Accepted solution]

Hi Tim,

Thanks for trying IDP. For my better understanding, could you provide answers to the following:
1. From your activated conda environment (idp), could you run "where python" from the location where you're running this script from, and paste the output?
2. Also, have you tried calling "arbtest.py" with python from the activated environment : "python arbtest.py"? Does that invoke the correct python?

Thanks,
Rohit

[Oleksandr_P_Intel]

Hi Hal, 

Please report which processor you are running on, as well as the output of running

python -c "import scipy.optimize.moduleTNC"

Thank you,
Oleksandr

[Tim_G_1]

Rohit J. (Intel) wrote:

2. Also, have you tried calling "arbtest.py" with python from the activated environment : "python arbtest.py"? Does that invoke the correct python?

Thanks! This was it! Explicitly specifying python on the commandline bypasses the error, and my script is now working. Oddly enough, my reason for trying out the Intel distribution was to resolve a similar missing-dependency issue in another precompiled 64-bit SciPy distribution ("no lapack/blas resources found" / "from ._ufuncs import * ImportError: DLL load failed: %1 is not a valid Win32 application."), but calling it as above failed in the unique way above.

To answer additional questions:

The results of 'where python': 

(idp) P:\proj\Design\Electronic\Software\pc_serial_test>where python
C:\Users\me\AppData\Local\Continuum\Miniconda2\envs\idp\python.exe
C:\Python27\python.exe
C:\Users\me\AppData\Local\Continuum\Miniconda2\python.exe

Result of grepping for 'moduleTNC':

c:\Python27\Lib\site-packages\scipy\optimize\moduletnc.pyd
c:\Users\me\AppData\Local\Continuum\Miniconda2\envs\idp\Lib\site-packages\scipy\optimize\moduleTNC.pyd
c:\Users\me\AppData\Local\Continuum\Miniconda2\pkgs\scipy-0.18.1-np111py27_intel_10\Lib\site-packages\scipy\optimize\moduleTNC.pyd

This suggests maybe it was using a preinstalled default SciPy version from miniconda, not the Intel distribution. Still, I didn't think Miniconda included big packages like SciPy (they say, "These Miniconda installers contain the conda package manager and Python").

Output of python -c "import scipy.optimize.moduleTNC":

Under Conda "idp" environment: No errors / output

Under existing (non-Conda non-Intel) environment:

C:\Users\me>python -c "import scipy.optimize.moduleTNC"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python27\lib\site-packages\scipy\optimize\__init__.py", line 221, in
<module>
    from .optimize import *
  File "C:\Python27\lib\site-packages\scipy\optimize\optimize.py", line 37, in <
module>
    from .linesearch import (line_search_wolfe1, line_search_wolfe2,
  File "C:\Python27\lib\site-packages\scipy\optimize\linesearch.py", line 18, in
 <module>
    from scipy.optimize import minpack2
ImportError: DLL load failed: %1 is not a valid Win32 application.

Note, this is the original issue that caused me to try out the Intel distribution.

Thanks again!

[Tim_G_1]

I did run into one additional snag so far: This script also uses matplotlib, which works correctly under my pre-Intel configuration. However, in the Intel distribution, trying to use it produces the error:

Traceback (most recent call last):
  File "arbtest.py", line 12, in <module>
    import matplotlib.pyplot as plt
  File "C:\Users\me\AppData\Local\Continuum\Miniconda2\envs\idp\lib\site-pa
ckages\matplotlib\pyplot.py", line 114, in <module>
    _backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()

  File "C:\Users\me\AppData\Local\Continuum\Miniconda2\envs\idp\lib\site-pa
ckages\matplotlib\backends\__init__.py", line 32, in pylab_setup
    globals(),locals(),[backend_name],0)
  File "C:\Users\me\AppData\Local\Continuum\Miniconda2\envs\idp\lib\site-pa
ckages\matplotlib\backends\backend_webagg.py", line 30, in <module>
    raise RuntimeError("The WebAgg backend requires Tornado.")
RuntimeError: The WebAgg backend requires Tornado.

Ordinarily, matplotlib auto-selects a backend based on availability, avoiding any with an unmet dependency. To get it working, I had to manually hardcode/force one like so:

import matplotlib
backend = 'TkAgg'
matplotlib.use(backend)
import matplotlib.pyplot as plt

But after that, matplotlib works, at least for the functions this script uses.

[Rohit_J_Intel]

Tim,

It seems that in your conda environment, there's no "tornado". From your activated environment, could you provide the output of "conda list"? In the presence of "tornado" in your environment, you wouldn't have run into this issue. If you're okay with using another backend, that's okay as well.

Thanks,
Rohit

[Tim_G_1]

Rohit J. (Intel) wrote:

It seems that in your conda environment, there's no "tornado". From your activated environment, could you provide the output of "conda list"? In the presence of "tornado" in your environment, you wouldn't have run into this issue. If you're okay with using another backend, that's okay as well.

I understand that; I definitely have not installed Tornado on this system (within the Conda environment or otherwise). My reason for mentioning it is this seems to be a behavioral discrepancy between the Intel and "stock" matplotlib distributions, which automatically choose a supported backend rather than one with special (unmet) dependencies. I've used the stock distribution up to this point without having to install Tornado or force a specific backend.

For my own local use, forcing the backend or installing the dependency are acceptable workarounds. But for eventual distribution of a python-based app to customers, the difference in behavior raises compatibility concerns about using the Intel distribution as a direct 1:1 replacement, so I thought I would mention it.

Thanks!