Intel® Distribution for Python*
Engage in discussions with community peers related to Python* applications and core computational packages.
423 Discussions

GDAL not working with Intel Distribution for Python

soman__kritik
Beginner
2,614 Views

I was trying to install GDAL with Intel Distribution for Python on anaconda using 

conda install -c conda-forge gdal 

After installation, I am unable to import gdal, it shows that a label in not present in hdf5 package. I then tried older versions of GDAL, but I was unable to install them due to dependencies on mkl_fft. 

Please update Intel Distribution for Python to support GDAL.

 

 

 

0 Kudos
1 Solution
Rohit_J_Intel
Employee
2,614 Views
Hi Kritik,
 
I was able to install gdal from conda-forge into IDP-2.7 using following steps:
<IDP_installation>/bin/conda create -n gdal_env -c intel python=2.7 scikit-learn -y
source <IDP_installation>/bin/activate gdal_env
conda install --no-deps -c conda-forge libgdal -y
conda install -c conda-forge gdal -y

 

For testing if gdal can be imported into the interpreter :
python -c "import gdal"

 

Thanks,
Rohit

View solution in original post

0 Kudos
8 Replies
DavidLiu
Employee
2,614 Views

Hi Kritik,

Using the Intel® Distribution for Python* with Python 3.6 nets me a very different issue in which libgdal.20.dylib having issues with a dynamic link. Can you post your conda info along with the subsequent error output?

Thanks,

David

0 Kudos
soman__kritik
Beginner
2,614 Views

Hi David

Thanks for the reply. I am actually trying to perform clustering on geospatial data by using intel’s sklearn. I want to import a geotif file for which I need gdal.

 

I was using python 2.7. I am not able to reproduce the last error that I was facing earlier, but no version of gdal seems to work in Intel distribution for python. Please find below the error I am getting now. My code was running on normal python 2.7. I shall be grateful if you could help setting up any version of gdal with intel for python.

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda install -c intel scikit-learn 

Fetching package metadata .............

Solving package specifications: .

 

Package plan for installation in environment /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo:

 

The following NEW packages will be INSTALLED:

 

    daal:         2018.0.1.20171012-1                 intel

    mkl_fft:      1.0.0-np113py27_intel_15            intel [intel]

    mkl_random:   1.0.0-np113py27_intel_6             intel [intel]

    pydaal:       2018.0.1.20171012-np113py27_intel_0 intel [intel]

    scikit-learn: 0.19.0-np113py27_intel_6            intel [intel]

    scipy:        0.19.1-np113py27_intel_23           intel [intel]

    tbb:          2018.0.1-py27_intel_3               intel [intel]

 

The following packages will be UPDATED:

 

    hdf5:         1.8.15.1-2                                --> 1.10.1-intel_0      intel [intel]

    libnetcdf:    4.3.3.1-3                                 --> 4.5.0-h42fd751_7         

    mkl:          2017.0.2-intel_1                    intel [intel] --> 2018.0.1-intel_5    intel

    numpy:        1.11.3-py27_0                             --> 1.13.3-py27_intel_6 intel [intel]

 

The following packages will be DOWNGRADED:

 

    gdal:         2.0.0-np111py27_3                         --> 1.10.1-py27_1            

    libgdal:      2.0.0-0                                   --> 1.11.2-1                 

 

Proceed (/n)? y

 

daal-2018.0.1. 100% |################################| Time: 0:01:58 653.65 kB/s

mkl-2018.0.1-i 100% |################################| Time: 0:02:30 897.26 kB/s

libnetcdf-4.5. 100% |################################| Time: 0:00:04 241.18 kB/s

tbb-2018.0.1-p 100% |################################| Time: 0:00:02 198.79 kB/s

mkl_fft-1.0.0- 100% |################################| Time: 0:00:00 250.12 kB/s

mkl_random-1.0 100% |################################| Time: 0:00:01 250.54 kB/s

numpy-1.13.3-p 100% |################################| Time: 0:00:10 550.26 kB/s

pydaal-2018.0. 100% |################################| Time: 0:01:02 780.27 kB/s

scipy-0.19.1-n 100% |################################| Time: 0:00:14   1.90 MB/s

scikit-learn-0 100% |################################| Time: 0:00:06 922.40 kB/s

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ python

Python 2.7.14 |Intel Corporation| (default, Oct 16 2017, 10:29:14) 

[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

Intel(R) Distribution for Python is brought to you by Intel Corporation.

Please check out: https://software.intel.com/en-us/python-distribution

>>> import gdal

>>> 

[15]+  Stopped                 python

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ python clustering.py 

Traceback (most recent call last):

  File "clustering.py", line 17, in <module>

    original_dem = original_dem.ReadAsArray()

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdal.py", line 781, in ReadAsArray

    import gdalnumeric

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdalnumeric.py", line 1, in <module>

    from gdal_array import *

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdal_array.py", line 25, in <module>

    _gdal_array = swig_import_helper()

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdal_array.py", line 17, in swig_import_helper

    import _gdal_array

ImportError: No module named _gdal_array

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda remove -n geo gdal

Fetching package metadata ...........

Solving package specifications: .

 

Package plan for package removal in environment /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo:

 

The following packages will be REMOVED:

 

    gdal: 1.10.1-py27_1

 

Proceed (/n)? y

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda install -c conda-forge gdal 

Fetching package metadata .............

Solving package specifications: .

 

Package plan for installation in environment /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo:

 

The following NEW packages will be INSTALLED:

 

    gdal:      2.2.3-py27_0       conda-forge

 

The following packages will be UPDATED:

 

    curl:      7.45.0-0                       --> 7.55.1-0   conda-forge

    freetype:  2.8-intel_0        intel       [intel] --> 2.8.1-0    conda-forge

    glib:      2.53.6-h33f6a65_2              --> 2.55.0-0   conda-forge

    kealib:    1.4.5-0                        --> 1.4.7-4    conda-forge

    libgdal:   1.11.2-1                       --> 2.2.3-2    conda-forge

    poppler:   0.60.1-hb654a48_0              --> 0.61.1-3   conda-forge

 

The following packages will be SUPERSEDED by a higher-priority channel:

 

    cairo:     1.14.10-h913ea44_6             --> 1.14.10-0  conda-forge

    libnetcdf: 4.5.0-h42fd751_7               --> 4.4.1.1-10 conda-forge

 

Proceed (/n)? y

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ python clustering.py 

Traceback (most recent call last):

  File "clustering.py", line 2, in <module>

    import gdal

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/gdal.py", line 2, in <module>

    from osgeo.gdal import deprecation_warn

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/__init__.py", line 21, in <module>

    _gdal = swig_import_helper()

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/__init__.py", line 17, in swig_import_helper

    _mod = imp.load_module('_gdal', fp, pathname, description)

ImportError: dlopen(/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/_gdal.so, 2): Library not loaded: @rpath/libpng16.16.dylib

  Referenced from: /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/libgdal.20.dylib

  Reason: Incompatible library version: libgdal.20.dylib requires version 51.0.0 or later, but libpng16.16.dylib provides version 49.0.0

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda remove -n geo gdal

Fetching package metadata ...........

Solving package specifications: .

 

Package plan for package removal in environment /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo:

 

The following packages will be REMOVED:

 

    gdal: 2.2.3-py27_0 conda-forge

 

Proceed (/n)? y

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda search gdal

Fetching package metadata ...........

gdal                         1.10.1                   py27_0  defaults        

                             1.10.1                   py26_1  defaults        

                             1.10.1                   py27_1  defaults        

                             1.10.1                   py33_1  defaults        

                             1.10.1               np18py26_2  defaults        

                             1.10.1               np18py27_2  defaults        

                             1.10.1               np18py33_2  defaults        

                             1.10.1               np18py34_2  defaults        

                             1.11.0               np19py26_0  defaults        

                             1.11.0               np19py27_0  defaults        

                             1.11.0               np19py33_0  defaults        

                             1.11.0               np19py34_0  defaults        

                             1.11.1               np19py26_0  defaults        

                             1.11.1               np19py27_0  defaults        

                             1.11.1               np19py33_0  defaults        

                             1.11.1               np19py34_0  defaults        

                             1.11.1               np19py26_1  defaults        

                             1.11.1               np19py27_1  defaults        

                             1.11.1               np19py33_1  defaults        

                             1.11.1               np19py34_1  defaults        

                             1.11.2               np19py26_0  defaults        

                             1.11.2               np19py27_0  defaults        

                             1.11.2               np19py33_0  defaults        

                             1.11.2               np19py34_0  defaults        

                             1.11.2               np19py26_1  defaults        

                             1.11.2               np19py27_1  defaults        

                             1.11.2               np19py33_1  defaults        

                             1.11.2               np19py34_1  defaults        

                             1.11.2               np19py26_2  defaults        

                             1.11.2               np19py27_2  defaults        

                             1.11.2               np19py33_2  defaults        

                             1.11.2               np19py34_2  defaults        

                             1.11.2               np19py26_3  defaults        

                             1.11.2               np19py27_3  defaults        

                             1.11.2               np19py33_3  defaults        

                             1.11.2               np19py34_3  defaults        

                             2.0.0               np110py27_0  defaults        

                             2.0.0               np110py34_0  defaults        

                             2.0.0               np110py35_0  defaults        

                             2.0.0                np19py26_0  defaults        

                             2.0.0                np19py27_0  defaults        

                             2.0.0                np19py33_0  defaults        

                             2.0.0                np19py34_0  defaults        

                             2.0.0                np19py35_0  defaults        

                             2.0.0               np110py27_3  defaults        

                             2.0.0               np110py34_3  defaults        

                             2.0.0               np110py35_3  defaults        

                             2.0.0               np111py27_3  defaults        

                             2.0.0               np111py34_3  defaults        

                             2.0.0               np111py35_3  defaults        

                             2.1.0                    py27_0  defaults        

                             2.1.0                    py34_0  defaults        

                             2.1.0                    py35_0  defaults        

                             2.2.2            py27h98c8286_0  defaults        

                             2.2.2            py36hff4f997_0  defaults        

                             2.2.2            py35h90dc150_0  defaults        

                             2.2.2            py36h503167d_0  defaults        

                             2.2.2            py35hd88591f_0  defaults        

                             2.2.2            py27hc303eaf_0  defaults        

                             2.2.2            py27h65d9da8_1  defaults        

                             2.2.2            py36h17ea0c9_1  defaults        

                             2.2.2            py35he67b238_1  defaults        

                             2.2.2            py35h7aa31b1_1  defaults        

                             2.2.2            py27h9d2a75c_1  defaults        

                             2.2.2            py36hd505dc6_1  defaults        

libgdal                      1.11.2                        0  defaults        

                             1.11.2                        1  defaults        

                             1.11.2                        2  defaults        

                             2.0.0                         0  defaults        

                             2.0.0                         4  defaults        

                             2.1.0                         0  defaults        

                             2.2.2                h2c60913_0  defaults        

                             2.2.2                h3aae6cd_0  defaults        

                             2.2.2                h7ea3a73_0  defaults        

                             2.2.2                hdcd1c41_0  defaults        

                             2.2.2                ha430587_1  defaults        

                             2.2.2                h839ae26_1  defaults        

                             2.2.2                h3559a57_1  defaults        

                             2.2.2                hc3eca24_1  defaults        

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda install gdal=2.0.0

Fetching package metadata ...........

Solving package specifications: .

 

UnsatisfiableError: The following specifications were found to be in conflict:

  - gdal 2.0.0*

  - pydaal

Use "conda info <package>" to see the dependencies for each package.

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda install gdal=2.1.0

Fetching package metadata ...........

Solving package specifications: .

 

UnsatisfiableError: The following specifications were found to be in conflict:

  - gdal 2.1.0* -> libgdal 2.1.0 -> geos 3.5.*

  - gdal 2.1.0* -> libgdal 2.1.0 -> proj4 4.9.2

  - libspatialite -> geos >=3.6.2,<3.6.3.0a0

Use "conda info <package>" to see the dependencies for each package.

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda install gdal=1.11.2

Fetching package metadata ...........

Solving package specifications: .

 

UnsatisfiableError: The following specifications were found to be in conflict:

  - gdal 1.11.2*

  - pydaal

Use "conda info <package>" to see the dependencies for each package.

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda install gdal=1.10.1

Fetching package metadata ...........

Solving package specifications: .

 

Package plan for installation in environment /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo:

 

The following NEW packages will be INSTALLED:

 

    gdal: 1.10.1-py27_1

 

Proceed (/n)? y

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ python clustering.py 

Traceback (most recent call last):

  File "clustering.py", line 17, in <module>

    original_dem = original_dem.ReadAsArray()

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdal.py", line 781, in ReadAsArray

    import gdalnumeric

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdalnumeric.py", line 1, in <module>

    from gdal_array import *

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdal_array.py", line 25, in <module>

    _gdal_array = swig_import_helper()

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdal_array.py", line 17, in swig_import_helper

    import _gdal_array

ImportError: No module named _gdal_array

(geo) Kritiks-MacBook-Air:clustering kritiksoman$  conda info

Current conda install:

 

               platform : osx-64

          conda version : 4.3.30

       conda is private : False

      conda-env version : 4.3.30

    conda-build version : 3.0.27

         python version : 3.6.3.final.0

       requests version : 2.18.4

       root environment : /Volumes/MacBookData/kritiksoman/anaconda3  (writable)

    default environment : /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo

       envs directories : /Volumes/MacBookData/kritiksoman/anaconda3/envs

                          /Volumes/MacBookData/kritiksoman/.conda/envs

          package cache : /Volumes/MacBookData/kritiksoman/anaconda3/pkgs

                          /Volumes/MacBookData/kritiksoman/.conda/pkgs

           channel URLs : https://repo.continuum.io/pkgs/main/osx-64

                          https://repo.continuum.io/pkgs/main/noarch

                          https://repo.continuum.io/pkgs/free/osx-64

                          https://repo.continuum.io/pkgs/free/noarch

                          https://repo.continuum.io/pkgs/r/osx-64

                          https://repo.continuum.io/pkgs/r/noarch

                          https://repo.continuum.io/pkgs/pro/osx-64

                          https://repo.continuum.io/pkgs/pro/noarch

            config file : /Volumes/MacBookData/kritiksoman/.condarc

             netrc file : None

           offline mode : False

             user-agent : conda/4.3.30 requests/2.18.4 CPython/3.6.3 Darwin/17.3.0 OSX/10.13.2    

                UID:GID : 501:20

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ 

 

0 Kudos
Rohit_J_Intel
Employee
2,615 Views
Hi Kritik,
 
I was able to install gdal from conda-forge into IDP-2.7 using following steps:
<IDP_installation>/bin/conda create -n gdal_env -c intel python=2.7 scikit-learn -y
source <IDP_installation>/bin/activate gdal_env
conda install --no-deps -c conda-forge libgdal -y
conda install -c conda-forge gdal -y

 

For testing if gdal can be imported into the interpreter :
python -c "import gdal"

 

Thanks,
Rohit
0 Kudos
soman__kritik
Beginner
2,614 Views

Hi Rohit

Thanks for the help. I also want to install opencv, matplotlib, skimage, and scipy after sklearn and gdal. But on doing so, I am running into the following errors

1) Incompatible library version: libgdal.20.dylib requires version 51.0.0 or later, but libpng16.16.dylib provides version 49.0.0

2)

conda install -c intel scikit-image
Fetching package metadata .............
Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:
  - geotiff
  - harfbuzz
  - libspatialite
  - mkl_fft
  - scikit-image
Use "conda info <package>" to see the dependencies for each package.

Please advise how these can be resolved.

Thanks

Kritik

Rohit J. (Intel) wrote:

Hi Kritik,

 

I was able to install gdal from conda-forge into IDP-2.7 using following steps:

<IDP_installation>/bin/conda create -n gdal_env -c intel python=2.7 scikit-learn -y
source <IDP_installation>/bin/activate gdal_env
conda install --no-deps -c conda-forge libgdal -y
conda install -c conda-forge gdal -y

 

For testing if gdal can be imported into the interpreter :

python -c "import gdal"

 

Thanks,

Rohit

0 Kudos
Rohit_J_Intel
Employee
2,614 Views
Hi Kritik,
 
Since you have "gdal_env" already setup with scikit-learn, scipy is already installed in your environment. We currently don't ship scikit-image and opencv on OSX, but you can install all your requirements through:
source <IDP_installation>/bin/activate gdal_env
conda install -c intel matplotlib -y
conda install -c conda-forge opencv scikit-image -y

 

Thanks,
Rohit
0 Kudos
soman__kritik
Beginner
2,614 Views

Hi Rohit

When I run 

conda install -c conda-forge opencv scikit-image -y

gdal 2.2.2 downgrades to 1.10.1, which is a problem because it gives error:ImportError: No module named _gdal_array

I want to run gdal >=2.2. I am attaching the terminal log.

Thanks

Kritik

Rohit J. (Intel) wrote:

Hi Kritik,

 

Since you have "gdal_env" already setup with scikit-learn, scipy is already installed in your environment. We currently don't ship scikit-image and opencv on OSX, but you can install all your requirements through:

source <IDP_installation>/bin/activate gdal_env
conda install -c intel matplotlib -y
conda install -c conda-forge opencv scikit-image -y

 

Thanks,

Rohit

0 Kudos
Rohit_J_Intel
Employee
2,614 Views
Hi Kritik,
 
It seems that opencv (from conda-forge) causes some conflict with one of dependencies of gdal. The following is a bit awkward but it should get what you require:
 
<IDP_installation>/bin/conda create -n gdal_env2 -c intel python=2.7 matplotlib scikit-learn -y
source <IDP_installation>/bin/activate gdal_env2
conda install -c conda-forge opencv scikit-image gdal -y

This should give you gdal==2.2.3 and libgdal==2.2.3.

 
Thanks,
Rohit
0 Kudos
soman__kritik
Beginner
2,614 Views

Hi Rohit

Thanks for the response. Your suggestion works well.

Thanks

Kritik

Rohit J. (Intel) wrote:

Hi Kritik,

 

It seems that opencv (from conda-forge) causes some conflict with one of dependencies of gdal. The following is a bit awkward but it should get what you require:

 

<IDP_installation>/bin/conda create -n gdal_env2 -c intel python=2.7 matplotlib scikit-learn -y
source <IDP_installation>/bin/activate gdal_env2
conda install -c conda-forge opencv scikit-image gdal -y

This should give you gdal==2.2.3 and libgdal==2.2.3.

 

Thanks,

Rohit

0 Kudos
Reply