Intel® Distribution for Python*
Engage in discussions with community peers related to Python* applications and core computational packages.
435 Discussions

GDAL not working with Intel Distribution for Python

soman__kritik
Beginner
‎01-08-2018 01:08 AM
4,456 Views
Solved Jump to solution

I was trying to install GDAL with Intel Distribution for Python on anaconda using 

conda install -c conda-forge gdal 

After installation, I am unable to import gdal, it shows that a label in not present in hdf5 package. I then tried older versions of GDAL, but I was unable to install them due to dependencies on mkl_fft. 

Please update Intel Distribution for Python to support GDAL.

 

 

 

0 Kudos
1 Solution
Rohit_J_Intel
Employee
‎01-09-2018 06:15 AM
4,456 Views
Solved Jump to solution
Hi Kritik,
 
I was able to install gdal from conda-forge into IDP-2.7 using following steps:
<IDP_installation>/bin/conda create -n gdal_env -c intel python=2.7 scikit-learn -y
source <IDP_installation>/bin/activate gdal_env
conda install --no-deps -c conda-forge libgdal -y
conda install -c conda-forge gdal -y

 

For testing if gdal can be imported into the interpreter :
python -c "import gdal"

 

Thanks,
Rohit

View solution in original post

0 Kudos
Copy link

Link Copied

8 Replies
DavidLiu
Employee
‎01-08-2018 12:30 PM
4,456 Views
Solved Jump to solution

Hi Kritik,

Using the Intel® Distribution for Python* with Python 3.6 nets me a very different issue in which libgdal.20.dylib having issues with a dynamic link. Can you post your conda info along with the subsequent error output?

Thanks,

David

0 Kudos
Copy link
soman__kritik
Beginner
‎01-08-2018 09:41 PM
4,456 Views
Solved Jump to solution

Hi David

Thanks for the reply. I am actually trying to perform clustering on geospatial data by using intel’s sklearn. I want to import a geotif file for which I need gdal.

 

I was using python 2.7. I am not able to reproduce the last error that I was facing earlier, but no version of gdal seems to work in Intel distribution for python. Please find below the error I am getting now. My code was running on normal python 2.7. I shall be grateful if you could help setting up any version of gdal with intel for python.

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda install -c intel scikit-learn 

Fetching package metadata .............

Solving package specifications: .

 

Package plan for installation in environment /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo:

 

The following NEW packages will be INSTALLED:

 

    daal:         2018.0.1.20171012-1                 intel

    mkl_fft:      1.0.0-np113py27_intel_15            intel [intel]

    mkl_random:   1.0.0-np113py27_intel_6             intel [intel]

    pydaal:       2018.0.1.20171012-np113py27_intel_0 intel [intel]

    scikit-learn: 0.19.0-np113py27_intel_6            intel [intel]

    scipy:        0.19.1-np113py27_intel_23           intel [intel]

    tbb:          2018.0.1-py27_intel_3               intel [intel]

 

The following packages will be UPDATED:

 

    hdf5:         1.8.15.1-2                                --> 1.10.1-intel_0      intel [intel]

    libnetcdf:    4.3.3.1-3                                 --> 4.5.0-h42fd751_7         

    mkl:          2017.0.2-intel_1                    intel [intel] --> 2018.0.1-intel_5    intel

    numpy:        1.11.3-py27_0                             --> 1.13.3-py27_intel_6 intel [intel]

 

The following packages will be DOWNGRADED:

 

    gdal:         2.0.0-np111py27_3                         --> 1.10.1-py27_1            

    libgdal:      2.0.0-0                                   --> 1.11.2-1                 

 

Proceed (/n)? y

 

daal-2018.0.1. 100% |################################| Time: 0:01:58 653.65 kB/s

mkl-2018.0.1-i 100% |################################| Time: 0:02:30 897.26 kB/s

libnetcdf-4.5. 100% |################################| Time: 0:00:04 241.18 kB/s

tbb-2018.0.1-p 100% |################################| Time: 0:00:02 198.79 kB/s

mkl_fft-1.0.0- 100% |################################| Time: 0:00:00 250.12 kB/s

mkl_random-1.0 100% |################################| Time: 0:00:01 250.54 kB/s

numpy-1.13.3-p 100% |################################| Time: 0:00:10 550.26 kB/s

pydaal-2018.0. 100% |################################| Time: 0:01:02 780.27 kB/s

scipy-0.19.1-n 100% |################################| Time: 0:00:14   1.90 MB/s

scikit-learn-0 100% |################################| Time: 0:00:06 922.40 kB/s

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ python

Python 2.7.14 |Intel Corporation| (default, Oct 16 2017, 10:29:14) 

[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

Intel(R) Distribution for Python is brought to you by Intel Corporation.

Please check out: https://software.intel.com/en-us/python-distribution

>>> import gdal

>>> 

[15]+  Stopped                 python

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ python clustering.py 

Traceback (most recent call last):

  File "clustering.py", line 17, in <module>

    original_dem = original_dem.ReadAsArray()

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdal.py", line 781, in ReadAsArray

    import gdalnumeric

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdalnumeric.py", line 1, in <module>

    from gdal_array import *

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdal_array.py", line 25, in <module>

    _gdal_array = swig_import_helper()

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdal_array.py", line 17, in swig_import_helper

    import _gdal_array

ImportError: No module named _gdal_array

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda remove -n geo gdal

Fetching package metadata ...........

Solving package specifications: .

 

Package plan for package removal in environment /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo:

 

The following packages will be REMOVED:

 

    gdal: 1.10.1-py27_1

 

Proceed (/n)? y

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda install -c conda-forge gdal 

Fetching package metadata .............

Solving package specifications: .

 

Package plan for installation in environment /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo:

 

The following NEW packages will be INSTALLED:

 

    gdal:      2.2.3-py27_0       conda-forge

 

The following packages will be UPDATED:

 

    curl:      7.45.0-0                       --> 7.55.1-0   conda-forge

    freetype:  2.8-intel_0        intel       [intel] --> 2.8.1-0    conda-forge

    glib:      2.53.6-h33f6a65_2              --> 2.55.0-0   conda-forge

    kealib:    1.4.5-0                        --> 1.4.7-4    conda-forge

    libgdal:   1.11.2-1                       --> 2.2.3-2    conda-forge

    poppler:   0.60.1-hb654a48_0              --> 0.61.1-3   conda-forge

 

The following packages will be SUPERSEDED by a higher-priority channel:

 

    cairo:     1.14.10-h913ea44_6             --> 1.14.10-0  conda-forge

    libnetcdf: 4.5.0-h42fd751_7               --> 4.4.1.1-10 conda-forge

 

Proceed (/n)? y

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ python clustering.py 

Traceback (most recent call last):

  File "clustering.py", line 2, in <module>

    import gdal

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/gdal.py", line 2, in <module>

    from osgeo.gdal import deprecation_warn

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/__init__.py", line 21, in <module>

    _gdal = swig_import_helper()

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/__init__.py", line 17, in swig_import_helper

    _mod = imp.load_module('_gdal', fp, pathname, description)

ImportError: dlopen(/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/_gdal.so, 2): Library not loaded: @rpath/libpng16.16.dylib

  Referenced from: /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/libgdal.20.dylib

  Reason: Incompatible library version: libgdal.20.dylib requires version 51.0.0 or later, but libpng16.16.dylib provides version 49.0.0

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda remove -n geo gdal

Fetching package metadata ...........

Solving package specifications: .

 

Package plan for package removal in environment /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo:

 

The following packages will be REMOVED:

 

    gdal: 2.2.3-py27_0 conda-forge

 

Proceed (/n)? y

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda search gdal

Fetching package metadata ...........

gdal                         1.10.1                   py27_0  defaults        

                             1.10.1                   py26_1  defaults        

                             1.10.1                   py27_1  defaults        

                             1.10.1                   py33_1  defaults        

                             1.10.1               np18py26_2  defaults        

                             1.10.1               np18py27_2  defaults        

                             1.10.1               np18py33_2  defaults        

                             1.10.1               np18py34_2  defaults        

                             1.11.0               np19py26_0  defaults        

                             1.11.0               np19py27_0  defaults        

                             1.11.0               np19py33_0  defaults        

                             1.11.0               np19py34_0  defaults        

                             1.11.1               np19py26_0  defaults        

                             1.11.1               np19py27_0  defaults        

                             1.11.1               np19py33_0  defaults        

                             1.11.1               np19py34_0  defaults        

                             1.11.1               np19py26_1  defaults        

                             1.11.1               np19py27_1  defaults        

                             1.11.1               np19py33_1  defaults        

                             1.11.1               np19py34_1  defaults        

                             1.11.2               np19py26_0  defaults        

                             1.11.2               np19py27_0  defaults        

                             1.11.2               np19py33_0  defaults        

                             1.11.2               np19py34_0  defaults        

                             1.11.2               np19py26_1  defaults        

                             1.11.2               np19py27_1  defaults        

                             1.11.2               np19py33_1  defaults        

                             1.11.2               np19py34_1  defaults        

                             1.11.2               np19py26_2  defaults        

                             1.11.2               np19py27_2  defaults        

                             1.11.2               np19py33_2  defaults        

                             1.11.2               np19py34_2  defaults        

                             1.11.2               np19py26_3  defaults        

                             1.11.2               np19py27_3  defaults        

                             1.11.2               np19py33_3  defaults        

                             1.11.2               np19py34_3  defaults        

                             2.0.0               np110py27_0  defaults        

                             2.0.0               np110py34_0  defaults        

                             2.0.0               np110py35_0  defaults        

                             2.0.0                np19py26_0  defaults        

                             2.0.0                np19py27_0  defaults        

                             2.0.0                np19py33_0  defaults        

                             2.0.0                np19py34_0  defaults        

                             2.0.0                np19py35_0  defaults        

                             2.0.0               np110py27_3  defaults        

                             2.0.0               np110py34_3  defaults        

                             2.0.0               np110py35_3  defaults        

                             2.0.0               np111py27_3  defaults        

                             2.0.0               np111py34_3  defaults        

                             2.0.0               np111py35_3  defaults        

                             2.1.0                    py27_0  defaults        

                             2.1.0                    py34_0  defaults        

                             2.1.0                    py35_0  defaults        

                             2.2.2            py27h98c8286_0  defaults        

                             2.2.2            py36hff4f997_0  defaults        

                             2.2.2            py35h90dc150_0  defaults        

                             2.2.2            py36h503167d_0  defaults        

                             2.2.2            py35hd88591f_0  defaults        

                             2.2.2            py27hc303eaf_0  defaults        

                             2.2.2            py27h65d9da8_1  defaults        

                             2.2.2            py36h17ea0c9_1  defaults        

                             2.2.2            py35he67b238_1  defaults        

                             2.2.2            py35h7aa31b1_1  defaults        

                             2.2.2            py27h9d2a75c_1  defaults        

                             2.2.2            py36hd505dc6_1  defaults        

libgdal                      1.11.2                        0  defaults        

                             1.11.2                        1  defaults        

                             1.11.2                        2  defaults        

                             2.0.0                         0  defaults        

                             2.0.0                         4  defaults        

                             2.1.0                         0  defaults        

                             2.2.2                h2c60913_0  defaults        

                             2.2.2                h3aae6cd_0  defaults        

                             2.2.2                h7ea3a73_0  defaults        

                             2.2.2                hdcd1c41_0  defaults        

                             2.2.2                ha430587_1  defaults        

                             2.2.2                h839ae26_1  defaults        

                             2.2.2                h3559a57_1  defaults        

                             2.2.2                hc3eca24_1  defaults        

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda install gdal=2.0.0

Fetching package metadata ...........

Solving package specifications: .

 

UnsatisfiableError: The following specifications were found to be in conflict:

  - gdal 2.0.0*

  - pydaal

Use "conda info <package>" to see the dependencies for each package.

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda install gdal=2.1.0

Fetching package metadata ...........

Solving package specifications: .

 

UnsatisfiableError: The following specifications were found to be in conflict:

  - gdal 2.1.0* -> libgdal 2.1.0 -> geos 3.5.*

  - gdal 2.1.0* -> libgdal 2.1.0 -> proj4 4.9.2

  - libspatialite -> geos >=3.6.2,<3.6.3.0a0

Use "conda info <package>" to see the dependencies for each package.

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda install gdal=1.11.2

Fetching package metadata ...........

Solving package specifications: .

 

UnsatisfiableError: The following specifications were found to be in conflict:

  - gdal 1.11.2*

  - pydaal

Use "conda info <package>" to see the dependencies for each package.

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ conda install gdal=1.10.1

Fetching package metadata ...........

Solving package specifications: .

 

Package plan for installation in environment /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo:

 

The following NEW packages will be INSTALLED:

 

    gdal: 1.10.1-py27_1

 

Proceed (/n)? y

 

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ python clustering.py 

Traceback (most recent call last):

  File "clustering.py", line 17, in <module>

    original_dem = original_dem.ReadAsArray()

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdal.py", line 781, in ReadAsArray

    import gdalnumeric

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdalnumeric.py", line 1, in <module>

    from gdal_array import *

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdal_array.py", line 25, in <module>

    _gdal_array = swig_import_helper()

  File "/Volumes/MacBookData/kritiksoman/anaconda3/envs/geo/lib/python2.7/site-packages/osgeo/gdal_array.py", line 17, in swig_import_helper

    import _gdal_array

ImportError: No module named _gdal_array

(geo) Kritiks-MacBook-Air:clustering kritiksoman$  conda info

Current conda install:

 

               platform : osx-64

          conda version : 4.3.30

       conda is private : False

      conda-env version : 4.3.30

    conda-build version : 3.0.27

         python version : 3.6.3.final.0

       requests version : 2.18.4

       root environment : /Volumes/MacBookData/kritiksoman/anaconda3  (writable)

    default environment : /Volumes/MacBookData/kritiksoman/anaconda3/envs/geo

       envs directories : /Volumes/MacBookData/kritiksoman/anaconda3/envs

                          /Volumes/MacBookData/kritiksoman/.conda/envs

          package cache : /Volumes/MacBookData/kritiksoman/anaconda3/pkgs

                          /Volumes/MacBookData/kritiksoman/.conda/pkgs

           channel URLs : https://repo.continuum.io/pkgs/main/osx-64

                          https://repo.continuum.io/pkgs/main/noarch

                          https://repo.continuum.io/pkgs/free/osx-64

                          https://repo.continuum.io/pkgs/free/noarch

                          https://repo.continuum.io/pkgs/r/osx-64

                          https://repo.continuum.io/pkgs/r/noarch

                          https://repo.continuum.io/pkgs/pro/osx-64

                          https://repo.continuum.io/pkgs/pro/noarch

            config file : /Volumes/MacBookData/kritiksoman/.condarc

             netrc file : None

           offline mode : False

             user-agent : conda/4.3.30 requests/2.18.4 CPython/3.6.3 Darwin/17.3.0 OSX/10.13.2    

                UID:GID : 501:20

(geo) Kritiks-MacBook-Air:clustering kritiksoman$ 

 

0 Kudos
Copy link
Rohit_J_Intel
Employee
‎01-09-2018 06:15 AM
4,457 Views
Solved Jump to solution
Hi Kritik,
 
I was able to install gdal from conda-forge into IDP-2.7 using following steps:
<IDP_installation>/bin/conda create -n gdal_env -c intel python=2.7 scikit-learn -y
source <IDP_installation>/bin/activate gdal_env
conda install --no-deps -c conda-forge libgdal -y
conda install -c conda-forge gdal -y

 

For testing if gdal can be imported into the interpreter :
python -c "import gdal"

 

Thanks,
Rohit
0 Kudos
Copy link
soman__kritik
Beginner
‎01-10-2018 08:58 PM
4,456 Views
Solved Jump to solution

Hi Rohit

Thanks for the help. I also want to install opencv, matplotlib, skimage, and scipy after sklearn and gdal. But on doing so, I am running into the following errors

1) Incompatible library version: libgdal.20.dylib requires version 51.0.0 or later, but libpng16.16.dylib provides version 49.0.0

2)

conda install -c intel scikit-image
Fetching package metadata .............
Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:
  - geotiff
  - harfbuzz
  - libspatialite
  - mkl_fft
  - scikit-image
Use "conda info <package>" to see the dependencies for each package.

Please advise how these can be resolved.

Thanks

Kritik

Rohit J. (Intel) wrote:

Hi Kritik,

 

I was able to install gdal from conda-forge into IDP-2.7 using following steps:

<IDP_installation>/bin/conda create -n gdal_env -c intel python=2.7 scikit-learn -y
source <IDP_installation>/bin/activate gdal_env
conda install --no-deps -c conda-forge libgdal -y
conda install -c conda-forge gdal -y

 

For testing if gdal can be imported into the interpreter :

python -c "import gdal"

 

Thanks,

Rohit

0 Kudos
Copy link
Rohit_J_Intel
Employee
‎01-11-2018 05:44 AM
4,456 Views
Solved Jump to solution
Hi Kritik,
 
Since you have "gdal_env" already setup with scikit-learn, scipy is already installed in your environment. We currently don't ship scikit-image and opencv on OSX, but you can install all your requirements through:
source <IDP_installation>/bin/activate gdal_env
conda install -c intel matplotlib -y
conda install -c conda-forge opencv scikit-image -y

 

Thanks,
Rohit
0 Kudos
Copy link
soman__kritik
Beginner
‎01-11-2018 06:59 AM
4,456 Views
Solved Jump to solution

Hi Rohit

When I run 

conda install -c conda-forge opencv scikit-image -y

gdal 2.2.2 downgrades to 1.10.1, which is a problem because it gives error:ImportError: No module named _gdal_array

I want to run gdal >=2.2. I am attaching the terminal log.

Thanks

Kritik

Rohit J. (Intel) wrote:

Hi Kritik,

 

Since you have "gdal_env" already setup with scikit-learn, scipy is already installed in your environment. We currently don't ship scikit-image and opencv on OSX, but you can install all your requirements through:

source <IDP_installation>/bin/activate gdal_env
conda install -c intel matplotlib -y
conda install -c conda-forge opencv scikit-image -y

 

Thanks,

Rohit

Terminal log.txt
(Virus scan in progress ...)
0 Kudos
Copy link
Rohit_J_Intel
Employee
‎01-11-2018 11:41 AM
4,456 Views
Solved Jump to solution
Hi Kritik,
 
It seems that opencv (from conda-forge) causes some conflict with one of dependencies of gdal. The following is a bit awkward but it should get what you require:
 
<IDP_installation>/bin/conda create -n gdal_env2 -c intel python=2.7 matplotlib scikit-learn -y
source <IDP_installation>/bin/activate gdal_env2
conda install -c conda-forge opencv scikit-image gdal -y

This should give you gdal==2.2.3 and libgdal==2.2.3.

 
Thanks,
Rohit
0 Kudos
Copy link
soman__kritik
Beginner
‎01-20-2018 08:05 AM
4,456 Views
Solved Jump to solution

Hi Rohit

Thanks for the response. Your suggestion works well.

Thanks

Kritik

Rohit J. (Intel) wrote:

Hi Kritik,

 

It seems that opencv (from conda-forge) causes some conflict with one of dependencies of gdal. The following is a bit awkward but it should get what you require:

 

<IDP_installation>/bin/conda create -n gdal_env2 -c intel python=2.7 matplotlib scikit-learn -y
source <IDP_installation>/bin/activate gdal_env2
conda install -c conda-forge opencv scikit-image gdal -y

This should give you gdal==2.2.3 and libgdal==2.2.3.

 

Thanks,

Rohit

0 Kudos
Copy link
Reply

Community support is provided Monday to Friday. Other contact methods are available here.

Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

For more complete information about compiler optimizations, see our Optimization Notice.