Compiler issues compiling pybind 11

[ShmarvDogg]

When attempting to compile a c++ project intended to be a python module made using pybind 11, intels compiler fails to produce valid binaries, the project builds and successfully loads in python when compled using MSVCC but when compiled using either intels c++ compiler or DPC++ compiler the binaries produced are not able to be loaded by python. The build is successfull and visual studio reports no errors, the project is fully compiled but python does not like it and complains DLL load failed DLL not found.

I am using the latest compiler from the latest OneAPI toolkit version 2022.1.3 and c++ std 17.

[HemanthCH_Intel]

Hi,

Thank you for posting in Intel Communities.

Could you please try running the project after adding the python39.dll file path in post-Build events as shown below screenshot?

For adding the python39.dll file, please follow below steps:

1) Open Visual Studio and navigate to the below path:

Project>>properties>>configuration properties>>Build events>>Post-Build events>>Command Line 

2)In the command line add the "path of python39.dll file" and click on apply.
Example:

xcopy /y /d "C:\Program Files (x86)\Intel\oneAPI\intelpython\python3.9\envs\<version>\python39.dll" "$(OutDir)"

3)After adding the path, rebuild the project.

If this resolves your issue, make sure to accept this as a solution. This would help others with a similar issue. Thank you!

Thanks & Regards,

Hemanth.

[ShmarvDogg]

Hi there, sorry for late reply. 

This does not resolve the issue, The issue is specifically python cant find the symbols to link to in the .pyd DLL.

[HemanthCH_Intel]

Hi,

Could you please provide the sample reproducer code and steps to reproduce your issue?

Could you please provide a screenshot of the error?

Could you please provide the Visual Studio version you are using?

Thanks & Regards,

Hemanth

[ShmarvDogg]

Here is a link to the github repo which contains source code.

https://github.com/Gavin-Development/GavinBackendDatasetUtils/tree/Intel_OneAPI_Support

The build fails to link to python both when compiled using Visual studio 2022 and when using cmake on linux.

[HemanthCH_Intel]

Hi,

We haven't heard back from you. Could you please provide an update on your issue?

Thanks & Regards,

Hemanth

[HemanthCH_Intel]

Hi,

Could you please confirm if you are getting the same error in Visual Studio as shown in the below screenshot? If not, Could you please provide the steps to reproduce your issue in Visual Studio and provide a screenshot of the error?

Could you please provide the steps to reproduce your issue with CMake in Ubuntu and provide a screenshot of the error?

Thanks & Regards,

Hemanth

[ShmarvDogg]

Hi there, sorry for late reply, Website would not let me login for a few days.

I do not get that error, that error seems to be because you are trying to launch the .pyd, it is a module for python and not an executable so testing has to be done in python. As for cmake on linux i do not know how it was achieved(i do not work on the linux version of this module), only that it is same issue as windows, compiles perfectly fine but when imported to python script python fails to load it.

The issue is with the fact the compiler does not produce valid binary not that we are unable to compile it.

[HemanthCH_Intel]

Hi,

Could you please provide the steps for reproducing your issue at our end?

Also, please share with us the screenshot of the error on Windows Machine.

Thanks & Regards

Hemanth.

[ShmarvDogg]

To reproduce the error, you will need to pull the repo, switch to the Intel_OneAPI_Support branch, build the module from source using visual studio. It should compile successfully. Then you should create a python file to import the module, place in same directory as output binary. Upon execution and import of the compiled module python gives a DLL load error as shown below.

The module does compile successfully but the binary the compiler produces is not valid. As switching back to MSVCC with the same code leads to a correctly working module but with both intels DPC++ and regular C++ compiler included in my version of the toolkit it does not produce a valid module.

[HemanthCH_Intel]

Hi,

We can able to import the GavinBackendDatasetUtils project successfully by following the below steps and please let me know if we need to add/change any steps in this process.

1) We have cloned the GavinBackendDatasetUtils project and switched to the intel oneAPIsupport branch.

2) We have compiled the project in Visual Studio 2022 and with oneAPI 2022.0.0 version 

3) After compiling the project we can see .pyd, .lib, .pbd files in the output folder

4) In the output folder we have created a file called test.py with the "import GavinBackendDatasetUtils" statement as attached in the screenshot.

5) By using Intel oneAPI command prompt we are able to run the python program by importing the module as attached in the screenshot.

Thanks & Regards,

Hemanth

[ShmarvDogg]

So you have followed all the steps correctly but got it working, so its clearly an issue on my end.

Can i please have a copy of the compiled binary from you so that i can compare it to the one produced by my compiler?

[HemanthCH_Intel]

Hi,

We haven't heard back from you. Could you please provide any updates on your issue?

Thanks & Regards,

Hemanth.

[HemanthCH_Intel · Accepted solution]

Hi,

Thanks for your update.

Please find the attachment for the binary. Since your issue has been resolved. Can we go ahead and close the thread?

Thanks & Regards,

Hemanth

[ShmarvDogg]

Yep, thank you very much for the help, i have narrowed this down to some sort of issue on my end NOT to do with the compiler, I will continue to investigate for a fix on my end but for now i have ruled out all issues with the OneAPI compiler and toolkit.

[ShmarvDogg]

As a short follow up, I have managed to get VS 2022 build fully working on my end, it seems the .pyd will only load correctly for intelpython and not vanilla python. Is there any reason for this or solution to allow the binary to work with regular, normal python distrobutions?

[HemanthCH_Intel]

Hi,

Could you please let us know the vanilla python version you are using? Also, please provide the details if you have tried any other distributions of Python.

Thanks & Regards,

Hemanth

[HemanthCH_Intel]

Hi,

We haven't heard back from you. Could you please provide an update on your issue?

Thanks & Regards,

Hemanth

[HemanthCH_Intel]

Hi,

We assume that your issue is resolved. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Thanks & Regards,

Hemanth