Intel® oneAPI Threading Building Blocks
Ask questions and share information about adding parallelism to your applications when using this threading library.
2478 Discussions

How to ship a package using TBB on PyPI / manylinux

wenzeljakob
Novice
2,269 Views

Dear all,

I'm interested in publishing a mixed C++/Python package using TBB on PyPI, the Python Package Index. The way that binary packages are deployed on Python (e.g. on Linux) is by compiling them on a very old version of CentOS dubbed "manylinux", which establishes base versions of standard libraries like 'libc', 'libm', etc.

However, TBB is not part of the "manylinux" core libraries, and so by default any Python project requiring TBB must bundle it. Bundling TBB is easy, no problems there. However, I am concerned what would happen if multiple Python extensions bundle their own TBB libraries with different versions and load them into the same interpreter. That's not only wasteful but also sounds like it would cause the program to segfault shortly thereafter.

Another way to  accomplish the same goal would be if there was a "tbb" Python PyPI package that ships TBB so that different Python extensions can compile against a consistent version. And, that actually exists:  [tbb]

But looking more closely, I don't understand this package at all. It only covers a subset of platforms (linux, win64, no macos/...), and it doesn't include header files, making it impossible to use to in a package compilation step. There is another package [tbb-devel], which appears deserted/out of sync with [tbb].

For many years, I've been maintaining a CMake overlay over TBB (https://github.com/wjakob/tbb), and I'm tempted to  publish the build output of that to PyPI with more complete coverage + headers. But that would create yet another package and likely add to the confusion.

Before this step, I was wondering if anybody knows the role of those existing Python packages? Is anyone looking into maintaining them? How are they used by Intel, and are they even meant to be used by others?

Thanks,
Wenzel

1 Solution
Olga_M_Intel
Employee
2,179 Views

Hi Wenzel, 

Thanks for your interest to Intel TBB product. 

Please, check the latest packages at 

https://pypi.org/project/tbb/#files 

https://pypi.org/project/tbb-devel/#files

https://pypi.org/project/tbb4py/#files

They should be in sync now and contain the latest oneTBB.

View solution in original post

0 Kudos
2 Replies
Olga_M_Intel
Employee
2,180 Views

Hi Wenzel, 

Thanks for your interest to Intel TBB product. 

Please, check the latest packages at 

https://pypi.org/project/tbb/#files 

https://pypi.org/project/tbb-devel/#files

https://pypi.org/project/tbb4py/#files

They should be in sync now and contain the latest oneTBB.

0 Kudos
wenzeljakob
Novice
2,177 Views
0 Kudos
Reply