Intel® Distribution for Python*
Engage in discussions with community peers related to Python* applications and core computational packages.
424 Discussions

Any plans to add optimizations for Python Web frameworks?

gaston-hillar
Valued Contributor I
747 Views

Django, Flask and other Python Web frameworks might benefit from the usage of Intel Threading Building Blocks. It would be great to have optimized versions of one or two popular Python Web frameworks. I think this would boost Intel Distribution for Python adoption in the Python community. Are there any plans to add optimizations to any Python Web framework?

0 Kudos
2 Replies
Anton_M_Intel
Employee
747 Views

Usually, I'd answer that TBB is designed for numeric computations and thus it does not work well with blocking I/O calls since it restricts the number of threads in order to avoid oversubscription. With modern asynchronous I/O frameworks, it might still be useful in other languages and for other applications.. but the nature of Python's web applications, I assume, implies that any computations are rather short and rely on Python interpreter often and heavily. Here, Python's GIL becomes a bottleneck, preventing applications from scaling and rendering TBB pretty useless. If these frameworks could run on multiple interpreter instances so that each thread would run a separate Python interpreter, it might make sense to revisit this conclusion.

0 Kudos
gaston-hillar
Valued Contributor I
747 Views

Anton,

Thanks for the detailed response. Python's GIL is the great bottleneck. It would be great to have better options for Python Web Framework when you run them on multicore VM's or multicore hardware.

0 Kudos
Reply