topic Multiple process using qatzip+qatlib in Intel® QuickAssist Technology (Intel® QAT)

Multiple process using qatzip+qatlib

Bakuto — Tue, 03 Sep 2024 02:13:16 GMT

One hardware compression engine(or hardware queue) can only used by one thread at a time, this is easy to find in qatzip where it locks instances. However when mutiple process uses the same qat card, how does it ensure that the one engine is not used by two or more processes?

Re:Multiple process using qatzip+qatlib

Ronny_G_Intel — Wed, 04 Sep 2024 23:04:44 GMT

Hi Bakuto,

The QAT hardware is designed to support concurrent access by multiple processes. The QAT driver is responsible for managing access to the hardware resources. It acts as an intermediary between the application (or process) and the hardware. When a process wants to use a QAT engine, it communicates with the QAT driver. The driver uses locking mechanisms to ensure that once an engine is allocated to a process, it is not available to others until the process has finished using it and releases it back to the pool.

I hope this answers your question but please let me know if you need more information.

Regards,

Ronny G

Re: Re:Multiple process using qatzip+qatlib

Bakuto — Thu, 05 Sep 2024 01:36:38 GMT

Hi Ronny,

Thanks for responding, I understand your explanation. However in qatzip+qatlib code i see that process scans all engines in startup and store their access point in memory, the software created lock for each engine and the lock is only protected against other thread in the same process. I kinda don’t get how driver locks take effect to process that want to use them(is it like all process have access to one engine but if one process submit request first, other process have to wait?)

Re:Multiple process using qatzip+qatlib

Ronny_G_Intel — Fri, 06 Sep 2024 22:28:48 GMT

Hi Bakuto,

The Intel QuickAssist Technology (QAT) driver manages hardware resources, including locking instances to prevent concurrent access by multiple threads or processes.

The specific implementation details can vary and may not be fully disclosed in public documentation, the general concept involves using synchronization primitives provided by the operating system's kernel.

Are you experiencing any particular issue with the QAT setup? Any specific functionality not working as expected? Performance issues perhaps?

If you do have issues, please provide me with the icp_dump.

To execute an icp_dump, please run the script located here: $ICP_ROOT/quickassist/utilities/debug_tool/icp_dump.sh

This will generate a tar file that will provide us with your full system setup, including your configuration files.

Also, please provide me with the following details:

Operating System
QAT hardware
QAT driver
QAT software stack
QAT configuration details (this will be included in the icp_dump), please include the qatzip+qatlib code that you mentioned.

Please include any additional relevant information or configuration file.

Thanks,

Ronny G

Re:Multiple process using qatzip+qatlib

Ronny_G_Intel — Tue, 10 Sep 2024 21:12:07 GMT

Hi Bakuto,

I am just checking if you have any update to my previous post.

Regards,

Ronny G

Re:Multiple process using qatzip+qatlib

Ronny_G_Intel — Mon, 16 Sep 2024 20:00:34 GMT

Hi Bakuto,

Have you had the opportunity to review my previous post?

Please let me know if you need additional information.

Regards,

Ronny G

Re:Multiple process using qatzip+qatlib

Ronny_G_Intel — Wed, 18 Sep 2024 15:13:49 GMT

Hi Bakuto,

I hope all is well with you.

It's been some time since our last communication, so I will be marking this case as conditionally closed.

Please don't hesitate to reopen this case or initiate a new inquiry if assistance is still needed.

Regards,

Ronny G