topic Re:Implementation of GZIP support with QATLib DC API in asynchronous approach in Intel® QuickAssist Technology (Intel® QAT)

Implementation of GZIP support with QATLib DC API in asynchronous approach

OttoChow — Mon, 16 Sep 2024 03:39:12 GMT

I am asking this question behalf of a customer who is evaluating QAT (Gen4) accelerator of Xeon 6 (E-Core) 6746E based server.

The current implementation of QATZIP is based synchronous approach. I checked with the developer of QATZIP and there will be plan for async. approach in the future but no time line. The customer believes that async. approach of data compression should improve performance.

He asked me the question on QATLIB. Will he be able to implement GZIP using QATLib compression API in async approach? I read the documentation of QAT API, there is code sample of GZip with QAT Compression API in sync. approach but very few info. on the asynchronous approach.

Thank you.

Re:Implementation of GZIP support with QATLib DC API in asynchronous approach

Ronny_G_Intel — Tue, 17 Sep 2024 01:16:31 GMT

Hi OttoChow,

As I look into this matter, could you please provide details on the specific real-world application or scenario the customer intends to implement using the asynchronous compression feature of QATZIP? This information will help me better understand the request and advocate for this feature if necessary

Regards,

Ronny G

Re: Implementation of GZIP support with QATLib DC API in asynchronous approach

OttoChow — Tue, 17 Sep 2024 05:46:33 GMT

Currently, customer is using QAT compression (8960 QAT Adaptors on 2nd and 3rd Gen Xeon based servers) on NGINX + QATZIP by compressing the web traffic. From the evaluation of 6746E, the customer is aware that most SKU of Xeon 6 - 6700E has 2 QAT devices only.

Due to increase in the core count of Xeon 6 - 6700E, customer would like to explore various way to increase the performance of compression in order to match the total compute capacity of 6700E. Customer believes that doing data compression in async mode may help. Please correct me if the customer is wrong.

Thank you.

Regards,

Otto

Re:Implementation of GZIP support with QATLib DC API in asynchronous approach

Ronny_G_Intel — Fri, 20 Sep 2024 23:31:25 GMT

Hi OttoChow,

Allow me a couple of extra days to double check on this question and I will get back to you.

Regards,

Ronny G

Re: Implementation of GZIP support with QATLib DC API in asynchronous approach

OttoChow — Mon, 23 Sep 2024 08:36:40 GMT

The first question from the customer is .... Is data compression throughput better with async mode than sync mode? In crypto, async mode performs much better than sync mode. Customer wonders that this is the same with data compression.

Will there be any sample code to demonstrate the performance difference between async mode data compression and sync mode data compression?

If async mode data compression delivers better data compression throughput than sync mode data compression, the customer may spend effort to develop a async mode data compression plug-in for NGINX (nothing committed at this stage).

Thank you.

Regards,

Otto

Re:Implementation of GZIP support with QATLib DC API in asynchronous approach

Ronny_G_Intel — Tue, 24 Sep 2024 18:17:40 GMT

Hi Otto,

I am still investigating this request, please allow me a couple of extra days.

Regards,

Ronny G

Re:Implementation of GZIP support with QATLib DC API in asynchronous approach

Ronny_G_Intel — Fri, 27 Sep 2024 20:05:49 GMT

Hi Otto,

See my answers below:

Is there an intention to introduce an asynchronous method for data compression in the future? Yes, there are plans to introduce an asynchronous QATzip API. We are currently working on it, but the timeline for public availability is not yet clear.

Can we expect an increase in data compression throughput when using asynchronous operations compared to synchronous operations? Yes, there is an expected performance improvement, particularly beneficial for compressing small blocks of data (below 64KB).

Will there be any sort of sample code available that demonstrates the performance differences between asynchronous and synchronous data compression methods? Perhaps including telemetry? Yes, the QATzip test application will be capable of displaying the differences between asynchronous and synchronous methods.

I hope this information is helpful, and I apologize for the delay in getting these answers back to you.

Regards,

Ronny G

Re:Implementation of GZIP support with QATLib DC API in asynchronous approach

Ronny_G_Intel — Tue, 01 Oct 2024 20:02:00 GMT

Hi Otto,

I am just checking, did you have a chance to look into my previous post? Do you need further information regarding this issue?

Regards,

Ronny G