need to replicate QAT dpdk perf test on x86

[JCK1]

Hi,

Based on https://fast.dpdk.org/doc/perf/DPDK_21_11_Intel_crypto_performance_report.pdf

I am having problems to replicate Intel's test on x86, the best I can get is 7.9gbps on Xeon(R) Gold 6338N CPU @ 2.20GHz from SuperMicro. I tried to follow your instructions of configuring BIOS and kernel settings, etc on your report.

The VFs were assigned with vfio-pci driver not QAT's VF driver.

Anything I could be missing?

Thanks in advance.

JC

[JCK1]

Hi Ronny

When I ran  service qat_service start

dmesg shows:

[492530.945327] c6xx 0000:c5:00.0: Starting acceleration device qat_dev0.
[492531.468019] c6xx 0000:c5:00.0: qat_dev0 started 10 acceleration engines
[492531.489563] c6xx 0000:c5:00.0: Cannot use PF with IOMMU enabled
[492531.490503] c6xx 0000:c6:00.0: Starting acceleration device qat_dev1.
[492532.012023] c6xx 0000:c6:00.0: qat_dev1 started 10 acceleration engines
[492532.033565] c6xx 0000:c6:00.0: Cannot use PF with IOMMU enabled
[492532.034539] c6xx 0000:c7:00.0: Starting acceleration device qat_dev2.
[492532.560026] c6xx 0000:c7:00.0: qat_dev2 started 10 acceleration engines
[492532.581568] c6xx 0000:c7:00.0: Cannot use PF with IOMMU enabled

is this expected? what is the message Cannot use PF with IOMMU enabled?

service qat_service status
● qat_service.service - LSB: modprobe the QAT modules, which loads dependant modules, before calling the user space utility to pass configuration parameters
Loaded: loaded (/etc/init.d/qat_service; generated)
Active: active (exited) since Mon 2024-02-12 18:19:29 UTC; 3min 37s ago
Docs: man:systemd-sysv-generator(8)
Process: 47570 ExecStart=/etc/init.d/qat_service start (code=exited, status=0/SUCCESS)
CPU: 362ms

Feb 12 18:19:27 xeon-sm-01 qat_service[47617]: Restarting all devices.
Feb 12 18:19:27 xeon-sm-01 qat_service[47617]: Processing /etc/c6xx_dev0.conf
Feb 12 18:19:28 xeon-sm-01 qat_service[47617]: Processing /etc/c6xx_dev1.conf
Feb 12 18:19:29 xeon-sm-01 qat_service[47617]: Processing /etc/c6xx_dev2.conf
Feb 12 18:19:29 xeon-sm-01 qat_service[47621]: Checking status of all devices.
Feb 12 18:19:29 xeon-sm-01 qat_service[47621]: There is 3 QAT acceleration device(s) in the system:
Feb 12 18:19:29 xeon-sm-01 qat_service[47621]: qat_dev0 - type: c6xx, inst_id: 0, node_id: 0, bsf: 0000:c5:00.0, #accel: 5 #engines: 10 state: up
Feb 12 18:19:29 xeon-sm-01 qat_service[47621]: qat_dev1 - type: c6xx, inst_id: 1, node_id: 0, bsf: 0000:c6:00.0, #accel: 5 #engines: 10 state: up
Feb 12 18:19:29 xeon-sm-01 qat_service[47621]: qat_dev2 - type: c6xx, inst_id: 2, node_id: 0, bsf: 0000:c7:00.0, #accel: 5 #engines: 10 state: up
Feb 12 18:19:29 xeon-sm-01 systemd[1]: Started LSB: modprobe the QAT modules, which loads dependant modules, before calling the user space utility to pass configuration parameters.

How to configure VF so we can run the Intel dpdk perf test on QAT?

Thanks

JCK

[JCK1]

Hello Ronny

I think I finally got your QAT running on my system:
./configure --enable-icp-sriov=host
make install

then:

service qat_service restart

service qat_service_vfs stop
service qat_service_vfs start

root@xeon-sm-01:/home/ubuntu/dpdk-qat-test# service qat_service status
● qat_service.service - LSB: modprobe the QAT modules, which loads dependant modules, before calling the user space utility to pass configuration parameters
Loaded: loaded (/etc/init.d/qat_service; generated)
Active: active (exited) since Tue 2024-02-13 18:18:50 UTC; 13min ago
Docs: man:systemd-sysv-generator(8)
Process: 61421 ExecStart=/etc/init.d/qat_service start (code=exited, status=0/SUCCESS)
CPU: 613ms

Feb 13 18:18:46 xeon-sm-01 qat_service[61469]: Processing /etc/c6xx_dev0.conf
Feb 13 18:18:47 xeon-sm-01 qat_service[61469]: Processing /etc/c6xx_dev1.conf
Feb 13 18:18:47 xeon-sm-01 qat_service[61469]: Processing /etc/c6xx_dev2.conf
Feb 13 18:18:48 xeon-sm-01 qat_service[61421]: enable sriov
Feb 13 18:18:50 xeon-sm-01 qat_service[61577]: Checking status of all devices.
Feb 13 18:18:50 xeon-sm-01 qat_service[61577]: There is 3 QAT acceleration device(s) in the system:
Feb 13 18:18:50 xeon-sm-01 qat_service[61577]: qat_dev0 - type: c6xx, inst_id: 0, node_id: 0, bsf: 0000:c5:00.0, #accel: 5 #engines: 10 state: up
Feb 13 18:18:50 xeon-sm-01 qat_service[61577]: qat_dev1 - type: c6xx, inst_id: 1, node_id: 0, bsf: 0000:c6:00.0, #accel: 5 #engines: 10 state: up
Feb 13 18:18:50 xeon-sm-01 qat_service[61577]: qat_dev2 - type: c6xx, inst_id: 2, node_id: 0, bsf: 0000:c7:00.0, #accel: 5 #engines: 10 state: up
Feb 13 18:18:50 xeon-sm-01 systemd[1]: Started LSB: modprobe the QAT modules, which loads dependant modules, before calling the user space utility to pass configuration parameters.
root@xeon-sm-01:/home/ubuntu/dpdk-qat-test# service qat_service_vfs status
● qat_service_vfs.service - LSB: modprobe the QAT VF modules
Loaded: loaded (/etc/init.d/qat_service_vfs; generated)
Active: active (exited) since Tue 2024-02-13 18:19:24 UTC; 12min ago
Docs: man:systemd-sysv-generator(8)
Process: 61601 ExecStart=/etc/init.d/qat_service_vfs start (code=exited, status=0/SUCCESS)
CPU: 323ms

Feb 13 18:19:24 xeon-sm-01 qat_service_vfs[61601]: Starting qat_dev42
Feb 13 18:19:24 xeon-sm-01 qat_service_vfs[61601]: Starting qat_dev43
Feb 13 18:19:24 xeon-sm-01 qat_service_vfs[61601]: Starting qat_dev44
Feb 13 18:19:24 xeon-sm-01 qat_service_vfs[61601]: Starting qat_dev45
Feb 13 18:19:24 xeon-sm-01 qat_service_vfs[61601]: Starting qat_dev46
Feb 13 18:19:24 xeon-sm-01 qat_service_vfs[61601]: Starting qat_dev47
Feb 13 18:19:24 xeon-sm-01 qat_service_vfs[61601]: Starting qat_dev48
Feb 13 18:19:24 xeon-sm-01 qat_service_vfs[61601]: Starting qat_dev49
Feb 13 18:19:24 xeon-sm-01 qat_service_vfs[61601]: Starting qat_dev50
Feb 13 18:19:24 xeon-sm-01 systemd[1]: Started LSB: modprobe the QAT VF modules.

Now my question is what steps required to run the Intel DPDK QAT Perf test based on https://fast.dpdk.org/doc/perf/DPDK_21_11_Intel_crypto_performance_report.pdf?

I ran this :
cat bind-intel-qat-vf.sh
#!/bin/sh
DEVBIND_PATH=/home/ubuntu/dpdk-stable-21.11.4/usertools/dpdk-devbind.py
for device in 5 6 7; do \
for fn in $(seq 0 7); do \
sudo ${DEVBIND_PATH} --bind=c6xxvf c${device}:01.${fn}; \
sudo ${DEVBIND_PATH} --bind=c6xxvf c${device}:02.${fn}; \
echo 0000:c${device}:01.${fn} 0000:c${device}:02.${fn};
done;
done

ll /sys/bus/pci/drivers/c6xxvf/
total 0
drwxr-xr-x 2 root root 0 Feb 13 18:19 ./
drwxr-xr-x 38 root root 0 Feb 7 01:30 ../
--w------- 1 root root 4096 Feb 13 18:37 bind
lrwxrwxrwx 1 root root 0 Feb 13 18:37 module -> ../../../../module/qat_c62xvf/
--w------- 1 root root 4096 Feb 13 18:37 new_id
--w------- 1 root root 4096 Feb 13 18:37 remove_id
--w------- 1 root root 4096 Feb 13 18:19 uevent
--w------- 1 root root 4096 Feb 13 18:35 unbind

if I used qat_c6xxvf or c6xxvf it will always report 

Warning: no supported DPDK kernel modules are loaded
Error: Driver 'c6xxvf' is not loaded.

or 

Warning: no supported DPDK kernel modules are loaded
Error: bind failed for 0000:c7:01.1 - Cannot open /sys/bus/pci/drivers/qat_c62xvf/bind: [Errno 2] No such file or directory: '/sys/bus/pci/drivers/qat_c62xvf/bind'

do we have DPDK supported kernel module for QAT?

thanks

JCK

[Ronny_G_Intel]

Hi JCK1,

It is great to see that QAT is running.

On the other hand, It looks like you haven't allocated a VF to run the DPDK application

• Bind the QAT VF to vfio-pci:
• # modprobe vfio && modprobe vfio-pci

Verify you have vfio-pci avaialble with: # lsmod | grep vfio

vfio_pci       16384 0

vfio_pci_core     69632 1 vfio_pci

vfio_virqfd      16384 1 vfio_pci_core

vfio_iommu_type1   40960 0

vfio         49152 2 vfio_pci_core,vfio_iommu_type1

irqbypass       16384 2 vfio_pci_core,kvm

• Run the DPDK python script to bind the chosen VF device i.e.: # /dpdk/dpdk-22.07/usertools/dpdk-devbind.py -b vfio-pci 0000:01:01.0
• Device status can be verified with the following i.e.: # /dpdk/dpdk-22.07/usertools/dpdk-devbind.py --status

Crypto devices using DPDK-compatible driver

0000:01:01.0 '200xx Series QAT Virtual Function 18ef' drv=vfio-pci unused=qat_200xxvf

Please let me know the results.

Regards,

Ronny G

[JCK1]

Hi Ronny

Thanks for your quick reply.

So basically my old script does exactly what you have described by using vfio-pci driver.

However the DPDK QAT perf test results still shows similar results (I attached here)

I checked the QAT card speed here:

LnkCap: Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-

Any idea?

Thanks

JCK

[Ronny_G_Intel]

Hi JCK1,

Let me rerun DPKD on my testing system, compare results and depending on that, I will escalate this case.

I will be back in 2 business days.

Regards,

Ronny G

[JCK1]

Hi Ronny

Appreciated very much if you could verify from your working system. I would like to see your BIOS, Linux kernel setting and version or anything  else required to make it have the reported throughput. We can procure  similar HW you have in order to speed up the evaluation. 

Best regards

JCK

[JCK1]

Hi Ronny

Just check if you can provide more guidance on this? 

Thanks

JCK

[Ronny_G_Intel]

Hi JCK1,

I have been working on this issue but I am getting mixed results and I will need the help of a DPDK expert, I am QAT support engineer and this issue goes beyond QAT, please allow me an extra day to try to reach out to my DPDK internal contacts.

I apologize for the inconvenience.

Regards,

Ronny G

[Ronny_G_Intel]

Hi JCK1,

As mentioned before I provide support for QAT, DPDK optimization is not my area of knowledge, so I am trying to find the appropriate support options for this issue, I apologize for the inconvenience.

See below my DPDK basic results binding one device: 0000:3d:01.0

[root@wolfpass3-fedora36 app]# ./dpdk-test-crypto-perf -l 1-2 -a 0000:3d:01.0 -- --devtype crypto_qat --buffer-sz 128 --cipher-algo aes-cbc --cipher-key-sz 16 --cipher-iv-sz 16 --cipher-op encrypt --optype cipher-only --silent --ptest throughput --total-ops 20000

EAL: Detected 72 lcore(s)

EAL: Detected 2 NUMA nodes

EAL: Detected static linkage of DPDK

EAL: Multi-process socket /var/run/dpdk/rte/mp_socket

EAL: Selected IOVA mode 'VA'

EAL: No available hugepages reported in hugepages-1048576kB

EAL: Probing VFIO support...

EAL: VFIO support initialized

EAL:  using IOMMU type 1 (Type 1)

EAL: Probe PCI driver: qat (8086:37c9) device: 0000:3d:01.0 (socket 0)

CRYPTODEV: Creating cryptodev 0000:3d:01.0_qat_sym

CRYPTODEV: Initialisation parameters - name: 0000:3d:01.0_qat_sym,socket id: 0, max queue pairs: 0

CRYPTODEV: Creating cryptodev 0000:3d:01.0_qat_asym

CRYPTODEV: Initialisation parameters - name: 0000:3d:01.0_qat_asym,socket id: 0, max queue pairs: 0

EAL: No legacy callbacks, legacy socket not created

Allocated pool "priv_sess_mp_0" on socket 0

CRYPTODEV: elt_size 0 is expanded to 240

Allocated pool "sess_mp_0" on socket 0

   lcore id   Buf Size Burst Size   Enqueued   Dequeued Failed Enq Failed Deq       MOps       Gbps Cycles/Buf

          2        128         32      20000      20000       3233       5165     7.0799     7.2498     324.86

Regards,

Ronny G

[JCK1]

Hi Ronny

Many thanks for your testing.

I have tried your test with a bit addition for different sizes:

dpdk-test-crypto-perf -l 1-2 -a 0000:c5:01.0 -- --devtype crypto_qat --buffer-sz 64,128,256,512,1024,2048 --cipher-algo aes-cbc --cipher-key-sz 16 --cipher-iv-sz 16 --cipher-op encrypt --optype cipher-only --silent --ptest throughput --total-ops 30000000
EAL: Detected CPU lcores: 64
EAL: Detected NUMA nodes: 1
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available 2048 kB hugepages reported
EAL: VFIO support initialized
EAL: Using IOMMU type 1 (Type 1)
EAL: Probe PCI driver: qat (8086:37c9) device: 0000:c5:01.0 (socket 0)
CRYPTODEV: Creating cryptodev 0000:c5:01.0_qat_sym
CRYPTODEV: Initialisation parameters - name: 0000:c5:01.0_qat_sym,socket id: 0, max queue pairs: 0
CRYPTODEV: Creating cryptodev 0000:c5:01.0_qat_asym
CRYPTODEV: Initialisation parameters - name: 0000:c5:01.0_qat_asym,socket id: 0, max queue pairs: 0
Allocated pool "priv_sess_mp_0" on socket 0
CRYPTODEV: elt_size 0 is expanded to 384
Allocated pool "sess_mp_0" on socket 0
lcore id Buf Size Burst Size Enqueued Dequeued Failed Enq Failed Deq MOps   Gbps    Cycles/Buf

   2         64         32   30000000 30000000 46336996   41002399   6.5919 3.3750  333.74
   2        128         32   30000000 30000000 44924829   39301224   6.5739 6.7316  334.66
   2        256         32   30000000 30000000 39289695   32896332   6.5958 13.5081 333.55
   2        512         32   30000000 30000000 35686881   28074496   6.5686 26.9052 334.92
   2        1024        32   30000000 30000000 56776310   46850295   4.8532 39.7578 453.31
   2        2048        32   30000000 30000000 196942525  189059223  2.5404 41.6215 866.01

So this is for 1VF of QAT and got 41GBPS with buffer size of 2K!

We got better throughput now! before I only could get max 8GBPS. I guess there is something with the Intel DPDK QAT PERF test (https://fast.dpdk.org/doc/perf/DPDK_22_11_Intel_crypto_performance_report.pdf) 
not configured correctly? If you could contact your colleagues in DPDK group, that would be great.

Thanks!

JCK  

[JCK1]

Hi Ronny

I tried to post a reply, your system failed, sounds that it marked my reply as SPAM.

Please help

Thanks

JCK

[JCK1]

This is post I have tried to post and failed many many times. hope this time it will go through.

Hi Ronny

Thanks for your test.

I made a bit modification of your test and ran on my HW:

dpdk-test-crypto-perf -l 1-2 -a 0000:c5:01.0 -- --devtype crypto_qat --buffer-sz 64,128,256,512,1024,2048 --cipher-algo aes-cbc --cipher-key-sz 16 --cipher-iv-sz 16 --cipher-op encrypt --optype cipher-only --silent --ptest throughput --total-ops 30000000

Allocated pool "sess_mp_0" on socket 0
lcore id Buf Size Burst Size Enqueued Dequeued Failed Enq Failed Deq MOps   Gbps    Cycles/Buf

       2       64         32 30000000 30000000 46336996   41002399   6.5919 3.3750  333.74
       2      128         32 30000000 30000000 44924829   39301224   6.5739 6.7316  334.66
       2      256         32 30000000 30000000 39289695   32896332   6.5958 13.5081 333.55
       2      512         32 30000000 30000000 35686881   28074496   6.5686 26.9052 334..92
       2     1024         32 30000000 30000000 56776310   46850295   4.8532 39.7578 453.31
                 2             2048                      32   30000000   30000000  196942525    189059223      2.5404    41.6215    866.01

It is getting better throughput than I got before but still far less the one reported.

If you could get in touch with Intel DPDK QAT folks, that would be great.. Hope to close the gap soon.

Thanks and Best!

JCK

[Ronny_G_Intel]

Hi JCK1,

can you please provide with the command that you are running for the below tests as stated in your  test-crypto-qat-2024-02-13-22-23-58.txt report?

• QAT perf for AES-CBC-128/SHA1-HMAC
• QAT perf for AES-CBC-128/SHA2-256-HMAC
• QAT perf for AES-GCM-128

I ran the equivalent test as you mentioned:

$./dpdk-test-crypto-perf -l 1-2 -a 0000:3d:01.0 -- --devtype crypto_qat --buffer-sz 64,128,256,512,1024,2048 --cipher-algo aes-cbc --cipher-key-sz 16 --cipher-iv-sz 16 --cipher-op encrypt --optype cipher-only --silent --ptest throughput --total-ops 30000000

...and I got the following results, very similar to your report:

Thanks,

Ronny G

[JCK1]

Hi Ronny

I attached three scripts basically are three tests published by Intel ( https://fast.dpdk.org/doc/perf/DPDK_23_03_Intel_crypto_performance_report.pdf)

Thanks

JCK

[JCK1]

Hi Ronny

Is there any update from your side (or the DPDK support side)?

Thanks

JCK

[Ronny_G_Intel]

Hi JCK1,

I hope to have an update before the week ends.

I apologize for the delay and appreciate your patience as we work on this issue.

Regards,

Ronny G

[JCK1]

Hi Ronny

Looking forward to it, thanks again for your support.

Best

JCK

[Ronny_G_Intel]

Hi JCK1,

I apologize for the delay in getting back to you.

I filed an internal case with the DPDK team, they will look into this issue and will get back to me as soon as possible and I will contact you immediately.

I hope this issue gets resolved soon.

Regards,

Ronny G

[Ronny_G_Intel]

Hi JCK1,

I have been working with the DPDK team and we have a few questions for you, please see below:

we checked the Test Cases you provided via "intel-qat-dpdk-test-scripts.tar.gz"

1. intel-cryptodev-crypto-scheduler-tests.sh

2. intel-cryptodev-qat-tests.sh

3. intel-cryptodev-sw-aesni-mb-test.sh

Do you have a sense whether the QAT results or the AESNI-MB results are higher priority for the debug effort? The reason for the ask is that the former uses hardware offload while the latter uses the software path so we want to prioritized debugging resources.

For the aes-cbc-128-sha1-hmac test case with QAT it appears that you are using the executing script:

sudo $DPDK_TEST_CRYPTO_PERF/dpdk-test-crypto-perf \

--socket-mem 2048,0 --legacy-mem \

-a ${QAT_PF0}.0 -a ${QAT_PF0}.1 -a ${QAT_PF0}.2 -a ${QAT_PF0}.3 \

-l 4,5,13,6,14 -n 4 \

-- --buffer-sz 64,128,256,512,1024,2048 \

--optype cipher-then-auth --ptest throughput --auth-key-sz 64 --cipher-key-sz 16 \

--devtype crypto_qat --cipher-iv-sz 16 --auth-op generate --burst-sz 32 \

--total-ops 30000000 --digest-sz 20 --auth-algo sha1-hmac --cipher-algo aes-cbc --cipher-op encrypt

In which case, it is somewhat confusing for us due to (1) the variables are labeled QAT_PF while the PCIe address is for the QAT_VF, and (2) you are using 4x QAT VFs as opposed to 1x QAT VFs.  

In order to be able to reproduce this issue in our lab we can plan to setup an ICX server with the 6338N SKU. However, you also mentioned chipset C62x, can you please provide the exact chipset model? Refer to https://ark.intel.com/content/www/us/en/ark/products/series/98470/intel-c620-series-chipsets.html 

Please confirm that you are not using any QAT add-in-card.

In the meantime, is it possible for you to align more closely with the details for the SKX test case in the performance report? For example, using 18x QAT VFs as opposed to only 1x QAT VF, specifying the round-robin option, etc?

As additional reference, please keep in mind that Test Case 1 - Cryptodev QAT PMD performance (https://fast.dpdk.org/doc/perf/DPDK_21_11_Intel_crypto_performance_report.pdf), uses a Xeon Platinum 8180 Processor with 18x QAT VFs

Regards,

Ronny G

[JCK1]

Hi Ronny,

Thank you for your effort to reach out internally to help us out! Please see my replies inlined below:

I have been working with the DPDK team and we have a few questions for you, please see below:

we checked the Test Cases you provided via "intel-qat-dpdk-test-scripts.tar.gz"

1. intel-cryptodev-crypto-scheduler-tests.sh

2. intel-cryptodev-qat-tests.sh

3. intel-cryptodev-sw-aesni-mb-test.sh

Do you have a sense whether the QAT results or the AESNI-MB results are higher priority for the debug effort? The reason for the ask is that the former uses hardware offload while the latter uses the software path so we want to prioritized debugging resources.

JCK> Yes we can just focus on QAT enabled test, since we are not focusing on software based solution yet. We are more interested in QAT offload.

For the aes-cbc-128-sha1-hmac test case with QAT it appears that you are using the executing script:

sudo $DPDK_TEST_CRYPTO_PERF/dpdk-test-crypto-perf \

--socket-mem 2048,0 --legacy-mem \

-a ${QAT_PF0}.0 -a ${QAT_PF0}.1 -a ${QAT_PF0}.2 -a ${QAT_PF0}.3 \

-l 4,5,13,6,14 -n 4 \

-- --buffer-sz 64,128,256,512,1024,2048 \

--optype cipher-then-auth --ptest throughput --auth-key-sz 64 --cipher-key-sz 16 \

--devtype crypto_qat --cipher-iv-sz 16 --auth-op generate --burst-sz 32 \

--total-ops 30000000 --digest-sz 20 --auth-algo sha1-hmac --cipher-algo aes-cbc --cipher-op encrypt

In which case, it is somewhat confusing for us due to (1) the variables are labeled QAT_PF while the PCIe address is for the QAT_VF, and (2) you are using 4x QAT VFs as opposed to 1x QAT VFs.  

JCK> Here QAT_PF# is more a prefix label, because it was followed immediately by number 0~7 which represent the VF functions, so in its entirty ${QAT_PF0}.0 is the VF function.

JCK> The above script options were exactly copied from the published report from Intel, we did not change anything other than codifying them. You are free to use just one VF in your test. Please see Intel report page. 12

In order to be able to reproduce this issue in our lab we can plan to setup an ICX server with the 6338N SKU. However, you also mentioned chipset C62x, can you please provide the exact chipset model? Refer to https://ark.intel.com/content/www/us/en/ark/products/series/98470/intel-c620-series-chipsets.html 

Please confirm that you are not using any QAT add-in-card.

JCK> Here is the info I can collect, not sure how to get the chipset information:

c5:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)

Subsystem: Intel Corporation QuickAssist Adapter 8970

Flags: bus master, fast devsel, latency 0, IRQ 16, NUMA node 0, IOMMU group 7

Memory at fb340000 (64-bit, non-prefetchable) [size=256K]

Memory at fb300000 (64-bit, non-prefetchable) [size=256K]

Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+

Capabilities: [60] MSI-X: Enable+ Count=17 Masked-

Capabilities: [6c] Power Management version 3

Capabilities: [74] Express Endpoint, MSI 00

Capabilities: [100] Advanced Error Reporting

Capabilities: [138] Alternative Routing-ID Interpretation (ARI)

Capabilities: [140] Single Root I/O Virtualization (SR-IOV)

Capabilities: [1b0] Access Control Services

Kernel driver in use: c6xx

Kernel modules: qat_c62x

In the meantime, is it possible for you to align more closely with the details for the SKX test case in the performance report? For example, using 18x QAT VFs as opposed to only 1x QAT VF, specifying the round-robin option, etc?

JCK> Are you referring the test case 1)? Indeed we are using 18 VFs and round-robin in that test case. That was taken exactly from Intel published report.

As additional reference, please keep in mind that Test Case 1 - Cryptodev QAT PMD performance (https://fast.dpdk.org/doc/perf/DPDK_21_11_Intel_crypto_performance_report.pdf), uses a Xeon Platinum 8180 Processor with 18x QAT VFs

JCK> We will find a similar Intel platform, or try the latest x86 HW like Rapid Sapphire, the numbers should be better with latest HW?

Thanks again

JCK

[Ronny_G_Intel]

Hi JCK1,

Thanks for the additional information provided.

Can you please provide the most recent result you have trying to stick as much as possible as described in the Cryptodev QAT PMD performance test?

As mentioned before, we are only going to look into Cryptodev QAT PMD performance test.

Thanks,

Ronny G