quickAssist 8970 card on redhat8: no acceleration

[denisski]

Hello,

I installed a quickAssist card on a Dell R750 server.
Redhat 8 is installed.
I can't find any performance improvement.
With Redhat 8, I can't remove the aes-ni module.

If you have an idea for the card to be taken into account.

thanks in advance.

Dennis

[BrusC_Intel]

Hello, denisski.

Thank you for posting on the Intel community Support Forum.

I received your ticket regarding QAT, and I will be assisitng you.

Please allow us to review this and I will contact you back as soon as possible, or in case more details are required.

Regards,

Bruce C.

Intel Customer Support Technician

[BrusC_Intel]

Hello, denisski.

Thank you for your patience.

There are some additional details we would like to obtain to make sure we provide accurate suggestions.

1. How is QAT being tested to conclude there are no performance improvements?
2. Can you provide the exact steps so we can get a better idea?
3. What do you mean by "removing the aes-ni module"?
4. What QAT driver is being used?

I will follow up on July 27th just in case

Regards,

Bruce C.

Intel Customer Support Technician

[denisski]

hi,

I will try to be as clear as possible.

I have 2 DELL R740 servers each with a qat 8970 card.
One server is in Redhat 8 and the other in Debian 10.4.
My 2 servers are connected with a 10G fiber thanks to their Intel X710 Ethernet cards.
I configured an IPsec tunnel with strongswan.
I am using iperf3 -c 10.0.0.1 -u -b0 --cport 17000 on a server
I use iperf3 -s on the other server.
I can unload the qat_c62x module (modprobe -r qat_c62x) while sending the stream in the tunnel. This doesn't bother me at all and leads me to think that it is not used by the ipsec service.
Whereas if I unload the aesni_intel module, it's not possible because "in use"

I read elsewhere on https://www.servethehome.com/intel-quickassist-at-40gbe-speeds-ipsec-vpn-testing/

“For our test, we used CentOS 7 with strongswan to manage our VPN connection and iperf3 for our tests.
We also disabled aesni_intel so that the CentOS kernel uses the QAT engine. If you don't, CentOS will default to AES-NI and you will NOT use the QAT card. »

So I tried not to load the aesni_intel module.
On DELL R740 servers, it is not possible to disable aesni_intel in the BIOS.
In Redhat 8, aesni_intel is no longer a module but it is integrated into the kernel so impossible to unload.
In Debian, it is possible to unload the aesni_intel module.
When I run the ipsec service, I get a QAT error:
QAT: Could not find a device on node 0.

And above all the tunnel no longer goes up.

I had also read that there was a QAT service, I don't know if I should set it up?

When the accelerator card is actually supported, how do we ensure that it is used correctly?
Should we see "offload" when we type ip xfrm status or ip xfrm policy or elsewhere?

Thanks in advance

Dennis

[Allan_A_Intel]

Hello Dennis,

Thank you for reaching out to us and providing detailed information about your setup. I understand that you have two Dell* PowerEdge R740 Rack Servers with Intel® QuickAssist Adapter 8970 cards, one running Red Hat Enterprise Linux* 8 and the other Debian* 10.4. You've set up an IPsec tunnel with strongSwan* and are using iperf3 for testing.

I want to assure you that I'm here to assist you with this matter on behalf of my coworker, Bruce. I'll do my best to investigate the issue you're facing and provide you with a helpful solution. Please allow me some time to review your configuration thoroughly.

I will look into the specific points you've raised, including the possibility of the aesni_intel module causing conflicts and the QAT error you encountered. Additionally, I'll explore whether the QAT service needs to be set up and how to ensure that the accelerator card is being utilized correctly.

Once I have gathered all the necessary information and potential solutions, I will get back to you as soon as possible. I appreciate your patience and understanding in this matter. If you have any further details or updates, feel free to share them, as they could be helpful in the investigation.

Thank you again for bringing this to our attention, and I'll be in touch shortly with further guidance.

Best regards,

Allan A.

Intel Customer Support

[Allan_A_Intel]

Dear Dennis,

Thank you for patiently waiting for my response. I sincerely appreciate your understanding and cooperation throughout this process. I'd like to provide you with some insights and guidance.

Regarding the implementation of QAT with the IPsec tunnel, it's important to note that QAT implementations are quite user-specific and may go beyond the scope of this support channel. However, if you could share the guide (if any) you are trying to follow for the QAT implementation, I may be able to provide some helpful suggestions.

Now, concerning the issue with not being able to unload the aesni-intel module, it appears that this is not directly related to QAT. Therefore, I regret to inform you that we cannot provide direct assistance with this matter on this community forum, as it is intended for QAT-related issues.

However, I'd like to be helpful and provide you with some resources that might address the aesni-intel module issue. Red Hat* has a self-help material that you can refer to for this matter. You can find it at the following link: "How do I prevent a kernel module from loading automatically?" https://access.redhat.com/solutions/41278.

Now, let's focus on the QAT error message you encountered: "QAT: Could not find a device on node 0." To troubleshoot this, it's essential to ensure that the QAT driver is properly installed and that the sample code is able to run. This will help confirm whether QAT is functioning correctly in your system.

For detailed instructions on setting up and running the sample code, I recommend referring to the "Intel QuickAssist Technology Software for Linux - Getting Started Guide - Customer Enabling Release." You can find the guide at the following link: https://www.intel.com/content/www/us/en/content-details/710059/intel-quickassist-technology-software-for-linux-getting-started-guide-customer-enabling-release.html.

To ensure you have the right QAT driver for your hardware, please download the "Intel QuickAssist Technology Driver for Linux* – HW Version 1.X" from the following link: https://www.intel.com/content/www/us/en/download/19734/intel-quickassist-technology-driver-for-linux-hw-version-1-x.html.

Once you've installed the correct QAT driver and followed the instructions in the "Getting Started Guide" to set up and run the sample code, please check for any issues along the way. By confirming that the sample code runs successfully, we can be certain that QAT is functioning correctly in your system.

If you encounter any difficulties during this process, please let me know, and we can focus on resolving those specific issues first. Once we confirm that QAT is working fine, we can proceed with troubleshooting any further integration levels in your system.

Thank you once again for your understanding and cooperation. I look forward to hearing about your progress. If you have any further questions or updates, feel free to share them. We're here to assist you throughout this process.

Best regards,

Allan A.

Intel Customer Support

[denisski]

Hi

Thanks for your feedback.

For unloading aesni_intel, I'll manage.

I just wanted to know whether to unload the module
aesni_intel to avoid conflict with qat module?

BR

Dennis

[denisski]

Hi,

Here are some logs:

Jul 27 10:27:13 dell154 pluto[1640]: | kernel: NIC esp-hw-offload offload for connection 'ipsec-16105' enabled on interface bond2
Jul 27 10:27:13 dell154 pluto[1640]: | xfrm: enabling tunnel mode
Jul 27 10:27:13 dell154 pluto[1640]: | netlink_add_sa() adding IPsec SA with reqid 16409
Jul 27 10:27:13 dell154 pluto[1640]: | xfrm: setting IPsec SA replay-window to 32 using old-style req
Jul 27 10:27:13 dell154 pluto[1640]: | xfrm: esp-hw-offload set via interface bond2 for IPsec SA
Jul 27 10:27:13 dell154 pluto[1640]: | xfrm: send_netlink_msg() sending 16
Jul 27 10:27:13 dell154 pluto[1640]: "ipsec-16105" #6: ERROR: netlink response for Add SA esp.44065126@xxx.xx.50.10 included errno 22: Invalid argument
Jul 27 10:27:13 dell154 pluto[1640]: | xfrm: enabling tunnel mode
Jul 27 10:27:13 dell154 pluto[1640]: | netlink_add_sa() adding IPsec SA with reqid 16409
Jul 27 10:27:13 dell154 pluto[1640]: | xfrm: setting IPsec SA replay-window to 32 using old-style req
Jul 27 10:27:13 dell154 pluto[1640]: | xfrm: esp-hw-offload not set for IPsec SA

the kernel tries to activate esp-hw-offload on my network card but it is refused (X710 card does not support offload).

Do I need to use a network card that accepts esp-hw-offload for the QuickAssist 8970 card to work?

BR

Dennis

[denisski]

Hi,

Unable to implement this qat card.
We will remove it from our configuration.

BR

Dennis