XL710 too many irq

[AndriiV]

Hello!

I have a problem with Intel XL710,  after 22 Gbps of traffic some cores of processor loads 100% IRQ and traffic goes down.

There are latast firmware and driver:

# ethtool -i ens2
driver: i40e
version: 2.24.6
firmware-version: 9.40 0x8000ecc0 1.3429.0
expansion-rom-version:
bus-info: 0000:d8:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

I read this guide

And made all possible configurations.  But nothing changes

382  set_irq_affinity local ens2
  384  set_irq_affinity all ens2
  387  ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 0 tx-usecs 0
  389  ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 5000 tx-usecs 20000
  391  ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 10000 tx-usecs 20000
  393  ethtool -g ens2
  394  ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 84 tx-usecs 84
  396  ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 62 tx-usecs 62
  398   ethtool -S ens2 | grep drop
  399   ethtool -S ens2 | grep drop
  400   ethtool -S ens2 | grep drop
  401   ethtool -S ens2 | grep drop
  402  ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 336 tx-usecs 84
  403   ethtool -S ens2 | grep drop
  404   ethtool -S ens2 | grep drop
  406   ethtool -S ens2 | grep drop
  407  ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 672 tx-usecs 84
  408   ethtool -S ens2 | grep drop
  409   ethtool -S ens2 | grep drop
  411   ethtool -S ens2 | grep drop
  412   ethtool -S ens2 | grep drop
  425   ethtool -S ens2 | grep drop
  426   ethtool -S ens2 | grep drop
  427  ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 8400 tx-usecs 840
  428  ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 4200 tx-usecs 840
  430   ethtool -S ens2 | grep drop
  431   ethtool -S ens2 | grep drop
  432   ethtool -S ens2 | grep drop
  433   ethtool -S ens2 | grep drop
  434  ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 4200 tx-usecs 1680
  435   ethtool -S ens2 | grep drop
  436   ethtool -S ens2 | grep drop
  439  ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 3200 tx-usecs 3200
  469  ethtool -a ens2
  472  ethtool ens2
  473  ethtool -i ens2
  475  ethtool -i ens2
  476  ethtool ens2
  482  ethtool -C ens2 adaptive-rx on
  484  ethtool -c ens2
  486  ethtool -C ens2 adaptive-tx on
  487  ethtool -c ens2
  492  history | grep ens2
  494  ethtool -m ens2
  499  ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 4200 tx-usecs 1600
  501  history | grep ens2

Server configuration:

80 cores  Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz

RAM 960 GB

8 x SAMSUNG MZQLB7T6HMLA-000AZ NVME disks

Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)

What can be done with the settings of this network card to solve the problem?

On another server with a similar configuration, but a different network cards, everything is fine

Server configuration:

88 cores Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz

RAM 512 GB

8 x SAMSUNG MZQLB7T6HMLA-00007 NVME disks

4 x 82599ES 10-Gigabit SFI/SFP+ Network Connection

[Azeem_Intel]

Hello AndriiV,

Greeting for the day!

Would it be helpful to set up a live debug call with you? If so, please let us know a suitable day and time this week. We would suggest Thursday or Friday, but we're flexible and can accommodate your availability.

Regards,

Azeem Ulla

Intel Customer Support Technician

intel.com/vroc

[AndriiV]

Hello Azeem!

You can call me thursday via phone, Facetime, telegram, viber, signal or write to this messangers. I will wait  your call from 10 to 16 GMT+2

[Simon-Intel]

Hello AndriiV,

As discussed over the call, we will arrange the callback accordingly.

Best regards,

Simon

Intel Support Engineer

[Sazirah]

Hi AndriiV,

Thank you for patiently waiting for us.

Unfortunately, the performance engineering team is not available until next week. We will be checking with them when they will be available and try to set up a call. We will share the date time if it is convenient with you.

Regards,

Sazzy_Intel

[AndriiV]

Ok, I will wait

[Sachinks]

Hello AndriiV,

Greetings!

This is a follow up. Kindly let us know if you were able to find the details provided in the last email.

Kindly reply back so that we can continue checking this further.

Regards,

Sachin KS

Intel Customer Support Technician

intel.com/vroc

[AndriiV]

Hello Sachin!

Sorry for late answer.

About cables,  I wrote the names in this topic. Also we used before XXV710 -  XL710 card, there was the same problems.

I also wrote about this earlier, we have another server with the XXB710 card connected directly to the same switch in the same port (in 4x25 mode) (2x25 in one server and 2x25 in the other). But in the other server there is rarely more than 1 Gbit/s of incoming traffic and therefore everything is fine there even with 35 Gbit/s of outgoing traffic.

[Simon-Intel]

Hi AndriiV,

Good Day!

Thank you for your response.

We will check this internally and get back to you as soon as we have an update.

Regards,

Simon

Intel Customer Support Technician

intel.com/vroc