Ethernet Products
Intel® Ethernet products and technologies
3880 Discussions

intel igb i354 issue with lack of multiqueue (intel atom C2758 board)

ITria
Beginner
1,503 Views

Hi there,

I've been having some trouble getting igb/i354 to enable multiple queues.

Despite compiling the latest igb module (5.2.9.4 at the time of writing this) I have not been able to enable it.

I have ensured that MSI-X interrupts are used, and even set RSS values to 8 (and IntMode to 2 to be safe again) and rebuilt initrd and have googled extensively as well as followed all the information/advice I've seen in intel's documentation for installing the modules. The kernel I am running is 2.6.32 on a proxmox installation (though I tried with a 3.10 kernel as well with same results).

The motherboard is a Supermicro A1SRi-2758F atom server board and there is nothing I can see in the BIOS that looks like it could be causing this issue.

I am starting to think that I am missing something obvious here...

Any help/advice would be much appreciated.

Regards,

Iakovos

From lspci ( lspci -vv at the bottom of this post due to size):

00:00.0 Host bridge: Intel Corporation Atom processor C2000 SoC Transaction Router (rev 02)

00:01.0 PCI bridge: Intel Corporation Atom processor C2000 PCIe Root Port 1 (rev 02)

00:02.0 PCI bridge: Intel Corporation Atom processor C2000 PCIe Root Port 2 (rev 02)

00:03.0 PCI bridge: Intel Corporation Atom processor C2000 PCIe Root Port 3 (rev 02)

00:0b.0 Co-processor: Intel Corporation Atom processor C2000 nCPM (rev 02)

00:0e.0 Host bridge: Intel Corporation Atom processor C2000 RAS (rev 02)

00:0f.0 IOMMU: Intel Corporation Atom processor C2000 RCEC (rev 02)

00:13.0 System peripheral: Intel Corporation Atom processor C2000 SMBus 2.0 (rev 02)

00:14.0 Ethernet controller: Intel Corporation Ethernet Connection I354 (rev 03)

00:14.1 Ethernet controller: Intel Corporation Ethernet Connection I354 (rev 03)

00:14.2 Ethernet controller: Intel Corporation Ethernet Connection I354 (rev 03)

00:14.3 Ethernet controller: Intel Corporation Ethernet Connection I354 (rev 03)

00:16.0 USB controller: Intel Corporation Atom processor C2000 USB Enhanced Host Controller (rev 02)

00:17.0 SATA controller: Intel Corporation Atom processor C2000 AHCI SATA2 Controller (rev 02)

00:18.0 SATA controller: Intel Corporation Atom processor C2000 AHCI SATA3 Controller (rev 02)

00:1f.0 ISA bridge: Intel Corporation Atom processor C2000 PCU (rev 02)

00:1f.3 SMBus: Intel Corporation Atom processor C2000 PCU SMBus (rev 02)

01:00.0 PCI bridge: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge (rev 03)

02:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 30)

03:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller (rev 03)

From dmesg:

igb 0000:00:14.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20

igb 0000:00:14.0: setting latency timer to 64

igb 0000:00:14.0: irq 35 for MSI/MSI-X

igb 0000:00:14.0: irq 36 for MSI/MSI-X

igb 0000:00:14.0: Intel(R) Gigabit Ethernet Network Connection

igb 0000:00:14.0: eth0: (PCIe:integrated:integrated)

igb 0000:00:14.0: eth0: MAC: 0c:c4:7a:xx:xx:xx

igb 0000:00:14.0: eth0: PBA No: 010A00-000

igb 0000:00:14.0: LRO is disabled

igb 0000:00:14.0: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s)

igb 0000:00:14.1: PCI INT B -> GSI 21 (level, low) -> IRQ 21

igb 0000:00:14.1: setting latency timer to 64

igb 0000:00:14.1: irq 38 for MSI/MSI-X

igb 0000:00:14.1: irq 39 for MSI/MSI-X

igb 0000:00:14.1: Intel(R) Gigabit Ethernet Network Connection

igb 0000:00:14.1: eth1: (PCIe:integrated:integrated)

igb 0000:00:14.1: eth1: MAC: 0c:c4:7a:xx:xx:xx

igb 0000:00:14.1: eth1: PBA No: 010A00-000

igb 0000:00:14.1: LRO is disabled

igb 0000:00:14.1: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s)

igb 0000:00:14.2: PCI INT C -> GSI 22 (level, low) -> IRQ 22

igb 0000:00:14.2: setting latency timer to 64

igb 0000:00:14.2: irq 41 for MSI/MSI-X

igb 0000:00:14.2: irq 42 for MSI/MSI-X

igb 0000:00:14.2: Intel(R) Gigabit Ethernet Network Connection

igb 0000:00:14.2: eth2: (PCIe:integrated:integrated)

igb 0000:00:14.2: eth2: MAC: 0c:c4:7a:xx:xx:xx

igb 0000:00:14.2: eth2: PBA No: 010A00-000

igb 0000:00:14.2: LRO is disabled

igb 0000:00:14.2: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s)

igb 0000:00:14.3: PCI INT D -> GSI 23 (level, low) -> IRQ 23

igb 0000:00:14.3: setting latency timer to 64

igb 0000:00:14.3: irq 43 for MSI/MSI-X

igb 0000:00:14.3: irq 44 for MSI/MSI-X

igb 0000:00:14.3: Intel(R) Gigabit Ethernet Network Connection

igb 0000:00:14.3: eth3: (PCIe:integrated:integrated)

igb 0000:00:14.3: eth3: MAC: 0c:c4:7a:xx:xx:xx

igb 0000:00:14.3: eth3: PBA No: 010A00-000

igb 0000:00:14.3: LRO is disabled

igb 0000:00:14.3: Using MSI-X interrupts. 1 rx queue(s), 1 tx queue(s)

igb 0000:00:14.0: eth0: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX

igb 0000:00:14.1: eth1: igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX

igb 0000:00:14.2: eth2: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX

igb 0000:00:14.3: eth3: igb: eth3 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX

From cat /proc/interrupts |grep eth:

35: 0 0 0 0 0 0 1 0 PCI-MSI-edge eth0

36: 919 901 926 916 920 936 882 904 PCI-MSI-edge eth0-TxRx-0

38: 0 1 0 0 0 0 0 0 PCI-MSI-edge eth1

39: 589 629 635 616 609 582 609 615 PCI-MSI-edge eth1-TxRx-0

41: 0 0 0 0 0 1 0 0 PCI-MSI-edge eth2

42: 644 620 577 604 602 616 603 615 PCI-MSI-edge eth2-TxRx-0

43: 0 0 1 0 0 0 0 0 PCI-MSI-edge eth3

44: 599 593 601 618 600 592 640 630 PCI-MSI-edge eth3-TxRx-0

Values I am passing on modules.conf & modprobe.d/options (for 2.6/3.x):

igb RSS=8,8,8,8 IntMode=2,2,2,2

# modinfo igb

filename: /lib/modules/2.6.32-32-pve/kernel/drivers/net/igb/igb.ko

version: 5.2.9.4

license: GPL

description: Intel(R) Gigabit Ethernet Network Driver

author: Intel Corporation, <</span>mailto:e1000-devel@lists.sourceforge.net e1000-devel@lists.sourceforge.net>

srcversion: E377200391EBF74638FEDA2

alias: pci:v00008086d000010D6sv*sd*bc*sc*i*

alias: pci:v00008086d000010A9sv*sd*bc*sc*i*

alias: pci:v00008086d000010A7sv*sd*bc*sc*i*

alias: pci:v00008086d000010E8sv*sd*bc*sc*i*

alias: pci:v00008086d00001526sv*sd*bc*sc*i*

alias: pci:v000080...

0 Kudos
2 Replies
PAUL_S_Intel
Employee
280 Views

Hi Iakovos -

I am an Intel Applications engineer. Could you please send your information to one of the following for a response:

1) File a bug on the e1000 SourceForge mailing list (e1000.sourceforge.net) or

2) email mailto:Linux-nics@intel.com Linux-nics@intel.com

Then we should be able to get you some help with your issue.

thanks;

Paul

ITria
Beginner
280 Views

Hi Paul,

Many thanks for your reply. It's much appreciated and nice to see that intel employees show interest in customers' problems (makes a nice change sometimes)!

As I needed a working implementation within a certain timeframe I was left with little option but to reinstall using a different distro. My troubleshooting pretty much proved that the issue was related with some options set within the kernels packaged with proxmox. I am afraid I run out of time to troubleshoot this further at this point. At a guess if you had similar hardware at hand you shouldn't have much trouble reproducing the issue

Using the very same driver/module within a debian vanilla kernel (both stable and testing) works a treat. As a matter of fact I had multiqueue support out of the box with the debian kernel with an igb versions of 5.0.5-k (if memory serves) though I compiled the latest version of the module regardless.

I will however make notes of the above information for future reference.

Many thanks for your time,

Iakovos

Reply