epct64e Error "Cannot initialize adapter"

[Brambo]

Hello Intel support,

I have a very similar situation:

% sudo ./epct64e -devices
Ethernet Port Configuration Tool
EPCT version: v1.41.03.01
Copyright 2019 - 2024 Intel Corporation.

Cannot initialize port: [00:010:00:00] Intel(R) Ethernet Network Adapter E810-C-Q1
NIC Seg:Bus:Fun Ven-Dev Connector Ports Speed Quads Lanes per PF
=== ============= ========= ========= ===== ======== ====== ============
1) 000:010:00 8086-1592 N/A 1 N/A N/A N/A

Error: Cannot initialize adapter.

My Setup:

Intel E810-CQDA1 card

% ethtool -i enp10s0np0
driver: ice
version: 6.8.6-100.fc38.x86_64
firmware-version: 4.40 0x8001c979 1.3534.0
expansion-rom-version:
bus-info: 0000:0a:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

Fedora 38

0a:00.0 Ethernet controller: Intel Corporation Ethernet Controller E810-C for QSFP (rev 02)
Subsystem: Intel Corporation Ethernet Network Adapter E810-C-Q1
Flags: bus master, fast devsel, latency 0, IRQ 126, IOMMU group 18
Memory at e4000000 (64-bit, prefetchable) [size=32M]
Memory at e6000000 (64-bit, prefetchable) [size=64K]
Expansion ROM at f6a00000 [disabled] [size=1M]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [70] MSI-X: Enable+ Count=2048 Masked-
Capabilities: [a0] Express Endpoint, IntMsgNum 0
Capabilities: [e0] Vital Product Data
Capabilities: [100] Advanced Error Reporting
Capabilities: [148] Alternative Routing-ID Interpretation (ARI)
Capabilities: [150] Device Serial Number 30-3e-a7-ff-ff-05-d0-78
Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
Capabilities: [1a0] Transaction Processing Hints
Capabilities: [1b0] Access Control Services
Capabilities: [1d0] Secondary PCI Express
Capabilities: [200] Data Link Feature <?>
Capabilities: [210] Physical Layer 16.0 GT/s <?>
Capabilities: [250] Lane Margining at the Receiver
Kernel driver in use: ice
Kernel modules: ice

dmesg:

...
[ 9.559916] ice 0000:0a:00.0: The DDP package was successfully loaded: ICE OS Default Package version 1.3.36.0
[ 9.588210] NET: Registered PF_QIPCRTR protocol family
[ 9.843641] ice 0000:0a:00.0: 63.008 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x8 link at 0000:00:03.2 (capable of 252.048 Gb/s with 16.0 GT/s PCIe x16 link)
[ 9.861927] ice 0000:0a:00.0: PTP init successful
[ 9.998466] ice 0000:0a:00.0: DCB is enabled in the hardware, max number of TCs supported on this port are 8
[ 10.000145] ice 0000:0a:00.0: FW LLDP is disabled, DCBx/LLDP in SW mode.
[ 10.001187] ice 0000:0a:00.0: Commit DCB Configuration to the hardware
[ 10.251209] nvidia-nvlink: Nvlink Core is being initialized, major device number 234

[ 10.253591] nvidia 0000:09:00.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=none:owns=io+mem
[ 10.258789] ice 0000:0a:00.0 enp10s0np0: renamed from eth0
[ 10.342381] ice 0000:0a:00.0 enp10s0np0: NIC Link is up 10 Gbps Full Duplex, Requested FEC: NONE, Negotiated FEC: NONE, Autoneg Advertised: Off, Autoneg Negotiated: False, Flow Control: None
...
[ 10.396347] i40e: Intel(R) Ethernet Connection XL710 Network Driver
[ 10.397235] i40e: Copyright (c) 2013 - 2019 Intel Corporation.

Originally the card didn't recognize the QSFP module (fs.com item 36281) at all, but it did after updating the NVM.

I can currently connect at 10 Gbit/s while this QSFP should do 40 Gbit/s or 4 x 10 Gbit/s.

Regards, Bram

[Sazirah]

Hi Brambo,

Good day.

Thank you for posting in Intel Ethernet Communities.

Regarding your issue, please give us some time to check on this and we will get back to you.

Regards,

Sazzy

[Sazirah]

Hi Brambo,

Regarding your issue, please see below for us to investigate further on this:

1.Kindly confirm if you are using latest driver. If no, please update the driver as per the link below:

https://www.intel.com/content/www/us/en/download/19630/intel-network-adapter-driver-for-e810-series-devices-under-linux.html

2.Please share with us the QSFP module and the model.

3.Please confirm if you have done AC power cycle after NIC firmware update.

Regards,

Sazzy_Intel

[Brambo]

Hi Sazirah,

Thank you for looking into this issue. Let me first mention that in the mean time I realized that I was confused a little by the specifications: I see now that the card supports a number of rates, including 10, 25, 50 Gb/s, but not 40.
Somewhere I've seen mentions of 4 x 10 Gb/s, and in my mind this implied 40 was also possible.

Regarding your questions:

1. The driver I expected to be fairly up-to-date, as I'm using kernel 6.8.6; when I try to build the ice-1.13.7 I see it is actually meant for older versions, so I'd have to downgrade the kernel in order to use this driver, that seems illogical to me.

2. The QSFP is from FS.com, P/N QSFP-SR4-40G coded for Intel, aka item #36281.

3. Indeed I did a full AC power cycle after the firmware update, only after that the QSFP module was recognized.

Regards,

Bram

[Brambo]

So ... I booted back into the old 6.6.13 kernel, built and installed the driver, rebooted.

And, yes, now it works:

# ./epct64e -get -nic=1
Ethernet Port Configuration Tool
EPCT version: v1.41.03.01
Copyright 2019 - 2024 Intel Corporation.

Available Port Options:
==========================================================================
        Port                             Quad 0           Quad 1          
Option  Option (Gbps)                    L0  L1  L2  L3   L4  L5  L6  L7  
======= =============================    ================ ================
Active  100                           -> 100   -   -   -    -   -   -   -
        2x50                          ->  50   -  50   -    -   -   -   -
        4x25                          ->  25  25  25  25    -   -   -   -

All actions succeeded.

In the end, this won't help me, as I need 40 or 4 x 10, so I'll need to downgrade the card to the old XL710 family I think anyway.

It does mean that there some work to do for the Linux kernel guys.

Regards, Bram

[Brambo · Accepted solution]

Anyway, I did boot back into the old 6.6.13 kernel, built and installed the ice-1.13.7 driver, and now the epct64e works!

Then I set the card to 4x25 speed, and it looks like it now can successfully connect at 10 Gbit speed on the 4 lines.

In the end, it appears there's still work for the Linux kernel guys.

Regards, Bram

[Sazirah]

Hi Brambo,

Thank you for your reply.

We are glad the issue seems to be fixed at your end. Therefore, may I know if you need any further assistance or we can proceed with closure for this case?

Thank you.

Regards,

Sazzy_Intel

[Brambo]

Hi Sazirah,

Please note that while the issue is effectively solved for me, this is still a work around.
A proper solution requires:

* epct64e to be improved to issue a sensible message when the driver is not supporting the required functions;

* the standard Linux driver to get updated to support this tool.

But I'm sure this is already on the road map of your software team.

Regards, Bram

[Sachinks]

Hello Brambo,

Good day!

Thank you for the update. We will convey it to our internal team.

Let us know if we can close this thread as of now.

Regards,

Sachin KS

[Brambo]

This thread can be close now, yes.