e810-cqda2 for OCP3.0 behaves differently than full height, receives power error

[chadb]

We have an an E810-CQDA2 for OCP3.0 and Full Height cards in a PCIe Gen4 slot. They connect to a Trident4 32x400G switch, using a Dell AEC 400G QSFP28 breakout (4x100G).

Bios settings have the NIC at 2x1x100G, with all other settings at defaults from the ice driver.

Plugging the QSFP into the OCP slot results in a link

Plugging the QSFP into full height card yields the error "The module’s power requirements exceed the device’s power supply. Cannot start link"

Datasheets do not differentiate any differences between the OCP and full height version of the card. The module in ice_main.c seems to indicate that this error is printing because the module power level is unsupported (code below). Yet, the power level IS supported by the OCP version and NOT by the full height.??

Is there documentation to justify this error, and is there a way to increase the threshold?

~line 1048

/**

 * ice_check_module_power

 * @PF: pointer to PF struct

 * @Link_cfg_err: bitmap from the link info structure

 *

 * check module power level returned by a previous call to aq_get_link_info

 * and print error messages if module power level is not supported

 */

static void ice_check_module_power(struct ice_pf *pf, u8 link_cfg_err)

{

                /* if module power level is supported, clear the flag */

                if (!(link_cfg_err & (ICE_AQ_LINK_INVAL_MAX_POWER_LIMIT |

                                                      ICE_AQ_LINK_MODULE_POWER_UNSUPPORTED))) {

                                clear_bit(ICE_FLAG_MOD_POWER_UNSUPPORTED, pf->flags);

                                return;

                }

                /* if ICE_FLAG_MOD_POWER_UNSUPPORTED was previously set and the

                 * above block didn't clear this bit, there's nothing to do

                 */

                if (test_bit(ICE_FLAG_MOD_POWER_UNSUPPORTED, pf->flags))

                                return;

                if (link_cfg_err & ICE_AQ_LINK_INVAL_MAX_POWER_LIMIT) {

                                dev_err(ice_pf_to_dev(pf), "The installed module is incompatible with the device's NVM image. Cannot start link\n");

                                set_bit(ICE_FLAG_MOD_POWER_UNSUPPORTED, pf->flags);

                } else if (link_cfg_err & ICE_AQ_LINK_MODULE_POWER_UNSUPPORTED) {

                                dev_err(ice_pf_to_dev(pf), "The module's power requirements exceed the device's power supply. Cannot start link\n");

                                set_bit(ICE_FLAG_MOD_POWER_UNSUPPORTED, pf->flags);

                }

}

[chadb]

NIC Pics.

[Faez_Intel]

Hi chadb,

Good day and thank you for reaching out to the Intel Community Support.

For this issue, can you verify if your QSFP is compatible to run on the E810? You can refer the link below for the reference of compatible optic modules.

https://compatibleproducts.intel.com/ProductDetails?activeModule=Intel%C2%AE%20Ethernet&prdName=Intel%C2%AE%20Ethernet%20Network%20Adapter%20E810-CQDA1%20/%20CQDA2%20for%20OCP3_0%20(Pull%20Tab/Interlock)

Also, have you tried to use different QSFP and see if the same issue persisted?

Thanks,

Faez_Intel

[chadb]

We have successfully used this adapter/PCIe version with DAC cables. The AEC/ACC cable is the only format that presents this power error. That said, the optics listed in the compatibility tool have never been used in any of our deployments : )

[chadb]

Firmware Version 2.30 does not complain about power, brings up the link and connectivity succeeds. This points more to a driver issue than cable or OS. This firmware version works with kernels 5.1* and 6.3* on Rocky 9.2, and with the default and 1.12(7) ice driver.

After downgrading a couple nics to 2.30 and repeating success, an attempt to step up firmware to find the breaking version began.  2.42 and 3.10 fail. 2.42 complains that the included .cfg file is invalid. 3.10 fails otherwise. 

[Faez_Intel]

Hi Chadb,

Good day and noted on this.

For us to proceed further, please kindly help to gather the SSU logs for us analyze further. Refer to the link below on how to gather the logs:

https://www.intel.com/content/www/us/en/support/articles/000008563/ethernet-products.html

Kindly let us know if you have any other questions.

Thanks,

Faez_Intel

[Faez_Intel]

Hi Chadb,

Good day. This is a follow up on this thread. Please kindly provide the details inquiry for us to assist you further. Let us know if you have any other questions.

Thanks,

Faez_Intel

[Faez_Intel]

Hi Chadb,

I hope you're having a wonderful day. Since we haven;t heard from you for quite some time, we will proceed with the closure of the thread..

If you want to continue support, please reply to this community thread and we will reopen your case or create a new one so that we can continue to support you. 

Thanks,

Faez_Intel