Re: Re:When enable Intel E810-2CQDA2 to 50G*2 mode,only one port can up,the other one is down,why？

蓝月飞雪 · ‎03-13-2023

1.Insert Intel NIC E810-2CQDA2 into my server.

2.Power on server then press DEL to enter to BIOS setup.

3.Change port option to 2*50G mode and save and reset.(Connect to 50G*2 cable to switch which support breakout to 50G speed.)

4.Enter to BIOS,found port 1 is disconnect and port 2 is connected.

5.Enter to OS,same phenormenon,one port up and one port down.

NIC version:

BIOS NIC version: optionrom version:1.3346.0

NVM version:4.20.

Faez_Intel · ‎03-15-2023

Hi Blue,

Good day and thank you for contacting Intel Customer Support.

For your issue, please kindly help to provide the details as below in order for us to assist you further.

1) May I know if you did any changes to the speed settings in OS mode? Is the NIC currently running on auto negotiation or full duplex?

2) If you change to a different speed to 25G or below, are you able to use both of those ports?

3) May I know what is the current NIC driver that you are using? And the OS that you are running on the system as well.

Kindly provide the details as above and let us know if you have any other inquiries.

Thanks,

Faez_Intel

蓝月飞雪 · ‎03-15-2023

Hi Faez,

Thanks a lot for your quickly response.Pls check my comment below:

1) May I know if you did any changes to the speed settings in OS mode?

---Haven't do any change in OS mode.Just in BIOS.

Is the NIC currently running on auto negotiation or full duplex?

---Output in OS:(one port up and the other down) from the output below,the up port is Duplex: Full and Auto-negotiation: on:

ethtool ens18f0
Settings for ens18f0:
Supported ports: [ Backplane ]
Supported link modes: 10000baseT/Full
25000baseCR/Full
25000baseKR/Full
25000baseSR/Full
50000baseCR2/Full
50000baseSR2/Full
10000baseSR/Full
10000baseLR/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Supported FEC modes: None RS BASER
Advertised link modes: 10000baseT/Full
25000baseCR/Full
25000baseKR/Full
50000baseCR2/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: None RS BASER
Speed: Unknown!
Duplex: Unknown! (255)
Auto-negotiation: off
Port: None
PHYAD: 0
Transceiver: internal
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: no
]0;ndcs_hz@AutumnLaurel:/home/ndcs_hz[root@AutumnLaurel ndcs_hz]# ethtool ens18f0[K1
Settings for ens18f1:
Supported ports: [ FIBRE ]
Supported link modes: 10000baseT/Full
25000baseCR/Full
25000baseKR/Full
25000baseSR/Full
50000baseCR2/Full
50000baseSR2/Full
10000baseSR/Full
10000baseLR/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: Yes
Supported FEC modes: None RS BASER
Advertised link modes: 10000baseT/Full
25000baseCR/Full
25000baseKR/Full
50000baseCR2/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: None RS BASER
Link partner advertised link modes: Not reported
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 50000Mb/s
Duplex: Full
Auto-negotiation: on
Port: Direct Attach Copper
PHYAD: 0
Transceiver: internal
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes

2) If you change to a different speed to 25G or below, are you able to use both of those ports?

---Haven't try 25G,just 50G and 100G ,for 100G ,both ports can up,for 50G only one port up.

3) May I know what is the current NIC driver that you are using? And the OS that you are running on the system as well.

---OS is Rocky 8.6. About the version which used test,actually I had try on 2 different versions below ,both have issue.

BR,

jing

Faez_Intel · ‎03-16-2023

Hi Jing,

Good day and thank you for the response.

As per our checking, in order for both to be working, you may need to review the bifurcation settings in BIOS to ensure the card is working with PCIe x16 bifurcation enabled. Please kindly refer to the link below:

https://www.intel.com/content/www/us/en/support/articles/000090670/ethernet-products/800-series-network-adapters-up-to-100gbe.html

Although the link stated that for a DELL servers, may I know what model of servers or motherboard are you running with NIC?

Please kindly try to review and enable the PCIE x16 bifurcation settings in your BIOS accordingly.

Let us know if you have any further questions or inquiries.

Thanks,

Faez_Intel

蓝月飞雪 · ‎03-19-2023

Hi Faez,

Thanks a lot for your reply.

I had checked BIOS in my side,no options for me to enable the PCIE x16 bifurcation settings .

What I used is NOKIA server which use AMI chipset I think.

Checked in os,the nic shown as below:

So you mean the pcie slot in my side not support or not enable pcie X16 cause the issue,right?Maybe I can try to find a MB which support pcieX16 and install the same NIC to see the issue can be resolved or not.Then back to you later.

BR,

Jing

蓝月飞雪 · ‎03-20-2023

Hi Faez,

checked with BIOS support team,

Default setting is x16 now.But in OS it shown x8 downgrade as above screenshot,what may cause this?

BR,

jing

Faez_Intel · ‎03-20-2023

Hi Jing,

Good day and thank you for the info provided.

For the bifurcation PCIE 16x, can you help to screenshot the BIOS settings if it was enabled? It should be configured as below sequence:

BIOS -> Advanced -> Chipset Configuration -> North Bridge -> IIO Configuration

It may be a different sequence from the server that you are currently running but to enable the bifurcation PCIE 16x, it should be under the IIO configuration.

As for the auto downgrade to running PCIE 8x, please kindly verify if the card is compatible to be running on the system board itself. An incompatible system board may caused the NIC to be running at PCIE 8x due to the limitations of the board itself.

Thanks,

Faez_Intel

蓝月飞雪 · ‎03-22-2023

Hi Faez,

Thank for reply.

About BIOS setting,I had checked in my side:

Checked AVL list,only this one type MB support this NIC.I had do a cross check in my side:

In my server ,insert 3 different NICs:check linkstatus in OS:

Intel E810 :

LnkSta: Speed 16GT/s (ok), Width x8 (downgraded)

TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

MCX512F-ACAT:

LnkSta: Speed 8GT/s (ok), Width x16 (ok)

TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-\

MCX562A-ACAB:

LnkSta: Speed 8GT/s (ok), Width x16 (ok)

TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

Then change PCIE slot of MCX512F and Intel E810,found Mellanox MCX512F can still linsta as WidthX16(OK),but E810 always X8 downgrade,both 100G*1 and 50G*2 work as linksta:Speed 16GT/s (ok), Width x8 (downgraded).

So seems the Intel E810 NIC can not work as X16 ,I think it's not related MotherBoard,but NIC,right?Attached my logs for details.

BR,

wangjing

Faez_Intel · ‎03-24-2023

Hi WangJing,

Good day and apologies for the delayed response.

Thank you for the info provided. For us to check further on this issue, can you please help to gather the SSU logs? Please refer the below link on how to gather the SSU logs:

https://www.intel.com/content/www/us/en/download/18895/26735/intel-system-support-utility-for-the-linux-operating-system.html?

Also, if it's possible, can you help to provide picture of the E810 card for our verification?

Kindly let us know if you have any other inquiries and we will assist you accordingly.

Thanks,

Faez_Intel

蓝月飞雪 · ‎03-28-2023

Hi Faez,

sorry for late reply,I am out of office these days and will back to office tomorrow.

I will upload the logs and NIC picture tomorrow.Thanks for support.

BR,

wangjing

蓝月飞雪 · ‎03-28-2023

Hi Faez,

attached debug logs and nic pictures.Hope it's OK for you.Thanks.

BR,

wangjing

Faez_Intel · ‎03-29-2023

Hi WangJing,

Good day and thank you for the details provided.

Please allow us some time to check on this and we will get back to you accordingly. Kindly let us know if you have any other inquiries or questions regarding on this matter.

Thanks,

Faez_Intel

蓝月飞雪 · ‎03-30-2023

Hi Faez,

Thanks for reply.No further questions from my side.Anything needed,pls let me know.Thanks.

BR,

wangjing

Faez_Intel · ‎04-03-2023

Hi WangJing,

This adapter is essentially 2 adapters on a single board.. For the card to work fully, it needs to be in a x16 physical slot that is bifurcated into 2 x8 slots electrically.

This will then show as 2 adapters to the system and there will be 1 port for each. If the slot in the system is not capable of bifurcation, it will only see half of the card and only 1 port will work.

As you mentioned earlier, you are able to see the x16 slot enabled on the BIOS but no option for the bifurcation, which may be the cause why the card is able to only run on 1 port only.

Kindly let us know if you have any other inquiries.

Thanks,

Faez_Intel

蓝月飞雪 · ‎04-07-2023

Hi Faez,

I had asked BIOS support vendor,they will provide a test image which can change settings to x8/x8 bifurcation in BIOS ,then try my test again.Pls expect an update next Monday.Thanks a lot.

BR,

wangjing

蓝月飞雪 · ‎04-09-2023

Hi Faez,

I had changed to x8x8 pcie in BIOS and same result as you mentioned above：

---It show as 2 adapters to the system and there are 1 port for each. I think everything it's clear for me.We can close the ticket here.Thanks a lot for your support.

BR,

wangjing

Faez_Intel · ‎04-11-2023

Hi WangJing,

Good day and I hope this message finds you well.

Glad to help you on this matter. We will proceed to close this thread for now. If you have any other inquiries please do not hesitate to contact us in the future.

Best regards,

Faez.

Intel Customer Support