For intel x540 card, is there a way to force a link up without a cable plugged in?

[SB13]

We have a Dell server w/ an intel x540 card. we're using the SR-IOV functionality on the 2 ports on the x540. However, to get the links up on the Virtual Functions, we need the link up on the physical function, and hence had to plug in a cable between the 2 ports. Is there a way to force the link up on one or both of the ports so that we don't need a cable plugged in?

[Adolfo_S_Intel]

Hello sbhanujan

We are consulting this issue, and we will reply back as soon as possible.

Best Regards,

Adolfo

[Lawrence_P_Intel · Accepted solution]

Link Detection is a function directly associated with having a cable attached.

There is no way to force Link Up with there being a cable attached.

A virtual port will not work if the physical port is not attached to anything.

[SB13]

Hi Larry, the ixgbe driver (which is the driver for the X540), describes an API for the loopback test.

ixgbe-4.3.13/src/ixgbe_ethtool.c

<<...snip...>>

/* X540 needs to set the MACC.FLU bit to force link up */

switch (adapter->hw.mac.type) {

case ixgbe_mac_X550:

case ixgbe_mac_X550EM_x:

case ixgbe_mac_X540:

reg_data = IXGBE_READ_REG(hw, IXGBE_MACC);

reg_data |= IXGBE_MACC_FLU;

IXGBE_WRITE_REG(hw, IXGBE_MACC, reg_data);

break;

<<...snip...>>

the above leads me to believe that it should be possible to force the link to be up. Also, I have not been able to download the LANCONF tool. My understanding is that I could either force the port or the NIC to be in loopback mode, or just force the link up. Please note, that by default, it doesn't seem that plugging in a dongle brings up the link.

[Adolfo_S_Intel]

Shiva B wrote:

For the x540 card, can somebody point out how to turn loopback on? What we basically want to do, is to plug in a (loopback) dongle in one of the ports so that the link comes up. This is for using SR-IOV. We are running XenServer and a CentOS VM on it. We create virtual functions and pass one of them to the CentOS VM. However, for the link to be up on the VF, we need the physical function link to be up.

I believe the lanconf tool has to be used to set the appropriate register in the NIC. However, I haven't been able to find appropriate documentation of how to do this.

Hello sbhanujan

The LANCONF tool is Intel Confidential material, in order to configure it you must have a CNDA with Intel. Please use the following link to apply for privilege account: https://www-ssl.intel.com/content/www/us/en/forms/design/registration-privileged.html Register for a Privileged Intel Account

Regarding your question about loop-back test, we are investigating and will reply as soon as possible.

Regards,

Adolfo.

[Adolfo_S_Intel]

Hello sbhanujan

Could you please indicate the specific model of your x540 card?

Best Regards,

Adolfo Sanchez.

[SB13]

Hi AdolfoS

The model is X540-AT2. here's the output of lspci.

04:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)

04:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)

Regards,

Shiva

[Adolfo_S_Intel]

Hello sbhanujan

Here is the feedback I have got from our engineers:

The card needs to have an active link to allow for the VF to use it.

This can be achieved by attaching the link to a switch, or if you have dual ports you can connect the ports back to back.

Loopback is a test mode and cannot be used for normal traffic.

Attaching a loopback dongle will not give an active link to the OS.

I hope this information is useful for your case.

Regards,

Adolfo

[SB13]

Hi AdolfoS,

thanks for the update. Currently, we do run a cable on the 2 ports, so that the link is up on each of the ports. As per section 2.10 in http://www.intel.co.uk/content/dam/www/public/us/en/documents/faqs/ethernet-x540-faq.pdf http://www.intel.co.uk/content/dam/www/public/us/en/documents/faqs/ethernet-x540-faq.pdf, it seems that even running a cable between the 2 ports would need to be a minimum of 1 meter in length. We've been using a 6 inch cable. Can you please let me know if this will be an issue?

Regards,

Shiva

[Adolfo_S_Intel]

Hi sbhanujan

Thanks for your patience, according to my sources at the division the reason that you require a minimum length of 1m is because the receiver expects a minimum amount of attenuation by the cable, if the cable is too short, the sending signal is basically washed out at the receiver because it's to strong. This can result in a variety of errors including link loss, link flap, corrupted data, or lost data.

Best Regards,

Adolfo Sanchez.

[SB13]

thanks AdolfoS. One last question - can you please let me know if I have a two VMs communicating w/ VFs on the same port, guaranteeing that there is no traffic on the wire, have any issues with regards to traffic loss? I'm inclined to think that there shouldn't be any issue, but still would like some sort of confirmation?

[Adolfo_S_Intel]

Hello sbhanujan

I received confirmation from our engineers, that the set up you describe is a purely software process so there shouldn't be any problems related to traffic loss through the cable.

Best Regards,

Adolfo Sanchez

[SB13]

thanks AdolfoS, this has been very helpful.