i210 SerDes with external SFP fails to transmit

[MSanz]

Hi,

We are working on an i210 flash-less design in which the i210 operates in Serdes/1000Base-BX mode attached to an external optical module.

We want the i210 to transmit regardless of the presence of a valid RX signal.

To accomplish this, we have disabled auto-negotiation and forced the link following these specification sections:

1. 3.7.4.1.4 Forcing Link-up in SerDes/1000BASE-BX Mode
2. 4.5.7.3 MAC/SERDES Link Setup (CTRL_EXT.LINK_MODE = 11b)
3. 4.5.7.3.2 Auto-Negotiation Skipped (PCS_LCTL. AN ENABLE = 0b; CTRL.FRCSPD = 1b; CTRL.FRCDPLX = 1b)

The design also keeps the SRDS_[n]_SIG_DET signal always asserted.

Although with this configuration TX packets appear on the NIC statistics, no signal is detected on SerDes TX output. (When a valid signal is connected to RX input, the controller transmits without problem.)

Could someone please provide some clue as to what we could be missing?

Is it mandatory in order for the PCS to transmit, to receive a valid signal on RX input? (Although all link parameters are forced)?

Thanks in advance.

Manuel.

[CarlosAM_INTEL]

Hello, ManuelS:

Thank you for contacting Intel Embedded Community.

We suggest you review that your implementation fulfills the requirements stated in section 3.7.4.1, specifically in section 3.7.4.1.5 on page 92 of the https://www.intel.com/content/dam/www/public/us/en/documents/datasheets/i210-ethernet-controller-datasheet.pdf Intel(R) Ethernet Controller I210 Datasheet document # 333016.

We hope that this information may help you.

Best regards,

Carlos_A.

[MSanz]

Hi Carlos,

Thank you for your quick response.

We discarded the auto-negotiation timeout approach described in section 3.7.4.1.5 on page 92 because we need to transmit even when the RX line is disconnected, so no signal (idle code groups) will be received in this case. (We have tested it and this approach works fine if a RX signal is received).

Paragraph 3.7.4.1.4 on the specification includes following note:

"When link is forced via the CTRL.SLU bit, the link does not come up unless the SRDS_[n]_SIG_DET signal is asserted or an internal energy indication is received from the SerDes receiver, implying that there is a valid signal being received by the optical module or SerDes circuitry."

According to this note we expect the link to come up and transmit regardless of the presence of a valid signal on RX input). (When SRDS_[n]_SIG_DET signal is asserted as it is on our design).

The behavior we observe is as if both conditions mentioned in paragraph 3.7.4.1.4 (SRDS_[n]_SIG_DET signal and energy indication) should be simultaneously met. (Logical AND instead of logical OR).

Thank you in advance.

[CarlosAM_INTEL]

Hello, ManuelS:

Thanks for your reply.

In order to be on the same page, we would like to address the following questions:

Could you please tell us if the affected project has been developed by you or by a third party company? In case that it is a third party one, could you please give us all the information related to it? If it has been developed by you, how many units show this behavior? could you please let us where are stated the guidelines to develop it? could you please tell us if it has been reviewed by Intel?

Please let us know all the information that may answer the previous questions.

We really appreciate your cooperation.

Best regards,

Carlos_A.

[MSanz]

Hi Carlos,

Once again, thank you for your prompt response.

I will try to answer your questions:

The project is still under development, it is in a prototype stage. It is being developed by our company. It has not been reviewed by Intel. We have followed all Intel guidelines. Of course we can provide any required information through a private channel. The application is very standard (IS model of i210, flashless, SerDes interface with external SFP) and as I already mentioned, it is working fine under normal circumstances.

We have to comply with an additional requirement, which states that the card has to be able to transfer when there is no RX fibre attached to it. All prototype units show the same behaviour, as it is rather a design not an implementation question. May be it's just the initialization values.

What we would like to know is if the observed behaviour is by design. Thoroughly reading the specification, we haven't found anything that states that RX and TX cannot work independently, when using the SerDes interface and an external optical transceiver.

Thank you very much.

Manuel.

[CarlosAM_INTEL]

Hello, ManuelS:

We really appreciate your cooperation.

Based on your previous communications, could you please tell us how do you configure this part for Serdes BX? Could you please let us know what module you are using? Finally, could you please inform us what is the Operating System (OS) and driver related to this situation?

Thanks again for your collaboration.

Best regards,

Carlos_A.

[MSanz]

We initialize the device with the PCI Device ID, 0x157C, and set the recommended Initialization Control Words. We can tell you the exact values if this helps.

We are using Debian 9 and the version of igb is 5.3.5.10.

We follow the link setup instructions of 4.5.7.3.2 Auto-Negotiation Skipped (PCS_LCTL. AN ENABLE = 0b; CTRL.FRCSPD = 1b;CTRL.FRCDPLX = 1b)

CTRL.FD = 1b

CTRL.SLU = 1bCTRL.RFCE = 0bCTRL.TFCE = 0bCTRL.SPEED = 10b

PCS_LCTL.FSD = 1b

PCS_LCTL.Force Flow Control = 1bPCS_LCTL.FSV = 10bPCS_LCTL.FDV = 1b

http://PCS_LCTL.AN/ PCS_LCTL.AN TIMEOUT EN = 0b

CONNSW.ENRGSRC = 1 (The SRDS_[n]_SIG_DET is forced to high in our design, and CTRL.ILOS is set accordingly).

We are testing different module models. We haven't observed differences in their behavior so far.

Thank you very much.

[CarlosAM_INTEL]

Hello, ManuelS:

Thanks for your reply.

By the way, could you please verify that the CONNSW.ENRGSRC is set to 1?

We hope that this information may help you.

Best regards,

Carlos_A.

[MSanz]

Hello Carlos_A.,

Yes, it is.

Thank you.

[CarlosAM_INTEL]

Hello, ManuelS:

Thanks for your reply.

There is no Serdes BX Flashless iNVM file that is why we want to know how you configure it.

The SFP modules generally use SGMII to talk so we need to know what module you are using.

However, it is important to let you know that the TX or RX only mode for the Intel(R) Ethernet Controller I210 should be tested and validated on our own.

We hope that this information may help you.

Best regards,

Carlos_A.

[MSanz]

Hello Carlos. Thank you for your response. I will try to answer your questions:

We used the Serdes KX initial configuration provided by INTEL (I210_Invm_SerDesKX_NoAPM_v0.6.txt) with just the changes commented below:

CSRALD 0x5ba0 = 0x00001541 LPG_RST

CSRALD 0x0000 = 0x00001A41 SW_RST ; Changed from 0x08100241

WALD 0x0a = 0x402F PCIE_RST

WALD 0x1b = 0x3400 PCIE_RST

WALD 0x28 = 0x1C60 PCIE_RST

WALD 0x0d = 0x157C PCIE_RST

WALD 0x21 = 0x0584 PCIE_RST

WALD 0x24 = 0x80B1 PCIE_RST ; Changed from 0x0290

;Ordered_Section_Start

CSRALD 0x5B44 = 0x00FF00A8 LPG_RST ; [Intel] Set NFTS to 0xA8.

CSRALD 0x5B40 = 0x5E000090 LPG_RST ; [Intel] cont.

;Ordered_Section_End

Apart from MAC address, subsystem ID, subsystem vendor ID, and LED configuration, we only added the following value:

WALD 0X0f = 0x7600 PCIE_RST

The following information may be useful:

With the force link approach, comparing the controller registers when the controller transmits (having a valid signal on RX input) and when it doesn't (having no signal on RX input) we only find differences on the PCS Link Status Register (PCS_LSTS): The 'SYNC OK' bit is not set if no RX signal is present. (All other relevant registers remain unchanged).

>> The SFP modules generally use SGMII to talk so we need to know what module you are using.

The SFPs we are using support the SERDES interface.

>> However, it is important to let you know that the TX or RX only mode for the Intel(R) Ethernet Controller I210 should be tested and validated on our own.

Sorry, I'm not sure of having understood this sentence correctly…

Thank you in advance.

[CarlosAM_INTEL]

Hello, ManuelS:

Thanks for your reply.

Please let me paraphrase the sentence that you misunderstand, the TX or RX only mode is not a supported feature of the listed part. You may be able to make it work, but Intel does not recommend or support this option.

We hope that this information may help you.

Best regards,

Carlos_A.

[MSanz]

Hello Carlos,

Thank you for your answer.

Could you please recommend us some Intel document that might be useful for the intended purpose?

Is there another Intel Ehternet controller of your knowledge that supports this feature?

Best regards,

Manuel.

[CarlosAM_INTEL]

Hello, ManuelS:

Thanks for your reply.

In order to find other Ethernet Controller than better fits to your needs, please address your request by filling out the https://plan.seek.intel.com/us_en_embedded_registration-form-contactsaleswebform_html Intel - Contact Sales form.

On the other hand, it is important to let you know that we have made a deep research to find any information that can help you with the intended purpose without results.

We hope that this information may help you.

Best regards,

Carlos_A.

[MSanz]

Hello Carlos,

We appreciate your help.

Should you come across any piece of information that could be useful for our purpose, please let us know.

Thank you very much,

Manuel.

[CarlosAM_INTEL]

Hello, ManuelS:

Thanks for your reply.

Actually, as we mentioned the suggestion of other product should be provided by the Intel Sales team because form our side we have provided all the information that we have.

We hope Sales team will contact you as soon as possible.

Best regards,

Carlos_A.