Community
cancel
Showing results for 
Search instead for 
Did you mean: 
TSuzu6
Beginner
2,433 Views

82599EN repeats Link Up and Down

Hi all

We use ixgbe driver for 82599EN on linux. driver version is 4.1.2

diriver is downloaded at https://downloadcenter.intel.com/download/25463/Network-Adapter-Driver-for-PCI-E-Intel-10-Gigabit-Et... https://downloadcenter.intel.com/download/25463/Network-Adapter-Driver-for-PCI-E-Intel-10-Gigabit-Et....

[test condition]:

OS:Linux kernel 3.10.31-ltsi

driver:Network Adapter Driver for PCI-E* Intel® 10 Gigabit Ethernet Network Connections under Linux* ver4.1.2

device:Intel 82599EN

PHY I/F:SFP module(FINISAR FTLX8573D3BTL) + MMF Fiber cable

EEPROM data:82599EN_SFI_NO_MNG_4.40.bin(modiy MAC address)

[steps]:

1.insmod mdio.ko

2.insmod ixgbe.ko

3.ifconfig eth2 up

4.ifconfig eth2 192.168.2.100

5.connect MMF cable to 10GbE Tester(SPIRENT C1).

[result]

------------------------------------------------------------------------------

mailto:root@socfpga:/usr/fsim root@socfpga:/usr/fsim# insmod ixgbe.ko

 

ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 3.13.10-k

 

ixgbe: Copyright (c) 1999-2013 Intel Corporation.

 

ixgbe_probe

 

PCI: enabling device 0000:01:00.0 (0140 -> 0142)

 

PCI resource c0000000 + 00020000

 

ixgbe 0000:01:00.0: Multiqueue Disabled: Rx Queue count = 1, Tx Queue count = 1

 

ixgbe 0000:01:00.0: (PCI Express:5.0GT/s:Width x4) 00:a0:c9:12:34:56

 

ixgbe 0000:01:00.0: MAC: 2, PHY: 12, SFP+: 5, PBA No: FFFFFF-0FF

 

ixgbe 0000:01:00.0: PCI-Express bandwidth available for this card is not sufficient for optimal performance.

 

ixgbe 0000:01:00.0: For optimal performance a x8 PCI-Express slot is required.

 

ixgbe 0000:01:00.0: Intel(R) 10 Gigabit Network Connection

 

platform leds.7: Driver leds-gpio requests probe deferral

---------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------

mailto:root@socfpga:/usr/fsim root@socfpga:/usr/fsim# ifconfig eth2

 

eth2 Link encap:Ethernet HWaddr 00:a0:c9:12:34:56

 

inet addr:192.168.2.100 Bcast:192.168.2.255 Mask:255.255.255.0

 

UP BROADCAST MULTICAST MTU:1500 Metric:1

 

RX packets:0 errors:0 dropped:0 overruns:0 frame:0

 

TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

 

collisions:0 txqueuelen:1000

 

RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------------

mailto:root@socfpga:/usr/fsim root@socfpga:/usr/fsim# ixgbe 0000:01:00.0 eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX

 

ixgbe 0000:01:00.0 eth2: NIC Link is Down

 

ixgbe 0000:01:00.0 eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX

 

ixgbe 0000:01:00.0 eth2: NIC Link is Down

 

ixgbe 0000:01:00.0 eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX

 

ixgbe 0000:01:00.0 eth2: NIC Link is Down

 

ixgbe 0000:01:00.0 eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX

 

ixgbe 0000:01:00.0 eth2: NIC Link is Down

 

ixgbe 0000:01:00.0 eth2: NIC Link is Up 10 Gbps, Flow Control: RX/TX

 

ixgbe 0000:01:00.0 eth2: NIC Link is Down

--------------------------------------------------------------------------------------------------

Our issue is that link up and down is repeated. How should we resolve this issue ?

BR,

Taira Suzuki

0 Kudos
16 Replies
idata
Community Manager
566 Views

Hi Taira,

 

 

Thank you for the post. Have you tried using the newer version 4.3.15

 

https://downloadcenter.intel.com/download/14687/Network-Adapter-Driver-for-PCI-E-Intel-10-Gigabit-Et...

 

 

rgds,

 

wb

 

TSuzu6
Beginner
566 Views

Hi wb

I tried to use ixgbe ver4.3.15. But the issue is occured again.

I can not resolve this issue. eth2(target eth) status is as below.

[staus result]

mailto:root@socfpga:/usr/fsim root@socfpga:/usr/fsim# ethtool eth2

Settings for eth2:

Supported ports: [ FIBRE ]

Supported link modes: 10000baseT/Full

Supported pause frame use: No

Supports auto-negotiation: No

Advertised link modes: 10000baseT/Full

Advertised pause frame use: Symmetric

Advertised auto-negotiation: No

Speed: Unknown!

Duplex: Unknown! (255)

Port: FIBRE

PHYAD: 0

Transceiver: external

Auto-negotiation: off

Supports Wake-on: d

Wake-on: d

Current message level: 0x00000007 (7)

drv probe link

Link detected: no

mailto:root@socfpga:/usr/fsim root@socfpga:/usr/fsim# ethtool -i eth2

driver: ixgbe

version: 4.3.15

firmware-version: 0x80000707

bus-info: 0000:01:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: no

mailto:root@socfpga:/usr/fsim root@socfpga:/usr/fsim#

I doubt that " Supports auto-negotiation: No" is a reason for repeating link up and down.

Could you tell a next step to resolve ?

BR,

Taira

idata
Community Manager
566 Views

Hi Taira,

 

 

Thank you for the update. Let me further check on this issue.

 

 

rgds,

 

wb

 

idata
Community Manager
566 Views

Hi Taira,

 

 

 

Checking on the fiber module used, FINISAR FTLX8573D3BTL is not a supported fiber module for Intel X520 network adapter. Here is the website for reference:

 

 

http://www.intel.com/content/www/us/en/support/network-and-i-o/ethernet-products/000005688.html

 

 

If you are using Intel X520 server adapter, you need to check on the supported fiber module. If not, then please confirm the following information:

 

 

1) Are you using an onboard NIC? If yes, what is the brand and model of your system?

 

2) If it is onboard NIC, have you contacted the system vendor to check with them?

 

3) With regards to the autonegotiation, as stated on our README.txt

 

https://downloadmirror.intel.com/13663/eng/README.txt

 

 

"An Intel(R) Network Adapter using fiber-based connections, however, will not

 

attempt to auto-negotiate with its link partner since those adapters operate

 

only in full duplex and only at their native speed."

 

 

4) Have you try testing on another switch to isolate the issue?

 

 

 

Please feel free to update me.

 

 

Thanks,

 

wb

 

TSuzu6
Beginner
566 Views

Hi wb

I answer your questions.

1) Are you using an onboard NIC? If yes, what is the brand and model of your system?

No, I use 82599EN for our PCIe exntended board. This PCI extended board is connected to mother board via PCI slot.

2) If it is onboard NIC, have you contacted the system vendor to check with them?

It is not onboard NIC.

3) With regards to the autonegotiation, as stated on our README.txt

I understand that fiber-based connections can not be auto-negotiated.

4) Have you try testing on another switch to isolate the issue?

No, but I used a tester(SPIRENT C1). The SFP modules and cable which I use can be linked between tester port A and tester port B.

Regurding to SFP module, I refer the site which you direct.

According to the site, FTLX8571D3BCV-IT works with 82599. So I order FTLX8571D3BCV without -IT.

When I will get FTLX8571D3BCV, I will try and update soon.

BR,

Taira

idata
Community Manager
566 Views

Hi Taira,

 

 

Thank you for the clarification. Please update me the test result of using FTLX8571D3BCV fiber module.

 

 

Looking forward to your update.

 

 

rgds,

 

wb

 

TSuzu6
Beginner
566 Views

Hi! wb

I tried FTLX8571D3BCV SFP module. But the issue can not be solved.

Same messages appear on linux console.

If you have another method for effecive debug, Please tell.

BR,

Taira

idata
Community Manager
566 Views

Hi Taira,

 

 

Thank you for the update, can you provide me the serial number of your X520 server adapter? You can locate the serial number on the network adapter's white sticker (NOT on the box).

 

Please provide me the number that has the format as follow:

 

 

12 alphanumeric + 5 digits + 6 digits -3 digits

 

 

Thanks,

 

wb

 

 

 

TSuzu6
Beginner
566 Views

Hi! wb

No, We use 82599EN on the board which is designed by ourself.

This board is connected to mother board via PCI-e.

ixgbe driver runs on Linux with ALTERA FPGA(ArriaV SoC).

We performed ethtool -t eth2. The result is below.

The test result is FAIL

The test extra info:

Register test (offline) 0

Eeprom test (offline) 0

Interrupt test (offline) 0

Loopback test (offline) 0

Link test (on/offline) 1

Link test is failed. What should we do for preventing to fail link test ?

BR,

Taira

idata
Community Manager
566 Views

Hi Taira,

 

 

Thank you for the clarification. As this is an ethernet controller used on the board designed by your company, which is a setup involves third party board. Please contact your Intel Field Application engineer for the support or contact the point of purchase .

 

 

Thanks,

 

wb
TSuzu6
Beginner
566 Views

Hi wb

Sorry for generating another thread.

I contacted to Inetl Japan and local distributor.

But they can not supprt my issue. Because they do not have experience to support a device which is implemented on non-Intel board.

So I wish you to help my issue.

I constructed test enviroment for 1Gb ether link as below,

*[82599EN]-->[FNISAR FTLX8571D3BCV(1G/10G)]-->[Fiber cable]-->[FNISAR FTLX8571D3BCV(1G/10G)]-->[Tester(SPIRENT)]

82599EN can link up on this enviroment.

And next,I constructed test enviroment for 10Gb ether link as below,

*[82599EN]-->[FNISAR FTLX8573D3BTL(10G)]-->[Fiber cable]-->[FNISAR FTLX8573D3BTL(10G)]-->[Tester(SPIRENT)]

The issue(link up/down) is occured on this enviroment.

[Detail]

When I run an ixgbe dirver and enter "ifconfig eth2 up", link up and down are displaied repeatly in linux console.

Then, I read the Link Status Register — LINKS (0x042A4; RO).

read value is 0x79081113=0111_1001_0000_1000_0001_0001_0001_0011.

I organize above value to the register field of LINKS.

[11:8]KX4_SIG_DET 0001b

Signal Detect of 10 GbE Parallel (KX4, CX4 or XAUI).Bit[11, 10, 9, 8] shows lane <3,2,1,0> status, respectively.

For each bit:0b = A signal is not present (failed). 1b = A signal is present (good).

[12]KR_SIG_DET 1b

Signal Detect of 10 GbE serial (KR or SFI) success.

[17]10G Align Status 0b

10 GbE Align_status failed (deskew process not complete).

I change SFP module to FINISAR FTLX8571D3BCV. FTLX8571D3BCV can work for 1GbE or 10GbE.

When I connect to 1GbE tester via this SFP module, 82599EN can keep a link up status.

If I should confirm the some registers, please tell.

Could you estimate causes which occur link up/down repeatly ?

BR,

Taira

idata
Community Manager
566 Views

HiTaira,

 

 

Thank you for the update and taking time to check. Let me further check for you.

 

 

rgds,

 

wb

 

TSuzu6
Beginner
566 Views

Hi wb

Thank you for your support continuously.

I notice that the register "LINKS" has strange value.

bit 30 "Link up" is 1b which means "Link is up."

bit 7 "Link Status is 0b which means "Link is currently down or link was down since last time read."

I read register value as below, when the issue was occured.

reg[00004200] = 80000008

reg[00004208] = 0204000e

reg[0000420c] = 0000000e

reg[00004218] = 000001a0

reg[0000421c] = 00000000

reg[00004220] = 00000000

reg[00004224] = 00000000

reg[00004240] = 08002fff

reg[00004244] = 00000381

reg[00004248] = 0000ffff

reg[0000425c] = 10000000

reg[00004260] = 00000000

reg[00004268] = 05ee0000

reg[00004288] = 00000080

reg[0000428c] = 00008400

reg[00004290] = 04000000

reg[00004298] = 1b1b0000

reg[0000429c] = 461f02f0

reg[000042a0] = c09c6084

reg[000042a4] = 79081113

reg[00004324] = 00000012

reg[000042a8] = 000a0000

reg[000042b0] = 00090000

reg[000042b4] = 00000000

reg[000042d0] = 00000000

reg[000042d4] = 00000000

reg[000042d8] = 00000000

reg[000042e0] = 10010000

reg[000042e4] = 822ffff8

reg[000042e8] = 00000000

reg[000042ec] = 00000000

reg[00014f00] = 0000006d

reg[00014f10] = 00000000

reg[00004294] = 0000000a

reg[00004314] = 00020000

reg[00004000] = 00000000

reg[00004004] = 00000000

reg[00004008] = 00000000

reg[00004034] = 00000013

reg[00004038] = 0010e891

reg[00004040] = 00000000

reg[00008780] = 00000000

reg[000041a4] = 00000000

reg[000041a8] = 00000000

reg[00004140] = 00000000

reg[00004144] = 00000000

reg[00004148] = 00000000

reg[0000414c] = 00000000

reg[00004150] = 00000000

reg[00004154] = 00000000

reg[00004158] = 00000000

reg[0000415c] = 00000000

reg[00004160] = 00000000

reg[00004164] = 00000000

reg[00004168] = 00000000

reg[0000416c] = 00000000

reg[00004170] = 00000000

reg[00004174] = 00000000

reg[00004178] = 00000000

reg[0000417c] = 00000000

reg[0000405c] = 00000000

reg[00004060] = 00000000

reg[00004064] = 00000000

reg[00004068] = 00000000

reg[0000406c] = 00000000

reg[00004070] = 00000000

reg[00004078] = 00000000

reg[0000407c] = 00000000

reg[00004074] = 00000000

reg[00004088] = 00000000

reg[0000408c] = 00000000

reg[000041b0] = 00000000

reg[000041b4] = 00000000

reg[000041b8] = 00000000

reg[00002f50] = 00000000

reg[00002f54] = 00000000

reg[00002f58] = 00000000

reg[00002f5c] = 00000000

reg[00002f60] = 00000000

reg[00002f64] = 00000000

reg[00002f68] = 00000000

reg[00002f6c] = 00000000

reg[00002f70] = 00000000

reg[00002f74] = 00000000

reg[00002f78] = 00000000

reg[00002f7c] = 00000000

reg[00004080] = 00000000

reg[00004090] = 00000000

reg[00004094] = 00000000

reg[000087a0] = 00000000

reg[000087a4] = 00000000

reg[000087a8] = 00000000

reg[000040a4] = 00000000

reg[000040a8] = 00000000

reg[000040ac] = 00000000

reg[000040b0] = 00000000

reg[000040b4] = 00000000

reg[000040b8] = 00000000

reg[000040c0] = 00000000

reg[000040c4] = 00000000

reg[000040d0] = 00000000

reg[000040d4] = 00000000

reg[000040d8] = 00000000

reg[000040dc] = 00000000

reg[000040e0] = 00000000

reg[000040e4] = 00000000

reg[000040e8] = 00000000

reg[000040ec] = 00000000

reg[000040f0] = 00000000

reg[000040f4] = 00000000

reg[00004010] = 00000000

reg[00004120] = 00000000

reg[00002300] = 00000000

reg[00002304] = 00000000

reg[00002308] = 00000000

reg[0000230c] = 00000000

reg[00002310] = 00000000

reg[00002314] = 00000000

reg[00002318] = 00000000

reg[0000231c] = 00000000

reg[00002320] = 00000000

reg[00002324] = 00000000

reg[00002328] = 00000000

reg[0000232c] = 00000000

reg[00002330] = 00000000

reg[00002334] = 00000000

reg[00002338] = 00000000

reg[0000233c] = 00000000

reg[00002340] = 00000000

reg[00002344] = 00000000

reg[00002348] = 00000000

reg[0000234c] = 00000000

reg[00002350] = 00000000

reg[00002354] = 00000000

reg[00002358] = 00000000

reg[0000235c] = 00000000

reg[00002360] = 00000000

reg[00002364] = 00000000

reg[00002368] = 00000000

reg[0000236c] = 00000000

reg[00002f40] = 00000000

reg[00005118] = 00000000

reg[0000241c] = 00000000

reg[00002424] = 00000000

reg[00002428] = 00000000

reg[0000242c] = 00000000

reg[00008784] = 00000000

reg[00008788] = 00000000

I hope that above result is useful for yur analysis.

BR,

Taira

idata
Community Manager
566 Views

Hi Taira ,

 

 

Just want to clarify, I noticed that you mentioend.

 

"I change SFP module to FINISAR FTLX8571D3BCV. FTLX8571D3BCV can work for 1GbE or 10GbE.

 

When I connect to 1GbE tester via this SFP module, 82599EN can keep a link up status. "

 

 

Are you saying that you are able to keep the link up status with FINISAR FTLX8271D38CV?

 

 

If yes, then please continue used the supported fiber modules.

 

 

You may also tried newer driver package 4.4.6 available at sourcefore

 

 

https://sourceforge.net/projects/e1000/files/ixgbe%20stable/

 

 

 

 

Thanks,

 

wb

 

TSuzu6
Beginner
566 Views

Hi wb

Thank you for your response.

Yes, I keep the link up status with FINISAR FTLX8271D38CV on 1Gbps speed.

I suspected the reference clock because the link up status could be kept on1Gbps.

Then, I measured a reference clock and found the cause for this issue.

The reference clock had a noise. So I replaced clock device near a 82599EN.

I connect to SPIRENT C1 tester after this. And I confirmed that the link up status is kept on 10Gbps.

The issue is solved.The cause is noisy reference clock.

Thank you for your analysis and a lot of advice.

BR,

Taira.

idata
Community Manager
566 Views

Hi Taira,

 

 

Thank you for sharing your test result. I am glad the issue is fixed and we will take note of this useful information.

 

 

rgds,

 

wb

 

Reply