No link between X520-DA2 and Netgear GS752TXS using Direct Attach

[idata]

Hi,

I've tried to connect my X520-DA2 to a Netgear GS752TXS using a 7m direct attach cable (Cisco SFP-H10GB-ACU7M) but I get no link.

I've tried both NIC ports and all the SFP+ switch ports.

What could be the problem?

Cheers,

Filipe

[Mark_H_Intel]

Do you have no link layer connection or is this a case of not passing traffic? Are any messages being logged that indicate an error? What OS are you using?

I do not know why you cannot get a connection, but here are a few things you could look at to help troubleshoot:

1. Upgrade to the latest driver. Some driver updates do have bug fixes for direct attach cable issues.
2. Try a totally different cable model and length in case there is some issue that shows up with the particular model and length you are using.
3. You might also check the way you have the cable installed. Some cables might not work right if your bend radius is to great.

Mark H

[idata]

I believe I have no connection (there's no light on the switch side when I connect the cable). I do not see any error messages. I'm using Scientific Linux 6.3 (RHEL clone).

1. I can try to upgrade but this seem to be even more fundamental than a driver problem.

2. I would like to try that but unfortunately I currently only have that cable.

3. I will check the but I don't think there are any tight bends.

Here are the messages right after the modprobe ixgbe if it's of any help:

ixgbe 0000:82:00.1: complete

ixgbe 0000:82:00.1: PCI INT B disabled

ixgbe 0000:82:00.0: complete

ixgbe 0000:82:00.0: PCI INT A disabled

ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 3.6.7-k

ixgbe: Copyright (c) 1999-2012 Intel Corporation.

ixgbe 0000:82:00.0: PCI INT A -> GSI 50 (level, low) -> IRQ 50

ixgbe 0000:82:00.0: setting latency timer to 64

ixgbe 0000:82:00.0: irq 179 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 180 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 181 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 182 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 183 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 184 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 185 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 186 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 187 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 188 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 189 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 190 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 191 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 192 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 193 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 194 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 195 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 196 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 197 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 198 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 199 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 200 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 201 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 202 for MSI/MSI-X

ixgbe 0000:82:00.0: irq 203 for MSI/MSI-X

ixgbe 0000:82:00.0: Multiqueue Enabled: Rx Queue count = 24, Tx Queue count = 24

ixgbe 0000:82:00.0: (PCI Express:5.0GT/s:Width x8) 90:e2:ba:3a:6e:04

ixgbe 0000:82:00.0: MAC: 2, PHY: 10, SFP+: 7, PBA No: E68793-005

ixgbe 0000:82:00.0: Intel(R) 10 Gigabit Network Connection

ixgbe 0000:82:00.1: PCI INT B -> GSI 52 (level, low) -> IRQ 52

ixgbe 0000:82:00.1: setting latency timer to 64

ixgbe 0000:82:00.1: irq 204 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 205 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 206 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 207 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 208 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 209 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 210 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 211 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 212 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 213 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 214 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 215 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 216 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 217 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 218 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 219 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 220 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 221 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 222 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 223 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 224 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 225 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 226 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 227 for MSI/MSI-X

ixgbe 0000:82:00.1: irq 228 for MSI/MSI-X

ixgbe 0000:82:00.1: Multiqueue Enabled: Rx Queue count = 24, Tx Queue count = 24

ixgbe 0000:82:00.1: (PCI Express:5.0GT/s:Width x8) 90:e2:ba:3a:6e:05

ixgbe 0000:82:00.1: MAC: 2, PHY: 1, PBA No: E68793-005

ixgbe 0000:82:00.1: Intel(R) 10 Gigabit Network Connection

[Mark_H_Intel]

I agree that this doesn't seem like a simple driver problem, but I know there were some changes related to some features implemented in the SFP+ modules of some cables that can affect the timing. So a driver update might help.

I also recall one other place that was using the exact same model of cable you are using that was having problems with dropped connections. They switched to a 10 meter cable and the drops cleared up. That might have been a coincidence, but these are the ideas that come to mind.

As to bend radius, some brands of cables are more forgiving of sharp bends than others. I don't know anything specific about the Cisco cables and bend radius, but I thought I would throw that out there just in case.

I am not experienced in looking at the modprobe messages, but as far as I can tell, there is not anything that stands out there. However, if you have another available x8 PCIe slot, you might want to try switching slots just to see if anything changes. Sometimes the PCIe bus for different slots connects to different chips in the same server, so I never know what switching slots might do. There might not be any difference in your server.

Mark H