When SR-IOV enabled on FreeBSD 12.0R, X540-T2 will not pass the traffic. driver vesion is ix-3.3.10.

[WQian6]

Download lastest ix-3.3.10 driver, modify Makefile to enabled SR-IOV by setting following

# Enable SR-IOV PF support SRIOV_ENABLE = 1

Compile without error. copy if_ix.ko to /boot/modules/if_ix_intel.ko.

Modify /boot/loader.conf add following line

if_ix_intel_load="YES"

reboot and make sure driver is loaded by

root@x10drl:/boot # kldstat Id Refs Address Size Name 1 47 0xffffffff80200000 243d250 kernel 2 1 0xffffffff8263e000 11c68 ipmi.ko 3 2 0xffffffff82650000 2e18 smbus.ko 4 1 0xffffffff82653000 3a9a10 zfs.ko 5 2 0xffffffff829fd000 a4f0 opensolaris.ko 6 1 0xffffffff82a08000 58d10 if_ix_intel.ko 7 1 0xffffffff82c22000 81f0 tmpfs.ko 8 1 0xffffffff82c2b000 7078 ioat.ko 9 1 0xffffffff82c33000 1800 uhid.ko 10 1 0xffffffff82c35000 23a8 ums.ko 11 1 0xffffffff82c38000 537460 vmm.ko 12 1 0xffffffff83170000 b30 nmdm.ko 13 1 0xffffffff83171000 6fb3 if_bridge.ko 14 1 0xffffffff83178000 4410 bridgestp.ko 15 1 0xffffffff8317d000 2e77 if_tap.ko

and create a /etc/iovctl.conf as following

PF { device : ix1; num_vfs : 3; } DEFAULT { passthrough : true; } # VF for use by host VF-0 { passthrough : false; }

execute iovctl -C -f /etc/iovctl.conf to activate SR-IOV function.

ifconfig confirm VF-0 is created.

ixv0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether 2a:ca:77:cf:89:14 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Add IP to interface

root@x10drl:/boot # ifconfig ixv0 up root@x10drl:/boot # ifconfig ixv0 192.168.10.22/24

I can ping ixv0 and ix1 from this very host. but cannot ping this two addresses from outside machine, nor ping outside addresss from this very host.

am I missing something?

thanks.

[Caguicla_Intel]

Hello WQian6,

Thank you for posting in Intel Ethernet Communities. 

Before we proceed in checking your query, we'd like to double check if the ix1 can be accessed from outside machine even if SR-IOV is not configured.

We look forward to your response.

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[Caguicla_Intel]

Hello WQian6,

We'd like to follow up the information that we requested for us to proceed in checking your query.

Looking forward to your reply.

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[Caguicla_Intel]

Hello WQian6,

Please be informed that we will proceed closing this inquiry since we haven't receive any response from our previous follow up. Should you have any other concern or further assistance in the future, kindly post a new question.

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[WQian6]

Sorry for the late response. I make another test machine. with FreeBSD 12.1R.

my environment is

Host A(FreeBSD 12.0) -----------(Port 1, ix1) Host B(FreeBSD 12.1) (Port 0, ix0)---------- HOST C(Windows 8.1)

-------172.16.2.253 -------------- 172.16.2.239 172.16.1.239 -----------172.16.1.202

Host B is the test machine.

compile ix-3.3.10, load again.

root@moon:~ # kldstat Id Refs Address Size Name 1 51 0xffffffff80200000 2448d90 kernel 2 1 0xffffffff82649000 58f30 if_ix_intel.ko 3 1 0xffffffff826a2000 2380 cc_htcp.ko 4 1 0xffffffff826a5000 3a99a8 zfs.ko 5 2 0xffffffff82a4f000 a5b8 opensolaris.ko 6 1 0xffffffff82c21000 70a8 ioat.ko 7 1 0xffffffff82c29000 18a0 uhid.ko 8 1 0xffffffff82c2b000 2928 ums.ko 9 2 0xffffffff82c2e000 3f0c8 ctl.ko 10 1 0xffffffff82c6e000 61f8 cfiscsi.ko 11 1 0xffffffff82c75000 b0cc iscsi.ko 12 1 0xffffffff82c81000 acf mac_ntpd.ko

so I can Pint Host B from A

$ ping 172.16.2.239 PING 172.16.2.239 (172.16.2.239): 56 data bytes 64 bytes from 172.16.2.239: icmp_seq=0 ttl=64 time=0.150 ms 64 bytes from 172.16.2.239: icmp_seq=1 ttl=64 time=0.118 ms 64 bytes from 172.16.2.239: icmp_seq=2 ttl=64 time=0.061 ms ^C --- 172.16.2.239 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.061/0.110/0.150/0.037 ms

and from host C

C:\Users\user01>ping -t 172.16.1.239   172.16.1.239 に ping を送信しています 32 バイトのデータ: 172.16.1.239 からの応答: バイト数 =32 時間 <1ms TTL=64 172.16.1.239 からの応答: バイト数 =32 時間 <1ms TTL=64 172.16.1.239 からの応答: バイト数 =32 時間 <1ms TTL=64 172.16.1.239 からの応答: バイト数 =32 時間 <1ms TTL=64 172.16.1.239 からの応答: バイト数 =32 時間 <1ms TTL=64 172.16.1.239 からの応答: バイト数 =32 時間 <1ms TTL=64

without SR-IOV, all work fine

create /etc/iovctl.conf as previous post.

root@moon:~ # cat /etc/iovctl.conf PF { device : ix1; num_vfs : 3; } DEFAULT { passthrough : true; } # VF for use by host VF-0 { passthrough : false; }

active iov function by following command without any error

root@moon:~ # iovctl -C -f /etc/iovctl.conf root@moon:~ #

check demsg, ixv0 is view by host, and pci2 0.131 and 0.133 ppt are also created.

ixv0: <Intel(R) PRO/10GbE Virtual Function Network Driver> at device 0.129 on pci2 ixv0: Using 2048 TX descriptors and 2048 RX descriptors ixv0: Using 1 RX queues 1 TX queues ixv0: Using MSI-X interrupts with 2 vectors ixv0: allocated for 1 queues ixv0: allocated for 1 rx queues ixv0: Ethernet address: 52:2e:f0:01:39:12 ixv0: netmap queues/slots: TX 1/2048, RX 1/2048 pci2: <network, ethernet> at device 0.131 (no driver attached) pci2: <network, ethernet> at device 0.133 (no driver attached)

ifconfig of ixv0 and ix1

root@moon:~ # ifconfig ixv0 ixv0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether 52:2e:f0:01:39:12 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>root@moon:~ # ifconfig ix1 ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000 options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether a0:36:9f:83:cc:aa inet 172.16.2.239 netmask 0xffffff00 broadcast 172.16.2.255 media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

ping ix1 from host A again, no response.

$ ping 172.16.2.239 PING 172.16.2.239 (172.16.2.239): 56 data bytes ^C --- 172.16.2.239 ping statistics --- 7 packets transmitted, 0 packets received, 100.0% packet loss

assign ip to ixv0 and bring it up

root@moon:~ # ifconfig ixv0 172.16.2.240/24 up root@moon:~ # ifconfig ixv0 ixv0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether 52:2e:f0:01:39:12 inet 172.16.2.240 netmask 0xffffff00 broadcast 172.16.2.255 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

ping ixv0 from outside, no response also.

$ ping 172.16.2.240 PING 172.16.2.240 (172.16.2.240): 56 data bytes ^C --- 172.16.2.240 ping statistics --- 4 packets transmitted, 0 packets received, 100.0% packet loss

ping both ip from host B, it is fine.

root@moon:~ # ping 172.16.2.239 PING 172.16.2.239 (172.16.2.239): 56 data bytes 64 bytes from 172.16.2.239: icmp_seq=0 ttl=64 time=0.055 ms 64 bytes from 172.16.2.239: icmp_seq=1 ttl=64 time=0.051 ms ^C --- 172.16.2.239 ping statistics --- 2 packets transmitted, 2 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.051/0.053/0.055/0.002 ms root@moon:~ # ping 172.16.2.240 PING 172.16.2.240 (172.16.2.240): 56 data bytes 64 bytes from 172.16.2.240: icmp_seq=0 ttl=64 time=0.050 ms 64 bytes from 172.16.2.240: icmp_seq=1 ttl=64 time=0.037 ms ^C --- 172.16.2.240 ping statistics --- 2 packets transmitted, 2 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.037/0.043/0.050/0.006 ms

since ix0 is not touch, it works fine.