- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi All,
I am trying to compile DPDK with Mellanox driver support and test pktgen on Ubuntu 18.04/16.04
on Azure. I have created the VM Ubuntu 18.04/16.04 with two interfaces with accelerated networking
enabled. I have compiled DPDK with MLX4/5 enabled successfully followed by PKTGEN with appropriate
targets. However when the Mellanox interfaces are bound to mlx4_core using dpdk-devbind.py
the interfaces doesn't show up as DPDK network devices and rather show up as using Network Devices.
However when the interfaces are bound to igb_uio driver the interfaces show up as DPDK compatible
network devices.
DPDK version used: dpdk-19.02
PKTGEN: pktgen-3.6.5
After installing all dependencies, DPDK is compiled with the following steps
Make config T=x86_64-native-linuxapp-gcc
To enable MLX driver:
sed -ri 's,(MLX._PMD=)n,\1y,' build/.config.
and
make install T=x86_64-native-linuxapp-gcc DESTDIR=<PKTGEN_DIR>
Please note that I have installed libibverbs and have not installed full version of OFED i.e. MLNX_OFED_LINUX.
Installation in /home/<>/dpdk-19.02// complete
Similarly PKTGEN is also compiled and built with the already compiled DPDK SDK.
What I notice is that these interfaces show up as DPDK compatible
interfaces when bound to igb_uio but when bound to mlx4_uio, they show up as Kernel
Output of dpdk-devbind.py with -b with igb_uio and mlx4_core
When bound to igb_uio
Network devices using DPDK-compatible driver
============================================
0001:00:02.0 'MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 1004' drv=igb_uio unused=
0002:00:02.0 'MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 1004' drv=igb_uio unused=
0003:00:02.0 'MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 1004' drv=igb_uio unused=
When bound to mlx4_core
Network devices using kernel driver
===================================
0001:00:02.0 'MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 1004' if=rename20 drv=mlx4_core unused=igb_uio
0002:00:02.0 'MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 1004' if=enP2s2 drv=mlx4_core unused=igb_uio
0003:00:02.0 'MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] 1004' if=enP3s3 drv=mlx4_core unused=igb_uio
Please note that the devices are using kernel driver and not DPDK MLX driver.
~/dpdk-19.02$ lspci
0000:00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (AGP disabled) (rev 03)
0000:00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 01)
0000:00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
0000:00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 02)
0000:00:08.0 VGA compatible controller: Microsoft Corporation Hyper-V virtual VGA
0001:00:02.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
0002:00:02.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
0003:00:02.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
dpdk-19.02$ ifconfig -a
enP1p0s2 Link encap:Ethernet HWaddr 00:0d:3a:f2:08:6b
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:9773 errors:0 dropped:0 overruns:0 frame:0
TX packets:15939 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7116204 (7.1 MB) TX bytes:3611453 (3.6 MB)
enP2p0s2 Link encap:Ethernet HWaddr 00:0d:3a:f2:f5:2c
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:15 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:1690 (1.6 KB)
enP3p0s2 Link encap:Ethernet HWaddr 00:0d:3a:f2:f0:7c
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:15 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:1690 (1.6 KB)
eth0 Link encap:Ethernet HWaddr 00:0d:3a:f2:08:6b
inet addr:10.0.0.4 Bcast:10.0.0.255 Mask:255.255.255.0
inet6 addr: fe80::20d:3aff:fef2:86b/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:11519 errors:0 dropped:0 overruns:0 frame:0
TX packets:15414 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:8502266 (8.5 MB) TX bytes:3579927 (3.5 MB)
eth1 Link encap:Ethernet HWaddr 00:0d:3a:f2:f5:2c
inet addr:10.0.1.4 Bcast:10.0.1.255 Mask:255.255.255.0
inet6 addr: fe80::20d:3aff:fef2:f52c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2 errors:0 dropped:0 overruns:0 frame:0
TX packets:15 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:762 (762.0 B) TX bytes:1690 (1.6 KB)
Do I need to do anything else to get this working? I mean bind Mellanox interfaces
As DPDK compatible interfaces? Packet generation is not working which I think is due
To this issue. In summary while the compilation goes through, DPDK MLX drivers doesn’t
Seem to get loaded and attach to the interfaces.
I have referred to https://doc.dpdk.org/guides/nics/mlx4.html as well while working on compiling DPDK.
Other thing I would like to know is if installing MLNX_OFED_LINUX is a must in this environment. I am facing
Issues while installing OFED drivers but DPDK compilation seems to go through fine with just libibverbs. Please
Let me know if you would like to know info. Any help will be greatly appreciated.
- Tags:
- DPDK
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have a similar issue on CentOs as well ( on Azure ). Post compilation I see that interfaces are using mlx4_core drivers but am unable to bind the
interfaces to DPDK drivers. However this time I get an error that DPDK drivers are not at all loaded. Seems like I am missing something basic.
[root@phanidpdkcentos usertools]# ./dpdk-devbind.py -b igb_uio 0002:00:02.0
Error - no supported modules(DPDK driver) are loaded
Any hints at what probably is missing?
I am following the instruction from
https://docs.microsoft.com/en-us/azure/virtual-network/setup-dpdk and
https://doc.dpdk.org/guides/nics/mlx4.html
for compilation
[root@phanidpdkcentos usertools]# lspci -s 0002:00:02.0 -vv
0002:00:02.0 Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
Subsystem: Mellanox Technologies Device 61b0
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0
Region 2: Memory at fe0800000 (64-bit, prefetchable) [size=8M]
Capabilities: [60] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset+ SlotPowerLimit 0.000W
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- FLReset-
MaxPayload 128 bytes, MaxReadReq 128 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x8, ASPM not supported, Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed unknown, Width x0, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [9c] MSI-X: Enable+ Count=24 Masked-
Vector table: BAR=2 offset=00002000
PBA: BAR=2 offset=00003000
Capabilities: [40] Power Management version 0
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Kernel driver in use: mlx4_core
Kernel modules: mlx4_core
![](/skins/images/872293744008A34B36F8ABF94A46CC66/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page