- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello!
I have a problem with Intel XL710, after 22 Gbps of traffic some cores of processor loads 100% IRQ and traffic goes down.
There are latast firmware and driver:
# ethtool -i ens2
driver: i40e
version: 2.24.6
firmware-version: 9.40 0x8000ecc0 1.3429.0
expansion-rom-version:
bus-info: 0000:d8:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
I read this guide
And made all possible configurations. But nothing changes
382 set_irq_affinity local ens2
384 set_irq_affinity all ens2
387 ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 0 tx-usecs 0
389 ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 5000 tx-usecs 20000
391 ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 10000 tx-usecs 20000
393 ethtool -g ens2
394 ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 84 tx-usecs 84
396 ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 62 tx-usecs 62
398 ethtool -S ens2 | grep drop
399 ethtool -S ens2 | grep drop
400 ethtool -S ens2 | grep drop
401 ethtool -S ens2 | grep drop
402 ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 336 tx-usecs 84
403 ethtool -S ens2 | grep drop
404 ethtool -S ens2 | grep drop
406 ethtool -S ens2 | grep drop
407 ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 672 tx-usecs 84
408 ethtool -S ens2 | grep drop
409 ethtool -S ens2 | grep drop
411 ethtool -S ens2 | grep drop
412 ethtool -S ens2 | grep drop
425 ethtool -S ens2 | grep drop
426 ethtool -S ens2 | grep drop
427 ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 8400 tx-usecs 840
428 ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 4200 tx-usecs 840
430 ethtool -S ens2 | grep drop
431 ethtool -S ens2 | grep drop
432 ethtool -S ens2 | grep drop
433 ethtool -S ens2 | grep drop
434 ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 4200 tx-usecs 1680
435 ethtool -S ens2 | grep drop
436 ethtool -S ens2 | grep drop
439 ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 3200 tx-usecs 3200
469 ethtool -a ens2
472 ethtool ens2
473 ethtool -i ens2
475 ethtool -i ens2
476 ethtool ens2
482 ethtool -C ens2 adaptive-rx on
484 ethtool -c ens2
486 ethtool -C ens2 adaptive-tx on
487 ethtool -c ens2
492 history | grep ens2
494 ethtool -m ens2
499 ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 4200 tx-usecs 1600
501 history | grep ens2
Server configuration:
80 cores Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz
RAM 960 GB
8 x SAMSUNG MZQLB7T6HMLA-000AZ NVME disks
Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
What can be done with the settings of this network card to solve the problem?
On another server with a similar configuration, but a different network cards, everything is fine
Server configuration:
88 cores Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz
RAM 512 GB
8 x SAMSUNG MZQLB7T6HMLA-00007 NVME disks
4 x 82599ES 10-Gigabit SFI/SFP+ Network Connection
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi AndriiV,
Thank you for reaching out to the Intel community. We sincerely apologize for the delay in our response.
We acknowledge receipt of your concern and want to assure you that assisting you is our top priority. To effectively troubleshoot the issue you're facing, we require some additional information from your end.
Could you kindly provide the following details?
- System Model:
- Operating System:
- Is the Ethernet card embedded on the board?
- Was the Ethernet card working before? Have any changes been made recently?
Your prompt response with this information will greatly assist us in diagnosing and resolving the issue as quickly as possible.
We look forward to hearing from you soon.
Best Regards,
Simon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Simon!
1. Supermicro SuperServer 1029U-TRTP2
2. Debian 10.13
# uname -a
Linux 15224 4.19.0-26-amd64 #1 SMP Debian 4.19.304-1 (2024-01-09) x86_64 GNU/Linux
3. No
4. No, I am using XL710 card for a first time
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi AndriiV,
Thank you for sharing the requested details.
As you mentioned, there is another server with a similar configuration, but with different network cards. Could you please share the server motherboard and the network card details?
We look forward to hearing from you soon.
Best Regards,
Simon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Simon!
Supermicro SuperServer 1028U-TR4+
Motherboard: X10DRU-i+
Debian 10.13
# uname -a
Linux 157540 4.19.0-25-amd64 #1 SMP Debian 4.19.289-2 (2023-08-08) x86_64 GNU/Linux
4x10Gbps network cards:
87:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
87:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
88:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
88:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 200
802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Andrii,
Good day!
Thank you for providing the requested details.
We are currently investigating this inquiry with our resources, and we will provide you with an update at the earliest opportunity.
Thank you for your patience and understanding.
Best regards,
Simon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Andrii,
Good Day!
Thank you for your patience as we work to resolve the issue you've encountered. We at Intel are fully committed to ensuring your satisfaction.
To address your concern, please refer to the following link for detailed instructions: Intel Ethernet 700 Series Linux Performance Tuning Guide - Section 4.1.
Your understanding and cooperation are greatly appreciated.
Best regards,
Simon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Simon!
As you can see my first message in this topic. I already read this guide and made all tunnings. I tried
382 set_irq_affinity local ens2
384 set_irq_affinity all ens2
With "local" configuration 100% IRQ on some cores at 19Gbps yet. You can see atop screen after
set_irq_affinity local ens2
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think the problem is that the traffic is unevenly distributed across the queues
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Andrii,
Good Day!
Firstly, I'd like to extend our apologies for any inconvenience caused by our previous communication. Your detailed explanation of the issue and server configuration was immensely helpful, and we appreciate your patience and cooperation.
I'm writing to inform you that we've escalated the matter to our next level team for further investigation and resolution. Rest assured, we're prioritizing this issue to ensure a swift resolution.
We will keep you updated on the progress and provide any necessary information as soon as it becomes available.
Once again, thank you for your understanding and cooperation throughout this process.
Best regards,
Simon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi AndriiV,
Greetings for the day!
Thank you for your patience. We attempted to replicate the issue in the lab with the latest drivers/NVM (FVL9.5) but were unable to do so.
Note: Results are the same whether I tune the interface level or not. This indicates that I can achieve 37.7Gbps throughput with or without any interface level tuning, and there are also no packet drops.
Results and tunings can be found in the attached document. Please have a look.
Please clarify below question
- According to the link you provided, there are two server configurations with different processor details. Does this mean that the issue persists with both configurations?
- The customer experimented with different Tx/Rx interrupt coalescing times. Did you observe consistent results each time?
- You also attempted to see if there were any packet drops. Are there any drops? We are not seeing any drops.
- Are you still experiencing a maximum throughput of 22 Gbps even after multiple tunings?
- Could you please share the OS and kernel details of the setup provided by you
Your configuration vs our configuration:
you used -> Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz
We used -> Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz
Both CPUs offer high performance , but differences between them may affect performance depending on the workload. The details are as follows::
- No.of cores and threads
- Clock speed
- TDP (Thermal Design Power)
- Cache size.
Could you please try the tunings recommended in the attached document?
Best Regards,
Azeem_Intel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Azeem!
Thanks for your job!
1. As you can see, in the second configuration there are completely different network cards that use a different driver. I provided the second configuration rather to show that these two servers are used in the same data center, in the same project, almost the same configuration (in particular disks), they are caching servers, the load is distributed between them the same.
2. Yes, I didn’t notice any special changes.
3. 14.05.2024 (3 days ago) I updated driver and firmware for the latest.
# ethtool -i ens2
driver: i40e
version: 2.25.7
firmware-version: 9.50 0x8000f163 1.3597.0
expansion-rom-version:
bus-info: 0000:d8:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
Packet count for these 3 days:
# ethtool -S ens2
NIC statistics:
rx_packets: 137882815105
tx_packets: 274937740003
rx_bytes: 84537446775570
tx_bytes: 401450641546052
rx_errors: 2022092
tx_errors: 0
rx_dropped: 229385
tx_dropped: 0
collisions: 0
rx_length_errors: 0
rx_crc_errors: 0
rx_unicast: 137882060073
tx_unicast: 274936560996
rx_multicast: 0
tx_multicast: 0
rx_broadcast: 343515
tx_broadcast: 0
rx_unknown_protocol: 0
tx_linearize: 2313
tx_force_wb: 0
tx_busy: 0
tx_stopped: 5303
rx_alloc_fail: 0
rx_pg_alloc_fail: 0
rx_cache_reuse: 368417417274
tx-0.packets: 25837685
tx-0.bytes: 33979734149
rx-0.packets: 168279842
rx-0.bytes: 170602932069
rx-0.xdp.pass: 0
rx-0.xdp.drop: 0
rx-0.xdp.tx: 0
rx-0.xdp.unknown: 0
rx-0.xdp.redirect: 0
rx-0.xdp.redirect_fail: 0
tx-1.packets: 5355399756
tx-1.bytes: 7812404286883
rx-1.packets: 2516813387
rx-1.bytes: 1774770706455
rx-1.xdp.pass: 0
rx-1.xdp.drop: 0
rx-1.xdp.tx: 0
rx-1.xdp.unknown: 0
rx-1.xdp.redirect: 0
rx-1.xdp.redirect_fail: 0
tx-2.packets: 21682826
tx-2.bytes: 28480770887
rx-2.packets: 161801610
rx-2.bytes: 164209951123
rx-2.xdp.pass: 0
rx-2.xdp.drop: 0
rx-2.xdp.tx: 0
rx-2.xdp.unknown: 0
rx-2.xdp.redirect: 0
rx-2.xdp.redirect_fail: 0
tx-3.packets: 5301254825
tx-3.bytes: 7734386600593
rx-3.packets: 2494644834
rx-3.bytes: 1756602843454
rx-3.xdp.pass: 0
rx-3.xdp.drop: 0
rx-3.xdp.tx: 0
rx-3.xdp.unknown: 0
rx-3.xdp.redirect: 0
rx-3.xdp.redirect_fail: 0
tx-4.packets: 15224069
tx-4.bytes: 19652026211
rx-4.packets: 150379555
rx-4.bytes: 152052109771
rx-4.xdp.pass: 0
rx-4.xdp.drop: 0
rx-4.xdp.tx: 0
rx-4.xdp.unknown: 0
rx-4.xdp.redirect: 0
rx-4.xdp.redirect_fail: 0
tx-5.packets: 5254224120
tx-5.bytes: 7666284532673
rx-5.packets: 2478075581
rx-5.bytes: 1743109229012
rx-5.xdp.pass: 0
rx-5.xdp.drop: 0
rx-5.xdp.tx: 0
rx-5.xdp.unknown: 0
rx-5.xdp.redirect: 0
rx-5.xdp.redirect_fail: 0
tx-6.packets: 11111160
tx-6.bytes: 14072825797
rx-6.packets: 147614285
rx-6.bytes: 151208273615
rx-6.xdp.pass: 0
rx-6.xdp.drop: 0
rx-6.xdp.tx: 0
rx-6.xdp.unknown: 0
rx-6.xdp.redirect: 0
rx-6.xdp.redirect_fail: 0
tx-7.packets: 5231649268
tx-7.bytes: 7633880383497
rx-7.packets: 2467490966
rx-7.bytes: 1727873025410
rx-7.xdp.pass: 0
rx-7.xdp.drop: 0
rx-7.xdp.tx: 0
rx-7.xdp.unknown: 0
rx-7.xdp.redirect: 0
rx-7.xdp.redirect_fail: 0
tx-8.packets: 9928118
tx-8.bytes: 12342185197
rx-8.packets: 144579783
rx-8.bytes: 147087173052
rx-8.xdp.pass: 0
rx-8.xdp.drop: 0
rx-8.xdp.tx: 0
rx-8.xdp.unknown: 0
rx-8.xdp.redirect: 0
rx-8.xdp.redirect_fail: 0
tx-9.packets: 5204698936
tx-9.bytes: 7595022219312
rx-9.packets: 2449855836
rx-9.bytes: 1707250231577
rx-9.xdp.pass: 0
rx-9.xdp.drop: 0
rx-9.xdp.tx: 0
rx-9.xdp.unknown: 0
rx-9.xdp.redirect: 0
rx-9.xdp.redirect_fail: 0
tx-10.packets: 8175661
tx-10.bytes: 10073920773
rx-10.packets: 142968142
rx-10.bytes: 146695366930
rx-10.xdp.pass: 0
rx-10.xdp.drop: 0
rx-10.xdp.tx: 0
rx-10.xdp.unknown: 0
rx-10.xdp.redirect: 0
rx-10.xdp.redirect_fail: 0
tx-11.packets: 5157512915
tx-11.bytes: 7526504135406
rx-11.packets: 2427772204
rx-11.bytes: 1684990668038
rx-11.xdp.pass: 0
rx-11.xdp.drop: 0
rx-11.xdp.tx: 0
rx-11.xdp.unknown: 0
rx-11.xdp.redirect: 0
rx-11.xdp.redirect_fail: 0
tx-12.packets: 7585912
tx-12.bytes: 9201941239
rx-12.packets: 143219886
rx-12.bytes: 146971694227
rx-12.xdp.pass: 0
rx-12.xdp.drop: 0
rx-12.xdp.tx: 0
rx-12.xdp.unknown: 0
rx-12.xdp.redirect: 0
rx-12.xdp.redirect_fail: 0
tx-13.packets: 5153818605
tx-13.bytes: 7521098070323
rx-13.packets: 2433463509
rx-13.bytes: 1692067903822
rx-13.xdp.pass: 0
rx-13.xdp.drop: 0
rx-13.xdp.tx: 0
rx-13.xdp.unknown: 0
rx-13.xdp.redirect: 0
rx-13.xdp.redirect_fail: 0
tx-14.packets: 6733993
tx-14.bytes: 8021367196
rx-14.packets: 141110460
rx-14.bytes: 144718927156
rx-14.xdp.pass: 0
rx-14.xdp.drop: 0
rx-14.xdp.tx: 0
rx-14.xdp.unknown: 0
rx-14.xdp.redirect: 0
rx-14.xdp.redirect_fail: 0
tx-15.packets: 5146347509
tx-15.bytes: 7510402462613
rx-15.packets: 2429800226
rx-15.bytes: 1686744528732
rx-15.xdp.pass: 0
rx-15.xdp.drop: 0
rx-15.xdp.tx: 0
rx-15.xdp.unknown: 0
rx-15.xdp.redirect: 0
rx-15.xdp.redirect_fail: 0
tx-16.packets: 6288014
tx-16.bytes: 7422482221
rx-16.packets: 142737163
rx-16.bytes: 147757208628
rx-16.xdp.pass: 0
rx-16.xdp.drop: 0
rx-16.xdp.tx: 0
rx-16.xdp.unknown: 0
rx-16.xdp.redirect: 0
rx-16.xdp.redirect_fail: 0
tx-17.packets: 5153273892
tx-17.bytes: 7520748979663
rx-17.packets: 2431759357
rx-17.bytes: 1682905350979
rx-17.xdp.pass: 0
rx-17.xdp.drop: 0
rx-17.xdp.tx: 0
rx-17.xdp.unknown: 0
rx-17.xdp.redirect: 0
rx-17.xdp.redirect_fail: 0
tx-18.packets: 5917693
tx-18.bytes: 6783176550
rx-18.packets: 144656301
rx-18.bytes: 149198188478
rx-18.xdp.pass: 0
rx-18.xdp.drop: 0
rx-18.xdp.tx: 0
rx-18.xdp.unknown: 0
rx-18.xdp.redirect: 0
rx-18.xdp.redirect_fail: 0
tx-19.packets: 5133828421
tx-19.bytes: 7492497965828
rx-19.packets: 2426018808
rx-19.bytes: 1680498388729
rx-19.xdp.pass: 0
rx-19.xdp.drop: 0
rx-19.xdp.tx: 0
rx-19.xdp.unknown: 0
rx-19.xdp.redirect: 0
rx-19.xdp.redirect_fail: 0
tx-20.packets: 32021056
tx-20.bytes: 42713819553
rx-20.packets: 174510718
rx-20.bytes: 172945330141
rx-20.xdp.pass: 0
rx-20.xdp.drop: 0
rx-20.xdp.tx: 0
rx-20.xdp.unknown: 0
rx-20.xdp.redirect: 0
rx-20.xdp.redirect_fail: 0
tx-21.packets: 4908767776
tx-21.bytes: 7155880834589
rx-21.packets: 2262681036
rx-21.bytes: 1694548329911
rx-21.xdp.pass: 0
rx-21.xdp.drop: 0
rx-21.xdp.tx: 0
rx-21.xdp.unknown: 0
rx-21.xdp.redirect: 0
rx-21.xdp.redirect_fail: 0
tx-22.packets: 22942526
tx-22.bytes: 30257850037
rx-22.packets: 162688755
rx-22.bytes: 163256711274
rx-22.xdp.pass: 0
rx-22.xdp.drop: 0
rx-22.xdp.tx: 0
rx-22.xdp.unknown: 0
rx-22.xdp.redirect: 0
rx-22.xdp.redirect_fail: 0
tx-23.packets: 4823018783
tx-23.bytes: 7031703420140
rx-23.packets: 2227592449
rx-23.bytes: 1666421909016
rx-23.xdp.pass: 0
rx-23.xdp.drop: 0
rx-23.xdp.tx: 0
rx-23.xdp.unknown: 0
rx-23.xdp.redirect: 0
rx-23.xdp.redirect_fail: 0
tx-24.packets: 16041703
tx-24.bytes: 20913960784
rx-24.packets: 155601104
rx-24.bytes: 157432173356
rx-24.xdp.pass: 0
rx-24.xdp.drop: 0
rx-24.xdp.tx: 0
rx-24.xdp.unknown: 0
rx-24.xdp.redirect: 0
rx-24.xdp.redirect_fail: 0
tx-25.packets: 4764271626
tx-25.bytes: 6946701442665
rx-25.packets: 2205802501
rx-25.bytes: 1643520544503
rx-25.xdp.pass: 0
rx-25.xdp.drop: 0
rx-25.xdp.tx: 0
rx-25.xdp.unknown: 0
rx-25.xdp.redirect: 0
rx-25.xdp.redirect_fail: 0
tx-26.packets: 12405218
tx-26.bytes: 15942960396
rx-26.packets: 148659356
rx-26.bytes: 150386448593
rx-26.xdp.pass: 0
rx-26.xdp.drop: 0
rx-26.xdp.tx: 0
rx-26.xdp.unknown: 0
rx-26.xdp.redirect: 0
rx-26.xdp.redirect_fail: 0
tx-27.packets: 4756551502
tx-27.bytes: 6936202019721
rx-27.packets: 2197427464
rx-27.bytes: 1630471167551
rx-27.xdp.pass: 0
rx-27.xdp.drop: 0
rx-27.xdp.tx: 0
rx-27.xdp.unknown: 0
rx-27.xdp.redirect: 0
rx-27.xdp.redirect_fail: 0
tx-28.packets: 10151000
tx-28.bytes: 12774643708
rx-28.packets: 144960837
rx-28.bytes: 148599813147
rx-28.xdp.pass: 0
rx-28.xdp.drop: 0
rx-28.xdp.tx: 0
rx-28.xdp.unknown: 0
rx-28.xdp.redirect: 0
rx-28.xdp.redirect_fail: 0
tx-29.packets: 4732011900
tx-29.bytes: 6900790435536
rx-29.packets: 2189226865
rx-29.bytes: 1620663612882
rx-29.xdp.pass: 0
rx-29.xdp.drop: 0
rx-29.xdp.tx: 0
rx-29.xdp.unknown: 0
rx-29.xdp.redirect: 0
rx-29.xdp.redirect_fail: 0
tx-30.packets: 8358978
tx-30.bytes: 10380924736
rx-30.packets: 142710443
rx-30.bytes: 145145089022
rx-30.xdp.pass: 0
rx-30.xdp.drop: 0
rx-30.xdp.tx: 0
rx-30.xdp.unknown: 0
rx-30.xdp.redirect: 0
rx-30.xdp.redirect_fail: 0
tx-31.packets: 4723869582
tx-31.bytes: 6889695080524
rx-31.packets: 2185841250
rx-31.bytes: 1614799018158
rx-31.xdp.pass: 0
rx-31.xdp.drop: 0
rx-31.xdp.tx: 0
rx-31.xdp.unknown: 0
rx-31.xdp.redirect: 0
rx-31.xdp.redirect_fail: 0
tx-32.packets: 7424933
tx-32.bytes: 9092723292
rx-32.packets: 140971438
rx-32.bytes: 144927121001
rx-32.xdp.pass: 0
rx-32.xdp.drop: 0
rx-32.xdp.tx: 0
rx-32.xdp.unknown: 0
rx-32.xdp.redirect: 0
rx-32.xdp.redirect_fail: 0
tx-33.packets: 4713949797
tx-33.bytes: 6875416042641
rx-33.packets: 2180065654
rx-33.bytes: 1606146744706
rx-33.xdp.pass: 0
rx-33.xdp.drop: 0
rx-33.xdp.tx: 0
rx-33.xdp.unknown: 0
rx-33.xdp.redirect: 0
rx-33.xdp.redirect_fail: 0
tx-34.packets: 6645734
tx-34.bytes: 8032100529
rx-34.packets: 142889514
rx-34.bytes: 146829744580
rx-34.xdp.pass: 0
rx-34.xdp.drop: 0
rx-34.xdp.tx: 0
rx-34.xdp.unknown: 0
rx-34.xdp.redirect: 0
rx-34.xdp.redirect_fail: 0
tx-35.packets: 4705775181
tx-35.bytes: 6863749652811
rx-35.packets: 2179507232
rx-35.bytes: 1602543181560
rx-35.xdp.pass: 0
rx-35.xdp.drop: 0
rx-35.xdp.tx: 0
rx-35.xdp.unknown: 0
rx-35.xdp.redirect: 0
rx-35.xdp.redirect_fail: 0
tx-36.packets: 6110848
tx-36.bytes: 7275990490
rx-36.packets: 141626294
rx-36.bytes: 146083863407
rx-36.xdp.pass: 0
rx-36.xdp.drop: 0
rx-36.xdp.tx: 0
rx-36.xdp.unknown: 0
rx-36.xdp.redirect: 0
rx-36.xdp.redirect_fail: 0
tx-37.packets: 4697287850
tx-37.bytes: 6851595622014
rx-37.packets: 2173140049
rx-37.bytes: 1595182400285
rx-37.xdp.pass: 0
rx-37.xdp.drop: 0
rx-37.xdp.tx: 0
rx-37.xdp.unknown: 0
rx-37.xdp.redirect: 0
rx-37.xdp.redirect_fail: 0
tx-38.packets: 5680927
tx-38.bytes: 6687879567
rx-38.packets: 138503460
rx-38.bytes: 140511672601
rx-38.xdp.pass: 0
rx-38.xdp.drop: 0
rx-38.xdp.tx: 0
rx-38.xdp.unknown: 0
rx-38.xdp.redirect: 0
rx-38.xdp.redirect_fail: 0
tx-39.packets: 4670505028
tx-39.bytes: 6812661284648
rx-39.packets: 2163878145
rx-39.bytes: 1587768451384
rx-39.xdp.pass: 0
rx-39.xdp.drop: 0
rx-39.xdp.tx: 0
rx-39.xdp.unknown: 0
rx-39.xdp.redirect: 0
rx-39.xdp.redirect_fail: 0
tx-40.packets: 5582064
tx-40.bytes: 6417514804
rx-40.packets: 140648540
rx-40.bytes: 144846915227
rx-40.xdp.pass: 0
rx-40.xdp.drop: 0
rx-40.xdp.tx: 0
rx-40.xdp.unknown: 0
rx-40.xdp.redirect: 0
rx-40.xdp.redirect_fail: 0
tx-41.packets: 5202830816
tx-41.bytes: 7593307389024
rx-41.packets: 2453481355
rx-41.bytes: 1695051504856
rx-41.xdp.pass: 0
rx-41.xdp.drop: 0
rx-41.xdp.tx: 0
rx-41.xdp.unknown: 0
rx-41.xdp.redirect: 0
rx-41.xdp.redirect_fail: 0
tx-42.packets: 5320754
tx-42.bytes: 6121974895
rx-42.packets: 141213599
rx-42.bytes: 146686782915
rx-42.xdp.pass: 0
rx-42.xdp.drop: 0
rx-42.xdp.tx: 0
rx-42.xdp.unknown: 0
rx-42.xdp.redirect: 0
rx-42.xdp.redirect_fail: 0
tx-43.packets: 5209746697
tx-43.bytes: 7603607833291
rx-43.packets: 2457853475
rx-43.bytes: 1696080937943
rx-43.xdp.pass: 0
rx-43.xdp.drop: 0
rx-43.xdp.tx: 0
rx-43.xdp.unknown: 0
rx-43.xdp.redirect: 0
rx-43.xdp.redirect_fail: 0
tx-44.packets: 5053836
tx-44.bytes: 5761081932
rx-44.packets: 140662343
rx-44.bytes: 144393302376
rx-44.xdp.pass: 0
rx-44.xdp.drop: 0
rx-44.xdp.tx: 0
rx-44.xdp.unknown: 0
rx-44.xdp.redirect: 0
rx-44.xdp.redirect_fail: 0
tx-45.packets: 5214859333
tx-45.bytes: 7611359948938
rx-45.packets: 2461817439
rx-45.bytes: 1697943876889
rx-45.xdp.pass: 0
rx-45.xdp.drop: 0
rx-45.xdp.tx: 0
rx-45.xdp.unknown: 0
rx-45.xdp.redirect: 0
rx-45.xdp.redirect_fail: 0
tx-46.packets: 5051264
tx-46.bytes: 5734376370
rx-46.packets: 140476501
rx-46.bytes: 143676407191
rx-46.xdp.pass: 0
rx-46.xdp.drop: 0
rx-46.xdp.tx: 0
rx-46.xdp.unknown: 0
rx-46.xdp.redirect: 0
rx-46.xdp.redirect_fail: 0
tx-47.packets: 5220069975
tx-47.bytes: 7618932224810
rx-47.packets: 2462508791
rx-47.bytes: 1697330708846
rx-47.xdp.pass: 0
rx-47.xdp.drop: 0
rx-47.xdp.tx: 0
rx-47.xdp.unknown: 0
rx-47.xdp.redirect: 0
rx-47.xdp.redirect_fail: 0
tx-48.packets: 4937412
tx-48.bytes: 5549726840
rx-48.packets: 142152820
rx-48.bytes: 145886059390
rx-48.xdp.pass: 0
rx-48.xdp.drop: 0
rx-48.xdp.tx: 0
rx-48.xdp.unknown: 0
rx-48.xdp.redirect: 0
rx-48.xdp.redirect_fail: 0
tx-49.packets: 5215579828
tx-49.bytes: 7612462870273
rx-49.packets: 2461172863
rx-49.bytes: 1695531339111
rx-49.xdp.pass: 0
rx-49.xdp.drop: 0
rx-49.xdp.tx: 0
rx-49.xdp.unknown: 0
rx-49.xdp.redirect: 0
rx-49.xdp.redirect_fail: 0
tx-50.packets: 4915707
tx-50.bytes: 5600595784
rx-50.packets: 142573143
rx-50.bytes: 146988083692
rx-50.xdp.pass: 0
rx-50.xdp.drop: 0
rx-50.xdp.tx: 0
rx-50.xdp.unknown: 0
rx-50.xdp.redirect: 0
rx-50.xdp.redirect_fail: 0
tx-51.packets: 11237559696
tx-51.bytes: 16437725489984
rx-51.packets: 5810562202
rx-51.bytes: 2739136792069
rx-51.xdp.pass: 0
rx-51.xdp.drop: 0
rx-51.xdp.tx: 0
rx-51.xdp.unknown: 0
rx-51.xdp.redirect: 0
rx-51.xdp.redirect_fail: 0
tx-52.packets: 139105353
tx-52.bytes: 196440985943
rx-52.packets: 275597413
rx-52.bytes: 279146045900
rx-52.xdp.pass: 0
rx-52.xdp.drop: 0
rx-52.xdp.tx: 0
rx-52.xdp.unknown: 0
rx-52.xdp.redirect: 0
rx-52.xdp.redirect_fail: 0
tx-53.packets: 11150191691
tx-53.bytes: 16307118398062
rx-53.packets: 5744895513
rx-53.bytes: 2729355235029
rx-53.xdp.pass: 0
rx-53.xdp.drop: 0
rx-53.xdp.tx: 0
rx-53.xdp.unknown: 0
rx-53.xdp.redirect: 0
rx-53.xdp.redirect_fail: 0
tx-54.packets: 139861008
tx-54.bytes: 197334666026
rx-54.packets: 277073607
rx-54.bytes: 281314521879
rx-54.xdp.pass: 0
rx-54.xdp.drop: 0
rx-54.xdp.tx: 0
rx-54.xdp.unknown: 0
rx-54.xdp.redirect: 0
rx-54.xdp.redirect_fail: 0
tx-55.packets: 11100938191
tx-55.bytes: 16232183354138
rx-55.packets: 5712317173
rx-55.bytes: 2732976654371
rx-55.xdp.pass: 0
rx-55.xdp.drop: 0
rx-55.xdp.tx: 0
rx-55.xdp.unknown: 0
rx-55.xdp.redirect: 0
rx-55.xdp.redirect_fail: 0
tx-56.packets: 140880238
tx-56.bytes: 198918939751
rx-56.packets: 279225002
rx-56.bytes: 283203726294
rx-56.xdp.pass: 0
rx-56.xdp.drop: 0
rx-56.xdp.tx: 0
rx-56.xdp.unknown: 0
rx-56.xdp.redirect: 0
rx-56.xdp.redirect_fail: 0
tx-57.packets: 11144476094
tx-57.bytes: 16292860797206
rx-57.packets: 5734279270
rx-57.bytes: 2752886930988
rx-57.xdp.pass: 0
rx-57.xdp.drop: 0
rx-57.xdp.tx: 0
rx-57.xdp.unknown: 0
rx-57.xdp.redirect: 0
rx-57.xdp.redirect_fail: 0
tx-58.packets: 140597852
tx-58.bytes: 198633473413
rx-58.packets: 278895264
rx-58.bytes: 282489488881
rx-58.xdp.pass: 0
rx-58.xdp.drop: 0
rx-58.xdp.tx: 0
rx-58.xdp.unknown: 0
rx-58.xdp.redirect: 0
rx-58.xdp.redirect_fail: 0
tx-59.packets: 11051468184
tx-59.bytes: 16154765733849
rx-59.packets: 5688479502
rx-59.bytes: 2752148601359
rx-59.xdp.pass: 0
rx-59.xdp.drop: 0
rx-59.xdp.tx: 0
rx-59.xdp.unknown: 0
rx-59.xdp.redirect: 0
rx-59.xdp.redirect_fail: 0
tx-60.packets: 142356513
tx-60.bytes: 201320868538
rx-60.packets: 278013249
rx-60.bytes: 281575258605
rx-60.xdp.pass: 0
rx-60.xdp.drop: 0
rx-60.xdp.tx: 0
rx-60.xdp.unknown: 0
rx-60.xdp.redirect: 0
rx-60.xdp.redirect_fail: 0
tx-61.packets: 9336978756
tx-61.bytes: 13645828265901
rx-61.packets: 4736654625
rx-61.bytes: 2545320363258
rx-61.xdp.pass: 0
rx-61.xdp.drop: 0
rx-61.xdp.tx: 0
rx-61.xdp.unknown: 0
rx-61.xdp.redirect: 0
rx-61.xdp.redirect_fail: 0
tx-62.packets: 143438727
tx-62.bytes: 202809194383
rx-62.packets: 275486795
rx-62.bytes: 275787966018
rx-62.xdp.pass: 0
rx-62.xdp.drop: 0
rx-62.xdp.tx: 0
rx-62.xdp.unknown: 0
rx-62.xdp.redirect: 0
rx-62.xdp.redirect_fail: 0
tx-63.packets: 9288100048
tx-63.bytes: 13571778097557
rx-63.packets: 4701780623
rx-63.bytes: 2553460122246
rx-63.xdp.pass: 0
rx-63.xdp.drop: 0
rx-63.xdp.tx: 0
rx-63.xdp.unknown: 0
rx-63.xdp.redirect: 0
rx-63.xdp.redirect_fail: 0
tx-64.packets: 2172995
tx-64.bytes: 3013872297
rx-64.packets: 2338455
rx-64.bytes: 1646611932
rx-64.xdp.pass: 0
rx-64.xdp.drop: 0
rx-64.xdp.tx: 0
rx-64.xdp.unknown: 0
rx-64.xdp.redirect: 0
rx-64.xdp.redirect_fail: 0
tx-65.packets: 9201606819
tx-65.bytes: 13451504319538
rx-65.packets: 4420621152
rx-65.bytes: 2259150329806
rx-65.xdp.pass: 0
rx-65.xdp.drop: 0
rx-65.xdp.tx: 0
rx-65.xdp.unknown: 0
rx-65.xdp.redirect: 0
rx-65.xdp.redirect_fail: 0
tx-66.packets: 1610850
tx-66.bytes: 2223576415
rx-66.packets: 1855754
rx-66.bytes: 1407107993
rx-66.xdp.pass: 0
rx-66.xdp.drop: 0
rx-66.xdp.tx: 0
rx-66.xdp.unknown: 0
rx-66.xdp.redirect: 0
rx-66.xdp.redirect_fail: 0
tx-67.packets: 9231747566
tx-67.bytes: 13495294161025
rx-67.packets: 4429819735
rx-67.bytes: 2270325037100
rx-67.xdp.pass: 0
rx-67.xdp.drop: 0
rx-67.xdp.tx: 0
rx-67.xdp.unknown: 0
rx-67.xdp.redirect: 0
rx-67.xdp.redirect_fail: 0
tx-68.packets: 1393370
tx-68.bytes: 1896504785
rx-68.packets: 1709552
rx-68.bytes: 1410020474
rx-68.xdp.pass: 0
rx-68.xdp.drop: 0
rx-68.xdp.tx: 0
rx-68.xdp.unknown: 0
rx-68.xdp.redirect: 0
rx-68.xdp.redirect_fail: 0
tx-69.packets: 9180497539
tx-69.bytes: 13418984742271
rx-69.packets: 4406785411
rx-69.bytes: 2270700790686
rx-69.xdp.pass: 0
rx-69.xdp.drop: 0
rx-69.xdp.tx: 0
rx-69.xdp.unknown: 0
rx-69.xdp.redirect: 0
rx-69.xdp.redirect_fail: 0
tx-70.packets: 1205866
tx-70.bytes: 1638544790
rx-70.packets: 1546092
rx-70.bytes: 1293311625
rx-70.xdp.pass: 0
rx-70.xdp.drop: 0
rx-70.xdp.tx: 0
rx-70.xdp.unknown: 0
rx-70.xdp.redirect: 0
rx-70.xdp.redirect_fail: 0
tx-71.packets: 9235493303
tx-71.bytes: 13498947850921
rx-71.packets: 4432466509
rx-71.bytes: 2286754563991
rx-71.xdp.pass: 0
rx-71.xdp.drop: 0
rx-71.xdp.tx: 0
rx-71.xdp.unknown: 0
rx-71.xdp.redirect: 0
rx-71.xdp.redirect_fail: 0
tx-72.packets: 1020837
tx-72.bytes: 1398334919
rx-72.packets: 1290369
rx-72.bytes: 1107113237
rx-72.xdp.pass: 0
rx-72.xdp.drop: 0
rx-72.xdp.tx: 0
rx-72.xdp.unknown: 0
rx-72.xdp.redirect: 0
rx-72.xdp.redirect_fail: 0
tx-73.packets: 9231228324
tx-73.bytes: 13491926606844
rx-73.packets: 4428966870
rx-73.bytes: 2289089162198
rx-73.xdp.pass: 0
rx-73.xdp.drop: 0
rx-73.xdp.tx: 0
rx-73.xdp.unknown: 0
rx-73.xdp.redirect: 0
rx-73.xdp.redirect_fail: 0
tx-74.packets: 902171
tx-74.bytes: 1178268786
rx-74.packets: 1123631
rx-74.bytes: 999317218
rx-74.xdp.pass: 0
rx-74.xdp.drop: 0
rx-74.xdp.tx: 0
rx-74.xdp.unknown: 0
rx-74.xdp.redirect: 0
rx-74.xdp.redirect_fail: 0
tx-75.packets: 9240675397
tx-75.bytes: 13505896095485
rx-75.packets: 4429361136
rx-75.bytes: 2288051533556
rx-75.xdp.pass: 0
rx-75.xdp.drop: 0
rx-75.xdp.tx: 0
rx-75.xdp.unknown: 0
rx-75.xdp.redirect: 0
rx-75.xdp.redirect_fail: 0
tx-76.packets: 796770
tx-76.bytes: 1051531227
rx-76.packets: 1187761
rx-76.bytes: 1155412577
rx-76.xdp.pass: 0
rx-76.xdp.drop: 0
rx-76.xdp.tx: 0
rx-76.xdp.unknown: 0
rx-76.xdp.redirect: 0
rx-76.xdp.redirect_fail: 0
tx-77.packets: 9291563508
tx-77.bytes: 13580117214277
rx-77.packets: 4453232039
rx-77.bytes: 2296535519245
rx-77.xdp.pass: 0
rx-77.xdp.drop: 0
rx-77.xdp.tx: 0
rx-77.xdp.unknown: 0
rx-77.xdp.redirect: 0
rx-77.xdp.redirect_fail: 0
tx-78.packets: 723753
tx-78.bytes: 955646562
rx-78.packets: 1181701
rx-78.bytes: 1198728585
rx-78.xdp.pass: 0
rx-78.xdp.drop: 0
rx-78.xdp.tx: 0
rx-78.xdp.unknown: 0
rx-78.xdp.redirect: 0
rx-78.xdp.redirect_fail: 0
tx-79.packets: 9230915572
tx-79.bytes: 13490311718806
rx-79.packets: 4430181562
rx-79.bytes: 2287906566917
rx-79.xdp.pass: 0
rx-79.xdp.drop: 0
rx-79.xdp.tx: 0
rx-79.xdp.unknown: 0
rx-79.xdp.redirect: 0
rx-79.xdp.redirect_fail: 0
veb.rx_bytes: 0
veb.tx_bytes: 0
veb.rx_unicast: 0
veb.tx_unicast: 0
veb.rx_multicast: 0
veb.tx_multicast: 0
veb.rx_broadcast: 0
veb.tx_broadcast: 0
veb.rx_discards: 0
veb.tx_discards: 0
veb.tx_errors: 0
veb.rx_unknown_protocol: 0
veb.tc_0_tx_packets: 0
veb.tc_0_tx_bytes: 0
veb.tc_0_rx_packets: 0
veb.tc_0_rx_bytes: 0
veb.tc_1_tx_packets: 0
veb.tc_1_tx_bytes: 0
veb.tc_1_rx_packets: 0
veb.tc_1_rx_bytes: 0
veb.tc_2_tx_packets: 0
veb.tc_2_tx_bytes: 0
veb.tc_2_rx_packets: 0
veb.tc_2_rx_bytes: 0
veb.tc_3_tx_packets: 0
veb.tc_3_tx_bytes: 0
veb.tc_3_rx_packets: 0
veb.tc_3_rx_bytes: 0
veb.tc_4_tx_packets: 0
veb.tc_4_tx_bytes: 0
veb.tc_4_rx_packets: 0
veb.tc_4_rx_bytes: 0
veb.tc_5_tx_packets: 0
veb.tc_5_tx_bytes: 0
veb.tc_5_rx_packets: 0
veb.tc_5_rx_bytes: 0
veb.tc_6_tx_packets: 0
veb.tc_6_tx_bytes: 0
veb.tc_6_rx_packets: 0
veb.tc_6_rx_bytes: 0
veb.tc_7_tx_packets: 0
veb.tc_7_tx_bytes: 0
veb.tc_7_rx_packets: 0
veb.tc_7_rx_bytes: 0
port.rx_bytes: 85089469570155
port.tx_bytes: 121076154453173
port.rx_unicast: 137883084747
port.tx_unicast: 274937794383
port.rx_multicast: 9128
port.tx_multicast: 8467
port.rx_broadcast: 343517
port.tx_broadcast: 1
port.tx_errors: 0
port.rx_dropped: 354515
port.tx_dropped_link_down: 0
port.rx_crc_errors: 0
port.illegal_bytes: 0
port.mac_local_faults: 0
port.mac_remote_faults: 0
port.tx_timeout: 0
port.rx_csum_bad: 2022092
port.rx_length_errors: 0
port.link_xon_rx: 0
port.link_xoff_rx: 0
port.link_xon_tx: 0
port.link_xoff_tx: 0
port.rx_size_64: 21863233950
port.rx_size_127: 64037694392
port.rx_size_255: 17192125
port.rx_size_511: 16389377
port.rx_size_1023: 190328800
port.rx_size_1522: 51758598761
port.rx_size_big: 0
port.tx_size_64: 116808966
port.tx_size_127: 2546764083
port.tx_size_255: 168710866
port.tx_size_511: 519025380
port.tx_size_1023: 1553551118
port.tx_size_1522: 270032942462
port.tx_size_big: 0
port.rx_undersize: 0
port.rx_fragments: 0
port.rx_oversize: 0
port.rx_jabber: 0
port.VF_admin_queue_requests: 0
port.arq_overflows: 0
port.tx_hwtstamp_timeouts: 0
port.rx_hwtstamp_cleared: 0
port.tx_hwtstamp_skipped: 0
port.fdir_flush_cnt: 9079
port.fdir_atr_match: 129856552330
port.fdir_atr_tunnel_match: 0
port.fdir_atr_status: 1
port.fdir_sb_match: 0
port.fdir_sb_status: 1
port.tx_lpi_status: 0
port.rx_lpi_status: 0
port.tx_lpi_count: 0
port.rx_lpi_count: 0
port.tx_lpi_duration: 0
port.rx_lpi_duration: 0
port.tx_priority_0_xon_tx: 0
port.tx_priority_0_xoff_tx: 0
port.rx_priority_0_xon_rx: 0
port.rx_priority_0_xoff_rx: 0
port.rx_priority_0_xon_2_xoff: 0
port.tx_priority_1_xon_tx: 0
port.tx_priority_1_xoff_tx: 0
port.rx_priority_1_xon_rx: 0
port.rx_priority_1_xoff_rx: 0
port.rx_priority_1_xon_2_xoff: 0
port.tx_priority_2_xon_tx: 0
port.tx_priority_2_xoff_tx: 0
port.rx_priority_2_xon_rx: 0
port.rx_priority_2_xoff_rx: 0
port.rx_priority_2_xon_2_xoff: 0
port.tx_priority_3_xon_tx: 0
port.tx_priority_3_xoff_tx: 0
port.rx_priority_3_xon_rx: 0
port.rx_priority_3_xoff_rx: 0
port.rx_priority_3_xon_2_xoff: 0
port.tx_priority_4_xon_tx: 0
port.tx_priority_4_xoff_tx: 0
port.rx_priority_4_xon_rx: 0
port.rx_priority_4_xoff_rx: 0
port.rx_priority_4_xon_2_xoff: 0
port.tx_priority_5_xon_tx: 0
port.tx_priority_5_xoff_tx: 0
port.rx_priority_5_xon_rx: 0
port.rx_priority_5_xoff_rx: 0
port.rx_priority_5_xon_2_xoff: 0
port.tx_priority_6_xon_tx: 0
port.tx_priority_6_xoff_tx: 0
port.rx_priority_6_xon_rx: 0
port.rx_priority_6_xoff_rx: 0
port.rx_priority_6_xon_2_xoff: 0
port.tx_priority_7_xon_tx: 0
port.tx_priority_7_xoff_tx: 0
port.rx_priority_7_xon_rx: 0
port.rx_priority_7_xoff_rx: 0
port.rx_priority_7_xon_2_xoff: 0
vf000.rx_bytes: 0
vf000.rx_unicast: 0
vf000.rx_multicast: 0
vf000.rx_broadcast: 0
vf000.rx_discards: 0
vf000.rx_unknown_protocol: 0
vf000.tx_bytes: 0
vf000.tx_unicast: 0
vf000.tx_multicast: 0
vf000.tx_broadcast: 0
vf000.tx_discards: 0
vf000.tx_errors: 0
vf001.rx_bytes: 0
vf001.rx_unicast: 0
vf001.rx_multicast: 0
vf001.rx_broadcast: 0
vf001.rx_discards: 0
vf001.rx_unknown_protocol: 0
vf001.tx_bytes: 0
vf001.tx_unicast: 0
vf001.tx_multicast: 0
vf001.tx_broadcast: 0
vf001.tx_discards: 0
vf001.tx_errors: 0
vf002.rx_bytes: 0
vf002.rx_unicast: 0
vf002.rx_multicast: 0
vf002.rx_broadcast: 0
vf002.rx_discards: 0
vf002.rx_unknown_protocol: 0
vf002.tx_bytes: 0
vf002.tx_unicast: 0
vf002.tx_multicast: 0
vf002.tx_broadcast: 0
vf002.tx_discards: 0
vf002.tx_errors: 0
vf003.rx_bytes: 0
vf003.rx_unicast: 0
vf003.rx_multicast: 0
vf003.rx_broadcast: 0
vf003.rx_discards: 0
vf003.rx_unknown_protocol: 0
vf003.tx_bytes: 0
vf003.tx_unicast: 0
vf003.tx_multicast: 0
vf003.tx_broadcast: 0
vf003.tx_discards: 0
vf003.tx_errors: 0
vf004.rx_bytes: 0
vf004.rx_unicast: 0
vf004.rx_multicast: 0
vf004.rx_broadcast: 0
vf004.rx_discards: 0
vf004.rx_unknown_protocol: 0
vf004.tx_bytes: 0
vf004.tx_unicast: 0
vf004.tx_multicast: 0
vf004.tx_broadcast: 0
vf004.tx_discards: 0
vf004.tx_errors: 0
vf005.rx_bytes: 0
vf005.rx_unicast: 0
vf005.rx_multicast: 0
vf005.rx_broadcast: 0
vf005.rx_discards: 0
vf005.rx_unknown_protocol: 0
vf005.tx_bytes: 0
vf005.tx_unicast: 0
vf005.tx_multicast: 0
vf005.tx_broadcast: 0
vf005.tx_discards: 0
vf005.tx_errors: 0
vf006.rx_bytes: 0
vf006.rx_unicast: 0
vf006.rx_multicast: 0
vf006.rx_broadcast: 0
vf006.rx_discards: 0
vf006.rx_unknown_protocol: 0
vf006.tx_bytes: 0
vf006.tx_unicast: 0
vf006.tx_multicast: 0
vf006.tx_broadcast: 0
vf006.tx_discards: 0
vf006.tx_errors: 0
vf007.rx_bytes: 0
vf007.rx_unicast: 0
vf007.rx_multicast: 0
vf007.rx_broadcast: 0
vf007.rx_discards: 0
vf007.rx_unknown_protocol: 0
vf007.tx_bytes: 0
vf007.tx_unicast: 0
vf007.tx_multicast: 0
vf007.tx_broadcast: 0
vf007.tx_discards: 0
vf007.tx_errors: 0
vf008.rx_bytes: 0
vf008.rx_unicast: 0
vf008.rx_multicast: 0
vf008.rx_broadcast: 0
vf008.rx_discards: 0
vf008.rx_unknown_protocol: 0
vf008.tx_bytes: 0
vf008.tx_unicast: 0
vf008.tx_multicast: 0
vf008.tx_broadcast: 0
vf008.tx_discards: 0
vf008.tx_errors: 0
vf009.rx_bytes: 0
vf009.rx_unicast: 0
vf009.rx_multicast: 0
vf009.rx_broadcast: 0
vf009.rx_discards: 0
vf009.rx_unknown_protocol: 0
vf009.tx_bytes: 0
vf009.tx_unicast: 0
vf009.tx_multicast: 0
vf009.tx_broadcast: 0
vf009.tx_discards: 0
vf009.tx_errors: 0
vf010.rx_bytes: 0
vf010.rx_unicast: 0
vf010.rx_multicast: 0
vf010.rx_broadcast: 0
vf010.rx_discards: 0
vf010.rx_unknown_protocol: 0
vf010.tx_bytes: 0
vf010.tx_unicast: 0
vf010.tx_multicast: 0
vf010.tx_broadcast: 0
vf010.tx_discards: 0
vf010.tx_errors: 0
vf011.rx_bytes: 0
vf011.rx_unicast: 0
vf011.rx_multicast: 0
vf011.rx_broadcast: 0
vf011.rx_discards: 0
vf011.rx_unknown_protocol: 0
vf011.tx_bytes: 0
vf011.tx_unicast: 0
vf011.tx_multicast: 0
vf011.tx_broadcast: 0
vf011.tx_discards: 0
vf011.tx_errors: 0
vf012.rx_bytes: 0
vf012.rx_unicast: 0
vf012.rx_multicast: 0
vf012.rx_broadcast: 0
vf012.rx_discards: 0
vf012.rx_unknown_protocol: 0
vf012.tx_bytes: 0
vf012.tx_unicast: 0
vf012.tx_multicast: 0
vf012.tx_broadcast: 0
vf012.tx_discards: 0
vf012.tx_errors: 0
vf013.rx_bytes: 0
vf013.rx_unicast: 0
vf013.rx_multicast: 0
vf013.rx_broadcast: 0
vf013.rx_discards: 0
vf013.rx_unknown_protocol: 0
vf013.tx_bytes: 0
vf013.tx_unicast: 0
vf013.tx_multicast: 0
vf013.tx_broadcast: 0
vf013.tx_discards: 0
vf013.tx_errors: 0
vf014.rx_bytes: 0
vf014.rx_unicast: 0
vf014.rx_multicast: 0
vf014.rx_broadcast: 0
vf014.rx_discards: 0
vf014.rx_unknown_protocol: 0
vf014.tx_bytes: 0
vf014.tx_unicast: 0
vf014.tx_multicast: 0
vf014.tx_broadcast: 0
vf014.tx_discards: 0
vf014.tx_errors: 0
vf015.rx_bytes: 0
vf015.rx_unicast: 0
vf015.rx_multicast: 0
vf015.rx_broadcast: 0
vf015.rx_discards: 0
vf015.rx_unknown_protocol: 0
vf015.tx_bytes: 0
vf015.tx_unicast: 0
vf015.tx_multicast: 0
vf015.tx_broadcast: 0
vf015.tx_discards: 0
vf015.tx_errors: 0
vf016.rx_bytes: 0
vf016.rx_unicast: 0
vf016.rx_multicast: 0
vf016.rx_broadcast: 0
vf016.rx_discards: 0
vf016.rx_unknown_protocol: 0
vf016.tx_bytes: 0
vf016.tx_unicast: 0
vf016.tx_multicast: 0
vf016.tx_broadcast: 0
vf016.tx_discards: 0
vf016.tx_errors: 0
vf017.rx_bytes: 0
vf017.rx_unicast: 0
vf017.rx_multicast: 0
vf017.rx_broadcast: 0
vf017.rx_discards: 0
vf017.rx_unknown_protocol: 0
vf017.tx_bytes: 0
vf017.tx_unicast: 0
vf017.tx_multicast: 0
vf017.tx_broadcast: 0
vf017.tx_discards: 0
vf017.tx_errors: 0
vf018.rx_bytes: 0
vf018.rx_unicast: 0
vf018.rx_multicast: 0
vf018.rx_broadcast: 0
vf018.rx_discards: 0
vf018.rx_unknown_protocol: 0
vf018.tx_bytes: 0
vf018.tx_unicast: 0
vf018.tx_multicast: 0
vf018.tx_broadcast: 0
vf018.tx_discards: 0
vf018.tx_errors: 0
vf019.rx_bytes: 0
vf019.rx_unicast: 0
vf019.rx_multicast: 0
vf019.rx_broadcast: 0
vf019.rx_discards: 0
vf019.rx_unknown_protocol: 0
vf019.tx_bytes: 0
vf019.tx_unicast: 0
vf019.tx_multicast: 0
vf019.tx_broadcast: 0
vf019.tx_discards: 0
vf019.tx_errors: 0
vf020.rx_bytes: 0
vf020.rx_unicast: 0
vf020.rx_multicast: 0
vf020.rx_broadcast: 0
vf020.rx_discards: 0
vf020.rx_unknown_protocol: 0
vf020.tx_bytes: 0
vf020.tx_unicast: 0
vf020.tx_multicast: 0
vf020.tx_broadcast: 0
vf020.tx_discards: 0
vf020.tx_errors: 0
vf021.rx_bytes: 0
vf021.rx_unicast: 0
vf021.rx_multicast: 0
vf021.rx_broadcast: 0
vf021.rx_discards: 0
vf021.rx_unknown_protocol: 0
vf021.tx_bytes: 0
vf021.tx_unicast: 0
vf021.tx_multicast: 0
vf021.tx_broadcast: 0
vf021.tx_discards: 0
vf021.tx_errors: 0
vf022.rx_bytes: 0
vf022.rx_unicast: 0
vf022.rx_multicast: 0
vf022.rx_broadcast: 0
vf022.rx_discards: 0
vf022.rx_unknown_protocol: 0
vf022.tx_bytes: 0
vf022.tx_unicast: 0
vf022.tx_multicast: 0
vf022.tx_broadcast: 0
vf022.tx_discards: 0
vf022.tx_errors: 0
vf023.rx_bytes: 0
vf023.rx_unicast: 0
vf023.rx_multicast: 0
vf023.rx_broadcast: 0
vf023.rx_discards: 0
vf023.rx_unknown_protocol: 0
vf023.tx_bytes: 0
vf023.tx_unicast: 0
vf023.tx_multicast: 0
vf023.tx_broadcast: 0
vf023.tx_discards: 0
vf023.tx_errors: 0
vf024.rx_bytes: 0
vf024.rx_unicast: 0
vf024.rx_multicast: 0
vf024.rx_broadcast: 0
vf024.rx_discards: 0
vf024.rx_unknown_protocol: 0
vf024.tx_bytes: 0
vf024.tx_unicast: 0
vf024.tx_multicast: 0
vf024.tx_broadcast: 0
vf024.tx_discards: 0
vf024.tx_errors: 0
vf025.rx_bytes: 0
vf025.rx_unicast: 0
vf025.rx_multicast: 0
vf025.rx_broadcast: 0
vf025.rx_discards: 0
vf025.rx_unknown_protocol: 0
vf025.tx_bytes: 0
vf025.tx_unicast: 0
vf025.tx_multicast: 0
vf025.tx_broadcast: 0
vf025.tx_discards: 0
vf025.tx_errors: 0
vf026.rx_bytes: 0
vf026.rx_unicast: 0
vf026.rx_multicast: 0
vf026.rx_broadcast: 0
vf026.rx_discards: 0
vf026.rx_unknown_protocol: 0
vf026.tx_bytes: 0
vf026.tx_unicast: 0
vf026.tx_multicast: 0
vf026.tx_broadcast: 0
vf026.tx_discards: 0
vf026.tx_errors: 0
vf027.rx_bytes: 0
vf027.rx_unicast: 0
vf027.rx_multicast: 0
vf027.rx_broadcast: 0
vf027.rx_discards: 0
vf027.rx_unknown_protocol: 0
vf027.tx_bytes: 0
vf027.tx_unicast: 0
vf027.tx_multicast: 0
vf027.tx_broadcast: 0
vf027.tx_discards: 0
vf027.tx_errors: 0
vf028.rx_bytes: 0
vf028.rx_unicast: 0
vf028.rx_multicast: 0
vf028.rx_broadcast: 0
vf028.rx_discards: 0
vf028.rx_unknown_protocol: 0
vf028.tx_bytes: 0
vf028.tx_unicast: 0
vf028.tx_multicast: 0
vf028.tx_broadcast: 0
vf028.tx_discards: 0
vf028.tx_errors: 0
vf029.rx_bytes: 0
vf029.rx_unicast: 0
vf029.rx_multicast: 0
vf029.rx_broadcast: 0
vf029.rx_discards: 0
vf029.rx_unknown_protocol: 0
vf029.tx_bytes: 0
vf029.tx_unicast: 0
vf029.tx_multicast: 0
vf029.tx_broadcast: 0
vf029.tx_discards: 0
vf029.tx_errors: 0
vf030.rx_bytes: 0
vf030.rx_unicast: 0
vf030.rx_multicast: 0
vf030.rx_broadcast: 0
vf030.rx_discards: 0
vf030.rx_unknown_protocol: 0
vf030.tx_bytes: 0
vf030.tx_unicast: 0
vf030.tx_multicast: 0
vf030.tx_broadcast: 0
vf030.tx_discards: 0
vf030.tx_errors: 0
vf031.rx_bytes: 0
vf031.rx_unicast: 0
vf031.rx_multicast: 0
vf031.rx_broadcast: 0
vf031.rx_discards: 0
vf031.rx_unknown_protocol: 0
vf031.tx_bytes: 0
vf031.tx_unicast: 0
vf031.tx_multicast: 0
vf031.tx_broadcast: 0
vf031.tx_discards: 0
vf031.tx_errors: 0
vf032.rx_bytes: 0
vf032.rx_unicast: 0
vf032.rx_multicast: 0
vf032.rx_broadcast: 0
vf032.rx_discards: 0
vf032.rx_unknown_protocol: 0
vf032.tx_bytes: 0
vf032.tx_unicast: 0
vf032.tx_multicast: 0
vf032.tx_broadcast: 0
vf032.tx_discards: 0
vf032.tx_errors: 0
vf033.rx_bytes: 0
vf033.rx_unicast: 0
vf033.rx_multicast: 0
vf033.rx_broadcast: 0
vf033.rx_discards: 0
vf033.rx_unknown_protocol: 0
vf033.tx_bytes: 0
vf033.tx_unicast: 0
vf033.tx_multicast: 0
vf033.tx_broadcast: 0
vf033.tx_discards: 0
vf033.tx_errors: 0
vf034.rx_bytes: 0
vf034.rx_unicast: 0
vf034.rx_multicast: 0
vf034.rx_broadcast: 0
vf034.rx_discards: 0
vf034.rx_unknown_protocol: 0
vf034.tx_bytes: 0
vf034.tx_unicast: 0
vf034.tx_multicast: 0
vf034.tx_broadcast: 0
vf034.tx_discards: 0
vf034.tx_errors: 0
vf035.rx_bytes: 0
vf035.rx_unicast: 0
vf035.rx_multicast: 0
vf035.rx_broadcast: 0
vf035.rx_discards: 0
vf035.rx_unknown_protocol: 0
vf035.tx_bytes: 0
vf035.tx_unicast: 0
vf035.tx_multicast: 0
vf035.tx_broadcast: 0
vf035.tx_discards: 0
vf035.tx_errors: 0
vf036.rx_bytes: 0
vf036.rx_unicast: 0
vf036.rx_multicast: 0
vf036.rx_broadcast: 0
vf036.rx_discards: 0
vf036.rx_unknown_protocol: 0
vf036.tx_bytes: 0
vf036.tx_unicast: 0
vf036.tx_multicast: 0
vf036.tx_broadcast: 0
vf036.tx_discards: 0
vf036.tx_errors: 0
vf037.rx_bytes: 0
vf037.rx_unicast: 0
vf037.rx_multicast: 0
vf037.rx_broadcast: 0
vf037.rx_discards: 0
vf037.rx_unknown_protocol: 0
vf037.tx_bytes: 0
vf037.tx_unicast: 0
vf037.tx_multicast: 0
vf037.tx_broadcast: 0
vf037.tx_discards: 0
vf037.tx_errors: 0
vf038.rx_bytes: 0
vf038.rx_unicast: 0
vf038.rx_multicast: 0
vf038.rx_broadcast: 0
vf038.rx_discards: 0
vf038.rx_unknown_protocol: 0
vf038.tx_bytes: 0
vf038.tx_unicast: 0
vf038.tx_multicast: 0
vf038.tx_broadcast: 0
vf038.tx_discards: 0
vf038.tx_errors: 0
vf039.rx_bytes: 0
vf039.rx_unicast: 0
vf039.rx_multicast: 0
vf039.rx_broadcast: 0
vf039.rx_discards: 0
vf039.rx_unknown_protocol: 0
vf039.tx_bytes: 0
vf039.tx_unicast: 0
vf039.tx_multicast: 0
vf039.tx_broadcast: 0
vf039.tx_discards: 0
vf039.tx_errors: 0
vf040.rx_bytes: 0
vf040.rx_unicast: 0
vf040.rx_multicast: 0
vf040.rx_broadcast: 0
vf040.rx_discards: 0
vf040.rx_unknown_protocol: 0
vf040.tx_bytes: 0
vf040.tx_unicast: 0
vf040.tx_multicast: 0
vf040.tx_broadcast: 0
vf040.tx_discards: 0
vf040.tx_errors: 0
vf041.rx_bytes: 0
vf041.rx_unicast: 0
vf041.rx_multicast: 0
vf041.rx_broadcast: 0
vf041.rx_discards: 0
vf041.rx_unknown_protocol: 0
vf041.tx_bytes: 0
vf041.tx_unicast: 0
vf041.tx_multicast: 0
vf041.tx_broadcast: 0
vf041.tx_discards: 0
vf041.tx_errors: 0
vf042.rx_bytes: 0
vf042.rx_unicast: 0
vf042.rx_multicast: 0
vf042.rx_broadcast: 0
vf042.rx_discards: 0
vf042.rx_unknown_protocol: 0
vf042.tx_bytes: 0
vf042.tx_unicast: 0
vf042.tx_multicast: 0
vf042.tx_broadcast: 0
vf042.tx_discards: 0
vf042.tx_errors: 0
vf043.rx_bytes: 0
vf043.rx_unicast: 0
vf043.rx_multicast: 0
vf043.rx_broadcast: 0
vf043.rx_discards: 0
vf043.rx_unknown_protocol: 0
vf043.tx_bytes: 0
vf043.tx_unicast: 0
vf043.tx_multicast: 0
vf043.tx_broadcast: 0
vf043.tx_discards: 0
vf043.tx_errors: 0
vf044.rx_bytes: 0
vf044.rx_unicast: 0
vf044.rx_multicast: 0
vf044.rx_broadcast: 0
vf044.rx_discards: 0
vf044.rx_unknown_protocol: 0
vf044.tx_bytes: 0
vf044.tx_unicast: 0
vf044.tx_multicast: 0
vf044.tx_broadcast: 0
vf044.tx_discards: 0
vf044.tx_errors: 0
vf045.rx_bytes: 0
vf045.rx_unicast: 0
vf045.rx_multicast: 0
vf045.rx_broadcast: 0
vf045.rx_discards: 0
vf045.rx_unknown_protocol: 0
vf045.tx_bytes: 0
vf045.tx_unicast: 0
vf045.tx_multicast: 0
vf045.tx_broadcast: 0
vf045.tx_discards: 0
vf045.tx_errors: 0
vf046.rx_bytes: 0
vf046.rx_unicast: 0
vf046.rx_multicast: 0
vf046.rx_broadcast: 0
vf046.rx_discards: 0
vf046.rx_unknown_protocol: 0
vf046.tx_bytes: 0
vf046.tx_unicast: 0
vf046.tx_multicast: 0
vf046.tx_broadcast: 0
vf046.tx_discards: 0
vf046.tx_errors: 0
vf047.rx_bytes: 0
vf047.rx_unicast: 0
vf047.rx_multicast: 0
vf047.rx_broadcast: 0
vf047.rx_discards: 0
vf047.rx_unknown_protocol: 0
vf047.tx_bytes: 0
vf047.tx_unicast: 0
vf047.tx_multicast: 0
vf047.tx_broadcast: 0
vf047.tx_discards: 0
vf047.tx_errors: 0
vf048.rx_bytes: 0
vf048.rx_unicast: 0
vf048.rx_multicast: 0
vf048.rx_broadcast: 0
vf048.rx_discards: 0
vf048.rx_unknown_protocol: 0
vf048.tx_bytes: 0
vf048.tx_unicast: 0
vf048.tx_multicast: 0
vf048.tx_broadcast: 0
vf048.tx_discards: 0
vf048.tx_errors: 0
vf049.rx_bytes: 0
vf049.rx_unicast: 0
vf049.rx_multicast: 0
vf049.rx_broadcast: 0
vf049.rx_discards: 0
vf049.rx_unknown_protocol: 0
vf049.tx_bytes: 0
vf049.tx_unicast: 0
vf049.tx_multicast: 0
vf049.tx_broadcast: 0
vf049.tx_discards: 0
vf049.tx_errors: 0
vf050.rx_bytes: 0
vf050.rx_unicast: 0
vf050.rx_multicast: 0
vf050.rx_broadcast: 0
vf050.rx_discards: 0
vf050.rx_unknown_protocol: 0
vf050.tx_bytes: 0
vf050.tx_unicast: 0
vf050.tx_multicast: 0
vf050.tx_broadcast: 0
vf050.tx_discards: 0
vf050.tx_errors: 0
vf051.rx_bytes: 0
vf051.rx_unicast: 0
vf051.rx_multicast: 0
vf051.rx_broadcast: 0
vf051.rx_discards: 0
vf051.rx_unknown_protocol: 0
vf051.tx_bytes: 0
vf051.tx_unicast: 0
vf051.tx_multicast: 0
vf051.tx_broadcast: 0
vf051.tx_discards: 0
vf051.tx_errors: 0
vf052.rx_bytes: 0
vf052.rx_unicast: 0
vf052.rx_multicast: 0
vf052.rx_broadcast: 0
vf052.rx_discards: 0
vf052.rx_unknown_protocol: 0
vf052.tx_bytes: 0
vf052.tx_unicast: 0
vf052.tx_multicast: 0
vf052.tx_broadcast: 0
vf052.tx_discards: 0
vf052.tx_errors: 0
vf053.rx_bytes: 0
vf053.rx_unicast: 0
vf053.rx_multicast: 0
vf053.rx_broadcast: 0
vf053.rx_discards: 0
vf053.rx_unknown_protocol: 0
vf053.tx_bytes: 0
vf053.tx_unicast: 0
vf053.tx_multicast: 0
vf053.tx_broadcast: 0
vf053.tx_discards: 0
vf053.tx_errors: 0
vf054.rx_bytes: 0
vf054.rx_unicast: 0
vf054.rx_multicast: 0
vf054.rx_broadcast: 0
vf054.rx_discards: 0
vf054.rx_unknown_protocol: 0
vf054.tx_bytes: 0
vf054.tx_unicast: 0
vf054.tx_multicast: 0
vf054.tx_broadcast: 0
vf054.tx_discards: 0
vf054.tx_errors: 0
vf055.rx_bytes: 0
vf055.rx_unicast: 0
vf055.rx_multicast: 0
vf055.rx_broadcast: 0
vf055.rx_discards: 0
vf055.rx_unknown_protocol: 0
vf055.tx_bytes: 0
vf055.tx_unicast: 0
vf055.tx_multicast: 0
vf055.tx_broadcast: 0
vf055.tx_discards: 0
vf055.tx_errors: 0
vf056.rx_bytes: 0
vf056.rx_unicast: 0
vf056.rx_multicast: 0
vf056.rx_broadcast: 0
vf056.rx_discards: 0
vf056.rx_unknown_protocol: 0
vf056.tx_bytes: 0
vf056.tx_unicast: 0
vf056.tx_multicast: 0
vf056.tx_broadcast: 0
vf056.tx_discards: 0
vf056.tx_errors: 0
vf057.rx_bytes: 0
vf057.rx_unicast: 0
vf057.rx_multicast: 0
vf057.rx_broadcast: 0
vf057.rx_discards: 0
vf057.rx_unknown_protocol: 0
vf057.tx_bytes: 0
vf057.tx_unicast: 0
vf057.tx_multicast: 0
vf057.tx_broadcast: 0
vf057.tx_discards: 0
vf057.tx_errors: 0
vf058.rx_bytes: 0
vf058.rx_unicast: 0
vf058.rx_multicast: 0
vf058.rx_broadcast: 0
vf058.rx_discards: 0
vf058.rx_unknown_protocol: 0
vf058.tx_bytes: 0
vf058.tx_unicast: 0
vf058.tx_multicast: 0
vf058.tx_broadcast: 0
vf058.tx_discards: 0
vf058.tx_errors: 0
vf059.rx_bytes: 0
vf059.rx_unicast: 0
vf059.rx_multicast: 0
vf059.rx_broadcast: 0
vf059.rx_discards: 0
vf059.rx_unknown_protocol: 0
vf059.tx_bytes: 0
vf059.tx_unicast: 0
vf059.tx_multicast: 0
vf059.tx_broadcast: 0
vf059.tx_discards: 0
vf059.tx_errors: 0
vf060.rx_bytes: 0
vf060.rx_unicast: 0
vf060.rx_multicast: 0
vf060.rx_broadcast: 0
vf060.rx_discards: 0
vf060.rx_unknown_protocol: 0
vf060.tx_bytes: 0
vf060.tx_unicast: 0
vf060.tx_multicast: 0
vf060.tx_broadcast: 0
vf060.tx_discards: 0
vf060.tx_errors: 0
vf061.rx_bytes: 0
vf061.rx_unicast: 0
vf061.rx_multicast: 0
vf061.rx_broadcast: 0
vf061.rx_discards: 0
vf061.rx_unknown_protocol: 0
vf061.tx_bytes: 0
vf061.tx_unicast: 0
vf061.tx_multicast: 0
vf061.tx_broadcast: 0
vf061.tx_discards: 0
vf061.tx_errors: 0
vf062.rx_bytes: 0
vf062.rx_unicast: 0
vf062.rx_multicast: 0
vf062.rx_broadcast: 0
vf062.rx_discards: 0
vf062.rx_unknown_protocol: 0
vf062.tx_bytes: 0
vf062.tx_unicast: 0
vf062.tx_multicast: 0
vf062.tx_broadcast: 0
vf062.tx_discards: 0
vf062.tx_errors: 0
vf063.rx_bytes: 0
vf063.rx_unicast: 0
vf063.rx_multicast: 0
vf063.rx_broadcast: 0
vf063.rx_discards: 0
vf063.rx_unknown_protocol: 0
vf063.tx_bytes: 0
vf063.tx_unicast: 0
vf063.tx_multicast: 0
vf063.tx_broadcast: 0
vf063.tx_discards: 0
vf063.tx_errors: 0
vf064.rx_bytes: 0
vf064.rx_unicast: 0
vf064.rx_multicast: 0
vf064.rx_broadcast: 0
vf064.rx_discards: 0
vf064.rx_unknown_protocol: 0
vf064.tx_bytes: 0
vf064.tx_unicast: 0
vf064.tx_multicast: 0
vf064.tx_broadcast: 0
vf064.tx_discards: 0
vf064.tx_errors: 0
vf065.rx_bytes: 0
vf065.rx_unicast: 0
vf065.rx_multicast: 0
vf065.rx_broadcast: 0
vf065.rx_discards: 0
vf065.rx_unknown_protocol: 0
vf065.tx_bytes: 0
vf065.tx_unicast: 0
vf065.tx_multicast: 0
vf065.tx_broadcast: 0
vf065.tx_discards: 0
vf065.tx_errors: 0
vf066.rx_bytes: 0
vf066.rx_unicast: 0
vf066.rx_multicast: 0
vf066.rx_broadcast: 0
vf066.rx_discards: 0
vf066.rx_unknown_protocol: 0
vf066.tx_bytes: 0
vf066.tx_unicast: 0
vf066.tx_multicast: 0
vf066.tx_broadcast: 0
vf066.tx_discards: 0
vf066.tx_errors: 0
vf067.rx_bytes: 0
vf067.rx_unicast: 0
vf067.rx_multicast: 0
vf067.rx_broadcast: 0
vf067.rx_discards: 0
vf067.rx_unknown_protocol: 0
vf067.tx_bytes: 0
vf067.tx_unicast: 0
vf067.tx_multicast: 0
vf067.tx_broadcast: 0
vf067.tx_discards: 0
vf067.tx_errors: 0
vf068.rx_bytes: 0
vf068.rx_unicast: 0
vf068.rx_multicast: 0
vf068.rx_broadcast: 0
vf068.rx_discards: 0
vf068.rx_unknown_protocol: 0
vf068.tx_bytes: 0
vf068.tx_unicast: 0
vf068.tx_multicast: 0
vf068.tx_broadcast: 0
vf068.tx_discards: 0
vf068.tx_errors: 0
vf069.rx_bytes: 0
vf069.rx_unicast: 0
vf069.rx_multicast: 0
vf069.rx_broadcast: 0
vf069.rx_discards: 0
vf069.rx_unknown_protocol: 0
vf069.tx_bytes: 0
vf069.tx_unicast: 0
vf069.tx_multicast: 0
vf069.tx_broadcast: 0
vf069.tx_discards: 0
vf069.tx_errors: 0
vf070.rx_bytes: 0
vf070.rx_unicast: 0
vf070.rx_multicast: 0
vf070.rx_broadcast: 0
vf070.rx_discards: 0
vf070.rx_unknown_protocol: 0
vf070.tx_bytes: 0
vf070.tx_unicast: 0
vf070.tx_multicast: 0
vf070.tx_broadcast: 0
vf070.tx_discards: 0
vf070.tx_errors: 0
vf071.rx_bytes: 0
vf071.rx_unicast: 0
vf071.rx_multicast: 0
vf071.rx_broadcast: 0
vf071.rx_discards: 0
vf071.rx_unknown_protocol: 0
vf071.tx_bytes: 0
vf071.tx_unicast: 0
vf071.tx_multicast: 0
vf071.tx_broadcast: 0
vf071.tx_discards: 0
vf071.tx_errors: 0
vf072.rx_bytes: 0
vf072.rx_unicast: 0
vf072.rx_multicast: 0
vf072.rx_broadcast: 0
vf072.rx_discards: 0
vf072.rx_unknown_protocol: 0
vf072.tx_bytes: 0
vf072.tx_unicast: 0
vf072.tx_multicast: 0
vf072.tx_broadcast: 0
vf072.tx_discards: 0
vf072.tx_errors: 0
vf073.rx_bytes: 0
vf073.rx_unicast: 0
vf073.rx_multicast: 0
vf073.rx_broadcast: 0
vf073.rx_discards: 0
vf073.rx_unknown_protocol: 0
vf073.tx_bytes: 0
vf073.tx_unicast: 0
vf073.tx_multicast: 0
vf073.tx_broadcast: 0
vf073.tx_discards: 0
vf073.tx_errors: 0
vf074.rx_bytes: 0
vf074.rx_unicast: 0
vf074.rx_multicast: 0
vf074.rx_broadcast: 0
vf074.rx_discards: 0
vf074.rx_unknown_protocol: 0
vf074.tx_bytes: 0
vf074.tx_unicast: 0
vf074.tx_multicast: 0
vf074.tx_broadcast: 0
vf074.tx_discards: 0
vf074.tx_errors: 0
vf075.rx_bytes: 0
vf075.rx_unicast: 0
vf075.rx_multicast: 0
vf075.rx_broadcast: 0
vf075.rx_discards: 0
vf075.rx_unknown_protocol: 0
vf075.tx_bytes: 0
vf075.tx_unicast: 0
vf075.tx_multicast: 0
vf075.tx_broadcast: 0
vf075.tx_discards: 0
vf075.tx_errors: 0
vf076.rx_bytes: 0
vf076.rx_unicast: 0
vf076.rx_multicast: 0
vf076.rx_broadcast: 0
vf076.rx_discards: 0
vf076.rx_unknown_protocol: 0
vf076.tx_bytes: 0
vf076.tx_unicast: 0
vf076.tx_multicast: 0
vf076.tx_broadcast: 0
vf076.tx_discards: 0
vf076.tx_errors: 0
vf077.rx_bytes: 0
vf077.rx_unicast: 0
vf077.rx_multicast: 0
vf077.rx_broadcast: 0
vf077.rx_discards: 0
vf077.rx_unknown_protocol: 0
vf077.tx_bytes: 0
vf077.tx_unicast: 0
vf077.tx_multicast: 0
vf077.tx_broadcast: 0
vf077.tx_discards: 0
vf077.tx_errors: 0
vf078.rx_bytes: 0
vf078.rx_unicast: 0
vf078.rx_multicast: 0
vf078.rx_broadcast: 0
vf078.rx_discards: 0
vf078.rx_unknown_protocol: 0
vf078.tx_bytes: 0
vf078.tx_unicast: 0
vf078.tx_multicast: 0
vf078.tx_broadcast: 0
vf078.tx_discards: 0
vf078.tx_errors: 0
vf079.rx_bytes: 0
vf079.rx_unicast: 0
vf079.rx_multicast: 0
vf079.rx_broadcast: 0
vf079.rx_discards: 0
vf079.rx_unknown_protocol: 0
vf079.tx_bytes: 0
vf079.tx_unicast: 0
vf079.tx_multicast: 0
vf079.tx_broadcast: 0
vf079.tx_discards: 0
vf079.tx_errors: 0
vf080.rx_bytes: 0
vf080.rx_unicast: 0
vf080.rx_multicast: 0
vf080.rx_broadcast: 0
vf080.rx_discards: 0
vf080.rx_unknown_protocol: 0
vf080.tx_bytes: 0
vf080.tx_unicast: 0
vf080.tx_multicast: 0
vf080.tx_broadcast: 0
vf080.tx_discards: 0
vf080.tx_errors: 0
vf081.rx_bytes: 0
vf081.rx_unicast: 0
vf081.rx_multicast: 0
vf081.rx_broadcast: 0
vf081.rx_discards: 0
vf081.rx_unknown_protocol: 0
vf081.tx_bytes: 0
vf081.tx_unicast: 0
vf081.tx_multicast: 0
vf081.tx_broadcast: 0
vf081.tx_discards: 0
vf081.tx_errors: 0
vf082.rx_bytes: 0
vf082.rx_unicast: 0
vf082.rx_multicast: 0
vf082.rx_broadcast: 0
vf082.rx_discards: 0
vf082.rx_unknown_protocol: 0
vf082.tx_bytes: 0
vf082.tx_unicast: 0
vf082.tx_multicast: 0
vf082.tx_broadcast: 0
vf082.tx_discards: 0
vf082.tx_errors: 0
vf083.rx_bytes: 0
vf083.rx_unicast: 0
vf083.rx_multicast: 0
vf083.rx_broadcast: 0
vf083.rx_discards: 0
vf083.rx_unknown_protocol: 0
vf083.tx_bytes: 0
vf083.tx_unicast: 0
vf083.tx_multicast: 0
vf083.tx_broadcast: 0
vf083.tx_discards: 0
vf083.tx_errors: 0
vf084.rx_bytes: 0
vf084.rx_unicast: 0
vf084.rx_multicast: 0
vf084.rx_broadcast: 0
vf084.rx_discards: 0
vf084.rx_unknown_protocol: 0
vf084.tx_bytes: 0
vf084.tx_unicast: 0
vf084.tx_multicast: 0
vf084.tx_broadcast: 0
vf084.tx_discards: 0
vf084.tx_errors: 0
vf085.rx_bytes: 0
vf085.rx_unicast: 0
vf085.rx_multicast: 0
vf085.rx_broadcast: 0
vf085.rx_discards: 0
vf085.rx_unknown_protocol: 0
vf085.tx_bytes: 0
vf085.tx_unicast: 0
vf085.tx_multicast: 0
vf085.tx_broadcast: 0
vf085.tx_discards: 0
vf085.tx_errors: 0
vf086.rx_bytes: 0
vf086.rx_unicast: 0
vf086.rx_multicast: 0
vf086.rx_broadcast: 0
vf086.rx_discards: 0
vf086.rx_unknown_protocol: 0
vf086.tx_bytes: 0
vf086.tx_unicast: 0
vf086.tx_multicast: 0
vf086.tx_broadcast: 0
vf086.tx_discards: 0
vf086.tx_errors: 0
vf087.rx_bytes: 0
vf087.rx_unicast: 0
vf087.rx_multicast: 0
vf087.rx_broadcast: 0
vf087.rx_discards: 0
vf087.rx_unknown_protocol: 0
vf087.tx_bytes: 0
vf087.tx_unicast: 0
vf087.tx_multicast: 0
vf087.tx_broadcast: 0
vf087.tx_discards: 0
vf087.tx_errors: 0
vf088.rx_bytes: 0
vf088.rx_unicast: 0
vf088.rx_multicast: 0
vf088.rx_broadcast: 0
vf088.rx_discards: 0
vf088.rx_unknown_protocol: 0
vf088.tx_bytes: 0
vf088.tx_unicast: 0
vf088.tx_multicast: 0
vf088.tx_broadcast: 0
vf088.tx_discards: 0
vf088.tx_errors: 0
vf089.rx_bytes: 0
vf089.rx_unicast: 0
vf089.rx_multicast: 0
vf089.rx_broadcast: 0
vf089.rx_discards: 0
vf089.rx_unknown_protocol: 0
vf089.tx_bytes: 0
vf089.tx_unicast: 0
vf089.tx_multicast: 0
vf089.tx_broadcast: 0
vf089.tx_discards: 0
vf089.tx_errors: 0
vf090.rx_bytes: 0
vf090.rx_unicast: 0
vf090.rx_multicast: 0
vf090.rx_broadcast: 0
vf090.rx_discards: 0
vf090.rx_unknown_protocol: 0
vf090.tx_bytes: 0
vf090.tx_unicast: 0
vf090.tx_multicast: 0
vf090.tx_broadcast: 0
vf090.tx_discards: 0
vf090.tx_errors: 0
vf091.rx_bytes: 0
vf091.rx_unicast: 0
vf091.rx_multicast: 0
vf091.rx_broadcast: 0
vf091.rx_discards: 0
vf091.rx_unknown_protocol: 0
vf091.tx_bytes: 0
vf091.tx_unicast: 0
vf091.tx_multicast: 0
vf091.tx_broadcast: 0
vf091.tx_discards: 0
vf091.tx_errors: 0
vf092.rx_bytes: 0
vf092.rx_unicast: 0
vf092.rx_multicast: 0
vf092.rx_broadcast: 0
vf092.rx_discards: 0
vf092.rx_unknown_protocol: 0
vf092.tx_bytes: 0
vf092.tx_unicast: 0
vf092.tx_multicast: 0
vf092.tx_broadcast: 0
vf092.tx_discards: 0
vf092.tx_errors: 0
vf093.rx_bytes: 0
vf093.rx_unicast: 0
vf093.rx_multicast: 0
vf093.rx_broadcast: 0
vf093.rx_discards: 0
vf093.rx_unknown_protocol: 0
vf093.tx_bytes: 0
vf093.tx_unicast: 0
vf093.tx_multicast: 0
vf093.tx_broadcast: 0
vf093.tx_discards: 0
vf093.tx_errors: 0
vf094.rx_bytes: 0
vf094.rx_unicast: 0
vf094.rx_multicast: 0
vf094.rx_broadcast: 0
vf094.rx_discards: 0
vf094.rx_unknown_protocol: 0
vf094.tx_bytes: 0
vf094.tx_unicast: 0
vf094.tx_multicast: 0
vf094.tx_broadcast: 0
vf094.tx_discards: 0
vf094.tx_errors: 0
vf095.rx_bytes: 0
vf095.rx_unicast: 0
vf095.rx_multicast: 0
vf095.rx_broadcast: 0
vf095.rx_discards: 0
vf095.rx_unknown_protocol: 0
vf095.tx_bytes: 0
vf095.tx_unicast: 0
vf095.tx_multicast: 0
vf095.tx_broadcast: 0
vf095.tx_discards: 0
vf095.tx_errors: 0
vf096.rx_bytes: 0
vf096.rx_unicast: 0
vf096.rx_multicast: 0
vf096.rx_broadcast: 0
vf096.rx_discards: 0
vf096.rx_unknown_protocol: 0
vf096.tx_bytes: 0
vf096.tx_unicast: 0
vf096.tx_multicast: 0
vf096.tx_broadcast: 0
vf096.tx_discards: 0
vf096.tx_errors: 0
vf097.rx_bytes: 0
vf097.rx_unicast: 0
vf097.rx_multicast: 0
vf097.rx_broadcast: 0
vf097.rx_discards: 0
vf097.rx_unknown_protocol: 0
vf097.tx_bytes: 0
vf097.tx_unicast: 0
vf097.tx_multicast: 0
vf097.tx_broadcast: 0
vf097.tx_discards: 0
vf097.tx_errors: 0
vf098.rx_bytes: 0
vf098.rx_unicast: 0
vf098.rx_multicast: 0
vf098.rx_broadcast: 0
vf098.rx_discards: 0
vf098.rx_unknown_protocol: 0
vf098.tx_bytes: 0
vf098.tx_unicast: 0
vf098.tx_multicast: 0
vf098.tx_broadcast: 0
vf098.tx_discards: 0
vf098.tx_errors: 0
vf099.rx_bytes: 0
vf099.rx_unicast: 0
vf099.rx_multicast: 0
vf099.rx_broadcast: 0
vf099.rx_discards: 0
vf099.rx_unknown_protocol: 0
vf099.tx_bytes: 0
vf099.tx_unicast: 0
vf099.tx_multicast: 0
vf099.tx_broadcast: 0
vf099.tx_discards: 0
vf099.tx_errors: 0
vf100.rx_bytes: 0
vf100.rx_unicast: 0
vf100.rx_multicast: 0
vf100.rx_broadcast: 0
vf100.rx_discards: 0
vf100.rx_unknown_protocol: 0
vf100.tx_bytes: 0
vf100.tx_unicast: 0
vf100.tx_multicast: 0
vf100.tx_broadcast: 0
vf100.tx_discards: 0
vf100.tx_errors: 0
vf101.rx_bytes: 0
vf101.rx_unicast: 0
vf101.rx_multicast: 0
vf101.rx_broadcast: 0
vf101.rx_discards: 0
vf101.rx_unknown_protocol: 0
vf101.tx_bytes: 0
vf101.tx_unicast: 0
vf101.tx_multicast: 0
vf101.tx_broadcast: 0
vf101.tx_discards: 0
vf101.tx_errors: 0
vf102.rx_bytes: 0
vf102.rx_unicast: 0
vf102.rx_multicast: 0
vf102.rx_broadcast: 0
vf102.rx_discards: 0
vf102.rx_unknown_protocol: 0
vf102.tx_bytes: 0
vf102.tx_unicast: 0
vf102.tx_multicast: 0
vf102.tx_broadcast: 0
vf102.tx_discards: 0
vf102.tx_errors: 0
vf103.rx_bytes: 0
vf103.rx_unicast: 0
vf103.rx_multicast: 0
vf103.rx_broadcast: 0
vf103.rx_discards: 0
vf103.rx_unknown_protocol: 0
vf103.tx_bytes: 0
vf103.tx_unicast: 0
vf103.tx_multicast: 0
vf103.tx_broadcast: 0
vf103.tx_discards: 0
vf103.tx_errors: 0
vf104.rx_bytes: 0
vf104.rx_unicast: 0
vf104.rx_multicast: 0
vf104.rx_broadcast: 0
vf104.rx_discards: 0
vf104.rx_unknown_protocol: 0
vf104.tx_bytes: 0
vf104.tx_unicast: 0
vf104.tx_multicast: 0
vf104.tx_broadcast: 0
vf104.tx_discards: 0
vf104.tx_errors: 0
vf105.rx_bytes: 0
vf105.rx_unicast: 0
vf105.rx_multicast: 0
vf105.rx_broadcast: 0
vf105.rx_discards: 0
vf105.rx_unknown_protocol: 0
vf105.tx_bytes: 0
vf105.tx_unicast: 0
vf105.tx_multicast: 0
vf105.tx_broadcast: 0
vf105.tx_discards: 0
vf105.tx_errors: 0
vf106.rx_bytes: 0
vf106.rx_unicast: 0
vf106.rx_multicast: 0
vf106.rx_broadcast: 0
vf106.rx_discards: 0
vf106.rx_unknown_protocol: 0
vf106.tx_bytes: 0
vf106.tx_unicast: 0
vf106.tx_multicast: 0
vf106.tx_broadcast: 0
vf106.tx_discards: 0
vf106.tx_errors: 0
vf107.rx_bytes: 0
vf107.rx_unicast: 0
vf107.rx_multicast: 0
vf107.rx_broadcast: 0
vf107.rx_discards: 0
vf107.rx_unknown_protocol: 0
vf107.tx_bytes: 0
vf107.tx_unicast: 0
vf107.tx_multicast: 0
vf107.tx_broadcast: 0
vf107.tx_discards: 0
vf107.tx_errors: 0
vf108.rx_bytes: 0
vf108.rx_unicast: 0
vf108.rx_multicast: 0
vf108.rx_broadcast: 0
vf108.rx_discards: 0
vf108.rx_unknown_protocol: 0
vf108.tx_bytes: 0
vf108.tx_unicast: 0
vf108.tx_multicast: 0
vf108.tx_broadcast: 0
vf108.tx_discards: 0
vf108.tx_errors: 0
vf109.rx_bytes: 0
vf109.rx_unicast: 0
vf109.rx_multicast: 0
vf109.rx_broadcast: 0
vf109.rx_discards: 0
vf109.rx_unknown_protocol: 0
vf109.tx_bytes: 0
vf109.tx_unicast: 0
vf109.tx_multicast: 0
vf109.tx_broadcast: 0
vf109.tx_discards: 0
vf109.tx_errors: 0
vf110.rx_bytes: 0
vf110.rx_unicast: 0
vf110.rx_multicast: 0
vf110.rx_broadcast: 0
vf110.rx_discards: 0
vf110.rx_unknown_protocol: 0
vf110.tx_bytes: 0
vf110.tx_unicast: 0
vf110.tx_multicast: 0
vf110.tx_broadcast: 0
vf110.tx_discards: 0
vf110.tx_errors: 0
vf111.rx_bytes: 0
vf111.rx_unicast: 0
vf111.rx_multicast: 0
vf111.rx_broadcast: 0
vf111.rx_discards: 0
vf111.rx_unknown_protocol: 0
vf111.tx_bytes: 0
vf111.tx_unicast: 0
vf111.tx_multicast: 0
vf111.tx_broadcast: 0
vf111.tx_discards: 0
vf111.tx_errors: 0
vf112.rx_bytes: 0
vf112.rx_unicast: 0
vf112.rx_multicast: 0
vf112.rx_broadcast: 0
vf112.rx_discards: 0
vf112.rx_unknown_protocol: 0
vf112.tx_bytes: 0
vf112.tx_unicast: 0
vf112.tx_multicast: 0
vf112.tx_broadcast: 0
vf112.tx_discards: 0
vf112.tx_errors: 0
vf113.rx_bytes: 0
vf113.rx_unicast: 0
vf113.rx_multicast: 0
vf113.rx_broadcast: 0
vf113.rx_discards: 0
vf113.rx_unknown_protocol: 0
vf113.tx_bytes: 0
vf113.tx_unicast: 0
vf113.tx_multicast: 0
vf113.tx_broadcast: 0
vf113.tx_discards: 0
vf113.tx_errors: 0
vf114.rx_bytes: 0
vf114.rx_unicast: 0
vf114.rx_multicast: 0
vf114.rx_broadcast: 0
vf114.rx_discards: 0
vf114.rx_unknown_protocol: 0
vf114.tx_bytes: 0
vf114.tx_unicast: 0
vf114.tx_multicast: 0
vf114.tx_broadcast: 0
vf114.tx_discards: 0
vf114.tx_errors: 0
vf115.rx_bytes: 0
vf115.rx_unicast: 0
vf115.rx_multicast: 0
vf115.rx_broadcast: 0
vf115.rx_discards: 0
vf115.rx_unknown_protocol: 0
vf115.tx_bytes: 0
vf115.tx_unicast: 0
vf115.tx_multicast: 0
vf115.tx_broadcast: 0
vf115.tx_discards: 0
vf115.tx_errors: 0
vf116.rx_bytes: 0
vf116.rx_unicast: 0
vf116.rx_multicast: 0
vf116.rx_broadcast: 0
vf116.rx_discards: 0
vf116.rx_unknown_protocol: 0
vf116.tx_bytes: 0
vf116.tx_unicast: 0
vf116.tx_multicast: 0
vf116.tx_broadcast: 0
vf116.tx_discards: 0
vf116.tx_errors: 0
vf117.rx_bytes: 0
vf117.rx_unicast: 0
vf117.rx_multicast: 0
vf117.rx_broadcast: 0
vf117.rx_discards: 0
vf117.rx_unknown_protocol: 0
vf117.tx_bytes: 0
vf117.tx_unicast: 0
vf117.tx_multicast: 0
vf117.tx_broadcast: 0
vf117.tx_discards: 0
vf117.tx_errors: 0
vf118.rx_bytes: 0
vf118.rx_unicast: 0
vf118.rx_multicast: 0
vf118.rx_broadcast: 0
vf118.rx_discards: 0
vf118.rx_unknown_protocol: 0
vf118.tx_bytes: 0
vf118.tx_unicast: 0
vf118.tx_multicast: 0
vf118.tx_broadcast: 0
vf118.tx_discards: 0
vf118.tx_errors: 0
vf119.rx_bytes: 0
vf119.rx_unicast: 0
vf119.rx_multicast: 0
vf119.rx_broadcast: 0
vf119.rx_discards: 0
vf119.rx_unknown_protocol: 0
vf119.tx_bytes: 0
vf119.tx_unicast: 0
vf119.tx_multicast: 0
vf119.tx_broadcast: 0
vf119.tx_discards: 0
vf119.tx_errors: 0
vf120.rx_bytes: 0
vf120.rx_unicast: 0
vf120.rx_multicast: 0
vf120.rx_broadcast: 0
vf120.rx_discards: 0
vf120.rx_unknown_protocol: 0
vf120.tx_bytes: 0
vf120.tx_unicast: 0
vf120.tx_multicast: 0
vf120.tx_broadcast: 0
vf120.tx_discards: 0
vf120.tx_errors: 0
vf121.rx_bytes: 0
vf121.rx_unicast: 0
vf121.rx_multicast: 0
vf121.rx_broadcast: 0
vf121.rx_discards: 0
vf121.rx_unknown_protocol: 0
vf121.tx_bytes: 0
vf121.tx_unicast: 0
vf121.tx_multicast: 0
vf121.tx_broadcast: 0
vf121.tx_discards: 0
vf121.tx_errors: 0
vf122.rx_bytes: 0
vf122.rx_unicast: 0
vf122.rx_multicast: 0
vf122.rx_broadcast: 0
vf122.rx_discards: 0
vf122.rx_unknown_protocol: 0
vf122.tx_bytes: 0
vf122.tx_unicast: 0
vf122.tx_multicast: 0
vf122.tx_broadcast: 0
vf122.tx_discards: 0
vf122.tx_errors: 0
vf123.rx_bytes: 0
vf123.rx_unicast: 0
vf123.rx_multicast: 0
vf123.rx_broadcast: 0
vf123.rx_discards: 0
vf123.rx_unknown_protocol: 0
vf123.tx_bytes: 0
vf123.tx_unicast: 0
vf123.tx_multicast: 0
vf123.tx_broadcast: 0
vf123.tx_discards: 0
vf123.tx_errors: 0
vf124.rx_bytes: 0
vf124.rx_unicast: 0
vf124.rx_multicast: 0
vf124.rx_broadcast: 0
vf124.rx_discards: 0
vf124.rx_unknown_protocol: 0
vf124.tx_bytes: 0
vf124.tx_unicast: 0
vf124.tx_multicast: 0
vf124.tx_broadcast: 0
vf124.tx_discards: 0
vf124.tx_errors: 0
vf125.rx_bytes: 0
vf125.rx_unicast: 0
vf125.rx_multicast: 0
vf125.rx_broadcast: 0
vf125.rx_discards: 0
vf125.rx_unknown_protocol: 0
vf125.tx_bytes: 0
vf125.tx_unicast: 0
vf125.tx_multicast: 0
vf125.tx_broadcast: 0
vf125.tx_discards: 0
vf125.tx_errors: 0
vf126.rx_bytes: 0
vf126.rx_unicast: 0
vf126.rx_multicast: 0
vf126.rx_broadcast: 0
vf126.rx_discards: 0
vf126.rx_unknown_protocol: 0
vf126.tx_bytes: 0
vf126.tx_unicast: 0
vf126.tx_multicast: 0
vf126.tx_broadcast: 0
vf126.tx_discards: 0
vf126.tx_errors: 0
vf127.rx_bytes: 0
vf127.rx_unicast: 0
vf127.rx_multicast: 0
vf127.rx_broadcast: 0
vf127.rx_discards: 0
vf127.rx_unknown_protocol: 0
vf127.tx_bytes: 0
vf127.tx_unicast: 0
vf127.tx_multicast: 0
vf127.tx_broadcast: 0
vf127.tx_discards: 0
vf127.tx_errors: 0
As you can see, there are packet losses
4. Yes. Below are the graphs of these two servers for the last 2 days. As you can see, on the second server with 4x10G cards the traffic in peaks was 23 Gbit/s, and on the first with the XL710 card it was only 21 Gbit/s. Although in theory it should be the same.
5. Linux 15224 4.19.0-26-amd64 #1 SMP Debian 4.19.304-1 (2024-01-09) x86_64 GNU/Linux
You can download it from https://packages.debian.org/buster/linux-image-4.19.0-26-amd64
# cat /etc/debian_version
10.13
You can download it from https://cdimage.debian.org/cdimage/archive/10.13.0/amd64/iso-cd/debian-10.13.0-amd64-netinst.iso
Thank you for your document. I'll try the settings from it. I’m really not sure that it will be better with 32 queues.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi AndriiV,
Please let us know the outcome once you tried tunings recommended in the attached document.
Regards,
Dinesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, Dinesh! Hi, Azeem!
Yesterday was the day with the highest traffic for the whole week.
I made changes 17.05.2024 at the evening
1. disable irqbalance
systemctl disable irqbalance
systemctl stop irqbalance
was allready stoped and disabled
2. Set the number of hardware receive (RX) and transmit (TX) queues
ethtool -L ens2 combined 40
was
ethtool -L ens2 combined 80
I made 40 queues becase CPU has 40 kernels
3. Set the interrupt moderation and packet coalescing for a network interface
ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 100 tx-usecs 100 rx-usecs-high 100
was
ethtool -C ens2 adaptive-rx off adaptive-tx off rx-usecs 2048 tx-usecs 2048
4. Set the receive (RX) and transmit (TX) ring parameters for a network interface
ethtool -G ens2 rx 2048 tx 2048
was
ethtool -G ens2 tx 4096 rx 4096
5. Disable GRO
ethtool -K ens2 gro off
was
ethtool -K ens2 tx on sg on tso on rx on
6. Disable auto-negotiation for RX and TX features
ethtool -A ens2 rx off tx off
already was disabled
7. Set the IRQ Affnity (so IRQ affinity to be local to each CPU)
./ set_irq_affinity -x local ens2
was
./ set_irq_affinity -x all ens2
8. Set the network socket buffer sizes to max
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
already was in sysctl.conf
9. Set the tuning profile
tuned-adm profile network-throughput
allready was the same. Also I tried (long time ago)
tuned-adm profile throughput-performance
10. Disable Firewall
service firewalld stop; systemctl mask firewalld
There are no firewalld in my system. Iptables has some fail2ban rulles for sshd and pure-ftpd
iptables -L -n -v
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
2787K 7817M f2b-pure-ftpd tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 21,20,990,989
3299K 381M f2b-sshd tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 22
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain f2b-sshd (1 references)
pkts bytes target prot opt in out source destination
1835K 276M RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
Chain f2b-pure-ftpd (1 references)
pkts bytes target prot opt in out source destination
2739K 7814M RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
Results
ethtool -S ens2 | grep dropped | column
rx_dropped: 10215 tx_dropped: 0 port.rx_dropped: 174490 port.tx_dropped_link_down: 0
I made atop screenshots of both servers yeasterday at 23:23 GMT+2 and 22:27 GMT+1 (Almost at the same time)
XL710:
4х 82599ES:
As you can see, first server has 21 Gbit/s, the second has 26 Gbit/s
IRQ loading with XL710 from 47% to 79% at the same time IRQ loading at 4x82599ES from 0% to 42%
You might think that the interrupts are due to the disks, but these servers have the same disks.
Graphs as a whole with yesterday's peak (GMT+2):
XL710:
4х 82599ES:
You may also be interested in Nginx statistics for this time:
XL710:
4х 82599ES:
As you can see, at approximately 21:30 with 15k connections on the xl710 problems begin.
A synthetic test in iperf for 10 connections is of course good, but it is far from real conditions.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi AndriiV,
Thank you for your prompt response.
We appreciate you sharing your concern with us. Please rest assured that we understand the importance of your situation and are fully committed to assisting you.
Please allow us some time to check this with the internal team, and we will get back to you as soon as we have an update.
If you have any additional information or questions, please feel free to share them with us. We're here to support you every step of the way.
Best regards,
Simon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello!
If it helps in any way. We are using optical twinax cable:
# ethtool -m ens2
Identifier : 0x0d (QSFP+)
Extended identifier : 0x10
Extended identifier description : 1.5W max. Power consumption
Extended identifier description : No CDR in TX, No CDR in RX
Extended identifier description : High Power Class (> 3.5 W) not enabled
Connector : 0x23 (No separable connector)
Transceiver codes : 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
Encoding : 0x05 (64B/66B)
BR, Nominal : 10300Mbps
Rate identifier : 0x00
Length (SMF,km) : 0km
Length (OM3 50um) : 0m
Length (OM2 50um) : 0m
Length (OM1 62.5um) : 0m
Length (Copper or Active cable) : 3m
Transmitter technology : 0x00 (850 nm VCSEL)
Laser wavelength : 0.000nm
Laser wavelength tolerance : 0.000nm
Vendor name : CISCO-FINISAR
Vendor OUI : 00:90:65
Vendor PN : FCBN410QE2C03-C2
Vendor rev : A
Vendor SN : FIW231301GE-B
Date code : 190326
Revision Compliance : Revision not specified
Module temperature : 0.00 degrees C / 32.00 degrees F
Module voltage : 0.0000 V
On other side is Juniper QFX5200-32C
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi AndriiV,
Greetings for the day!
You observed the problem in the setup with the "Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz" but not in the setup with the "Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz".
Since we did not have both processors available, we tested with an Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz and were unable to reproduce the issue.
Here is the small comparison b/w your setups:
Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz: It is a part of the Skylake microarchitecture. It has 20 cores and 40 threads, a clock speed of 2.10 GHz, a cache of 27.5 MB, and a TDP (termal design power) of 125 watts.
Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz: It is based on Broadwell microarchitecture. It has 22 cores and 44 threads, a clock speed of 2.20 GHz, a cache of 55 MB, and a TDP of 145 watts.
Conclusion : Despite the Xeon E5-2699 v4 having slightly higher speed and cache, the Xeon Gold 6230 might still perform well or even better in certain tasks due to its architectural improvements.
Please clarify the below question:
Is this the same pattern used for network traffic, where one setup has four 10-gigabit ports and the other has a 40-gigabit NIC (XL710) Did you initiate traffic on every port simultaneously in the 10-gigabit setup
Some tunings: Please try using these
# Disable IRQ Balance
systemctl stop irqbalance
# Set IRQ Affinity
${pathtoi40epackage}/scripts/set_irq_affinity -X local <interface>
Note: “-x all” to use all the cores,”-x local” to use only the local NUMA socket
# Increase the maximum number of packets allowed to queue when a particular interface receives packets faster than the kernel can process them
sysctl -w net.core.netdev_max_backlog=8192
# Enable interrupt coalescing to aggregate and process multiple interrupts as a single event, reducing CPU overhead
ethtool -C $NIC_INTERFACE adaptive-rx off adaptive-tx off rx-usecs 50 tx-usecs 50 rx-usecs-high 50
# Enable Multi-Queue support on NIC, to distribute network traffic processing to across multiple CPU cores.
ethtool -L ethX combined Y (ex: ethtool -L ethX combined 32 this is to set 32 Tx/Rx queues)
# set max Tx/RX buffer size
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
#Set the tuning profile
tuned-adm profile network-throughput
#Disable Firewall
service firewalld stop; systemctl mask firewalld
# Set the receive (RX) and transmit (TX) ring parameters for a network interface (These parameters are to optimize the buffer space allocated for handling incoming and outgoing network packets)
ethtool -G $NIC_INTERFACE rx 2048 tx 2048
Best Regards,
Azeem_Intel
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Azeem, hi Simon!
Is this the same pattern used for network traffic, where one setup has four 10-gigabit ports and the other has a 40-gigabit NIC (XL710) Did you initiate traffic on every port simultaneously in the 10-gigabit setup
As you can see in my second message in this topic, there are Linux software bonding
cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 200
802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
The atop screenshot shows that the traffic is divided evenly between all 4 ports.
Actually, that’s why they installed the second server with 10G network cards, because there were problems with 40G.
# Disable IRQ Balance
systemctl stop irqbalance
Alredy Stopped
# Set IRQ Affinity
${pathtoi40epackage}/scripts/set_irq_affinity -X local <interface>
Note: “-x all” to use all the cores,”-x local” to use only the local NUMA socket
Made it. Was set_irq_affinity -x all ens2
# Increase the maximum number of packets allowed to queue when a particular interface receives packets faster than the kernel can process them
sysctl -w net.core.netdev_max_backlog=8192
Made it. Was net.core.netdev_max_backlog = 300000
# Enable interrupt coalescing to aggregate and process multiple interrupts as a single event, reducing CPU overhead
ethtool -C $NIC_INTERFACE adaptive-rx off adaptive-tx off rx-usecs 50 tx-usecs 50 rx-usecs-high 50
Made it
# Enable Multi-Queue support on NIC, to distribute network traffic processing to across multiple CPU cores.
ethtool -L ethX combined Y (ex: ethtool -L ethX combined 32 this is to set 32 Tx/Rx queues)
Made it before set_irq_affinity -x loacal ens2 I think it's better this way.
# set max Tx/RX buffer size
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
Alredy was the same. I did it again just in case
#Set the tuning profile
tuned-adm profile network-throughput
Alredy was the same. I did it again just in case
#Disable Firewall
service firewalld stop; systemctl mask firewalld
# service firewalld stop; systemctl mask firewalld
Failed to stop firewalld.service: Unit firewalld.service not loaded.
Unit firewalld.service does not exist, proceeding anyway.
Created symlink /etc/systemd/system/firewalld.service → /dev/null.
# Set the receive (RX) and transmit (TX) ring parameters for a network interface (These parameters are to optimize the buffer space allocated for handling incoming and outgoing network packets)
ethtool -G $NIC_INTERFACE rx 2048 tx 2048
Made it.
Now the picture is like this
There are still a lot of IRQ compared to 10G network cards.
Peak day is still Sunday. On weekdays, traffic rarely exceeds 22 Gbps
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Azeem! Yesterday was highest traffic of the week. Graphs below (after tunnings):
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello AndriiV,
Thank you for choosing Intel.
This is the first follow-up regarding the issue you reported to us.
We wanted to inquire whether you had the opportunity to review the plan of action (POA) we provided.
Feel free to reply to this message, and we'll be more than happy to assist you further.
Regards,
Simon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello AndriiV,
Thank you for contacting Intel.
This is a follow-up regarding the issue you reported. We are committed to ensuring a swift resolution and would appreciate any updates or additional information you can provide.
Please respond to this message at your earliest convenience. If we do not receive any correspondence from you by tomorrow, we will need to close this case.
We look forward to your reply.
Best regards,
Simon
![](/skins/images/45C6C2D737ED71F4C51F0145C8CB1E9C/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page