Poor UDP performance over ad-hoc network

[YYan8]

Hi,

I am using iperf3 to send UDP packets over ad-hoc network with Atheros AR9280 in 2.4GHz.

The performance is pretty poor, see the following screenshots:

1.

The Galileo can only send about 700 packets within 10 sends for ad-hoc network, which is supposed to send much more than this.

2. Then, I tested it through Ethernet, two Galileos.

The Galileo can send over 50k packets within 10 sends, which demonstrates that Galileo can send and handle this many packets.

We are expecting that Galileo can send at least 500 packets per second over ad-hoc network. However, the performance is quite limited. It cannot achieve what we need.

I have no idea why the performance is so poor. Is it because Galileo's limited hardware capability, wireless network configuration or something else?

Any ideas are appreciated.

P.S. 1. The two Galileos are vary close, and both are connected with proper antennas, so there shouldn't be much interference between them.

2. The option -l 12 with iperf3 means the length of a packet. It has been the shortest length.

3. The default bandwidth is 1M configured for iperf3, it should be more than enough to send 500 or more packets in a second.

Updated:

We tested the same scheme with old devices, FreeBSD, Atom x86 32bit, Atheros AR9280.

Old devices are working fine.

Old device is powerful than Galileo, but the performance should not be a such difference.

[FTinetti]

Hi AU_yzy0050 ,

What happens with larger packets, e.g. 400 or 500?

What Yocto image (and Atheros driver) are you using?

Fernando.

[YYan8]

Hi, FGT

I haven't tried larger packets, but tried default packets with the length of 1470 bytes, it was not working as well as old devices. It send about 200 packets within 10 seconds, while old devices could send much more than this number, not exactly remembered the number.

For the image, I am using latest one, BSP 1.2.1, and I believe we are using ath9k driver pre-installed in BSP 1.2.1. Only did minor modifications, i.e. some kernel printings to trace some messages.

Thank you.

[FTinetti]

AU_yzy0050 wrote:

Hi, FGT

I haven't tried larger packets, but tried default packets with the length of 1470 bytes, it was not working as well as old devices. It send about 200 packets within 10 seconds, while old devices could send much more than this number, not exactly remembered the number.

For the image, I am using latest one, BSP 1.2.1, and I believe we are using ath9k driver pre-installed in BSP 1.2.1. Only did minor modifications, i.e. some kernel printings to trace some messages.

Thank you.

What do you mean by "old devices"? If you refer to Atom as an "old device":

http://ark.intel.com/products/family/29035/Intel-Atom-Processor# @All Intel® Atom™ Processor

Then you could try:

a) Easy experiments, at least for documentation: different packet sizes and compare those with other processors and Ethernet

b) Given you have already recompiled the kernel maybe you should try with a better network driver.

Fernando.

[YYan8]

Hi, FGT

Yes, by old devices, I mean our devices with Atom Processor N270: http://ark.intel.com/products/36331/Intel-Atom-Processor-N270-512K-Cache-1_60-GHz-533-MHz-FSB Intel® Atom™ Processor N270 (512K Cache, 1.60 GHz, 533 MHz FSB) Specifications

In the old devices (FreeBSD), we are also using the same driver.

And I will try other packet sizes.

Thank you.

[FTinetti]

AU_yzy0050 wrote:

Hi, FGT

Yes, by old devices, I mean our devices with Atom Processor N270: http://ark.intel.com/products/36331/Intel-Atom-Processor-N270-512K-Cache-1_60-GHz-533-MHz-FSB Intel® Atom™ Processor N270 (512K Cache, 1.60 GHz, 533 MHz FSB) Specifications

In the old devices (FreeBSD), we are also using the same driver.

I see... I know it does not necessarilly helps for the problem you have, but

a) At http://www.intel.com/content/www/us/en/embedded/products/quark/x1000/overview.html Intel® Quark™ SoC X1000 Series—Get Started , about Intel® Quark™ SoC X1000 Series:

Intel® Pentium® processor instruction set architecture (ISA) enables applications to scale from the Intel Quark SoC to platforms based on Intel® Atom™ and Intel® Core™ processors, without recompiling code.

b) And comparing http://ark.intel.com/products/36331/Intel-Atom-Processor-N270-512K-Cache-1_60-GHz-533-MHz-FSB Intel® Atom™ Processor N270 (512K Cache, 1.60 GHz, 533 MHz FSB) Specifications with http://ark.intel.com/products/79084/Intel-Quark-SoC-X1000-16K-Cache-400-MHz?q=Intel%C2%AE%20Quark%E2... Intel® Quark™ SoC X1000 (16K Cache, 400 MHz) Specifications you are right taking into account Lithography, for example, but taking into account GHz, Quark is 1/4 behind Atom...

Anyhow, now that you mention that you are using the same driver maybe there is some other I/O and/or CPU-memory interaction that strongly penalizes the device in the ntel Galileo and not in other platforms... would you please send the details of those old devices (PC? laptop?, memory size?).

Fernando.

[YYan8]

Hi, FGT

Anyhow, now that you mention that you are using the same driver maybe there is some other I/O and/or CPU-memory interaction that strongly penalizes the device in the ntel Galileo and not in other platforms... would you please send the details of those old devices (PC? laptop?, memory size?).

The memory is DDR2 4GB, and we are using SATA3 SSD.

As you mentioned I/O writes and reads, I guess it could be the cause. The Galileo now frequently reads and writes kernel messages, so the speed of SD card could be a limit. Anyway, I don't know much about this hardware specification.

I will test a pre-compiled binary image by Intel to see if there is much difference.

Thank you.

[idata]

Hi AU,

Have you been able to keep working on this? Do you have any updates? Regards,-Pablo

[YYan8]

Hi, Pablo,

Thanks for your reply.

I'm not sure if I/O reads and writes limit the performance, but as far as I understand, once the OS is loaded, kernel messages should be handled in memory instead of I/O reads and writes.

Anyway, as I mentioned in the above reply, I will test a pre-compiled binary image by Intel, which does not have frequent reads and writes to kernel messages.

Will let you know the results.

Thank you.

[YYan8]

Hi, Pablo and FGT

Just tested the same scheme with a pre-compiled kernel image without any kernel modification, the performance is acceptable, see the below screenshot:

Except for our kernel modification, which are some printings, the rest should be the same.

In our modification, every packet sent and received will issue some printings in kernel. Therefore, if there are a lot of packets, there will be a lot of printings in kernel.

Now my question is: are these kernel messages I/O reads and writes? If so, are they associated with SD card? Frequent SD card reads and writes will decrease performance for sure.

Like I said in the previous reply: as far as I understand, once the OS is loaded, kernel messages should be handled in memory. Please let me know if I was wrong.

Thank you.

[FTinetti]

AU_yzy0050 wrote:

Hi, Pablo and FGT

Just tested the same scheme with a pre-compiled kernel image without any kernel modification, the performance is acceptable, see the below screenshot:

Except for our kernel modification, which are some printings, the rest should be the same.

In our modification, every packet sent and received will issue some printings in kernel. Therefore, if there are a lot of packets, there will be a lot of printings in kernel.

Now my question is: are these kernel messages I/O reads and writes? If so, are they associated with SD card? Frequent SD card reads and writes will decrease performance for sure.

Like I said in the previous reply: as far as I understand, once the OS is loaded, kernel messages should be handled in memory. Please let me know if I was wrong.

Thank you.

Experiment seems to indicate that's not true (or a kernel expert should explain what really happens, but kernel experts are not usually at hand). I'd sujest you modify your printings, so that you do not print something on each packet... e.g. average values, or store data in some array to print it all at once (you should be careful about array length, though...), or something similar...

HTH,

Fernando.

[idata]

Hi AU,

I'm wondering if you have updates on this case.Regards,-Pablo

[YYan8]

So far, no updates. The issue could be from too many I/O writes and reads.

Will update later on if possible.

Thanks

[idata]

Hi Au,

Were you able to run some more tests? Regards, -Pablo