Programmable Devices
CPLDs, FPGAs, SoC FPGAs, Configuration, and Transceivers
20638 Discussions

Reading HPS EMAC counters not reliable

Dzar
New Contributor I
2,037 Views

UPDATE: Seems EMAC1 is looking OK. The problem, there, was understanding what's meant by "good" in some cases. But EMAC0 looks like it's almost not counting anything. It's quite sporadic when I see anything increment, and this is while sending thousands of packets in/out of it.

 

------------------- original message, below ---------------

 

I'm using both HPS EMACs on a design and want to use the counters in them for various purposes. For example, 0xff700180 and 0xff702180 should be frame counters for EMAC0 and EMAC1, respectively.

 

But when I send a packet to EMAC0, I don't see this counter increment. And when I send, say 10, packets to EMAC1, it counts 3. I know the packets are being sent as they arrive at the destinations just fine, but the EMAC is not counting.

 

The same for other counters. In the above example, they are UDP packets. But the IP4 and UDP counters are not incrementing by the right number, either.

 

I'm using DS-5 debugger to read these registers/memory, as well as saving them in variables in my program code and these agree with each other, but are clearly wrong.

 

Any ideas? Has anyone seen these counters work, properly?

 

(I am able to read/write status registers and such, just fine, but these counters seem broken.)

0 Kudos
1 Solution
Dzar
New Contributor I
863 Views

Final answer... Set the PR bit, not the RA bit. One does not need both, but the PR bit is the correct one if one wants counters to work. No documentation on this, sadly, but that's what we've deduced.

View solution in original post

0 Kudos
15 Replies
Fawaz_Al-Jubori
Employee
863 Views

Hello,

Which device is this?

Thanks

0 Kudos
Dzar
New Contributor I
863 Views

These are Cyclone V SE parts: 5CSEBA2U23I7N for example. Dual ARM.

0 Kudos
Ahmed_H_Intel1
Employee
863 Views

Hi,

Can you please ensure your design includes the necessary Intel settings to configure the HPS EMAC outputs for the required delays? This might be the reason.

 

Here you can find very useful guidelines

https://www.intel.com/content/www/us/en/programmable/documentation/nau1499473596388/loi1508376022247/lcb1501781141048/cqv1501781332717.html

 

 

0 Kudos
Dzar
New Contributor I
863 Views

I guess I don't understand why that link is useful. First, it's for Stratix 10 devices, and I'm using Cyclone V. But assuming the EMAC designs are identical, that document is about how to integrate the PHY and configure the system.

My system "works." That is, I can send and receive millions of packets, at line rate (100 Mb/s in my case) with no issues. The only problem I seem to have is that when I read the counters in EMAC0, I see obviously wrong numbers. EMAC1 counters seem correct, however.

Is there a known issue with EMAC0 counters? What are some configuration issues with it that I may have overlooked repeated to the counters. The both seem to work very well in TX/RX, otherwise. I use the same configuration code for both devices. One works and the other doesn't.

0 Kudos
Fawaz_Al-Jubori
Employee
863 Views

Hello Dzar,

this is not the expected behavior. By default, the counter register should reflect the number of frames, whether good or bad.

Let me investigate about these counters. I will post my feedback once I find more information about it,

 

thanks

 

0 Kudos
Dzar
New Contributor I
863 Views

OK,

 

I've also tried reading the counters in code (using alt_emac.h and the structures in there) and I get the same results. For the attached PCAP files, I see 0 frames received, 12 transmitted on EMAC0, and 12 Frames received, 28 frames transmitted on EMAC1. These are all correct, except the 0 on RX for EMAC0. My PCAP files that show the traffic in/out are attached. (Note, the SSDP traffic on EMAC1 happened before I reset the EMAC counters so are not reflected, here).

 

I would love to attach my PCAP files (16 KB each) but when I try, I see only a folder icon (no file name) and the "Answer" button is grayed out until I delete the file. I cannot attach multiple files. Seems upload is not working. But it's 100% clear to me that the EMAC0 RX counters (I reported frames, but octets, IP packets, etc. on the RX side are all wrong - like 0 in this case).

 

The RX counters on EMAC0 do count, however... seems they count ARP packets. But that's all I can see will increment them. And, again, this is the ENET FRAME counter so it should increment on every frame, should it not? How can it tell what's what?

 

Let me know if there is a way to get you my PCAP files if you want them.

 

(Here's my code)

 

void ShowMacCtrs()

{

char message[PROGRESS_STORAGE_WIDTH];

struct ALT_EMAC_GMAC_s *MAC0 = (struct ALT_EMAC_GMAC_s *) 0xff700000;

struct ALT_EMAC_GMAC_s *MAC1 = (struct ALT_EMAC_GMAC_s *) 0xff702000;

 

show_message("MAC\tFrms RX\tFrms TX\n");

sprintf(message, "0:\t%d\t%d\n", MAC0->rxframecount_gb.cnt, MAC0->txframecount_gb.cnt );

show_message(message);

sprintf(message, "1:\t%d\t%d\n", MAC1->rxframecount_gb.cnt, MAC1->txframecount_gb.cnt );

show_message(message);

}

 

output looks like this:

 

MAC Frms RX Frms TX

0:  0     12

1:  12    28    

 

0 Kudos
Fawaz_Al-Jubori
Employee
863 Views

Hello,

We were looking at the Synopsys Errata in case there is a known issue for this counter. One interesting thing that might affect the counters.

 

Operate the RxFIFO in the store-and-forward mode.

Operation_Mode Register: located at address:

0xFF701018 (for EMAC0)

0xFF703018 (for EMAC1)

Set bit 25 (“rsf” bit) to 0x1.

 

 

We looked at this register in the uboot that we are using in the system, and it is NOT set. It might be needed to be set before we can read from the counters.

 

I will test this and let you know the result.

 

Thanks

 

 

0 Kudos
Dzar
New Contributor I
863 Views

I am setting the fsf bit (bit 25) on both EMAC's DMA controller. Always have been.

 

I do see that on a Linux system, it seems to be able to read the RX counters on EMAC0. (Assuming that ifconfig and /dev use those). So I've been pouring over the driver (supplied by Altera/Intel) for Linux to see if I'm doing anything wrong/different and have not found that, yet. So it's still confounding.

0 Kudos
Dzar
New Contributor I
863 Views

I am able to reproduce the non-working RX counter on uboot:

 

  1. Boot my board to uboot prompt (only)
  2. configure emac0 to add its IP/MAC address
    1. setenv ethaddr 00:11:22:33:44:55
    2. setenf ipaddr 10.10.7.90
  3. md.l 0xff700180 1 (and see the result is a low number... or maybe 0)
  4. on a PC connected to that port, do iperf -B 10.10.7.88 -c 10.10.7.90 -u -l 1000 -n 10000 (should send ten UDP packets, with the last re-transmitted ten times as there is no server running on my hardware)
  5. md.l 0xff700180 1 will now show three more frames than before. If I wireteap the line, I see three ARPs sent (no UDP packets).
  6. On PC, put in a static ARP entry for the hardware
  7. do iperf -B 10.10.7.88 -c 10.10.7.90 -u -l 1000 -n 10000
  8. See UDP packets being sent on the wire (and the light flashing on the port)
  9. md.l 0xff700180 1 and see no change in the counter value

 

This is uboot, folks. Built from code supplied by Intel. Can you repeat this? Do you have any hardware with EMAC0 connected to an RJ45 you can test?

0 Kudos
Fawaz_Al-Jubori
Employee
863 Views

ello Dzar,

I have tried some settings before reading from frame counter. Please note this is from EMAC1, since my dev kit uses EMAC1 :(

However, you might try set these registers first:

 

https://www.intel.com/content/www/us/en/programmable/hps/cyclone-v/hps.html#topic/sfo1410069057648.html

 

This is the control of MMC.

 

The interesting information here is, you can enable/disable broadcast frames, enable full/half preset counter, etc...

The interesting part when I tried memtool in linux was, each time I read, there was a reset to the counters. This MMC control allowed me to turn-off this feature, so I can read the counters without being reset.

 

This is the command I used: (this command initialize the MMC counters control)

memtool -32 0xFF702100=0x1f0

 

then I read the counter:

memtool -32 0xFF702188 1

 

And I can find the counter is increasing accordingly.

 

Please note the above bold addresses were tested since I am using EMAC1 only. You might need to change them to 0xFF700100 and 0xFF700188 for EMAC0, respectively.

 

Can you please try this on linux, if it works, then you need to apply these settings to DS-5 before you run your code.

 

I suspect mostly the reset configuration after each counter read you do. This option should be disabled to maintain the number of frames.

 

 

Thanks

 

 

 

0 Kudos
Dzar
New Contributor I
863 Views

I want to reiterate, as clearly as I can, that the registers are set exactly the same for EMAC0 and EMAC1. The same code is used to initialize both; the only difference is the base address passed in, essentially.

 

I, too, I have tried various settings and I do see that the register get reset to 0 if that setting is set. But then it never counts as I stream packets. It will increase if an ARP happens. But not if I send IP packets.

 

I see that Linux uses the reset on read (presumably because the counters are 64-bit yet the EMAC counters are 32 so that's how they deal with potential overflow). But there you are.

 

I will do another test with Linux on Monday. I do believe that I saw valid counts for IP packets, but I will try sending packets with errors to see if they get counted. It could be that the Linux implementation is not actually using the MAC counters but the stack; if that's the case, it will not see errors that the MAC will throw out so that will be telling.

 

My analysis of the Linux drivers is that they do nothing differently than what I am doing, myself.

 

0 Kudos
NNauf
Partner
863 Views

Can Intel route EMAC0 via the Fabric to a connector on the board and do the experiment using EMAC0?

0 Kudos
Fawaz_Al-Jubori
Employee
863 Views

Hello NNauf,

This is not applicable since there is no Hard PHY connected to EMAC0.

 

Thanks

0 Kudos
Dzar
New Contributor I
863 Views

It seems that I've uncovered and fixed the problem.

 

The short answer is that when using the MAC in promiscuous mode (RA bit set in the filter config register), one also needs to set the PR bit in the same register.

 

Bottom line is that the documentation is wrong on how the counters work, at best, and the MAC has a bug that it only counts when the MAC filters match even when in promiscuous mode, in my opinion.

 

 

0 Kudos
Dzar
New Contributor I
864 Views

Final answer... Set the PR bit, not the RA bit. One does not need both, but the PR bit is the correct one if one wants counters to work. No documentation on this, sadly, but that's what we've deduced.

0 Kudos
Reply