Bug in Avalon-MM PCIe core for Non-Qword aligned writes

[corestar]

The "Avalon-MM Cyclone V Hard IP for PCI Express" does not appear to handle Non-Qword aligned writes correctly. I can't find anything in the datasheet that says only Qword aligned writes are allowed.

I do the following for writes from the root complex using iowrote32:

What shows up in SignalTap is the the attached.

For the qword aligned addresses, everything is fine. The address is in the bar2_address signal and the data is in the bar2_write_data signal.

For non-qword aligned addresses, the bar2_address is 0 and it puts that address in the lower 32-bits of bar2_write_data.

The data value is correct in all cases.

So is the Avalon-MM core unable to deal with non-qword aligned addresses. I was sort of hoping it would take care of such complexities as described in the Avalon-ST manual. But apparently not.

[FvM]

Hi,
the Signaltap recordings don't look incorrect at first sight.
You are performing 32 bit writes, they are translated to partial enabled 64 bit writes. Data position, 64-bit address (a[2:0] masked out) and byte enables are matching for all four writes. The only confusing point is address copied to inactive data part, but it shouldn't cause trouble. Interesting question would be if unaligned 64 bit writes and reads are translated correctly.

Does Cyclone V Bar2 support bursting operation? Then we would expect unaligned 64 bit write split into two aligned writes, otherwise failure.

Regards
Frank

[corestar]

Hello @FvM 

Thanks for looking at this. I completely agree that data is fine as I noted.

But I have to disagree that the address is not a problem since I have no way of knowing when it is putting it in the bar2_write_data and when it is putting it in the bar2_address. Note: it is not just copied; the bar2_address is wrong. For example, on the third write (after the third bus_enable), it says the address is 0x08 (which apparently is just a hold over from the previous TLP,  but has the correct address, 0x0C, in the low dword of write_data. If it had just copied it, I agree it would not be a big deal. But the bar2_address for the first write is 0 (should be 4) and for the third is 0x8 and should be 0xC. I see no way to disentangle this. 

I did not include it, but I captured the 64-bit TLP and they in fact look as expected.

I agree it would be interesting to see what 64-bit writes do. Unfortunately, as near as I can tell drivers on the root complex have no way of doing that other than DMA. And the writes I included are the 32-bit writes to setup the DMA. Clearly I could get around it, but the whole point of the Avalon-MM vs the Avalon-ST PCI was to simplify things. The Avalon-ST interface is quite clumsy, but I at least have it working and so have gone back to it.

Thanks again

[FvM]

Hi,
on the 64 bit BAR2 interface, address bits 2:0 are always zero, the interface can only perform aligned access. Unaligned accesses from the host are translated to aligned accesses by shifting data and setting byte enables. When connecting 64 bit BAR2 to 32 bit slaves, interconnect will take care of the back-translation.
Do you see wrong data in 32 bit slave access?
Regards,
Frank

[corestar]

Hello,

I have both BAR0 and BAR2 set to 32-bit as below:

So based on this:

I had expected it to accept a DWORD aligned address.

The data itself is ok in all cases.

Thanks

[FvM · Accepted solution]

Hi,
I see the misunderstanding. BAR 32/64-bit setting refers to address width. Data width is the native master width, can be 64 to 256 bit, in your design 64 bit. To see address and data presented to a 32-bit slave, tap the slave interface directly.

Regards,
Frank

[corestar]

Hello @FvM ,

You are right, I did misread that. I had made my bridges 64-bit data width since I assumed they had to be in order to connect to the 64-bit wide PCI data. Just to see what would happen, I changed them to 32-bit and it works! Non-qword aligned 0xC shows up in the address and 0x80 in the data.

I don't quite see how a 64-bit interface can feed a 32-bit interface at the same clock speed. What if the TLP had 20 dwords? I assume Qsys is creating a FIFO somewhere, but using the technology viewer proved an exercise in futility.

It's all a bit confusing, but at least I'm a step closer.

Thanks  for the help.

[Wincent_Altera]

Hi,

Thanks FVM for helping to answer this on Intel Forum.

So corestar do you still have any other question ?

Regards,

Wincent_Intel

[Wincent_Altera]

Hi

Kudos to FVM for helping to address user questions here. Also glad that you got the answer.

If you have a new question, feel free to open a new thread to get support from Intel experts.

Otherwise, the community users will continue to help you on this thread. Thank you

If your support experience falls below a 9 out of 10, I kindly request the opportunity to rectify it before concluding our interaction. If the issue cannot be resolved, please inform me of the cause so that I can learn from it and strive to enhance the quality of future service experiences. 

Regards,

Wincent_Intel