PCIE example issues

[PVanL]

I am using the PCIE Example as described in UG-20234. As i want to connect another application behind the hard-IP EP, i am modifying the testbench and APPS to allow transfers >4 bytes, which is how the example is written. The idea is: i verify in simulation, before mapping it onto the FPGA, debugging on a mapped design is way more time consuming then in simulation.

It is configured for 64 bit/250 Mhz / Atlanta interface between HardIP and APPS.

I managed to send 8 bytes to APPS, which sends those 8 bytes to memory, and returns them again. I then add 1 to each byte, to be sure i do not mix up with the sent information. The comparator at testbench detects the differences, and goes on playing (in the example it stops when send != receive. ) I adapted the header for the CmplD TLP from

|          77 RX |      MWr      | 0004    | 60000001_0000000F_00000001_00010000  |

|          77 RX |      MRd (00) | 0004    | 20000001_0000000F_00000001_00010000  |

|          77 TX |     CplD (00) | 0004    | 4A000001_01080004_00000000           |

|          77 RX |      MWr      | 0004    | 60000001_0000000F_00000001_00010000  |

|          77 RX |      MRd (00) | 0004    | 20000001_0000000F_00000001_00010000  |

|          77 TX |     CplD (00) | 0004    | 4A000001_01080004_00000000           |

(for 8 byte) to

|          77 RX |      MWr      | 0008    | 60000002_000000FF_00000001_00010000  |

|          77 RX |      MRd (00) | 0008    | 20000002_000000FF_00000001_00010000  |

|          77 TX |     CplD (00) | 0008    | 4A000002_01080008_00000000           |

The Header for 8 byte has the following TLP fields, which are to my knowledge correct:

For MWr:

DW0         60000002: length = 2, Fmt = 2’b11”, Type = 5’b00000 => strange: Fmt should be 2’b10 for MWr!!!??? (this is made by INTEL IP)

DW1         000000FF:  1stBE = F, Last_BE = F, Tag = 0, ReqID = 0 (Made by Intel IP)

DW2         00000001: why is DW1[0] = R=1, this should be 0  !!!!???? Address = 0 (Made by Intel IP) In reality it is 0000_0000 !!! (bug)

DW3         00010000 = ???? (should be data???!!!). But the data arrives correctly in the memory (Made by Intel IP) In reality it is data[31:0]

DW4: (not depicted in this table) data[63:32] works correctly

For Mrd:

DW0         20000002: Length = 2, Type = 5’b0000, Fmt = 2’b01 => strange: Fmt should be 2’b00 for Mrd (Made by Intel IP)

DW1         000000FF: :  1stBE = F, Last_BE = F, Tag = 0, ReqID = 0 (Made by Intel IP)

DW2:        00000000: why is DW1[0] = R=1, this should be 0  !!!!???? Address = 0 (Made by Intel IP)

For CplD

DW0         4A000002: Length = 2, Type = 5’b01010 (correct), Fmt = 2’b10 (correct) (Made by me)

DW1         000000FF: :  1stBE = F, Last_BE = F, Tag = 0, ReqID = 0 (Made by me)

DW2:        00000001: why is DW1[0] = R=1, this should be 0  !!!!???? Address = 0 (Made by Intel IP), in reality it is 0000_0000

DW3         data[31:0]

DW4         data[63:32]

For some reason this does not work:

The DUT PCIE hard IP does not produce nice continuous tx0..tx3 signals: they are interrupted with ‘x’ s.

In the testbench, the signal is routed via rx0..rx3 to altpcietb_bfm_rpvar_64b_x8_pipen1b. The output of rpvar: rx_data0 0000000000000000 rx_desc0 044a000002010800080000000078561011 rx_be f4 rx_dv 0 rx_dfr 0 rx_ack 0 rx_abort 0 rx_retry 0 rx_mask 0 rx_ws 0  Where the last 8 bytes are correct = data[31:0] sent!

My questions:

Why does this not work? Options:

1. the header for MRd is not correct, but this is generated by INTEL testbench?
2. The header for ClpD is not correct (this is generated by my modifications)?
3. The HardIP in DUT has a configuration that only allows 4 bytes?
4. The rpvar mimic can only handle 4 bytes?
5. The rpvar needs a modification of parameters?
6. And how do I make it working?

Thanks in advance! Pieter

[KhaiChein_Y_Intel]

Hi,

May I know if you have any updates?

Thanks

Best regards,

KhaiY

[KhaiChein_Y_Intel]

Hi,

We do not receive any response from you to the previous question/reply/answer that I have provided. This thread will be transitioned to community support. If you have a new question, feel free to open a new thread to get the support from Intel experts. Otherwise, the community users will continue to help you on this thread. Thank you

Best regards,

KhaiY

[PVanL]

Hi KhaiY,

Please do not close this thread. I did not see a clear end / conclusion in your last reply, other then:

1. please buy commercial available verification tools: it is not my intention to challenge INTEL's PICE block. i assume it works correctly.
2. INTEL will not update the PCIE example to handle more then 4 Byte
3. if you have a solution, please share it with INTEL
4. there is no example PCIE / LL MAC / Ethernet PHY, if you have one, please share it with INTEL.
   
   Last friday i finally managed to send 8 bytes (that worked already for some months), receive 8 bytes (that worked for like 4 weeks), and last friday to SEND 8 bytes, and RECEIVE them back again over the full chain in INTEL's APPS application: TLP parser, write controller, memorycontroller, read controller, TLP parser. So that is a good start to extend from 8 bytes to many bytes.
   
   My question: as this has been hard work, and as INTEL is not willing to spend their resources on upgrading the testbench up to the standard as described in the manual (sending and receiving up to the maxpayload), how much would INTEL be willing to pay me for this effort?

regards, Pieter

[KhaiChein_Y_Intel]

Hi Pieter,

I apologize for the miscommunication if there is any.

As I explained earlier, the IP can accept TLP up to max payload and the testbench provide a simple method to do basic testing of the Application Layer logic that interfaces to the variation. Corner cases and certain traffic profile stimuli are not covered. This is stated in the UG https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_a10_pcie_avst.pdf (Page 156).

Please do not share any information that is confidential or non-public accessible. Otherwise, you are always welcome to share in this Public forum.

Thanks

Best regards,

KhaiY

[PVanL]

Hi KhaiT,

I still feel we somewhere miss what we are saying to each other:

You say: the TLP can handle any traffic size, as long as it less than the max payload (which can be 128, 256, 512, 1048, 2096 Bytes). For special cornercases, please apply commercial verication IP is recommended.

I say: the testbench and APPS can ONLY handle traffic that is LESS or equal to 4 Bytes. Document UG-01145 says: $16, pag 157: "It can only handle received read requests that are less than or equal to the currently set Maximum payload size option specified under PCI Express/PCI Capabilities heading under the Device tab using the parameter editor. Many systems are capable of handling larger read requests that are then returned in multiple completions." THIS IS NOT TRUE!!!!
See figure 10 on page 21, where you clearly see the 4 bytes reported. IT IS NOT POSSIBLE TO MAKE THIS AS BIG AS MAX.PAYLOAD. I would not call anything larger then 4 bytes a corner case.

And yes, indeed, if you run the software test as in figure 12, then the application accepts traffic loadup to MAX.PAYLOAD. BUT NOT THE TESTBENCH / APPS in simulation!

So, i think i am on the edge of improving the testbench and APPS to payload = max.payload. My question: if i would share this code with you, would INTEL be willing to compensate me for this effort ? (i think INTEL does not deliver what it says it does as described in the manual)

regards, Pieter

[KhaiChein_Y_Intel · Accepted solution]

Hi Pieter,

And yes, indeed, if you run the software test as in figure 12, then the application accepts traffic loadup to MAX.PAYLOAD. BUT NOT THE TESTBENCH / APPS in simulation!

Yes. As in my previous reply, the testbench provides simple method to do basic testing and this does not cover all the traffic profile stimuli. If user wants to simulate with what is not covered in the testbench, user has to modify the testbench based on their requirements.

So, i think i am on the edge of improving the testbench and APPS to payload = max.payload. My question: if i would share this code with you, would INTEL be willing to compensate me for this effort ? (i think INTEL does not deliver what it says it does as described in the manual)

I apologize for the miscommunication if there is any. The reason I asked if you are willing to share the modification earlier is because there are some hobbyists share information, what they have created or method to solve problems in this public forum. Please do not share if the content is confidential or non-public accessible.

Thanks for your understanding. Do let me know if you have any questions or concerns.

Best regards,

KhaiY

[PVanL]

Hi KhaiY,

thanks for this clarification.

I do have a strong request: please adapt the manual accordingly, this is really very misleading!

Consider the case closed, thanks for your support.

Regards, Pieter

[KhaiChein_Y_Intel]

Hi Pieter,

Thank you for the valuable feedback. I now transition this thread to community support. If you have a new question, feel free to open a new thread to get the support from Intel experts. Otherwise, the community users will continue to help you on this thread. Thank you.

Best regards,

KhaiY

[PVanL]

Hi KhaiY,

I solved the testbench and APPS issues. The testbench now runs in simulation any length >=4, as long as smaller then max.payload. the code is not confidential, but it costed me a lot of time and i am willing to share it with INTEL, if INTEL will give me a compensation for the effort.

Regards, Pieter