A design based on the PCIe DMA transfer example design for Arria 10 device.

[Sijith]

Hi,

This is a extension of the discussion https://community.intel.com/t5/Programmable-Devices/Modifying-the-PCIe-DMA-transfer-example-design-for-Arria-10/m-p/1484799#M90698  and https://community.intel.com/t5/FPGA-Intellectual-Property/API-calls-failed-while-running-PCIe-DMA-transfer-example-design/m-p/1520494#M28014 where we could not get a solution to the problem we are facing.

We also tried trying to get a Intel Premium help (We are based on a University in USA, but purchased the FPGA and Intel Quartus Prime Pro with normal rate NOT through University reduced rate). But thats is rejected by Intel explaining that since, We are from a University, We are not eligible for Premium help. It would be great if you suggest a way to resolve the issue. 

Story in short: We were working on a project to use the FPGA to increase the data throughput in an experiment. We are planning to use FPGA as an intermediate in signal transmission through an optical fiber from electronic readout to the Data Acquisition System. So we need a way to transfer signal though the FPGA (input though the QSFP+ port and output through the PCIe). We were planning to develop a design based on the PCIe DMA transfer example design (that involve the DDR4 memory to store data). In the example design, data is created at a host computer and is written into the DDR4 memory through DMA write through PCIe. And then it read it back to the hot computer to verify the sending data is same as the receiving data.

What we need in our project to stream data from QSFP+ to the DDR4 memory and then DMA read through PCIe to host computer using the API provided. 

As an initial step (to test the working of the FIFO + PCIE DMA transfer example design, as our final aim is to get the data from QSFP+ to FIFO to goto the DDR4 element), we used a custom IP of data counter that counts upto 1000 (works on getting an trigger from a switch SW[0] ) to connect to Avalon FIFO IP. Then this design is integrated to the  PCIe DMA transfer example design using Platform Builder. The idea is to stream created at the counter though FIFO to the DDR4 memory element. Then DMA read though PCIE to a host computer.  But when we try to do DMA read from the host computer, we could not see the counter outputs (which we are suppose to get?).

Great if you could help us out. Any suggestion where to get help for making our design work is highly appreciated. 

[ventt]

Hi @Sijith,

In order to see the nodes, please ensure that you complete the correct pin assignment in the Pin Planner and compile the project beforehand.

Besides creating a Signal Tap Instance by HDL Instantiation, you can create the Signal Tap instance using the Signal Tap GUI.

Please refer to the steps in Section 2.3.1 to create a Signal Tap instance with the Signal Tap logic analyzer GUI.

User Guide: https://www.intel.com/content/www/us/en/docs/programmable/683819/23-4/faq.html

Thanks.

Best Regards,

VenTing_Intel

[Sijith]

Hi VenTing,

Correct pin assignment you meant is for the modified design, right? I have the pin assignment for the modified design (you can see that if you could open the ModifiedDesign file I shared you previously). And I assume I may not need to work on the pin assignment just after adding the Signal Tap IP into the design in the Platform Designer? 

I have a pin assignment (as I mentioned above I did not change any pin assignment after adding Signal Tap IP to the design) and did compile it too. But looks something is still missing.

I went through the document and what I could understand is, we can add Signal Tap instances using GUI (So here we dont need to add signal tap IP to the Platform designer system) . Please correct me if I got wrong?

Looking forward for your reply. Thank you.

[ventt]

Hi @Sijith,

Yes, you can run the Signal Tap by creating a Signal Tap instance with the Signal Tap logic analyzer GUI instead of using HDL instantiation (which is through the Signal Tap Logic Analyzer IP).

There are two ways to create a Signal Tap instance in this method:

1. Click Tools ➤ Signal Tap Logic Analyzer.

2. Click File ➤ New ➤ Signal Tap Logic Analyzer File.

You may follow the steps in Section 2.3.1 in the user guide.

Thanks.

Best Regards,

VenTing_Intel

[Sijith]

Thank you.  I am able to see the nodes. The problem was my version of Quartus Prime Pro (18.1) has the Signal tap GUI with node finder NOT having LIST option instead it has search nodes option. So I hd to search for the needed nodes.

There are still. some problems. After adding nodes to the instance and I could see thee message in Red in the Instance Manager of Signal Tap GUI "Start Rapid Recompile to Continue". But the problem is Rapid Recompile button is inactive as shown in attachment. Then I went for full compilation. 

After full compilation and programmed the board through JTAG. But I am getting the message in Instance manager "Program the device to continue".  I did it multiple time and again I gets this message. I tried programming using Programmer from the Quartus Prime also (I have seen an intel  community thread suggesting to do so https://community.intel.com/t5/Intel-Quartus-Prime-Software/Signal-tap-Program-the-device-to-continue/td-p/1276158 ), but it did not work for me as well. Screenshot attached.  I have seen a community thread suggesting to do so.

I have an additional question too. as I am trying to debug the counter fifo outputs at first, can I make an instance having signals that comes to fifo from the counter and from the counter-fifo to the DUT (PCIe DMA) as well? Or I need separate instances for them?

Also, as my design having counter starting when SW[0] =1 (pull up), do I want to set SW[0]=1 after programming the device just before Running the Signal Tap?

[ventt]

Hi @Sijith,

Sorry for the late reply. I was doing some experiments from my side. Here are some suggestions on the message "Program the device to continue": 

1. Reconnect the JTAG connection to the board and ensure the board is powered on when running the Signal Tap. 
2. Make sure the OPN that matches the board is set correctly in the Quartus Prime Pro software (Assignments > Device).
3. In the Signal Tap Window, check the Device connection and make sure it is connected to the right hardware device. You can click on the "Scan Chain" to select the correct device.
4. After you compile the project, make sure you open the signal tap file that is included in the "Files" section of the Project Navigator.
5. Check the QSF (Quartus Setting File) of the project to make sure the signal tap is enabled and the signal tap file is included in the QSF:

set_global_assignment -name SIGNALTAP_FILE stp_name.stp

set_global_assignment -name ENABLE_SIGNALTAP ON

set_global_assignment -name USE_SIGNALTAP_FILE stp_name.stp

You may include both signals in an instance to observe their behaviors.

You can set SW[0] = 1 before or after running the Signal Tap, as the Signal Tap will capture the live signal behavior.

Thanks.

Best Regards,

VenTing_Intel

[ventt]

Hi @Sijith,

May I know if you have any updates on the case?

Thanks.

Best Regards,

VenTing_Intel

[Sijith]

Hi VenTing,

I am afraid that something is still missing. I was thinking that if you gets sometime early next week for a short meeting, that would be perfect. I believe that will make debugging little bit more faster. If you are available to meet please message me in my email.  Eventhough  I will be available anytime to meet, CDT 9 AM to 5 PM would be perfect for me. Let me know if you have any questions. Thank you.

Regards

Sijith

[ventt]

Hi @Sijith,

Sure. I've contacted you through email. Please let me know if you're not receiving it.

Thanks.

Best Regards,

VenTing_Intel

[Sijith]

Thank you I have got it. And looking forward for the link for the meeting.

1) I have a quick question. The "JTag communucation error" while programming the device happens sometimes (some times it works fine too) do you have any idea regarding the potential reason for this unstable behavior? The hardware is well connected though. Any software crash?

2) Also, even in the scenarios where the JTAG is ready and  "Data acquisition in Progress" message is seen in the Instance manager tab, the instance status shows "waiting for the trigger" and it waits forever and no Data is being populated in Data Log plot ? What may be the reason for this?

Is it something to do with my trigger selection? If my counter module has an input ready that triggers the counter to output data. Also a switch to start count and a reset counter. What are the triggers I am suppose to use? Pls see one of the the attached pic to see my trigger selections.

3) If my code has some state selection for a signal (say input=1,) and can I somehow change it in the Signal Tap GUI? Like somehow by setting the Trigger or so?

[ventt]

Hi @Sijith,

In the sub-design (Counter_FIFO_SignalTap.zip), it consists of only the counter and FIFO. The FIFO does not have the input signals (avalonmm_read_slave_address and avalonmm_read_slave_read) received from other IP. Based on the FIFO coding, in order to get the dataflow at FIFO output (avalonmm_read_slave_readdata) from the counter, it requires the trigger of input signals (avalonmm_read_slave_address and avalonmm_read_slave_read). When (avalonmm_read_slave_address == 0 and avalonmm_read_slave_read == 1), the readdata signal will get the data from the counter and hence there is dataflow at FIFO output. Otherwise, the readdata signal will not get the data from the counter, instead it will get {32(1’b0)}.

This also explains the observations of the Signal Tap result. The condition:

fifo_0_out_address = 0 and fifo_0_out_read = 1, 

needs to be met to allow dataflow to happen when data_0_conduit_end_input is high. From the video recording, when you set the fifo_0_out_read to 1 or 0, you observed that no dataflow happened because the fifo_0_out_address is always set at 1. 

In short, connect the avalonmm_read_slave_address and avalonmm_read_slave_read with valid signal if you have only counter + FIFO and want to check dataflow from counter to FIFO in the sub-design. 

From the counter_fifo_signaltap.mp4 video, when sampled at a time instant, the fifo_0_out_readdata is observed to be different from the data_0_avalon_streaming_source data. 

This is because the FIFO core performs the endian conversion to conform to the output interface protocol.

For instance, at 00:00:23 in the video, perform the endian conversion for fifo_o_out_readdata[31..0], you will get 00000000000000000000010100010010b, which is still not the same as the source_data, while the value is smaller than the source_data. Can you the add clock signal that drives the source_data and fifo_o_out_readdata, and the signals that carry the data before passing it to the fifo_o_out_readdata signal in the Signal Tap to check that the correct data are passed to fifo_o_out_readdata?

Next, the Signal Tap logic analyzer is used as a debugging tool to capture and store the specific signal data continuously when the trigger conditions set are met. User only specify conditions that trigger the start or stop of data capture. Therefore, assigning variables in Signal Tap is not possible. Assigning variables should be done in the coding.

Besides, did you compile successfully on the original PCIe DMA transfer example design without adding Signal Tap? 

There is a similar issue discussed in the Intel Community Forum. Can you check out the forum thread? https://community.intel.com/t5/Intel-Quartus-Prime-Software/3-Wire-SPI-IP-Component-with-Arria-10/m-p/1476199/highlight/true

Thanks.

Best Regards,

VenTing_Intel

[ventt]

Hi @Sijith,

May I know if there are any updates from you?

Thanks.

Best Regards,

VenTing_Intel

[ventt]

Hi @Sijith,

We have not received any response from you on the previous answer that we provided. Please login to ‘https://supporttickets.intel.com’, view details of the desire request, and post a feed/response within the next 15 days to allow me to continue to support you.

After 15 days, this thread will be transitioned to community support.

The community users will be able to help you with your follow-up questions.

If you feel your support experience was less than a 4 or 5, please allow me to correct it before closing, or please let me know the cause so that I may improve your future support experience.

Thanks.

Best Regards,

VenTing_Intel

[Sijith]

Hi Ven Ting,

Sorry for late reply, I was bit busy with an another project. I would like to have your  support to complete this project successfully. 

1)In the sub-design (Counter_FIFO_SignalTap.zip), I have been setting avalonmm_read_slave_address == 0 and avalonmm_read_slave_read == 1 using the button in the FPGA board.

2)"Can you the add clock signal that drives the source_data and fifo_o_out_readdata, and the signals that carry the data before passing it to the fifo_o_out_readdata signal in the Signal Tap to check that the correct data are passed to fifo_o_out_readdata?" 

You mean I have to add th clock signal that drives the source_data and fifo_o_out_readdatain the SignalTap GUI window to view it? Then The signalTap should run in some different clock? I mean the default clock? (currently signal tap is running in the same clock as clock signal that drives the source_data and fifo_o_out_readdata .

3) "did you compile successfully on the original PCIe DMA transfer example design without adding Signal Tap?"

Yes I did

4)"There is a similar issue discussed in the Intel Community Forum."

In the thread you mentioned basically talks about some problem with connection while adding an IP to a design (so he can change it easily). But in our case we are adding Signal Tap to the design and the connections are done automatically, so may I know you are suggesting to go and change them manually?

Thank you and regards

Sijith