Stratix 10 MX HBM2 Example design with Avalon-MM user side interface and efficiency counters
Description
This article is dedicated to users that are looking for a Stratix 10 MX HBM2 Example design with Avalon-MM user side interfaces. Avalon-MM was introduced in Quartus Pro 20.2 as an option for the interface protocol. AXI4 had been the only option up to Quartus Pro 20.2. The design in this article leverages an example design that was created using Quartus Pro 20.2 using the High Bandwidth Memory (HBM2) Interface Intel® FPGA IP. The design was adjusted to work with the Stratix 10 MX development kit and adds efficiency counters to monitor efficiency in the physical hardware.
Documentation links
Refer to the following user guides to better understand the HBM2 IP:
Stratix 10 MX Support and Documentation
High Bandwidth Memory (HBM2) Interface Intel® FPGA IP User Guide
The Stratix 10 MX development kit used for this example contains a 4GB HMB2 on top and 4G HBM2 on the bottom. The example uses the 4GB HBM2 on the top.
Example
The design created in this article is here:
qts_hbm_top_AVMM_TG16_20_2_0_50.qar - See attached at the bottom of this article.
This example will run on the Stratix 10 MX development kit TOP HBM2. The design can be adjusted to operate on the BOTTOM HBM2 if needed.
If you would like to go through the process of creating the example design attached in this article, here are the steps:
- Open Quartus Prime Pro 20.2
- In the IP Catalog, in the Library -> Memory Interfaces and Controllers folder, open the High Bandwidth Memory (HBM2) Interface Intel FPGA IP
- Fill out the parameters as follows:
Note*: The IP parameter file used to create the example is also attached to this article as hbm2_avmm_TG16.ip
For controller configuration 2 through 7, do the same thing and set as "Is clone of: Controller 0".
Now click on the "Generate Example Design" button at the top right. Use the default location listed for the project creation.
- The example will be located at <your project dir>/hbm_0_example_design/qii
Note:* All file references below can be found after unarchiving the project attached to this article.
- In the ./hbm_0_example_design/qii directory there will be ed_synth qpf and qsf files. Rather than use those files, new qpf and qsf files are created as a wrapper top level file. In the example design attached to this article, they are named qts_hbm_top.qpf and qts_hbm_top.qsf.
- Create a new top level file qts_hbm_top.v
- Create an sdc file, hbm_top.sdc
- Create efficiency_counters.v
- Create a signal tap file stp1.stp
- Edit each ed_synth_tg*_*.v file and change the TEST_DURATION parameter from SHORT to INFINITE.
- Infinite patterns are used to saturate the total count on the efficiency counters and allow a user to test on on going pattern.
- The location from the qts_hbm_top project directory is ./ip/ed_synth_tg_0_0/synth/
- There are 16 traffic generators, tg_0_0 through tg_7_1.
- Edit the ed_synth.v file to add in the efficiency counters.
- The location of the file is ./ed_synth/synth/
**** If the example design is re-generated or if Quartus Pro is changed to "Always regenerate design files for IP cores" in Quartus Settings -> IP Settings, the ed_synth.v and ed_synth_tg*_*.v will be overwritten and will need to be edited again with the changes outlined above.
Running the example
To run the example on the Stratix 10 MX development kit, open Quartus Pro 20.2 and then open Tools -> Programmer. Turn on the development kit and make sure the micro USB on the development kit is connected to the computer being used to download the sof file. No clock changes are required from the default clock settings on the board after power up. Do an Auto Detect and then Add File to add the qts_hbm_top.sof file. The chain should look like this:
Hit Start and program the Stratix 10 MX development kit.
In Quartus, go to Tools -> Signal Tap Logic Analyzer to open Signal Tap. You will see 16 instances that are used to probe each traffic generator. Make sure the Hardware setup on the top right is set for the Intel Stratix 10 MX FPGA Development kit.
Click on each instance and clock on the continuous run button to see the Avalon-MM transactions occuring real time and to see the efficiency counters for the chosen test generator. The below picture shows a snapshot from tg_0_0.
The efficiency counters are 32 bits wide and count the number of times the read and write signals are asserted. Read wait request, write wait request, and read data valid are also counted to understand when the read and writes are being held off by the HBM2 controller. The total count is used to know the total number of clock periods and stops once the 32 bit counter is saturated. All efficiency counters in Signal Tap are setup to be in unsigned decimal mode. The counter numbers can be put into a spreadsheet to understand the overall efficiency rate of write and read transactions. Efficiency numbers will be highly dependent on the addressing scheme used to write and read data to/from the HBM2.
The HBM2 in Stratix 10 shows very large memory bandwidth in a smaller board space footprint compared to using external DDR4 or DDR5 memories.
