PCIe in-simulation status debug for PIPE interface

[#1]

PCIe in-simulation status debug for PIPE interface

This article demonstrates how to create post-simulation status flags for PCIe transaction/error messages on the PIPE interface. This will be helpful for those who would like to debug the TLP and any issues associated with it. The user can simply look at the status flags and know the type of transaction done by the TLP without actually having to decode it. The current files are known to be compatible with the Modelsim simulator and Synopsys VCS (with limitations) and support all Gen1 and Gen2 PCIe HIP configurations from Altera.

Overview

Supported Transaction/Error Messages

The current version supports creating and viewing flags for the following transactions and error messages for both Gen1 and Gen2 HIP.

Note: VCS does not support flags for the error messages. 

Signal name (TLP Type)	Transaction/Error type
MRd_tx / MRd_rx	Memory Read Request
MRd_Lk_tx / MRd_Lk_rx	Memory Read Request-Locked
MWr_tx / MWr_rx	Memory Write Request
IORd_tx / IORd_rx	I/O Read Request
IOWr_tx / IOWr_rx	I/O Write Request
CfgRd0_tx / CfgRd0_rx	Configuration Read Type 0
CfgWr0_tx / CfgWr0_rx	Configuration Write Type 0
CfgRd1_tx / CfgRd1_rx	Configuration Read Type 1
CfgWr1_tx / CfgWr1_rx	Configuration Write Type 1
Cpl_tx / Cpl_rx	Completion without Data
CplD_tx / CplD_rx	Completion with Data
CplLk_tx / CplLk_rx	Completion for Locked Memory Read without Data
CplDLk_tx / CplDLk_rx	Completion for Locked Memory Read with Data
ERR_COR_tx / ERR_COR_rx	Error message when a correctable error is detected on the PCIe interface
ERR_NONFATAL_tx / ERR_NONFATAL_rx	Error message when a nonfatal, uncorrectable error is detected on the PCIe interface
ERR_FATAL_tx / ERR_FATAL_rx	Error message when a fatal uncorrectable error is detected on the PCIe interface
ASRT_INT_tx / ASRT_INT_rx	Assert Interrupt virtual wire for all INT lumped together
DSRT_INT_tx / DSRT_INT_rx	De-assert Interrupt virtual wire for all INT lumped together
PMM_tx / PMM_rx	Power management messages lumped into one signal

How To:

This section provides a step by step guide on how to create and view the transaction/error status flags in the Modelsim simulator. The configuration of the files are similar for both Gen1 and Gen2 HIP. There is slight difference in configuration based on whether it is a x1, x4 or x8 lane HIP.

Modelsim

STEP 1:

The first step is to create a PCIe HIP using Altera Quatus II software and Qsys. Here is brief instruction on ow to creae a PCIe HIP using Qsys.

1. Create a project directory.
2. Create a PCIe HIP design using either Megawizard or Qsys. Follow the instructions given in the ug_s5_pcie.pdf to create the IP.
3. Open Qsys within Quartus II. In Qsys go to File > Open and choose the .qsys file for the PCIe configuration you want (for example: pcie_de_gen1_x1_ast64.qsys).
4. Navigate to the "Generation" Tab at the top of the Qsys GUI. Choose your preferred HDL language for the Simulation and Testbench Model, and choose "Standard BFMs for Standard Avalon Interfaces" for the Testbench Qsys System. Choose your project folder for the output directory and click "Generate.

Once the IP is generated there should be two subfolders (/testbench and /synthesis) under your <project_directory>/<pcie_de_gen1_x1_ast64>/ (The name depends on the type of IP that has been generated).

STEP 2:

To interpret the TLPs at the PIPE interface it is neccessary to make sure that the data scrambling has been disabled while compiling the design files which is enabled otherwise by default. To disable scrambling make the following modifications:

1. Goto the directory: <project_directory>/<pcie_de_....>/testbench/<pcie_de..._tb>/simulation/submodules/
2. Open the file named altpcietb_bfm_top_rp.v
3. Scroll down to the place where test_in [2:1] = 0 is declared. Change it to test_in[2] =1 and test_in[1]=0. Making test_in[2] = 1 disables the scrambling at the PIPE interface.
4. Save the file and close.

STEP 3:

Open modelsim and in the transcript window navigate to directory <project_directory>/<pcie_de_....>/testbench/mentor and do the following steps:

1. Source the file named msim_setup.tcl using command source msim_setup.tcl
2. Compile all design files and elaborate the top level design using the commnad ld_debug
3. After the compilation is done successfully add the PIPE interface signals to the wave window (To add the waves right click on <pcie_de_gen1_x1_ast64_inst> on the simulation window and select Add to wave > All items in region)
4. Run the simulation by typing the command run -all in the transcript window.

STEP 4:

Depending on the number of lanes in your design (x1, x4, x8) copy one of the below mentioned files to the directory named <project_directory>/<pcie_de_....>/testbench/mentor.

x1 lane: virtual_x1.do

x4 lane: virtual_x4.do

x8 lane: virtual_x8.do

The above mentioned files can be found in Virtual.zip (http://virtualvcs.zip/ ) file.

STEP 5:

In this step we modify the .do files as per our design. There are only two small modifications needed per design.

1. The path mentioned in these files are the default paths for the given design configurations. If using a different path please provide the correct path on line 1 of each file and then run the above command. Example: set path "/pcie_de_gen1_x8_ast128_tb/pcie_de_gen1_x8_ast128_inst".
2. The next part is to set the delay parameter of the virtual signals txd_d1, txd_d2, rxd_d1, rxd_d2 depending on the period of our simulation clock hip_pipe_sim_pipe_pclk_in. Below are examples on how the delays are calculated per design (x1, x4, x8).

Note: The period of the signal hip_pipe_sim_pipe_pclk_in can be seen from the modelsim waveform. This is the reason we execute run -all command before we actually add the virtual signals.

x1 lane: virtual_x1.do:

In this example we consider the clock period of signal hip_pipe_sim_pipe_pclk_in to be 4 ns.

The delay for txd_d1 is given by 3* (period:hip_pipe_sim_pipe_pclk_in) = 3* 4 ns = 12 ns.

Therefore we set the delay of signal driving txd_d1 to 12 ns : virtual signal -env $path {hip_pipe_txdata0'delayed(12 ns)} txd_d1.

Similarly, the delay for txd_d2 is given by 10* (period:hip_pipe_sim_pipe_pclk_in) = 10* 4 ns = 40 ns.

Therefore we set the delay of signal driving txd_d1 to 12 ns : virtual signal -env $path {hip_pipe_txdata0'delayed(40 ns)} txd_d2.

The delays for rxd_d1 and rxd_d2 are same as txd_d1 and txd_d2 respectively.

x4 lane: virtual_x4.do:

For 4 lane HIP, txd_d1 does not have any delayed parameter hence no change is required.

The delay for txd_d2 is given by 2* (period:hip_pipe_sim_pipe_pclk_in) = 2* 4 ns = 8 ns.

virtual signal -env $path {hip_pipe_txdata2'delayed(8 ns)} txd_d2

The delays for rxd_d1 and rxd_d2 are same as txd_d1 and txd_d2 respectively.

x8 lane: virtual_x8.do:

txd_d1 is given by1* (period:hip_pipe_sim_pipe_pclk_in) = 1* 4 ns = 4 ns.

virtual signal -env $path {hip_pipe_txdata2'delayed(4 ns)} txd_d1.

txd_d2 is given by 1* (period:hip_pipe_sim_pipe_pclk_in) = 1* 4 ns = 4 ns.

virtual signal -env $path {hip_pipe_rxdata6'delayed(4 ns)} txd_d2.

The delays for rxd_d1 and rxd_d2 are same as txd_d1 and txd_d2 respectively.

STEP 6:

Once the changes have been made to the .do file save the file and close. On the transcript window of the Modelsim simulator type the command do <filename>.do to create these waves and add it to your simulation wave window. 

Note: A virtual_xx.do file can only be run once in an active simulation if any modifications are made to the signals in the file the simulation has to be run again before runing the .do file or else the simulator would ingnore the changes.

STEP 7 (Optional):

Group the following signals along with the status flags for efficient interpretation of the data.

hip_pipe_txdata

hip_pipe_txdatak

hip_pipe_rxdata

hip_pipe_rxdatak

After correctly following these steps one can view the PCIe transaction/error message status flags on the Modelsim simulator wave window. Below is the screenshot of a 4 lane memory read TLP.

                                                                                           Figure 1. Memory Read TLP on a 4 lane PCIe HIP- Wave1.jpg 

Synopsys VCS

The PCIe debug status flags for the PCIe PIPE simulation can also be created on the VCS simulator (with limitations). Status flags for 1 lane PCIe HIP is not supported in VCS. Also error flags are not supported.

To create and view TLP transaction status flags for VCS follow the below steps.

STEP1 and STEP2 are same as that mentioned in the Modelsim howto.

STEP 3:

Add the below piece of code to the file pcie_de_gen?_x?_ast??_tb.v (the file can be found at <project directory>/pcie_de.../testbench/pcie_de..._tb/simulation). Add the code inside the module declaration.

initial begin

$dumpfile("dumpfile.vpd");

$dumpvars;

$dumpon;

end

save and close the file.

STEP 4:

Run the simulation using VCS

1. Go to the directory <project directory>/pcie_de.../testbench/synopsys/vcs and run the command sh vcs_setup.sh

2. Once done run the command ./simv

3. Once the simulation is done type the command dve -vpd dumpfile.vpd .This command opens the DVE gui to view the waveforms

4. In the hierarchy window of the DVE gui select pcie_de_gen?_x?_ast??_inst module under pcie_de_gen?_x?_ast??_tb. Right click on it and select Add to wave> New wave view.

STEP 5:

Depending on the number of lanes in the design (x4, x8) copy one of the files (virtual_x4.tcl, virtual_x8.tcl) to the directory <project directory>/pcie_de.../testbench/synopsys/vcs. The files can be found in VirtualVCS.zip file. To set the path for your design open the virtual_x?.tcl file and set the below two options appropriately.

gui_load_child_values {pcie_de_gen?_x?_ast??_tb.pcie_de_gen?_x?_ast??_inst}

set pcie_de_gen?_x?_ast??_inst pcie_de_gen?_x?_ast??_inst

STEP 6:

In the transcript window of the DVE gui (at the very bottom of the DVE gui where dve> is mentioned) type the command source virtual_x?.tcl . This adds the virtual signals to the waveform window.

Follow STEP7 as mentioned in the modelsim howto for better intertpretation.

Below is a screenshot of a 4 lane PCIe HIP Memory Read TLP on VCS simulator.

                                                                                      Figure 2. Memory Read TLP for 4 lane PCIe HIP- Vcs.jpg 

References

https://www.intel.com/content/dam/altera-www/global/en_US/uploads/b/b5/Stratix_V_hard_IP_For_PCIe.pdf

users.ece.cmu.edu/~kbiswas/modelsim/se_man.pdf

Disclaimer

© 2010 Altera Corporation. The material in this wiki page or document is provided AS-IS and is not supported by Altera Corporation. Use the material in this document at your own risk; it might be, for example, objectionable, misleading or inaccurate.