Community
cancel
Showing results for 
Search instead for 
Did you mean: 
RamaMohan
Novice
3,597 Views

Issue with PCIe HIP(A10) Gen3 model behavior in simulations with serial interface

Hi,

 

We are facing an issue in simualtions, with the data transmitted from PCIe hard-IP model during initialization while switching from Gen1 to Gen3 Speed. The model being used is the one for A10 FPGA device. Here’s the issue:

After moving Recovery.Speed state and sending TS ordered set, HIP is sending unknown symbols on Tx with HiZ on the lines.

The said behavior is seen repeatedly for a long time before HIP Tx enters electrical Idle.

The simulation has enabled fast_sims in serial mode.

 

Appreciate help/suggestions

 

Thanks,

RamaMohan

 

 

 

Tags (1)
22 Replies
Nathan_R_Intel
Employee
193 Views

Hie RamaMohan,

 

fast_sim does help to speed up the simulation related to analog functions to be shorter such as PLL locking,CDR locking and analogue calibration. Hence, its good you have enabled it.

However, you seem to be observing HiZ. This is typically caused by some of the IP's input ports floating upon starting simulation. I can't be certain this is the root cause, but firstly, please check if all you clocks are driven and resets have an initial value upon staring simulation.

 

Secondly, please check the FPGA is connected to the other end (if FPGA is endpoint, it is connected to a Root Port). This way the TX lines will observe a Rx and won't be showing HiZ for a long time.

 

Also do advice, which Arria 10 PCIe interface, configuration is being used (ex: AVMM Gen3x8 128bit)  and which simulator is showing this behavior.

 

Regards,

Nathan

 

RamaMohan
Novice
193 Views

Hi Nathan,

Thanks for the reply.  The HiZ being observed is at a point LTSSM on both (RC and EP) sides had proceeded to recovery speed state, which implies all the connectivity on serial interface side and input side is fine. We have physically verified the same in waveform as well.

Please refer to the attached snapshot of waveform showing A10 PCIe HIP signals which indicate this condition. The LTSSM state is also captured in the waveform.

We are using Arria10 PCIe Gen3, x8, 256bit is the configuration being used.

 

Thanks,

RamaMohan

RamaMohan
Novice
193 Views

​Hi Nathan, 

Here is another snapshot that shows more simulation window where issue had occurred.

 

Thanks,

RamaMohan

Nathan_R_Intel
Employee
193 Views

Hie,

 

Could you advice if you are not able to achieve Lo at Gen3 after running simulation for longer period due to this issue?

 

i want to understand the LTSSM if its stuck at Recovery.Speed

 

Regards,

Nathan​

RamaMohan
Novice
193 Views

Hi Nathan,

Yes, RC model LTSSM is not able to proceed ​to L0 at Gen3 and Endpoint monitor goes out of sync due to this issue. The RC model is stuck in phase2 equalization after this.

But at Gen1 we are able to proceed to L0 state.

 

Thanks,

RamaMohan

Nathan_R_Intel
Employee
193 Views

​Hie,

 

I will to duplicate the issue in-house to debug further. Could you provide your simulation files (zipped). In the meanwhile, can you the following:

 

  1. disable fast_sim
  2. ensure following are disabled in your Arria 10 Hard IP for PCIe under Phy Characteristics tab:
  • Enable Rx polarity inversion soft logic
  • enable soft DFE controller IP

If not, please disable it and retry simulation.

Also change the Requested equalization Far end Tx Preset to Preset 0 if you are recompiling.

 

Regards,

Nathan

 

 

RamaMohan
Novice
193 Views

Hi Nathan,

Sure I'll check these parameters and try out with values proposed.

When you say zipped simulation files​, do you mean HIP files?

Also, lease let me know how can I share these with you?

 

Thanks,

RamaMohan

Nathan_R_Intel
Employee
193 Views

Hie,

 

I have another question which PCIe interface are you using. Are you using AVMM_SRIOV?

 

I mean your .tcl and library files (PCIe IP) that you are currently using to run simulation. There is an attach file icon in the forum, you could try using that.

 

Regards,

Nathan

RamaMohan
Novice
193 Views

​Is there an option to configure AVMM with SRIOV enabled? My interactions with Intel confirmed that this configuration wasn't available.  So, is it fine if I attach just the .ip file?

 

Thanks,

RamaMohan

Nathan_R_Intel
Employee
193 Views

Hie,

 

Sorry, my mistake. I accidently typed AVMM_SRIOV instead of AVST-SRIOV. You are correct, we don't offer AVMM_SRIOV interface.

 

I needed to know which PCIe interface are you using currently, If you are using AVST-SRIO, then observed simulation is expected as the Intel-BFM don't support Gen3 Phase 2 and Phase 3 equalization.

 

Please advice accordingly.

 

Regards,

Nathan

RamaMohan
Novice
193 Views

Hi Nathan,

We have both the configurations with SRIOV enabled and without SRIOV enabled as we need 2 endpoints. I've attached the IP config files of the both the EPs.

 

We are seeing the same issue for both the endpoints. We are using the sriov sim files except for all the unique modules specific to non-sriov config.

 

But, I'm wondering how SRIOV would impact the LTSSM behavior in your BFM.

 

Thanks,

RamaMohan

Nathan_R_Intel
Employee
193 Views

Hie RamaMohan,

 

Well as long as you use the AVST-SRIOV Interface (regardless if SR-IOV is enabled), the Intel-BFM don't support Gen3 Phase 2 and Phase 3 equalization. This is covered in our user guide (section 10.5.4).

 

https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_a10_pcie_sriov.pdf

 

For the SRIOV interface, the root port BFM (used for simulation) does not support Phase 2 and Phase 3 of Gen3 equalization. Hence, to achieve Gen3, the LTSSM will move to Recovery.Speed after speed change is requested. At this point, the root port BFM in simulation does not perform equalization and return RcvrLock. Hence, LTSSM cannot move beyond this.

 

Hence using Intel BFM as your root port in simulation will cause this behavior. i just checked this issue even exist in AVMM-DMA interface.

Hence, to workaround this issue, you will need to use a third party BFM as root port.

 

Regards,

Nathan

 

 

RamaMohan
Novice
193 Views

Hi Nathan,

Sorry for the confusion. We are not using Intel BFMs in the bench. We are only using Intel PCIe HIP DUT ​ and all the bench are not from intel. We di go through the section referred by you in userguide regarding equalization and we are in sync with you.

 

  1. We came across this equalization issue where RC model is initiating Phase2, 3 equalization but DUT(which is Intel PCIe HIP) is stuck at phase3. And when we skipped the phase3 equalization step LTSSM went to L0 state. But, note we did all this EP monitor disabled. But for successful operation we need the Monitor as well in bench.
  2. The reason for EP monitor not working is HiZ being sent by DUT Tx which is a spec violation

 

We did the changes suggested by you to the IP and ran simulations. Still we see the same issue.

 

Were you reproduce the issue at your end with the IP files provided by me?

 

Thanks,

RamaMohan

Nathan_R_Intel
Employee
193 Views

Hie RamaMohan,

 

The files you have attached only contains the pcie_ep_x8.ip file. This will take some effort on my side to regenerate all the simulation files, find a root port BFM from a third party, create a top level and simulation script that can show the failure. Could you help provide zip all the EP library files and the root port library files so, I could reproduce the simulation issue on my sides. If you are unable to due to proprietary reasons, then you will need to allow me sometime to reproduce on my side.

 

Regards,

Nathan

 

 

RamaMohan
Novice
193 Views

Hi Nathan,

I don't think I will be able to share Root Port BFM now. I think generating EP library from .ip file​ is simple. Isn't it. We are using 17.1 version of Quartus Prime and target device is 10AX115N3F40E2SG.

Let me know if you still want to send you EP library files.

Thanks,

RamaMohan

Nathan_R_Intel
Employee
193 Views

Hie RamaMohan,

 

Yes it is not difficult to ​generate the End Point library files from .ip files; but wanted to check the exact library files. Even similar Quartus versions with patch or updates could have different library files. 

Anyway, I will need to update my available BFM to be Gen3 capable and attempt to replicate this issue on myside.

I will be out from 7th to 13th December. Hence, please expect my next response only after 14th December 2018.

 

I will need the following information from you.

i. Have you installed any patch or update to Quartus Prime 17.1

ii. Which tool are you using for simulation

 

Regards,

Nathan

 

 

 

 

RamaMohan
Novice
193 Views

Hi Nathan, We are currently using below version of Quartus. Simulation tool being used is vcs (Version O-2018.09) Thanks, RamaMohan
RamaMohan
Novice
193 Views

Here's the quartus version: 17.1.0 Build 240 10/25/2017 SJ Pro Edition

 

Thanks,

RamaMohan

RamaMohan
Novice
193 Views

Hi Nathan,

Can you please update on this issue. Our PCIe Gen3 progress is currently blocked because of this issue.

 

Thanks,

RamaMohan

Nathan_R_Intel
Employee
141 Views

​Hie RamaMohan,

 

I was unable to duplicate the issue yet on myside using Modelsim. I will be trying with VCS and updating you by tomorrow.

 

Regards,

Nathan

Reply