hi, i would like to implement a PCIe to PCIe bridge (ArriaV GX) between 2 Hosts (both Root Complex).

[OLevy1]

i am thinking of using 2 PCIe Cores, each will contain the same characteristics (device id, BARs...) of one of the HOSTs. basically the goal is to transfer the TLPs between one Host to another. i would like to know what is the best way and if there is any example design that might help me.

thanks.

[Abe]

The PCIe subsystem does not have a Bridge unlike its predecessors the PCI bus. PCIe devices are of 3 types only - Root Complex (RC), Switch (SW) and EndPoint( EP). The RC is the only initiator in this subsystem , while the Switch behaves like a routing mechanism that routes TLPs to devices (EP) on the downlink. EndPoints are typically implemented in devices such as Ethernet, SoundCards, etc..

In the PCIe subsystem there is no need for a Bridge as the PCIe subsystem can be expanded to support more devices (EPs) using a Switch.

For point to point communication, you can use one PCIe RP on one FPGA board, and a PCIe EP on another FPGA .

Two host PCIe RPs will not be able to talk to each other. A PCIe RP has to connect to a PCIe EP or SW downlink and uplink.

[OLevy1]

thank you for your answer.

i will describe the system i have today: i have a Root Complex Host and a dedicated EP Processor and they are connected to each other with the PCIe interface. the problem i have is that each time i restart the processor i have to shut down and power up the Host to initiate enumeration process. in order to avoid it i wish to change the processor to be Root Complex and add FPGA with 2 PCIe EP Cores to transfer the TLPs between them. this way the Host and the Processor, each will interface one PCIe EP Core and i will not have enumeration problems. the problem i am facing is to find a solution in which the Host and the Processor will not have to change their functionalities (accept changing the Processor to be Root Complex instead of EP) and to transfer the information between them in "Transparent" way. Do you think there is a way to receive TLP from the Host through its corresponding interfacing FPGA PCIe EP Core, manipulate the TLP header in some way, and forward it to the other FPGA PCIe EP Core to be transmitted to the Processor?

thanks,

Oren

[Abe]

When you restart the processor are you passing the same reset signal to the EP as well? If you do not reset the EP when the processor is being reset, the EP will not enumerate correctly.

To change the PCIe IP on the processor from EP to RC, you have to find out the following:

1. The current connection between processor and EP - Is this connection via the processors BUS interface or via PCIe or another specific interface?
2. Will the processor support interfacing to a PCIe RC, ie, does the design support it.
3. Is the processor and EP sitting in an FPGA?

 You can take a look at the PCIe reference designs here:

https://www.intel.com/content/www/us/en/programmable/support/support-resources/intellectual-property/interface-protocols/pci-express/m-pci-express-refdesigns.html

[OLevy1]

hi,

sorry for the misunderstanding.

the processor is not part of the FPGA. it is currently EP, located on another PCB outside the Host. It is connected to the Host with a PCIe Cable (to the PCIe card located in the Host).

i wish to change the Processor to be Root Complex and to switch the Host PCIe Card to the Card containing the FPGA with the 2 EP PCIe Cores. the Host will be interfacing one of the FPGA EP PCIe Cores and the Processor (which is in a different PCB outside the Host) will be connected with the PCIe Cable to the other FPGA EP PCIe Core.

this way the Processor and the Host, both as Root Complex, will interface, each one of them, one FPGA EP PCIe Core.

Oren

[OLevy1]

to make it simple, instead of the Processor as Root Complex you can refer as if it is another Host with PCIe Card.

so there are 2 Hosts, both Root Complex, one with Pcie Card and one Host with the Card containing the FPGA with 2 EP PCIe Cores in one of it Motherboard PCIe Slots.

Oren

[Abe]

Let me get this right..

1. The Processor and EP sit on a separate PCB outside. Processor is connected to EP via PCIe link internally
2. The Host system has a PCIe RP card in one of the PCIe slots.
3. The EP and RP are connected via PCIe cable.

You want to swap out the PCIe EP on the processor side PCB to a RP and also swap the RP PCIe card in the Host system to one with an FPGA which has 2 EPs in it.

[OLevy1]

attached.

i hope it will make the problem clearer.

as you can see in the 'Current System' drawing, the problem is that each time the Processor is rebooting, the Host must be rebooted as well so it will initiate Enumeration process and map the Processor EP PCIe to its devices.

Oren