Processors
Intel® Processors, Tools, and Utilities
14811 Discussions

PCIe switch inside the chipset

vmetodiev
Novice
3,326 Views

Hello Community,

Given that the PCIe controller is integrated inside the CPU, the chipset also provides PCIe lanes and the connection between the CPU and the chipset is DMI, I would like to ask the following:

  1. Where is the PCIe root complex? Is it inside the CPU?
  2. Since the CPU and the chipset  are connected via the DMI link, it makes me think that a lot of chipsets should contain a PCIe switch that provides the additional downstream lanes?
  3. If 2. is correct, can I get some code samples to use the PCIe switch for peer-to-peer communication between endpoints?
  4. Is the DMI link actually PCIe, or something else?

 

 

 

 

0 Kudos
6 Replies
JosueO_Intel
Moderator
3,227 Views

Hello vmetodiev,


Thank you for posting on the Intel communities.


We are sorry for the delay in our response. In order to have a better understanding of the issue, please share with us the following information: 


  1. CPU model. 
  2. Chipset series. 


Regards, 


Josue O.  

Intel Customer Support Technician



0 Kudos
vmetodiev
Novice
3,220 Views

Hi @JosueO_Intel ,

 

First of all, I would like to thank you and the whole Intel team for supporting me and sending me regular updates on my ticket.

 

Now, on the topic... I am interested in the following topology:

Intel.png

I want to use an Alder Lake CPU and the chipset to provide direct peer-to-peer communication between two PCIe endpoints. From most of the chipset diagrams (like the one of Z690 below), it's logical that since the CPU to chipset DMI link is x8 and the chipset offers much more PCIe lanes (12x PCIe 4.0, 16x PCIe 3.0), the chipset should contain a PCIe switch.

Z690_Block_Diagram.png

If my assumption is correct, the PCIe standard states that the PCIe switch should be able to do direct PCIe P2P linkage - for example, by BAR remapping of both endpoints.

 

Could you possibly tell me:

1) Is my assumption correct - that the chipset contains a PCIe switch indeed?

2) How could I figure out which PCIe switch-port does a specific PCIe slot belong to (different motherboard vendors may connect the slot to different lanes and PCIe switch ports respectively, IMO)?

3) If so, can you provide me some documentation for the its registers - so that I can program it and accomplish my PCIe peer-to-peer goal?

 

I know the question is tricky and maybe under NDA,  but I really need this information and I have been waiting for long!

And sometimes such topics could be crucial for start-ups that want to make a great product!

 

Thank you for your understanding!

 

Best regards,

Varban

0 Kudos
JosueO_Intel
Moderator
3,214 Views

Hello vmetodiev,


Thank you for the information provided.  


I will proceed to check your inquiries internally and post back soon with more details. 


Regards, 


Josue O.  

Intel Customer Support Technician



0 Kudos
vmetodiev
Novice
3,206 Views

Hello @JosueO_Intel ,

 

Thank you! Much appreciated!

0 Kudos
JosueO_Intel
Moderator
3,187 Views

Hello vmetodiev,


Hope you are doing great and staying safe, in this case, our recommendation is to contact the motherboard manufacturer to get a more precise response to these questions.


It is worth mentioning that the CPU gives the PCIe support, but all the configuration is on the motherboard. It is important for you to know that released Intel products cannot be modified, however, we will take your feedback into consideration for future CPUs. 


Hope this information is useful, if you need any additional information, please submit a new question as this thread will no longer be monitored. 


Regards, 


Josue O.  

Intel Customer Support Technician



0 Kudos
vmetodiev
Novice
3,182 Views

Hi @JosueO_Intel ,

 

Thank you for your time!

 

I must, however, disagree with your statement. Of course, I hope I do not sound rude or I am trying to offend you/Intel, of course... and below I am trying to explain my point:

 

The motherboard vendor designs the PCIe slots topology according to the lanes provided by the Intel components - in this case, both the CPU and the Chipset contain PCIe "lanes". The lanes get either bifurcated or switched.

 

In many diagrams, the CPU - to - Chipset DMI link seems impossible for bifurcation. Thus, the only logical lane "splitting" to more than one PCIe ports, originating from the chipset, is the PCIe switch option. Please correct me if my I am wrong, but I doubt it...

 

Another interesting topic is the bifurcation configuration and the PCIe switch configuration (e.g. the root complex and the chispet registers). Here, it should be the UEFI developer that has some access to the documentation of every CPU and chipset model/generation. The UEFI developer is not always the motherboard vendor - maybe companies like American Megatrends and Phoenix BIOS are the best example for this.

 

So, it is logical that if I contact the motherboard vendors, they will forward me to the UEFI developer who may explain me that the Intel documentation is under NDA...

 

Because of the upper-mentioned, I have the following request now from your side:

1) Do you have some program for UEFI developers (NDAs, application process for it)? 

2) Can you also inform  the oneAPI development team that such information could be interesting for a lot of "instructure"-level developers, mostly in the field of Ethernet and high-speed networking. Just take the exampe with two (Smart)NICs that need to pass packets to one another directly. In the case of 10G, 40G and 100G it will be rediculous to travel to the root-complex and make a virtual "patch" inside the main memory...

 

 

 

 

 

 

 

 

0 Kudos
Reply