- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We are using Xeon E3-1275v3 and C226 PCH on our board. I am aware that this system does not support I/OAT or NetDMA.
Is there any user programmable DMA controller?
We want to perform DMA transfers from the main memory (source) to PCIe devices (destination).
Note: PCIe devices do not have DMA controller. PCIe device (destination) can be either PCIe BAR in endpoint or multicast BAR residing inside PCIe switch.
I previously posted this question here: https://communities.intel.com/thread/58913 and got suggestion (from Intel) to come here...
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Most PCIe devices have DMA controllers -- processors don't have them. (They have been invented numerous times, but not implemented in high-volume commercial products.)
Do you have a specific PCIe device in mind that you know does not have a DMA controller?
If the PCIe device does not have its own DMA controller, then the fastest way to copy data from system memory to that IO device is to use a processor core. You would need to set up a memory-mapped IO range for the device with the write-combining attribute, then use a processor core (or thread) to read from (cacheable) system memory and write to the MMIO range using streaming stores.
For the Xeon E5-1275 v3, there should be no trouble saturating any outbound PCIe interface using a single thread. HyperThreading may be useful if you need all four cores to be doing other work at the same time -- the other thread on the core that you use to push the data to the IO device will have reduced effective memory bandwidth, but should retain most of its performance capability for computation, and much of its performance capability for memory accesses that hit in cache.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
John D. McCalpin wrote:My comment is that in Xeon family, E3 CPUs do not have DMA controller, but E5-1600/2400/2600/4600 family have multiple channel DMA controller(Intel QuickData technology). I was hoping that Xeon E5-1275 v3 could have something similar...
Most PCIe devices have DMA controllers -- processors don't have them. (They have been invented numerous times, but not implemented in high-volume commercial products.)
John D. McCalpin wrote:We are designing our own boards (both CPU and I/O). In our application, Xeon E5-1275 v3 is crunching data and sending results (samples) to a lot of PCIe devices living as endpoints connected to PCIe switch. You can imagine each PCIe endpoint being simple I/O card with digital-to-analog converters. Now, the real beauty would come from the fact that we could initiate only one DMA transfer from system memory to the multicast BAR residing inside the PCIe switch. Using this method, all PCIe endpoints would receive data from the system memory using single DMA transfer. PCIe switch would detect that destination address in incoming packet is hitting multicast BAR and redistribute all incoming packets to endpoints. In other words, CPU and memory controller would not be aware what is happening with data after PCIe switch, but we cannot use this method, because there is no DMA controller for Xeon E5-1275 v3/ C226.
Do you have a specific PCIe device in mind that you know does not have a DMA controller?
John D. McCalpin wrote:This sounds promising! In our case, we are reading from memory and writing to multicast BAR in PCIe switch. So, we first have to make multicast BAR to appear as MMIO? All HW access is done using Linux and drivers written in C. In order to implement above mentioned method, can we still use C and make our own Linux drivers or we have to use assembly for streaming stores? Can you please provide some link where we can educate ourselves more about this method? Thank you very much for your response. It seems like that there is solution for the problem. Regards, SlavisaIf the PCIe device does not have its own DMA controller, then the fastest way to copy data from system memory to that IO device is to use a processor core. You would need to set up a memory-mapped IO range for the device with the write-combining attribute, then use a processor core (or thread) to read from (cacheable) system memory and write to the MMIO range using streaming stores.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I tried to check if your processor supports Intel QuckData but I must confess that I'm puzzled by the processor name "Xeon E5-1275 v3".. Are you sure that this is not a typo?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thomas Willhalm (Intel) wrote:It is mistyped in the last reply and then spread over by using copy and paste. So, the correct name is Xeon E3-1275 v3. Regards, Slavisa
I tried to check if your processor supports Intel QuckData but I must confess that I'm puzzled by the processor name "Xeon E5-1275 v3".. Are you sure that this is not a typo?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page