Intel QuickData DMA Engine Data Transfers via PCIe NTB Port and Request for Detailed Documentation

[mpcukur]

We are experiencing issues with data transfers between two Intel XeonD-1500 processors using the Intel QuickData DMA engine in a system running VxWorks. Although the DMA engine reports successful completion of transfers, the data does not appear at the destination processor as expected. Additionally, we are seeking more detailed documentation on the DMA engine configuration.

Detailed Issue Description:

1. System Configuration:

   • Processors: Two Intel XeonD-1500 Series SoC
   • OS: VxWorks 7.0 SR640
   • PCIe Configuration: Data transfers are executed over the PCIe bus, facilitated through a PCIe switch using a Non-Transparent Bridge (NTB) port.

2. Observations:

   • Transfers between local memory regions on the same CPU using the DMA engine are successful.
   • Transfers from a CPU to an external endpoint (e.g., FPGA) over TB ports are successful.
   • Direct CPU writes (bypassing the DMA engine) to the destination CPU via the PCIe switch and NTB port are successful.

3. Problematic Scenario:

   • When using the Intel QuickData DMA engine to transfer data from one CPU to another over the NTB port, the DMA engine reports that it has successfully processed the descriptor list and completed the transfer. However, the expected data is not present at the destination CPU.

4. Specific Issue with BAR Configuration:

   • Our PCIe switch setup includes multiple Base Address Registers (BARs) used as message windows. Writing directly to BAR2 (using CPU instructions) successfully initiates and completes the data transfer to the second CPU. However, attempts to perform this operation via the DMA engine do not result in visible data at the destination, despite successful completion reports from the DMA engine.

5. Documentation Concern:

   • The available documentation for setting up and configuring the Intel QuickData DMA engine lacks specific details on the process sequence and register settings necessary for proper data transfer setup. This includes vague guidelines on the descriptor list format and a lack of clear instructions for the sequence of register writings for specific operational purposes using the DMA engine.

Request: We seek your guidance on resolving these discrepancies in DMA engine behavior, particularly concerning the NTB port and BAR configurations for cross-CPU data transfers. Additionally, we request more detailed and explicit documentation or guidance that clearly explains the required processes and register configurations for the Intel QuickData DMA engine, to better understand and utilize this technology in our systems.

[NormanS_Intel]

Hello mpcukur,

It's great to see your post in our community. For more specialized assistance, we're moving your query to the Server Products Forum. This way, you can expect a quicker and more focused reply.

Best regards,

Norman S.

Intel Customer Support Engineer

[Sreelakshmi1]

Hi mpcukur,

Good day!

Thank you for posting in Intel community.

As per our validation, your query is related to Intel XeonD-1500 processor.

We recommend posting your query in the Embedded Community Forum to receive a quicker response regarding the issue.

https://community.intel.com/t5/Embedded-Products/ct-p/embedded-products

Thank you for understanding.

Regards,

Sreelakshmi