Simple DMA Avalon-to-PCIe Address Translation help needed

[Altera_Forum]

See attached SOPC recipe that I am using. 

 I am using Simple DMA, and I am programming the registers directly to initiate a DMA transaction inside a Windows Driver.  My Windows Driver is using the Windows Driver Foundation framework, it's running on Windows XP 32bit.  I can successfully do a DMA transfer from device onboard SDRAM to device onboard SDRAM without a problem.  But I am now trying to use the DMA engine to transfer data on the PC host to the devices onboard SDRAM i.e. read from PCIe and write to Avalon. The transaction succeeds but the data put into the destination is not the same as the source address.  In the PCIe Compiler I am using Requester/Completer and Dynamic Translation Table, and 2 address pages, which have a size of 4KBytes - 12 bits.  I am only programming the first entry in the address translation table.  I have the TX_Interface at a base address of 0x0.  I'm guessing this is meant to go as the high bits in an Avalon-MM address which is going to be translated to a PCI Address (see section 4-26 of PCI Express Compiler) i.e. "Slave Base Address"...to select which slave the address is for.  Avalon-to-PCIe Address Translation - I interpret this as meaning when I give an address to the device which is in the range 0x0 to 0xFFF (i.e. 4Kbytes), this will cause it to refer to the translation table to obtain the "higher" bits to formulate the full PCIe address, which the DMA Read Master then accesses through the TX_Interface.  Thus I take my PCIe 32bit physical address:  * I mask off the lower 12 bits, and use those 12bits as the ReadMasterAddress.  I am guessing I don't need to supply any higher bits, because the Tx_Interface is 0x0, and the entry I want to use for translation is 0? Is this correct ?  * I use bitfields to refer to the A2P_ADDR_SPACE and A2P_ADDR_MAP_LO bits in the translation entry. But effectively I am masking off the lower 2 bits of the 32bit PCIe address, and ORing that with 2 bits specifying address space - to define the low entry.  The lower 12 bits of the PCIe address are insignificant and masked off by the hardware - because I specified a 4KByte (12 bit) page size.  I've used a common buffer (which is in non-paged memory) to hold the source data to read, so that I can rule out any paging out issues.  Burst Transfers are TURNED OFF in the SOPC (so the corruption issue due to wrongly aligned addresses shouldn't come into play).   Allowed Transation is ticked on all of them: byte, halfword, etc - I am trying to do a byte transfer with DMA.  Structures and offset: ---------------------- # define OFFSET_BAR0_LED 0x2000# define OFFSET_BAR0_DMA0_CONTROLLER 0x2080# define OFFSET_BAR0_ONCHIPMEMORY_2_1 0x4000# define OFFSET_BAR0_TX_INTERFACE 0x0000# define OFFSET_BAR0_PCIE_CONTROL_REGISTER_ACCESS 0xC000 # define OFFSET_DDR_SDRAM 0x2000000  // Because the PCIe addresses are aligned to 64bit rather than // having to add padding entries into all the structures, // let the compiler do the work. The top 32bits are insignificant.  typedef _int64 PACKED32BIT;  // Address Translation Table (0x1000 to 0x1FFF) # define PCIE_CRA_ADDRESS_TRANSLATION_TABLE_OFFSET 0x1000  typedef struct _LOW_ENTRY {  PACKED32BIT A2P_ADDR_SPACE : 2 ; // bit 0-1  PACKED32BIT A2P_ADDR_MAP_LO : 30 ; // bit 2-32  } LOW_ENTRY;  typedef struct _PCIE_CRA_TRANSLATION_TABLE_ENTRY {  union  {  LOW_ENTRY bits;  PACKED32BIT ulong;  } LOW;   PACKED32BIT A2P_ADDR_MAP_HI; // bit 0-31  } PCIE_CRA_TRANSLATION_TABLE_ENTRY;  typedef struct _PCIE_CRA_TRANSLATION_TABLE {  PCIE_CRA_TRANSLATION_TABLE_ENTRY entry1;  PCIE_CRA_TRANSLATION_TABLE_ENTRY entry2;   } PCIE_CRA_ADDRESS_TRANSLATION_TABLE, * PCIE_CRA_ADDRESS_TRANSLATION_TABLE;  When the BAR0 resources are mapped in I do this: ------------------------------------------------  (BAR0VirtualMapped and PCIeCRA are UCHAR pointers)  DevExt->PCIeCRA = (PUCHAR)(DevExt->BAR0VirtualMapped + OFFSET_BAR0_PCIE_CONTROL_REGISTER_ACCESS);  DevExt->PCIeCRAInterruptStatus = (PPCIE_CRA_INTERRUPT_STATUS)(DevExt->PCIeCRA + (1 * PCIE_CRA_INTERRUPT_STATUS_OFFSET));  DevExt->PCIeCRAInterruptEnable = (PPCIE_CRA_INTERRUPT_ENABLE)(DevExt->PCIeCRA + (1 * PCIE_CRA_INTERRUPT_ENABLE_OFFSET));  DevExt->PCIeCRAAddressTranslation = (PPCIE_CRA_ADDRESS_TRANSLATION_TABLE)(DevExt->PCIeCRA + (1 * PCIE_CRA_ADDRESS_TRANSLATION_TABLE_OFFSET));  In the drivers EvtProgramDmaWrite callback I am doing this: -----------------------------------------------------------  ULONG addressread = devExt->WriteCommonBufferBaseLA.LowPart;  //ULONG addressread = SgList->Elements[0].Address.LowPart; // when referring to incoming WriteFile data buffer  BYTE patternbyte = 0xFE;  if (devExt->WriteCommonBufferBaseVA > 0) {  *(devExt->WriteCommonBufferBaseVA) = patternbyte; // set the pattern in common buffer }  //device i/o address ULONG addresswrite = OFFSET_DDR_SDRAM;  PCIE_CRA_TRANSLATION_TABLE_ENTRY e1;  e1.LOW.bits.A2P_ADDR_SPACE = 0; // use 32bit addressing e1.LOW.bits.A2P_ADDR_MAP_LO = addressread >> 2; // there's only room for 30 bits in the field, other 2 are for the address space. e1.A2P_ADDR_MAP_HI = 0;   WRITE_REGISTER_ULONG((PULONG) &devExt->PCIeCRAAddressTranslation->entry1.LOW.ulong, e1.LOW.ulong); WRITE_REGISTER_ULONG((PULONG) &devExt->PCIeCRAAddressTranslation->entry1.A2P_ADDR_MAP_HI, e1.A2P_ADDR_MAP_HI);  addressread = (addressread & 0xFFF); // only keep least significant 12 bits  int length = 1;  PULONG preadmaster = (PULONG)&devExt->DMA0ControlPortSlave->ReadMasterStartAddress; PULONG pwritemaster = (PULONG)&devExt->DMA0ControlPortSlave->WriteMasterStartAddress; PULONG ptransactionlength = (PULONG)&devExt->DMA0ControlPortSlave->TransactionLength;  WRITE_REGISTER_ULONG((PULONG) &devExt->DMA0ControlPortSlave->ReadMasterStartAddress, addressread ); WRITE_REGISTER_ULONG((PULONG) &devExt->DMA0ControlPortSlave->WriteMasterStartAddress, addresswrite); WRITE_REGISTER_ULONG((PULONG) &devExt->DMA0ControlPortSlave->TransactionLength, length );  // Clear the status bits, so that old bits don't report wrong status WRITE_REGISTER_ULONG((PULONG) &devExt->DMA0ControlPortSlave->Status, 0 );  // Get the current DMA Control Register  union {  DMA_CPS_CONTROL bits;  ULONG ulong; } dmaControl1;  union {  DMA_CPS_STATUS bits;  ULONG ulong; } dmaStatus;  dmaControl1.ulong = READ_REGISTER_ULONG( (PULONG) &devExt->DMA0ControlPortSlave->Control ); TraceEvents(TRACE_LEVEL_INFORMATION, DBG_GENERAL, "Read DMA Control Value 0x%X", dmaControl1.ulong);  dmaStatus.ulong = READ_REGISTER_ULONG( (PULONG) &devExt->DMA0ControlPortSlave->Status ); TraceEvents(TRACE_LEVEL_INFORMATION, DBG_GENERAL, "Read DMA Status Value 0x%X", dmaStatus.ulong);  dmaControl1.bits.TXBytes = 1; dmaControl1.bits.TXHalfWord = 0; dmaControl1.bits.TXWord = 0; dmaControl1.bits.EnableTransaction = 1; dmaControl1.bits.EnableInterrupt = 1; dmaControl1.bits.EndReadTransation = 0; dmaControl1.bits.EndWriteTransaction = 0; dmaControl1.bits.EndWhenLengthIsZero = 1; dmaControl1.bits.ReadConstantAddress = 0; dmaControl1.bits.WriteConstantAddress = 0; dmaControl1.bits.TXDoubleWordation = 0; dmaControl1.bits.TXQuadWord = 0;  // Start the DMA transaction  TraceEvents(TRACE_LEVEL_INFORMATION, DBG_GENERAL, "Writing DMA Control Value 0x%X", dmaControl1.ulong); WRITE_REGISTER_ULONG( (PULONG) &devExt->DMA0ControlPortSlave->Control, dmaControl1.ulong );  -------------------------------------------------------------------------------

[Altera_Forum]

I've figured out what I was doing wrong. 

 Inside the Interrupt event handler (EvtInterruptIsr) I was looking at the memory after checking that the Done bit in the Status register was set, which seemed reasonable.  However, due to WDF using bounce buffers (to ensure a continuous buffer), I should have checked the memory after a WdfDmaTransactionRelease, as that's when the transfer buffers are copied to the destination memory.