mSGDMA DDR3 errors in data saved after X bytes

[Altera_Forum]

I have Qsys system consisting of NIOS (+ associated peripherals), mSGDMA dispatcher, mSGDMA write master and custom block with avalon stream output. 

 Dispatcher settings:  

• Streaming to Memory Mapped 
• Memory mapped response port 
• Fifo depth: 1024 

 Write master settings  

• Data width: 32 
• Fifo depth: 4096 
• Length Width: 26 
• Burst disabled 

 Custom block properties  

• 32 bit output of 0 - 4095 counter 
• 8 bit symbol 

 I am setting up my descriptors using the following code:   for(i = 0; i < NUMBER_OF_BUFFERS; i++){ // Hold until there is space to add another descriptor. while ((RD_CSR_STATUS(STREAM_DISPATCHER_CSR_BASE) & CSR_DESCRIPTOR_BUFFER_FULL_MASK) != 0) {} // Set up descriptor control bits on last buffer to interupt when finished. controlBits = (i == (NUMBER_OF_BUFFERS-1))? DESCRIPTOR_CONTROL_TRANSFER_COMPLETE_IRQ_MASK : DESCRIPTOR_CONTROL_EARLY_DONE_ENABLE_MASK ; construct_standard_st_to_mm_descriptor(pStdDescriptor,(alt_u32 *)writeAddress, BUFFER_SIZE, controlBits); write_standard_descriptor (STREAM_DISPATCHER_CSR_BASE, STREAM_DISPATCHER_DESCRIPTOR_SLAVE_BASE, pStdDescriptor); }   BUFFER_SIZE = 16kb NUMBER_OF_BUFFERS = 4  My intention is to move 16kb blocks of data into 4 separate buffers in DDR3 memory. At the end of the 4th buffer, I interrupt and reset the descriptor chain. The delay between the end of the chain and the start of the new one is a problem for me as it means losing data. But that is not my current concern. The problem that I am having at the minute is that there are a lot of errors in the data saved to the DDR3. The data looks perfect up until around the same point every time (~5100 bytes/ 1275 words). This is well inside the first descriptor transfer.  The second descriptor in the chain again starts error free for similar length before errors become common. This 2nd descriptor also seems to have a delay in starting transfer after the end of the first descriptor as instead of saving 0 (which the counter rolls over to) it saves 0x693 which is undesirable for my application.   On the 2nd descriptor chain (after restarting in the interrupt service routine), the errors are similar though some occur earlier. I've attached an image of the diff between the memory contents and what they should be. You can see how the number of errors gets progressively worse.    https://www.alteraforum.com/forum/attachment.php?attachmentid=8573   Has anyone any idea why I am getting these errors? I'm assuming it's something to do with the length of the FIFOs? At the moment I am just setting valid <= '1' on the stream and I am not doing anything with the ready signal (Though signal tapping shows it always to be high).  Any help or ideas appreciated   

[Altera_Forum]

Data cache issues?

[Altera_Forum]

Thanks for your reply, I have to admit I don't know what the issue could be with the Data cache but ill do a bit of research into it and see what I find.  

 Thanks for the point in a direction.

[Altera_Forum]

I've been looking into possible issues with the data cache but I haven't been able to find a solution. I tried increasing the data cache to 64kb from 4kb (I'm using a NIOS II/f core) but it didnt have any affect. I also wondered if I needed to flush the data cache after writing a descriptor using alt_dcache_flush_all(), but again to no affect.  

 It might be worth mentioning that the errors in the data are different every time I run but they do occur at very similar (but not equal) addresses. Invariably the first 4kb of data (1024 words) are correct. So I wonder if my issue is to do with the write master FIFO which is 4096 bytes deep.   Though the first error does tend to be 1275 words in so that is confusing me. Again, any further advice appreciated.    edit:spelling

[Altera_Forum]

The cpu write to the dma controller registers need to bypass the data cache. 

You need to make sure that any data you want the dma controller to read has been flushed from the data cache (to ensure the dma reads the correct data). You need to invalidate the data cache for the dma target buffer before the dma starts (to ensure the cpu doesn't write back dirty cache lines).

[Altera_Forum]

Thanks for another response! 

 My experience with DMA programming is fairly basic so forgive me if I am slow to understand.   The only write to the mSGDMA controller I perform is the write_standard_descriptor call. Following this method through to WR_DESCRIPTOR_WRITE_ADDRESS etc I can see it uses IOWR_32DIRECT to write to the mSGDMA so it's bypassing the cache. I think this applies to any reads by the mSGDMA controller too.   As for invalidating the cache before the DMA starts I flush the cache (alt_dcache_flush_all) before calling start_dispatcher(STREAM_DISPATCHER_CSR_BASE) so I think this accomplishes this.  To investigate whether it's a cache problem further I did switch over to a NIOS II/e so that I dont have a cache and my problem remains the same. Does that confirm the problem is not with the cache as I hope?

[Altera_Forum]

Just to add some more information to aid any opinions, As long as the descriptor chain is set up to transfer 4096 bytes or less, then no matter how many times I repeat that chain I get correct data.  

 So I keep coming back to the FIFO depth in the write master. Any opinions on whether this could be my issue?  Thanks  edit:  I've just realised that the offending addresses of DDR3 memory cannot be changed accurately from Eclipse memory debug window either. So it seems my problem may be more with the DDR3 than the mSGDMA.   When I write to the same DDR3 addresses from a loop in the NIOS I get almost the exact same pattern of incorrect data. The screencap attached shows the difference between what should just be FFFFFFFF written to a block of memory compared to what is actually found in the DDR3.  https://www.alteraforum.com/forum/attachment.php?attachmentid=8582

[Altera_Forum]

I've found my problem. Just wanted to put here to close off thread.  

 The timing values for my memory set by using the altera template did not follow the data sheet. I had also reduced the number of columns in the ddr3 controller which I think was a mistake. After adding these columns back in and fixing the timing, this problem went away.   My issue now is that once the write master Fifo has been filled, I start to get missed data. i.e. the first 4096 words of my streaming counter are saved correctly, but after that the values saved by the dma from my streaming counter block seem to jump 13 words each time. 49C -> 4a9 -> 485 etc. But perhaps that problem is for another thread.