mSGDMA: Incorrect operation but the behavior changes when SignalTapII is inserted.

[Altera_Forum]

Hi all, 

 Please help me. I am using two mSGDMA modules to read and write DDR3 on DE5-NET board with quartus ii ver 14.1.1. The mSGDMA is of the modules of Qsys ver 14.1.  The problem that I cannot solve is:  1) msgdma fails for some lengths of bytes, while it works for other lengths. This design is compiled without SignalTap II.  2) the behavior changes when i put signaltap ii to see registers and ports of the mSGDMA modules. Finally, the design works completely with a big signaltap ii.  The design includes:  1. PCIe-module -BAR2 is used to read and write csr, desc, and response of mSGDMAs 2. two ddr3 memories, ddr3_b and ddr3_a. ddr3_b is a master while ddr3_a shares oct, pll, and dll of ddr3_b as a slave. 3. ddr3_b and ddr3_a have 2GB each, and have memory address spaces of 0x0000_0000-0x7FFF_FFFF and 0x8000_0000 to 0xFFFF_FFFF, respectively. 4. msgdma_ur1 is of mm-to-st, and reads ddr3_b and ddr3_a. 5. msgdma_uw1 is of st-to-mm, and writes ddr3_b and ddr3_a. 6. st-src of msgdma_ur1 is connected directly to st-sink of msgdma_uw1.  The reason why I don't use a single mSGDMA of MM-to-MM, I will insert a stream-computing module between the two mSGDMAs.  The current design is for testing mSGDMAs and a control software. 7. The internal ports of ddr3_b and ddr3_a are mm-slave of a 512-bit width @ 200mhz. 8. The mm-master of msgdma_ur1 and msgdma_uw1 is of a 512-bit width @ 150mhz.  They each have configurations of   burst length is 0x20  command FIFO depth 128  descriptor FIFO depth 128 9. I put clk_crossing_bridge between the mSGDMAs and DDR3 memories. 10. The ddr3 memories works correctly when I read and write data to/from a host PC using other SGDMAs.   All the following data transfer tests satisfy 512-bit alignment and burst boundary. When I use the design compiled without SignalTap II, I get:  + 1 KB from 0x00000400 (ddr3_b) to 0x80000000 (ddr3_a) : passed + 4 KB from 0x00000400 (ddr3_b) to 0x80000000 (ddr3_a) : passed + 32 KB from 0x00000400 (ddr3_b) to 0x80000000 (ddr3_a) : passed + 64 KB from 0x00000400 (ddr3_b) to 0x80000000 (ddr3_a) : no data transferred + 128 KB from 0x00000400 (ddr3_b) to 0x80000000 (ddr3_a) : no data transferred + 256 KB from 0x00000400 (ddr3_b) to 0x80000000 (ddr3_a) : no data transferred + 512 KB from 0x00000400 (ddr3_b) to 0x80000000 (ddr3_a) : no data transferred + 1024 KB from 0x00000400 (ddr3_b) to 0x80000000 (ddr3_a) : no data transferred + 2048 KB from 0x00000400 (ddr3_b) to 0x80000000 (ddr3_a) : no data transferred + 32 MB from 0x00000400 (ddr3_b) to 0x80000000 (ddr3_a) : passed + 128 MB from 0x00000400 (ddr3_b) to 0x80000000 (ddr3_a) : passed These results are reproduced for repeated tests with various data lengths and addresses.  The error of "no data transferred" seems to be caused by the destination addresses corruption. For example, when I transfer data to 0x8000_0000, the actual destination can be 0x0000_0000.  When I use the design compiled with SignalTap II, tests for all the data length are passed. So I cannot know what are happen in the mSGDMA modules. (I observed correct descriptors and control values are written via BAR2 of PCIe.)  I doubted timing of the design, however, I made sure that sufficient constrains are given for all clocks including ddr3's 200MHz and mSGDMA's 150MHz  (generated by pll with external 50MHz). time-quest reports all the conditions are met.  So now I don't know what I should do to solve this problem. Any hints and suggestions are helpful.  -- Mickycat

[Altera_Forum]

--- Quote Start ---   8. The mm-master of msgdma_ur1 and msgdma_uw1 is of a 512-bit width @ 150mhz.  They each have configurations of   burst length is 0x20  command FIFO depth 128  descriptor FIFO depth 128   --- Quote End ---    When I disabled the burst transfer, then the mSGDMAs work completely. However, the DDR3 memory throughput gets very low. I need burst transfer.  For max burst count = 2, 8, and 64, the mSGDMA modules have the same failure as that for max burst count = 32.  Mickycat

[Altera_Forum]

Hi, 

 I found that   1) mSGDMA core gets easier to operate correctly when I turn off "burst enable." 2) However, sometimes the mSGDMA with burst disabled can fail to transfer data.   With signal-tap II, I found that this failure is caused by incorrect latch to length_counter and/or  address_counter in Write_master. For example, when descriptor port of Dispatcher receives  0x0001_4000 as a length, 0x0000_4000 is written into the length_counter in Write_master  of a ST-MM mSGDMA. Since MM-ST mSGDMA sends 0x0001_4000 elements of data to the  ST-MM mSGDMA, the former mSGDMA is kept busy. And then the next DMA transfer fails.  "src_write_master_data" (a command from Dispatcher to Write_mater) has a length of  0x0000_4000, instead of 0x0001_4000. So Dispatcher wrongly works. I doubt  the_write_signal_breakout module in the_descriptor_buffers module.  The both MM-ST and ST-MM mSGDMAs are operating at 200MHz of afi_clk of DDR3 Uniphy  memory controller. The DDR3 memories operate correctly, because I can transfer data between  a host PC and the FPGA via PCI-Express. So I doubt something like timing-violation, while  TimeQuest saids no negative slack found for all clocks and conditions. (I rechecked all the timing constraints are given sufficiently.)  Are there anyone who experienced similar phenomena? Your any information is very helpful.  -- Mickycat

[Altera_Forum]

--- Quote Start ---  Hi,  I found that   1) mSGDMA core gets easier to operate correctly when I turn off "burst enable." 2) However, sometimes the mSGDMA with burst disabled can fail to transfer data.   With signal-tap II, I found that this failure is caused by incorrect latch to length_counter and/or  address_counter in Write_master. For example, when descriptor port of Dispatcher receives  0x0001_4000 as a length, 0x0000_4000 is written into the length_counter in Write_master  of a ST-MM mSGDMA. Since MM-ST mSGDMA sends 0x0001_4000 elements of data to the  ST-MM mSGDMA, the former mSGDMA is kept busy. And then the next DMA transfer fails.  "src_write_master_data" (a command from Dispatcher to Write_mater) has a length of  0x0000_4000, instead of 0x0001_4000. So Dispatcher wrongly works. I doubt  the_write_signal_breakout module in the_descriptor_buffers module.    --- Quote End ---    I solved the problem by implementing my own fifo_with_byteeable that does not use alt_syncram with byteenable. So I conclude that incorrect write with byteenable of the altsyncram on Sratx V 5SGXEA7 causes the problem.  If you want to read my fifo_with_byteenables, please let me know.  Mickycat

[Altera_Forum]

It seems to me that I have the same issue. I’m using the mSGDMA core in a simple PCIe -> mSGDMA -> Avalon-ST DC FIFO qsys system (Quartus v14.0.2). The mSGDMA core is configured in MM to Stream mode (parameters are shown in the attachments). The mSGDMA core stalls with the busy flag asserted at random lengths of bytes: 

. . . Transferring 97517568 bytes ... Transferring 98566144 bytes ... Transferring 99614720 bytes ... Transferring 100663296 bytes ... Transferring 101711872 bytes ... Transferring 102760448 bytes ... Transferring 103809024 bytes ... Transferring 104857600 bytes ... Transferring 105906176 bytes ... Transferring 106954752 bytes ... Transferring 108003328 bytes ... Transferring 109051904 bytes ... Transferring 110100480 bytes ... Transferring 111149056 bytes ... Transferring 112197632 bytes ... Transferring 113246208 bytes ... Transferring 114294784 bytes ... Transferring 115343360 bytes ... Transferring 116391936 bytes ... Transferring 117440512 bytes ... <--- indefinitely  I have also tried with the last version of mSGDMA core from Altera Wiki, but the result is the same. Can you plz provide your fifo_with_byteenables?