External Memory Interface IP on Agilex.

[Big_Al]

Hi, I am new to Quartus Prime Pro. I have instantiated a external memory interface in a Qsys design for an Agilex part. I have also generated the example design and simulated it. Looking through the associated data sheet,  I can't find any description of how to use it! It has an avalon interface (AMM) plus a few other descrete signals (Lock, reset request etc). It is the use of the avalon interface which is not documented. I can't beleive it is just a memory mapped interface, ie you provide the DDR4 address on the avalon bus and read/write the data in this fashion, but maybe it is. It is impossible to say from the user guide or data sheet. It looks like I am missing some documentation. Having said that I have sat through hours of videos on the EMIF. I have designed Xilinx emifs before and all the information required to drive them was in the data sheet. So it looks line I am missing something, but I can't see what.

So anyone who knows how to drive the emif, please put me out of my mysery.

[Big_Al]

Having looked at the simulation, it appears there are no transaction on the aMM interface from the TG2. It is not clear from the documentation how to stimulate the TG2 to perform any transactions. It does say it should perform 1 transaction after power-up., and that this could be made infinite by chaging a parameter. That did not seem to work either.

Giving up on the simulation, I looked at the interface on Qsys. There appears to be a 576-bit rd/wr bus and a 29-bit address bus. As the DDR4 I chose is 4Gx72, that suggests a single transfer on 576-bits is 8x72, so the 29-address bits would enable me to get to the whole memory. It is not clear, but there is a 7-bit burst count field, so potentially I could send 128x576bits in a single burst (the max allowed by Avalon). I presume the emif interface would wait for all 128 576-bit words to be loaded into it's buffer before it tried to burst them at the DDR.

One thing which is not clear is why there appears to be no byteenables on the emif interface? As I will be writing an Avalon master I can add them (72 of them) and try to connect the interfaces together in Qsys. I suppose it will either throw an error or work.

Again, it strikes me I am missing some important document somewhere which describes how to use this interface.

Answers on a postcard please.

[sstrell]

Have you gone through the user guide?  There's a lot to setting up an EMIF:

https://www.intel.com/content/www/us/en/programmable/support/support-resources/support-centers/external-memory-interfaces-support.html

[Big_Al]

Yes, I have been through all the documentation and videos. The user guide seems to concentrate mainly on the phy side, which is all very interesting but what I really want to know more about from a user point of view is the Avalon interface. I could try to work it out from my own test bench, but I can't help but think it is probably documented somewhere, which would save me a lot of time.

[sstrell]

What aspect of Avalon?  Avalon in general or like the register map of the EMIF IP?  You can learn more about Avalon itself in the spec:

https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/manual/mnl_avalon_spec.pdf

[Big_Al]

I have read the Avalon specs and have used axi before. My specific issue is as I mentioned above on the 5 Nov. The Avalon interface on the emif is 576 bits wide. There are no byte enables and only 29 address bits. So if I want to write to a Specific byte, how can I do this? I would have thought this might be explained somewhere along with other questions like details on when a burst mode starts. Is it after the last bytes of the last 576 byte word is written?

[Big_Al]

Maybe I should be more specific. I have generated an external memory IP module for an Agilex fpga, which interfaces to a 32GB (4Gx72) DIMM. It has the following user Avalon interface (amm):

Avalon Interface:

Name	Role	Width	Direction	Description
amm_ready_0	waitrequest_n	1	Output	Wait-request is asserted when controller is busy
amm_read_0	read	1	Input	Read request signal
amm_write_0	write	1	Input	Write request signal
amm_address_0	address	29	Input	Address for the read/write request
amm_readdata_0	readdata	576	Output	Read data
amm_writedata_0	writedata	576	Input	Write data
amm_burstcount_0	burstcount	7	Input	Number of transfers in each read/write burst
amm_readdatavalid_0	readdatavalid	1	Output	Indicates whether read data is valid

So, on the face of it I there is a 29-bit address bus which allows unique access to 0.5G address space. So, I can access the full 32GB memory in increments of 576-bits (= 8x72).

There appears to be no byte enables, so it would suggest I can only access data in 576-bit resolution. Either that or if I want to modify a byte, it would require a 576-bit read-modify-write operation. This doesn't sound right!

Avalon clearly supports byte-enables, although in this case I would require 72-byte-enables. That doesn't sound right either.

So it looks like I am going wrong somewhere?

Unfortunately, the documentation doesn't discuss this aspect of the emif very much. There are many other aspects to this interface which are conspicuous by their absence from the documentation, which make me think I am missing a document somewhere. I just need a few simply examples of  writing to and reading from the amm.

I have watched the 4 videos on the subject, generated and simulated the example code and read the External Memory Interfaces Intel Agilex™ FPGA IP User Guide.  I also read the Avalon specs.

[Big_Al]

Once I get the Avalon issue out of the way, the next issue will be the ECC. The user guide allows you to 'enable ECC' and there are ECC status bits, so I assumed the generation and detection is built into the emif. I would therefore have thought that the write interface would be x64 not x72 and that the emif would generate the ECC bits in the 9th byte. On the read side, I would have only expected a x64 input and the status flags to indicate any errors. Clearly this is wrong as there is still a x72 interface in both directions. Maybe this is only so you could if that way inclined, make your own algorithm up and implement it. Unfortunately there is nothing in the user guide to clarify what is included and how it works.

Again, if you know, please share this with me.

[AdzimZM_Intel]

Hello Sir,

May I know if you're using Example Design or your custom design?

I can see the signals like below from the Example Design.

• tg_ctrl_amm_0_waitrequest;
• [575:0] tg_ctrl_amm_0_readdata;
• tg_ctrl_amm_0_read;
• [32:0] tg_ctrl_amm_0_address;
• [71:0] tg_ctrl_amm_0_byteenable;
• tg_ctrl_amm_0_readdatavalid;
• tg_ctrl_amm_0_write;
• [575:0] tg_ctrl_amm_0_writedata;
• [6:0] tg_ctrl_amm_0_burstcount;

I think those signals are from the traffic generator that interact with the external memory interface.

Please let me know if you need help on creating the Example Design.

I think that by enabling ECC, your data bit will become 64bit + 8bit ECC.

So there will be a total of 72bit.

Thanks,

Adzim

[Big_Al]

Hi Adzim,

As I mentioned above (11-07-2021) I simulated the example design. The interface signal that I showed were from the EMIF IP I generated in Qsys. My error was that I hadn't enabled the Data Mask option. Apparently this is required to get the byte enables.

Interestingly when I enable Data Masks I can't enable DBI to improve SI (they are mutually exclusive). When I try to create the IP it suggests I should use DBI as it is over 1200MHz clock speed. So that seems to suggest is you run a DDR4 RDIMM at 2666 or 3200 you shouldn't access anything less than 576-bit words at a time. So all byte changes would need to be a read-modify-write. That doesn't sound very efficient.

Next, I was planning on using a MTA36ASF4G72PZ – 32GB external RDIMM as simulated on our Intel development board schematics. This is a dual rank device with 4 DQ pins per DQS group. When I select Data Mask, an error pops up saying 'Data Masks are not supported in x4 modes'. So it looks like for this RDIMM, you cannot access anything less than 576-bits (or 512 if ECC is enabled). At least the questions I had about ECC seem to now be resolved. With ECC enabled, the interface drops down to 512-bits wide, so at least that now makes sense.

What is not clear from the documentation still is the timing of writes to the DDR4. I can write up to 128 512-bit words at a time to the Avalon interface. The question is when do they start to be transferred to the physical memory. Is it only when all 128 words have been written or as soon as each word is written. I suppose I will have to simulate it and find out.

Thanks for your help.

[Big_Al]

Another question arrises regarding access to the EMIF. I have an HPS with Arm in it which wants to load the DDR4 with test data. I then have some custom IP which reads that data out and sends it to a DAC via JESD, while also collecting data from an ADC and writing it to the EMIF.

1) The EMIF only has a single Avalon port (if you ignore the optional configuration port). I presume these 2 interfaces to the EMIF, one from the HPS and one from the Custom IP will need to go via an Avalon bridge to the EMIF. Is this correct?

2) In Qsys, does this happen automatically when I connect them directly together or do I have to add a bridge to the design?

3) Is it possible that HPS can have byte access to the physical DDR while the Custom IP does not. It sounds like that is not possible?

4) Does the bridge convert between the 512-bit wide data buses from the EMIF to the 32/64 bit interface on the HPS. I presume it must do. Can you point me to documentation which explains this please?

5) If say the HPS has a 64-bit interface, then how does writing to the EMIF work? Does the bridge implement some width conversion logic which collects 8 64-bit words before writing? What happens when fewer that 8 words are written?

[AdzimZM_Intel]

Hi Sir,

Thanks for your reply and feedback the issue.

I'm really appreciate it.

The question is when do they start to be transferred to the physical memory. Is it only when all 128 words have been written or as soon as each word is written.

I think it's will transferred immediately because the operation is required the location for it to store the data.

You can simulate it to find on how your data are been processed.

Thanks,

Adzim

[Big_Al]

Re my last post, I have found the video "Custom Component Development Using Avalon® and Arm* AMBA* AXI Interfaces" which has answered most of this.

[Timothy_W_Intel]

updated link: Custom Component using avalon and axi

[nelky]

Hi,

Do you have the sample of how I can read from and write to external SDRAM through Linux driver on Agilex 7?

I need to use the SDRAM as temporary storage and also to use the bridge to access to the peripherals, like UART, I2C, SPI.

I guess there are 2 parts here.

Firstly, we need to setup the connection using Quartus platform Designer.  From here, we can generate the avalon-mm addresses.  

Secondly, we need to write a Linux driver to access to these avalon-mm addresses in order to read and write.  

Am I correct to say that?

But I can't find the sample on this project.  Could someone please help direct me to the proper sample?  

I guess there is a Cyclone V one.  But its architecture isn't the same.  I can't use the Cyclone V one on Agilex 7.

Please help.

Regards