Data send thru ethernet

[Altera_Forum]

Hello, 

I am trying to do a simple task: I will form an UDP packet in hardware, then I want to move it to Nios, add IP header and send to ethernet.  I will split my problem into two parts:  I've created avalon streaming source component, so I am able to move the packet to the Nios memory using SGDMA stream to memory, however I am not sure how can I move the packet to the TSE core? TSE also use SGDMA (st-to-mem and mem-to-st), so I can't connect my component to one of those SGDMA inputs, since it is already in use. What are my options? Add another SGDMA Stream-to-memory core and connect to the same memory that TSE SGDMA use? How can I pick the data from the memory using another SGDMA there?  Lets say I already have a packet data in Nios, how can I add IP header to the packet using InterNiche? Maybe there are some examples of packet generation using InterNiche? Also I can't find NicheStack documentation anywhere, maybe someone could point me out? I suppose simple packet sending or receiving tasks can be used without RTOS?  I have chosen SGDMA because the data stream can reach 50mbps or more, so the Nios should be really fast to add a header to the whole packet, which is almost 1400 bytes.  Thank You.

[Altera_Forum]

No I don't think you'll find any documentation about that. If you really want to do it that way you'll have to look into the interniche code to see how a packet is sent, and find where to insert your data.

[Altera_Forum]

As support of Daixiwen's suggestion, we have a working design that streams UDP data directly to the TSE, in 1000Base-T mode, while a Nios handles all our higher level communication (ARP, ICMP, TCP, etc.). One particularly challenging part was sharing the Avalon-ST of the TSE between the Nios and our hardware UDP stream. One needs to understand the ST protocol very well to get this to work reliably; SignalTap is your friend for this. You'll certainly need a state machine of sorts to build up your IP and UDP headers from your data stream, so a solid understanding of these protocols is important too. We burst 800MBits/s on this UDP channel, so it can be done.

[Altera_Forum]

Well I've successfully done the IP and UDP header forming, checksums are ok. Did You use multiplexer before TSE? Maybe You are also working with video streams?

[Altera_Forum]

--- Quote Start ---  Well I've successfully done the IP and UDP header forming, checksums are ok. Did You use multiplexer before TSE? Maybe You are also working with video streams?  --- Quote End ---  If you're going to share the TSE directly between the Nios and your UDP streaming logic, you'll need something to direct traffic to the TSE Avalon-ST sink, without losing data, or corrupting packets from either ST source. If you're going through the Nios, I'm not sure how you'd do that, and I wonder if it would be able to keep up.

[Altera_Forum]

Well that was my plan, I would make Avalon-ST streaming source adapter to interface my logic to Avalon-ST bus, then stream the data through that bus. Now I don't know how can I have a full ethernet support: 

1) Connect my streaming source adapter and nios-to-tse adapter to multiplexer (the one from offload example) then connect that multiplexer to the TSE core (this is my main idea). This should multiplex the data between my data and Nios. However I am not sure, but I may face another problem: the incoming data is a stream, which I can't stop anytime I want except if I place FIFO between the MUX and adapter.  2) Connect my streaming source adapter to DMA, that has shared memory with TSE core, so it can stream the data to the memory through DMA and pick up that data from DMA in Nios. Then place it to InterNiche buffer (I still did not manage to find where it is, yet)  I can't use only hardware implementation. If I understood correctly - I need to instantiate TSE core in logic without Nios? Then it's a no-go :(, I need ARP and ICMP support now and maybe additional options later.

[Altera_Forum]

Yes 1) is the best solution. If you can't pause the stream you generate when you want it, then yes you'll probably need a fifo before the mux.

[Altera_Forum]

Right, this part is now clear. Another confusion now is how to receive such stream on the other endpoint? I need to initialize the tcp/ip stack to get the IP address, then the core will do the job for me? What if I need to grab multicast packets? E.g. 223.x.x.x address?

[Altera_Forum]

I don't remember the offload example in detail, but I think they have a kind of filter that can redirect packets addressed to a specific IP/port number to the hardware and leave the rest to the SGDMA and Nios CPU.

[Altera_Forum]

Ok I am moving forward. It is very easy to send udp data using udp_send() in InterNiche, it does everything You need, but I want to prepare UDP data in hardware, so I want to use ip_write only. 

That is also pretty easy, but I face some problems: I prepare the packet same as I would use in udp_send(), but it forms a packet without src/dest ip adresses and I don't know how to set them? The other parts (except adresses and payload) of the packets are OK. Another problem that the payload is always 10 bytes, no matter if I set the payload length 26 and add required octets to the packet. Maybe someone also had such problems?  here (http://pastebin.com/d1a4qtjg) is the source code.  EDIT: SOLVED -> Need to add src and dest ip addresses through ip struct:  own_address = ip_mymach(pkt->fhost); pip = (struct ip *)(pkt->nb_prot - sizeof(struct ip)); pip->ip_src = own_address; pip->ip_dest = pkt->fhost;  ----------------------------------------  Now moving towards to add DMA to move payload from hardware to pkt buffer.

[Altera_Forum]

So my first idea failed: 

Stream -> SGDMA -> INiche+uCOS -> TSE -> LAN everything up to DMA writes was correct (checked with memory content editor), but the interrupt of SGDMA in uCOS lasts too long and I get missed incoming stream packets. I suppose the logic is OK, but packets received during interrupt fills up FIFO.  So I moved to a different approach. My idea is to use avalon-st multiplexer to connect sgdma_tx used by TSE and my own stream. Here is how it looks like: http://i51.tinypic.com/6gabeg.png  Now the problem is that the stream is transferred to ethernet only a certain amount of time. Sometimes it is a few seconds, sometimes a minute, but usually no longer. If I do all the stuff with backpressure (use aso_ready signal), then ready becomes low after a few seconds after stream start. I suppose the problem could be in FIFO buffers of TSE, but I get no errors in InterNiche and my own stream is below 4mbps. Where is the problem?  I've added scfifo after stream input (dma_input is just old name, there is no dma), but that didn't help either.  P.S. Someone here in forum said that it is possible to achieve a few hundreds of mbps. How to build such interconnect? Connect stream directly to TSE_transmit? Then what to do with receive port and how to control the PHY? I suppose standard uCOS+InterNiche package will not even start without sgdma buffers. Anyone did that?

[Altera_Forum]

Socretes your posts have been very helpful to me. I plan on testing out some more of your work later tonight, however did you ever figure out a way to increase the size of the payload? I would suspect part of the speed problem results from not making the ethernet packet as large as possible.

[Altera_Forum]

The largest packet can be somewhere around 1500bytes. My packet is about 1350bytes, so I suppose this is quite good result.

[Altera_Forum]

Ah, alright I was going off of your payload size that you mentioned in June. I was wondering if you have made any changes to your source code that improved throughput I would love to take a look at it. 

 Edit: For getting hundreds of mbps, I would imagine that you would need to use RGMII and implement the send with gigabit Ethernet protocol, which should give you a payload of ~9000 bytes.

[Altera_Forum]

Well, as You can see, I don't use Nios to form IP packets anymore. All the layers MAC+IP+protocol are generated in hardware.

[Altera_Forum]

A small update: 

I've added fill level port to SC FIFO I use on Avalon-ST bus on the output, so after the transmit stop, that fifo is fully filled. Too bad I can't check internal TSE fifo fill level, but I suppose it is also filled to the max. How is this happening? I feed the packets at 50MHz, 32bit bus, but the valid signal occurs so rare, that the input stream is no more than 3.5mbps. I suppose after FIFO fill-up in TSE it drops the ready signal forever?

[Altera_Forum]

So I am completely stuck... If here are any guys done hardware ethernet packets forming and sending to lan please tell me if You used TSE core through SOPC or it was a fully hardware approach? Somehow TSE fifo gets full and I don't understand how MAC can't handle ~3-4mbps stream. 

 here is the code. (http://pastebin.com/mpbs2xdm) If aso_ready check is commented, then the data goes a little bit longer... Where could be the problem?

[Altera_Forum]

Can you run a signaltap probe on your streams, especially on the inputs and outputs of the multiplexer? 

You should use backpressure to be sure you don't loose data, and I agree that you should get a better throughput than this. One of the components must get stuck at one point, but I never experienced any problem with the TSE's own fifos.

[Altera_Forum]

I've solved the issue using 8->32 bit conversion using fifo. That allows backpressure and the signal is always valid if the packet sending is started when fifo is full enough. Probably if You don't send the packet in a certain amount of time, the tse fifo gets full.