How to stream video (and process) into FPGA

[Altera_Forum]

Hi, 

I want to take a camera... (IDS camera) and stream video into my FPGA. Camera interface will be GigE or USB3 depending on FPS i will decide. in the FPGA i would like to do (what i believe is) simple image processing of... lets say frame subtraction i.e. subtract frame k from frame k+1.  resolution: ideally i would like to work with 640x512 with 300 FPS gray scale 8 bit. I know its a lot of data and eventually i will probably downgrade significantly the SPEC but this is the ideal parameters i would like to work with.  I have a few questions: 1. Interface: how do stream the frames into the FPGA?   what component should i look for in order to take the GigE or USB3 output interface of my camera and connect it to my board so i can do something with it? 2.I assume i will need an external DDR? can you give me some pointers on how to choose what i need and the main principle on how to use it?  Maybe 2 DDR's and erasing the older frame every a new frame comes along? 3. can you give me an advice on which FPGA should i choose? i worked mainly with LATTICE so far and want to cross to ALTERA. 4. any suggestions/ reading material/ comment will be most welcome  Thanks!

[Altera_Forum]

FYI, i found the camera. 

https://www.arrow.com/en/products/emc9080o/byd-company  I, used the project from DECA to stream the video. Now i would like to get into the video handling part. I am looking into the project DECA had prepared and i am not really sure why they use the nios there? what is its purpose?