Image Processing Project on the DE2 board

[Altera_Forum]

Greetings! 

 I am making this project using the DE2 development board (with a CMOS Camera and LCD display attached) which has the following objectives:  1. Capture an image 2. Display the Image on the LCD 3. Store the Image in an SD card 4. Convert the image into a JPEG format. 5. Have a user interface to control these functions. 6. The user interface should be implemented on top of a controller 7. The controller should load the JPEG algorithm bit stream into the FPGA to configure it at runtime from a memory (like the SD card). *very important  My project also should have HDL codes to interface to an embedded RAM (where the Image captured will be stored temporarily), CMOS camera, and SD card.  Can the embedded RAM, the DE2 board has, generally hold the RAW image from the camera?  In reconfiguring the Cyclone II, does the whole chip need to be altered? or can only a portion of it alloted for JPEG compression be changed? I ask this because I plan to implement the controller in the FPGA chip instead of the NIOS II embedded controller.  Some of my mentors in school pointed out that the FPGA chip (Cyclone II) might run out of space if I implement the controller and the image processing there.  Thank you for your time, some advise, insights, referrals are highly appreciated:)

[Altera_Forum]

It looks like your projects needs a lot of software work. 

The only hardware seems to be the JPEG compression. If you don't have strong real time requirements this also can be done in software.   Probably the NIOS allows to better implement all this controller stuff.   Hope this helps

[Altera_Forum]

--- Quote Start ---   1. Capture an image 2. Display the Image on the LCD 3. Store the Image in an SD card 4. Convert the image into a JPEG format. 5. Have a user interface to control these functions. 6. The user interface should be implemented on top of a controller 7. The controller should load the JPEG algorithm bit stream into the FPGA to configure it at runtime from a memory (like the SD card).  --- Quote End ---    1. I don't have a DE2 board with me, so I don't know exactly how to do this. But, you can take a look at the CMOS imager specs and write your HDL based on it. It's basically just handshaking between the FPGA and this CMOS imager, and getting all the data correctly. 2/3. Again, take a look at the LCD's specs. For the SD card, you need to implement the SDIO interface in the FPGA. Think there are a few free implementations on this out there... 4. Haven't done JPEG, sorry... 5. Not sure if the LCD panel is also a touchscreen? If so, you can write HDL that renders the UI graphics to the touchscreen. Just superimpose the UI on top of the image that's currently on the screen. Or, you can play around with alpha-blending. 6. What is this controller for? I am guessing that this controller receives user inputs and processes those inputs. If so, for a simple touchscreen controller, just detect the coordinates of a user "touch". Be sure to add at least a few adjacent pixels to be certain it is the user who "touched" the screen, rather than some bug or dust. 7. I am thinking that you should put the JPEG compression algorithm in the HDL, which means it becomes active as soon as your machine becomes active. But if you really need to load this algorithm everytime you do a JPEG compression, then you can write your own scan chain (see below).    --- Quote Start ---   In reconfiguring the Cyclone II, does the whole chip need to be altered? or can only a portion of it alloted for JPEG compression be changed? I ask this because I plan to implement the controller in the FPGA chip instead of the NIOS II embedded controller.  --- Quote End ---   No, you don't need to reconfigure the whole chip. You can try the "partial reconfiguration" feature released by Altera a while ago, though I've never tried it yet. Or, what I'd do is write my own scan chain and have it hooked to an interface to receive data. The interface can be USB/Ethernet/Wi-Fi or whatever. For example, say you've made changes to your JPEG compression algorithm and you want your FPGA to reload the new algorithm, then you can scan in the new algorithm into your FPGA using the said interface + scan chain. The scan chain could be just some shift registers, or controls to internal/external memory. What the scan chain does is just receive data from the interface, and stores the data accordingly (usually in shift registers, but can be other internal/external memory as well). This would update your old algorithm with the new data given by the user. You could also implement your own protocols for storage/memory control or error detection in between the interface controller and the scan chain (if you wish).

[Altera_Forum]

--- Quote Start ---   What is this controller for? I am guessing that this controller receives user inputs and processes those inputs.   --- Quote End ---    The controller would be for: 1. Yes, handling commands such as "capture image", "store to sd card", "display on lcd", compress to <compression type>". For the compression type I would implement JPEG but the project leaves room for future developments of other compression types. 2. It would also serve as the configuring device during run time. The idea is when the "compress <>" option is chosen, the corresponding synthesized HDL code of the compression would be loaded to the portion of the FPGA to be reconfigured. The reconfiguring bit stream (compression HDL code) would come from a memory (SD Card for example) and loaded by the controller (NIOS or FPGA implementation) to the FPGA compression portion.    --- Quote Start ---   7. I am thinking that you should put the JPEG compression algorithm in the HDL, which means it becomes active as soon as your machine becomes active.   --- Quote End ---    I will implement the compression in Verilog but I need the capability to load it at run time so that future compression algorithms could be added during run time (as users need them).     --- Quote Start ---   But if you really need to load this algorithm everytime you do a JPEG compression, then you can write your own scan chain (see below).   Or, what I'd do is write my own scan chain and have it hooked to an interface to receive data.   you can scan in the new algorithm into your FPGA using the said interface + scan chain. The scan chain could be just some shift registers, or controls to internal/external memory.   --- Quote End ---    From what I understand about the scan chain, it looks like the concept of using a controller to load the compression algorithm bit stream to reconfigure the fpga. Am I correct?

[Altera_Forum]

--- Quote Start ---   The controller would be for: 1. Yes, handling commands such as "capture image", "store to sd card", "display on lcd", compress to <compression type>". For the compression type I would implement JPEG but the project leaves room for future developments of other compression types. 2. It would also serve as the configuring device during run time. The idea is when the "compress <>" option is chosen, the corresponding synthesized HDL code of the compression would be loaded to the portion of the FPGA to be reconfigured. The reconfiguring bit stream (compression HDL code) would come from a memory (SD Card for example) and loaded by the controller (NIOS or FPGA implementation) to the FPGA compression portion.   --- Quote End ---   okay, my guess was something like that too... :) You can write a controller in HDL to do those things.    --- Quote Start ---   I will implement the compression in Verilog but I need the capability to load it at run time so that future compression algorithms could be added during run time (as users need them).   --- Quote End ---   you very well need the shift register / scan chain logic then...    --- Quote Start ---   From what I understand about the scan chain, it looks like the concept of using a controller to load the compression algorithm bit stream to reconfigure the fpga. Am I correct?  --- Quote End ---   Yes, you're right. It probably would be just some shift registers, or some kind of matrix memory controlled by some logic. You might even like to write your HDL such that it uses the internal RAM (such as M4K, etc.) of the device. Well, you can write your own flavour using your own taste... :)

[Altera_Forum]

--- Quote Start ---  okay, my guess was something like that too... :) You can write a controller in HDL to do those things.  --- Quote End ---    So I can implement the controller in a portion of the FPGA chip OR I can use the NIOS right? If I go for the first one then I would have to retain the circuitry of the controller while reconfiguring the other portion for the image processing. If I go for the NIOS then I would have to implement an interface to the FGPA. Which do you suggest?:)    --- Quote Start ---   you very well need the shift register / scan chain logic then...  Yes, you're right. It probably would be just some shift registers, or some kind of matrix memory controlled by some logic. You might even like to write your HDL such that it uses the internal RAM (such as M4K, etc.) of the device. Well, you can write your own flavour using your own taste... :)  --- Quote End ---    Either choice of controller implementation I would have to use scan chain logic right? What would the memory be used for? I get the concept of scanning in bit stream to configure but I don't get why use a memory/shift register/RAM. Can't I use the controller to read the configuration bits from an external memory like an SD card and just load it to the FPGA? Is the memory you're referring to a memory that is holding the reconfiguration bits of the compression algorithm?   Oh yeah! regarding the reconfiguration of the FGPA, my HDL code would be compiled into a bit stream that would configure the chip right? I am in the assumption that I would always need the Quartus software to configure the chip. If that's the case then how can the chip be changed without the software?:)

[Altera_Forum]

--- Quote Start ---  So I can implement the controller in a portion of the FPGA chip OR I can use the NIOS right? If I go for the first one then I would have to retain the circuitry of the controller while reconfiguring the other portion for the image processing. If I go for the NIOS then I would have to implement an interface to the FGPA. Which do you suggest?:)   --- Quote End ---   Yes, you can implement the controller in a portion of the FPGA. Actually, even if you use NIOS, it will also be in a portion of the FPGA (Nios will also get compiled to use the FPGA's logic resources). Either way, you have to implement an interface to the FPGA.  Think of it very simply, it's just your controller + interface. Controller is a block in FPGA (yes, you need to retain this circuitry while reconfiguring the other portion), and interface logic is another block in the same FPGA. Your controller implementation can be of your own choice, you can write your own HDL, or use Nios. Either way, it's still your controller. You have to hook this controller (Nios or HDL) up to an interface, maybe RS232/USB/etc. Again, it's Controller + Interface. :) Of course, in Nios, you can use any of the interfaces available, and build your Controller + Interface totally in Nios. It's really your own preference. I personally would like everything in VHDL.     --- Quote Start ---   Either choice of controller implementation I would have to use scan chain logic right? What would the memory be used for? I get the concept of scanning in bit stream to configure but I don't get why use a memory/shift register/RAM. Can't I use the controller to read the configuration bits from an external memory like an SD card and just load it to the FPGA? Is the memory you're referring to a memory that is holding the reconfiguration bits of the compression algorithm?  --- Quote End ---   Yes, you are right about the concept of scanning in the bitstream. And yes, the bitstream can be drawn from external memory like an SD card. The contents of your bitstream can be your new JPEG compression algorithm, or whatever. Or, you could write a generic JPEG compressor, and only change some parameters during runtime. So, in this case, your bitstream would be just the new parameters to scan into your algorithm. Whatever it is, it deals with some kind of memory. You need to scan in the bitstream (e.g. from your SD card), and store it somewhere (RAM). Now, it's up to you to define which memory to use (could be shift registers, M4K, external RAM, etc.). It's really up to you. Well, at least you got the concept right already :)    --- Quote Start ---   Oh yeah! regarding the reconfiguration of the FGPA, my HDL code would be compiled into a bit stream that would configure the chip right? I am in the assumption that I would always need the Quartus software to configure the chip. If that's the case then how can the chip be changed without the software?:)  --- Quote End ---   No, you don't *always* need Quartus to configure the chip, but it's good to start with that first. Later on, you can always use a configuration device (such as EPCS), or configuration controller (like a MAX-II CPLD) to do the auto-configuration during power up of your board. Here, you don't need to hook your board up to Quartus.  Regarding your question on how to "change the chip" configuration without Quartus, actually all you need to do is to scan in your new bitstream to your FPGA board, and you don't need Quartus (or even JTAG) to do the scanning for you. If you've implemented the RS232 interface (and of course, hooked it up to your bitstream controller) for example, then you can even connect your computer to your FPGA board via your RS232 port, and use some tool (or write your own) to do the bitstream scanning. Of course, you might as well use other interfaces such as Ethernet/USB/etc.

[Altera_Forum]

--- Quote Start ---   Actually, even if you use NIOS, it will also be in a portion of the FPGA (Nios will also get compiled to use the FPGA's logic resources). Either way, you have to implement an interface to the FPGA.  --- Quote End ---    Okay, now I think I have the wrong idea about the NIOS embedded processor. I thought it was a separate chip that can be used as a general purpose microcontroller. I double checked and there's no chip like that in the board. What then is NIOS? :))  And what do you mean by "interface to the FPGA"?    --- Quote Start ---   Think of it very simply, it's just your controller + interface. Controller is a block in FPGA (yes, you need to retain this circuitry while reconfiguring the other portion), and interface logic is another block in the same FPGA.   --- Quote End ---    Is the interface you mention here the same with the interface in the next quote (RS232/USB/etc.)? If not, what is the interface logic here for?     --- Quote Start ---   Your controller implementation can be of your own choice, you can write your own HDL, or use Nios. Either way, it's still your controller. You have to hook this controller (Nios or HDL) up to an interface, maybe RS232/USB/etc. Again, it's Controller + Interface. :) Of course, in Nios, you can use any of the interfaces available, and build your Controller + Interface totally in Nios. It's really your own preference. I personally would like everything in VHDL.  --- Quote End ---    If HDL code is to be compiled and loaded in the FPGA, how then is the implementation done in NIOS?      --- Quote Start ---   You need to scan in the bitstream (e.g. from your SD card), and store it somewhere (RAM). Now, it's up to you to define which memory to use (could be shift registers, M4K, external RAM, etc.).   --- Quote End ---    When the bit stream is scanned why does it have to be stored somewhere? Can't it be loaded to the FPGA immediately? What is the other memory for?    --- Quote Start ---   Later on, you can always use a configuration device (such as EPCS), or configuration controller (like a MAX-II CPLD) to do the auto-configuration during power up of your board. Here, you don't need to hook your board up to Quartus.  --- Quote End ---    Doesn't the controller I'm planning to implement (with HDL or NIOS) handles the reconfiguring? Here, the "configuration controller" pertains to upon power up, not during run time?  Can this configuration controller be used as the reconfiguration controller during run time to load in the new scanned bit stream instead of the one discussed above?  Please bear with me:) Thank you very much.

[Altera_Forum]

--- Quote Start ---   And what do you mean by "interface to the FPGA"? Is the interface you mention here the same with the interface in the next quote (RS232/USB/etc.)? If not, what is the interface logic here for?   --- Quote End ---   Yes, I meant the same interface as in the next quote.    --- Quote Start ---  If HDL code is to be compiled and loaded in the FPGA, how then is the implementation done in NIOS?    --- Quote End ---   The "implementation" in the FPGA should as much as possible be similar (HDL and Nios), it is just the source files that are different. For HDL, you would use VHDL or Verilog, or other HDL formats, while for Nios, you would use C for the source code. The resulting RTL from both should be similar (though not always so - I still prefer HDL to Nios when improving area/efficiency).    --- Quote Start ---   When the bit stream is scanned why does it have to be stored somewhere? Can't it be loaded to the FPGA immediately? What is the other memory for?   --- Quote End ---   You can use the same external SD memory to do your RAW-to-JPEG manipulation, JPEG compression, etc., and use something like a temporary file for the manipulation of bits. But I personally would want to manipulate bits in a RAM (could be on-chip RAM like M4K/M8K, or external RAM) because it performs faster. So it depends on how fast you want your application to work.    --- Quote Start ---   Doesn't the controller I'm planning to implement (with HDL or NIOS) handles the reconfiguring? Here, the "configuration controller" pertains to upon power up, not during run time?   --- Quote End ---   No, this is a different controller. Your controller does not reconfigure the whole FPGA the way Quartus configures the FPGA. The built-in JTAG controller in the FPGA does the configuring. Now, you want to implement your own controller that configures a part of your whole design, so you need to write a separate HDL for it (your HDL, again, is not the same as the built-in JTAG configuration controller in the FPGA).    --- Quote Start ---   Can this configuration controller be used as the reconfiguration controller during run time to load in the new scanned bit stream instead of the one discussed above?   --- Quote End ---   Yes. But only to load the new bitstream for your JPEG compression/or whatever other function. You can't use this to configure the whole FPGA (i.e. you can't load a SOF/POF with this).    --- Quote Start ---  Please bear with me:) Thank you very much.  --- Quote End ---   No problem.

[Altera_Forum]

--- Quote Start ---   You can use the same external SD memory to do your RAW-to-JPEG manipulation, JPEG compression, etc., and use something like a temporary file for the manipulation of bits. But I personally would want to manipulate bits in a RAM (could be on-chip RAM like M4K/M8K, or external RAM) because it performs faster. So it depends on how fast you want your application to work.  --- Quote End ---    I think the memory here is for the image data itself, the data from the camera module? So, I have to implement 2 memories. (Correct me if I'm wrong):  1. One to hold the image data from the camera - this is the ram you were referring to?  2. One to hold the Algorithm bit stream to be loaded to the FPGA for it to be reconfigured to do the processing - I'll use an SD card for this.  So, I'm thinking the process will go like this: (Assuming that the image data from the camera is stored in a memory (RAM))  a. Choose compression option b. The bit stream from the SD Card is loaded to the FGPA c. The FPGA is reconfigured to do the compression d. The RAW data from the RAM is fed through the FPGA (I don't know if this is what really happens) e. The processed data is stored in the RAM as a compressed image data.  If I'm correct, what part of this process does the scan chain happen?  I got confused with the memory part because you pointed out that from the SD card, the bit stream must be loaded to an internal memory. I'm assuming now that the bit stream you were referring to here is the image data from the camera not the JPEG algorithm (HDL) bit stream for configuring the FPGA.  I know that a controller must be implemented to handle control and interface with the FPGA, memory, camera, etc. Also, to load the configuration bits from the SD to reconfigure the FPGA. However, from the quote below, the controller I mention here does not do this function (reconfiguring). So what do I do to reconfigure? Write THE controller to access the JTAG?      --- Quote Start ---   No, this is a different controller. Your controller does not reconfigure the whole FPGA the way Quartus configures the FPGA. The built-in JTAG controller in the FPGA does the configuring. Now, you want to implement your own controller that configures a part of your whole design, so you need to write a separate HDL for it (your HDL, again, is not the same as the built-in JTAG configuration controller in the FPGA).  --- Quote End ---  

[Altera_Forum]

--- Quote Start ---  I think the memory here is for the image data itself, the data from the camera module? So, I have to implement 2 memories. (Correct me if I'm wrong):  1. One to hold the image data from the camera - this is the ram you were referring to?  2. One to hold the Algorithm bit stream to be loaded to the FPGA for it to be reconfigured to do the processing - I'll use an SD card for this.   --- Quote End ---   The memory can be for image data, or also your compression algorithm. I'm not sure how you are going to load the algorithm into the FPGA as a bitstream - is it going to be the whole algorithm, or just parameters to the algorithm? I would actually write the algorithm in HDL (means this is part of your SOF), and only load the parameters as a bitstream. However, even with this I would load the bitstream into a RAM (best if internal RAM), so the algorithm doesn't need to access the external SD card all the time at every loop. Imagine, if your algorithm is running, and you remove the SD card, your whole system is going to fail. So, I'd always load the bitstream to (internal or external) RAM.    --- Quote Start ---   So, I'm thinking the process will go like this: (Assuming that the image data from the camera is stored in a memory (RAM))  a. Choose compression option b. The bit stream from the SD Card is loaded to the FGPA c. The FPGA is reconfigured to do the compression d. The RAW data from the RAM is fed through the FPGA (I don't know if this is what really happens) e. The processed data is stored in the RAM as a compressed image data.  If I'm correct, what part of this process does the scan chain happen?    --- Quote End ---   You're almost there. This process will start after you have already configured your FPGA using Quartus. The scan chain bitstream scanning will happen in (b), after your FPGA is already configured by Quartus. (c) is also related to (b), where after scanning in the bitstream, your FPGA automatically already gets configured with the new compression algorithm. For (d), usually RAW comes from the image sensor, but u will want to store it once in RAM before you do further processing (such as JPEG compression). The processing should work on the contents of the RAM, and not modify contents in the SD card (if for example, u want to load an image from SD instead of sensor and do image processing on it). For your JPEG compression, it is also better to have the compression parameters stored in RAM, so that your algorithm can use it directly from RAM instead of rely on SD card (again, SD card may be removed anytime during runtime).    --- Quote Start ---   So what do I do to reconfigure? Write THE controller to access the JTAG?  --- Quote End ---   Reconfigure the FPGA using the normal way, Quartus programmer via USB JTAG. Your design should contain another controller to scan in the bitstream via another interface (maybe another USB port, or others).

[Altera_Forum]

--- Quote Start ---  The memory can be for image data, or also your compression algorithm. I'm not sure how you are going to load the algorithm into the FPGA as a bitstream - is it going to be the whole algorithm, or just parameters to the algorithm?  --- Quote End ---    The idea is that I would have a library of compression or image processing algorithms in the SD card then as I choose an algorithm to implement, it will get loaded to the FPGA so that the FPGA (or at least a portion of it) would perform the desired algorithm. So to answer the question, the whole algorithm would be loaded essentially. In this way, do the FGPA still have to access the SD card over and over again? Or can it be just one time loading?   I understand why you say that its better to store it in a RAM first, but if it will be a one time loading to the fpga, can this be ommited for the compression algorithm?    --- Quote Start ---   I would actually write the algorithm in HDL (means this is part of your SOF), and only load the parameters as a bitstream. However, even with this I would load the bitstream into a RAM (best if internal RAM), so the algorithm doesn't need to access the external SD card all the time at every loop. Imagine, if your algorithm is running, and you remove the SD card, your whole system is going to fail. So, I'd always load the bitstream to (internal or external) RAM.   --- Quote End ---    I understand why it should be stored in a RAM (except the parameters part), however, if that's the case then is it not true that the FPGA just have to be changed then it doesn't need to access the algorithm bit stream used to configure it?     --- Quote Start ---   You're almost there. This process will start after you have already configured your FPGA using Quartus. The scan chain bitstream scanning will happen in (b), after your FPGA is already configured by Quartus.   --- Quote End ---    Yes it will start after the first configuration by quartus, but after that the FPGA must be reconfigured without quartus. That's why I didn't understand why you said the scan chain bitstream (compression algorithm) will happen after the FPGA is already configured. What if I change the current algorithm to a bmp compression without the use of quatrus? That's when the scan chain comes in right? The new algorithm is scanned and the controller reconfigures the FPGA.    --- Quote Start ---   (c) is also related to (b), where after scanning in the bitstream, your FPGA automatically already gets configured with the new compression algorithm.   --- Quote End ---    This should happen without the use of quartus.   Then why in the quote below, reconfiguring the FPGA using Quartus? Then the reconfiguration won't be implemented in real time? Furthermore, if the reconfiguration is done by quartus, what is the "another controller to scan in" for?    --- Quote Start ---   Reconfigure the FPGA using the normal way, Quartus programmer via USB JTAG. Your design should contain another controller to scan in the bitstream via another interface (maybe another USB port, or others).  --- Quote End ---  

[Altera_Forum]

--- Quote Start ---  The idea is that I would have a library of compression or image processing algorithms in the SD card then as I choose an algorithm to implement, it will get loaded to the FPGA so that the FPGA (or at least a portion of it) would perform the desired algorithm. So to answer the question, the whole algorithm would be loaded essentially. In this way, do the FGPA still have to access the SD card over and over again? Or can it be just one time loading?   I understand why you say that its better to store it in a RAM first, but if it will be a one time loading to the fpga, can this be ommited for the compression algorithm?   --- Quote End ---   Yes, you can do that, provided the FPGA doesn't keep accessing the SD card for any other info after the loading is completed, unless of course it needs to perform another loading.    --- Quote Start ---   Yes it will start after the first configuration by quartus, but after that the FPGA must be reconfigured without quartus. That's why I didn't understand why you said the scan chain bitstream (compression algorithm) will happen after the FPGA is already configured. What if I change the current algorithm to a bmp compression without the use of quatrus? That's when the scan chain comes in right? The new algorithm is scanned and the controller reconfigures the FPGA.   --- Quote End ---   Absolutely right! If you want to change the current algorithm to bmp for example, you just need to scan in the new algorithm bitstream using that interface, and you don't need quartus. Actually, you can even disconnect Quartus totally from your board (plug out USB-Blaster or disconnect the cable) after you've configured your FPGA the first time.    --- Quote Start ---   This should happen without the use of quartus.   --- Quote End ---   Absolutely.

[Altera_Forum]

Alright! Let me see if I get this correctly:) 

 I attached a block diagram I made to represent the system I want to implement.  The solid lines represent the control signals used for interfacing the devices and the dashed lines are the data paths. The red line would be the data path of the configuration bits (HDL algorithms such as the JPEG compression) used to configure the FPGA portion for the data processing block. The black one are the buses for the image data being manipulated.  For the red line, the scan chain I imagined is like that, that the configuration bit stream would be scanned and loaded by the controller into the FPGA. Can I do this as direct from the SD Card into the FPGA processing block with the controller just commanding the devices? OR am I missing something in this part? (I didn't include the RAM I can use to store the configuration bits anymore)  For the controller block, I divided it into two parts, the interface to the other devices and the scan-chain part. Is this logical?  Furthermore, You mentioned that I have to implement a controller for the reconfiguration when I unplugged the Quartus and the JTAG. Is the controller in the diagram one in the same?

[Altera_Forum]

--- Quote Start ---  Furthermore, You mentioned that I have to implement a controller for the reconfiguration when I unplugged the Quartus and the JTAG. Is the controller in the diagram one in the same?  --- Quote End ---   What I meant was in your design, you already included the controller to do the bitstream scanning (or let's say it JPEG algorithm reconfiguration). Your whole design (which includes this controller), is loaded once into the FPGA by Quartus, after which you can disconnect Quartus from your board. And your JPEG Algorithm Reconfiguration controller should work (with Quartus disconnected) by scanning in a new algorithm from PC to FPGA via some pre-defined interface. So yes, this controller is already included in your block diagram, named as "Controller (HDL or NIOS)". It has always been the same controller we were talking about. :)

[Altera_Forum]

--- Quote Start ---  And your JPEG Algorithm Reconfiguration controller should work (with Quartus disconnected) by scanning in a new algorithm from PC to FPGA via some pre-defined interface.  --- Quote End ---    And from SD Card to FPGA via the interface shown in the diagram can be done as well right?:)  So is the diagram I drew good enough? Does it miss something or is anything wrong with it?

[Altera_Forum]

Thank you very much for your help :)