Problem using TRDB-D5M Camera with SOPC Builder

[Altera_Forum]

I'm trying to use the TRDB-D5M camera connected to a DE2-115 with the UP video design examples. However I can't achieve what the example design file contained in following folder is supposed to do: 

IP_Core_Demos\Video_Demos_using_SOPC_Builder\DE2-115\Example_4_Video_In   The main modifications that I made in SOPC builder is to change the input of the 'Video-In Decoder' core to THDB_D5M camera, add 'Bayer Pattern Resampler', use 'Scaler' and 'Clipper' components to match VGA 640 x 480 specifications. The system was generated successfully.   Then I added the following lines to the main example_4_video_in.v file to be able to read from the camera:  .PIXEL_CLK_to_the_Video_In_Decoder(PIXEL_CLK),  .FRAME_VALID_to_the_Video_In_Decoder(FRAME_VALID),  .LINE_VALID_to_the_Video_In_Decoder(LINE_VALID),  .PIXEL_DATA_to_the_Video_In_Decoder (PIXEL_DATA),    This compiled without errors too. But when I download the .sof to the board, I don't get any display on the VGA output. Am I missing something there?     I can see that the 'Video-In Decoder' component has option THDB_D5M but the camera is called TRDB_D5m. Could that be a reason?     The camera system CD has an example file which works fine, but it does not make use of SOPC. I would like the video source to be an SOPC component. Has anybody used this camera in this way?

[Altera_Forum]

I'm thinking that it may best to take the verilog example and re-construct it to be faster. The reason is i'm afraid that the uni cores will restrict the manipulations of the data too much. As in having to scale down to the format acceptable by those cores, but they might be that restrictive for a reason. 

 From the SOPC perspective the FIFO does work as I have done the earlier Uni core examples and customised them so they would work on the DE2-115. I rebuilt the SOPCs with the DE2-115 modules and took the software into a Nios project for examples 1, 2 and 3. I think the issue is possibly the AV core, i never noticed that the camera model number was different until i saw your post.  I'm goig to look at the verilog example and try understand it, i may even try to modularise it for a schematic build or SOPC (but i got no idea about SOPC component creation).   Also check to ensure you changed the clocks to their appropriate ones based on the Example 4 of the uni program cores.

[Altera_Forum]

I managed to find a DE2 board from another student. I tried the TRDB-D5M camera with the example UP files for DE2 and they all worked! 

 - The Extra_D5M_Video_In worked fine apart from ouput is showing at slow frame-rate.  - The Extras_Edge_Detection also is detecting the edges of whatever the camera is reading in. The only flaw is that the output display is upside down.. don't know why.  So, we can say that the IP_Core_Demos are not fully DE2-115 compliant yet. The fact that no example SOPC design example 'Extra_D5M_Video_In' exists in DE2-115 folder should have been a hint to us I guess. Who writes these UP examples? Is it Terasic or Altera? I contacted Terasic but they say they can't help with the SOPC section, and told me to use their example verilog design file and SignalTap to sort it out myself.  I'm bit annoyed by this. I thought I could get up and running quickly connecting the camera to the DE2-115 board using SOPC builder because I only bought this kit after reading the following in the UP literature "The video decoder converts raw video input from videoin chips on Altera DE2/DE2-70/DE2-115 boards, or Terasic’s 5 megapixel CCD camera, into packets that can be processed by the video-processing cores."  Anyway I have to carry on with the DE2-115 and work using the verilog design file as AsValdr has suggested. I don't understand the verilog code fully. I think one potential mistake among many others is with the GPIO ports. DE-115 define the ports as  //////////// GPIO, GPIO connect to D5M - 5M Pixel Camera ////////// input D5M_D; input D5M_FVAL; input D5M_LVAL; input D5M_PIXLCLK; output D5M_RESET_N; output D5M_SCLK; inout D5M_SDATA; input D5M_STROBE; output D5M_TRIGGER; output D5M_XCLKIN; while DE2 would just do  inout [35:0] GPIO_0; // GPIO Connection 0  OK anyway, time to go back to work and figure out how to solve all this.

[Altera_Forum]

What difference in cores is there between the two? Remeber the DE2-115 has a seperate example 4. Also by using quartus and nios manually i got the earlier 3 examples going which proves all those included cores work. 

(Those examples being the char, pixel buffers and the combo of the two.   Are you able to get the Example 4 video going. It says the video it needs must be NTSC, but i'm from Australia and all our devices are PAL format; so i can't try it.    Most of the cores are have a setting for which DE2 your using. The thing i'm suss on is the config core. When you select the camera it greys out the board selection. Perhaps this matters not.   But if your shy to the verilog perhaps you could do some experimentation and determine which core/s is the problem. That way we can focus on replacing that core with our own.   I'm prbably going to make my own cores (long road) by changing up the verilog example, but i'm cool to help you plug the gaps for an SOPC design if i can along the way.   Don't get your hopes up, i'm a noob aswell, but there is only 1 way to un-nob yourself and thats to have a go.   in summary i think you should try: 1. The example 4 thats specific for the DE2-115. Confirm it works. 2. List the cores in example 1,2,3 and 4. All the cores there can be confirmed as working. 3. Compare the old DE2 example you got working with the camera and identify the cores that are different from the list you made. 4. Therefore those differing cores are the issue. 5. We can attack those cores ourselves and replace them.   Meanwhile i'll be experimenting with the verilog, which will hopefully allow me to understand it and develop seperate components. Then if someone knows how to convert these to SOPC, then "TADA"; we'll have a working SOPC DE2-115 D5M camera combo.

[Altera_Forum]

I have same problem. In my opinion Audio and Video Configuration ip module works wrong.  

 alt_u32 data; alt_up_av_config_dev* d5m=alt_up_av_config_open_dev(CAMERA_CONFIG_NAME); alt_up_av_config_read_D5M_cfg_register(d5m, 0,&data); printf("Data %x",data);   CAMERA_CONFIG_NAME - is Audio and Video Configuration ip name in SOPC   Register 0 is Chip Version register and its default value is 0x1801, but i have data = deadbeef so Audio and Video Configuration module cant read/write data from/to D5M   P.S. I have DE0 module and i cant use examples from D5M CD.  P.P.S Sorry for my English

[Altera_Forum]

Sorry, it was my mistake. Audio and Video Configuration ip works fine, 

the solution for DE0 was found =) I have attached my project. In this project i use 8-bit grayscale mode, 80x60 pixel buffer. I connect D5M module to GPIO_0 header. In the project folder a lot of unnecessary files, but the main thing here is the SOPC and the correct pinout (see bdf).

[Altera_Forum]

I've gotten an SOPC to work for the DE2-115, but its incredibly slow fps (unusably slow for video). 

How is the fps on yours evired?

[Altera_Forum]

FPS is not so good as I would like, but in my opinion is better than in the example without SOPC from terasic. In my project I dont need high resolution and color, so that the FPS is quite suits me.

[Altera_Forum]

Ok. For the DE2-115 the verilog example is exceptionally better.  

I haven't had the chance to look at your design but is it using the Uni program cores or the VIP?

[Altera_Forum]

University Program IP's

[Altera_Forum]

Now I have new problem. Video from D5M camera is normally displayed on the VGA screen. The problem arises when I start to write a program for NIOS.  

 # include "main.h"  int main() {  alt_up_av_config_dev* d5m=alt_up_av_config_open_dev(D5M_CONFIG_NAME); return 0;  }   This simple code build successfully. But when I run it in debugger I have wrong d5m (alt_up_av_config_dev) structure: base adress is correct, but type=17 (instead of 9 for TRDB_D5M_CONFIG). Since the alt_up_av_config_dev structure is wrong, any further attempts to read / write from/to configuration registers of D5M module do not worked out. What am I doing wrong?

[Altera_Forum]

hello everyone, 

 I am also trying to use the demo extras_video-in_D5M camers from UP demos. But that doesn't work for me. I am using DE2-115 cyclone IV E board and my quartas edition is 11.0 . do I need to make any changes to the verilog file. I don't see any software for the demo.. Do I need to write one..

[Altera_Forum]

hi , 

 i am graduate student at mtsu.. i am going to start doing a video processing project can any one tell me how to begin  i have DE2 board in university   please heap me

[Altera_Forum]

Evired 

 i try to downlod your mrdk_de0.qar file and open it but it is giving   following error  Error: Block Design File C:\Documents and Settings\vis174.FSA\My Documents\Downloads\mrdk_de0_restored\mrdk_de0.bdf has an unsupported version number 1.4  i am using Quartus ii  can you please help me

[Altera_Forum]

Thiruparan, 

 You might want to have a TRDB-D5M CMOS camera module from Terasic to plug it into your DE2 board and use the sample code they provide. I did just this and its been a good start.    I am trying to carry out some basic Image processing too.. currently trying to manipulate pixel values (using the VGA controller module) like colour inversion, tint creation etc.    Alternatively, you could have the R,G and B 10 bit values go to a NIOS and do some interesting stuffs there. Hope this helps.

[Altera_Forum]

Thank you very much for your response........ 

 ya i can buy TRDB-D5M .. ... i have only quartus ii web edtion which i downloded from altera website for free ...is that tool is enough to carry out the image /video processing ..  what are the tools and boards you are using ........

[Altera_Forum]

Yes Thirupan. The camera can be interfaced to the DE2 (through the GPIO pins) and the FPGA programmed with the sample code provided by Terasic. Quartus II will be fine, and the web edition (free license) version works.  

 I use the DE2 with TRDB-D5M. Currently trying to add an SOPC featuring NIOS to prossibly have the processing done. Meanwhile I would be interested to know about your progress...  Rgds, Balu

[Altera_Forum]

hi ,  

really Thank you very much for your response ,  i have design a processor using DE2 board which is a very basic processor i did that for my microprocessor design class   i did that using block diagrams . only thats is the experince i have in FPGA ...  to do image processing i am just started .. my target is devoleping a fpga which can detect objects    i have only the quartus ii web edtion and DE2 board , i am going to by cam ...   what you have done so far   http://freevideolectures.com/course/2316/digital-image-processing-iit-kharagpur its online lecture on image processing   it may help you   please share what you have done so far Thank you

[Altera_Forum]

I want to buy d5m module. can anyone send me the snapshot of the sample picture of this d5m camera

[Altera_Forum]

Hi, 

 Are you talking about a pic of the cam itself? It can be found on the terasic website.. Link below:  http://www.terasic.com.tw/cgi-bin/page/archive.pl?language=english&no=281

[Altera_Forum]

Thanks for the reply. I am talking about sample picture taken using the Terasic d5m camera module because I did not find yet any sample picture taken from this camera on the Terasic Website. I want to check the quality of the camera, which is essential in my project.  

 Please help me