Frame Reader Initialization

[Altera_Forum]

Hello, 

 I want to implement a Frame Reader which should read data from memory and send its AVALON Video protocol to an alpha blending mixer. My clocked video output settings for PAL are: Image Width/Active pixels: 720 Image Height/Active lines: 576 Bits per Pixel per color planes: 8 Color plane transmission format: sequence  Interlaced Video Pixel fifo size: 720 Fifo Level at which to start output: 719 Video in and out use the same Clock Use control port Sync signals: Embedded in video Active picture line: 23 Horizontal blanking: 24 F rising edge line: 313 F falling edge line: 1 Vertical blanking rising edge line: 311 Vertical blanking: 25 I have written the value 128 to all corresponding addresses of the ram. For test purposes I am using the onchip ram for this. The RAM has two masters, the NIOS II and the frame reader. The settings of the frame reader are: Bits per pixel per color plane: 8 Number of color planes in parallel: 1 Number of color planes in sequence: 2 Maximum Image width: 40 Maximum Image height: 40 Master port width: 64 Read master FIFO depth: 64 Read master FIFO burst target: 32 Use separate clock for the Avalon MM master interface: unchecked (The data-width of the onchip RAM is 64.)   For initialisation of the frame reader I have written the following function which is called in once the main function of my NIOS II software: void initFrameReader() { alt_printf("Initialize Frame Reader.\r\n"); // start Frame Reader IOWR(FRAMEREADER_BASE, 0, 1); // Frame Select IOWR(FRAMEREADER_BASE, 3, 0); // Frame 0 Base Address IOWR(FRAMEREADER_BASE, 4, ONCHIP_MEM_BASE); // Frame 0 Words IOWR(FRAMEREADER_BASE, 5, 800); // 40*40 / ( 64/(8*4) ) = 800 // Frame 0 Single Cycle Color Patterns IOWR(FRAMEREADER_BASE, 6, 6400); // 40*40*4 number of pixels in the frame multiplied by the number of single cycles required to represent one pixel // Frame 0 Reserved // IOWR(FRAMEREADER_BASE, 7, 0); // reserved for future use ? // Frame 0 Width IOWR(FRAMEREADER_BASE, 8, 40); // Frame 0 Height IOWR(FRAMEREADER_BASE, 9, 40); // Frame 0 Interlaced IOWR(FRAMEREADER_BASE, 10, 8); // See VIP Processing Suite - Table 4-4 } But as soon as I Activate the Layer the Frame Reader corresponds to, I do not get a valid picture. Even if I deactivate it on runtime, it does not get a valid picture anymore. I have to reset the cpu to get the alpha blending mixer get work again (with frame reader layer deactivated). What could be the problem? I have double-checked the BSP-Editor, none of my program memory addresses are placed on the onchip-mem. Could it be an arbitration problem? As far as I know the arbitration of the sopc-builder is round-robin and only requesting masters are included in the arbitration. But I am not writing data to the onchip memory while the frame reader is reading data from the RAM. Could someone double-check the initialisation settings of the frame reader? Did I miss something, or did I missunderstand anything?   I would be pleased if anyone can helb me.

[Altera_Forum]

You are enabling the frame reader before you actually configure it. Try moving the write to register 0 to after the other registers are configured.

[Altera_Forum]

Thank you kevin. Your tip was good. I have done following things: 

 - increased the cpu-frequency to 100 MHz - changed max width of the frame reader in the sopc builder to 720 - changed max hight of the frame reader in the sopc builder to 576 - put the start of the frame reader in the to the end of the subprogram, after configuring the other registers (what kevin told me)   now I at least get a grey picture (all values in my memory are 128 -> grey should be right), but not in the right size. The whole display is being grey.

[Altera_Forum]

I would suggest using the Test Pattern generator instead of the Frame Reader to test if the rest of your system is setup correctly. You may even want to try using just the Test Pattern Generator plus Clocked Video Output without anything else. Start with the simplest setup possible and build it up one step at a time.

[Altera_Forum]

Actually I have tried that, but the same problem occours. The Alpha Blending Mixer itself works. I can also show the layer of a Test Pattern. But as soon as I activate the pattern of the Frame Reader the system does not do what it is supposed to. 

 I have the feeling that there is something wrong with the setting of the Single Cycle Patterns and the number of Words read from memory. I have clocked video output with 4:2:2 sampling YCbCr. That means I actually need 2 single cycle patterns for one pixel instead of 3. (YCbYCr instead of RGB)   So the settings for the Single Cycle Patterns should be: 40*40*2 = 3200;   Since the width of the Onchip Memory is 64, eight (complete) single cycle bytes fit into one word of the memory. According to this the number of words that should be read from the memory should be the number of bytes needed for one picture divided by eight:   3200/8 = 400;   So the correct settings for single cycle pattern and number of words read from memory should be:   // Frame 0 Words IOWR(FRAMEREADER_BASE, 5, 400); // 40*40*2 / ( 64/8 ) = 300  // Frame 0 Single Cycle Color Patterns IOWR(FRAMEREADER_BASE, 6, 3200); // 40*40*2 number of pixels in the frame multiplied by the number of single cycles required to represent one pixel  Can anybody approve that?

[Altera_Forum]

The documentation for the frame reader is a bit confusing. They do not give any examples for color planes in sequence. I have only ever used it with color planes in parallel, so I couldn't say for sure if your settings are correct. 

 I noticed you have your output set to interlaced. Are you trying to output interlaced video from the frame reader as well? I don't see anywhere in the documentation where it explicitly mentions interlaced support for the frame reader. I don't know for sure, but I suspect the frame reader does not support interlaced output.

[Altera_Forum]

Thanks for your help kevin.  

 I think it does support interlaced video. The Video and image processing guide says: "Also, the Frame Reader must be configured with the starting address of the video frame in memory, and the width, height, and INTERLACED values of the control data packet to output before each video data packet." (page 5-29)   And in Table 5-11 (page 5-30) it says about the parameter interlaced/progressive: "Set via the Avalon Memory Mapped Slave control port, ALL VALUES SUPPORTED."   The corresponding register should be the one with offset 10 where I can set "the interlaced nibble used for the CONTROL PACKET associated with frame 0"   Examples for this control packet nibble are given in table 4-4. (page 4-8)    I have been browsing the datasheets a lot, but since my video output is not working I am afraid to have something gotten up the wrong way. 

[Altera_Forum]

I think some registers of the control interface are not set up properly but I may have misunderstood what you are trying to achieve. If you want the frame reader to output 40x40 interlaced video (ie, a pair of 40x20 fields) then I would suggest something like that: 

 IOWR(FRAMEREADER_BASE, 4, FIELD_0_BASE); IOWR(FRAMEREADER_BASE, 5, 200); // 40*20 / ( 64/(8*2) ) = 200 IOWR(FRAMEREADER_BASE, 6, 1600); // 40*20*2 IOWR(FRAMEREADER_BASE, 8, 40); IOWR(FRAMEREADER_BASE, 9, 20); IOWR(FRAMEREADER_BASE, 10, 8);  IOWR(FRAMEREADER_BASE, 11, FIELD_1_BASE); IOWR(FRAMEREADER_BASE, 12, 200); // 40*20 / ( 64/(8*2) ) = 200 IOWR(FRAMEREADER_BASE, 13, 1600); // 40*20*2 IOWR(FRAMEREADER_BASE, 15, 40); IOWR(FRAMEREADER_BASE, 16, 20); IOWR(FRAMEREADER_BASE, 17, 12);   You also need to switch the frame select register between field 0 and field 1 for every output field. IOWR(FRAMEREADER_BASE, 3, ??);  Be aware that the alpha blending mixer has limited support for interlaced video. There is no mechanism in the mixer to ensure that fields of the same type are mixed together so if a F0 is coming one input and a F1 is coming on the other one they will be mixed together which is probably not the desired behavior.

[Altera_Forum]

Thanks for your tips and hints vgs. 

 I have tried what you told me but have problems to changes the frame properly. I want do do it after the "end of frame" interrupt is being fired, but can't figure out how I can access it. Does anybody have experience with that?   The documentation says about the interrupt register:   "Bit 1 of this register is the end of frame interrupt bit. All other bits are unused. Writing a 1 to bit 1 resets the end of frame interrupt."   - "Bit 1": is this the first bit, or the second one (with the first one being Bit 0) ? I guess the first one.   - I know now how to reset it, but do I read it? Is it the first bit with a rising edge, when the documentation says "firing"? And do I have to reset it manually? 

[Altera_Forum]

My experience with the end of frame interrupt is that it does not work. 

 If you search for the irq signal in the post mapping net list viewer you will see it get's optimized out and that there is no interrupt connection between the frame reader and Nios. The source code for the frame reader is actually included with Quartus. I did manage to patch it to make the interrupt work. I sent all the details to my Altera FAE, but I never heard anything back about it.  In the end, I was able to write my own frame reader from scratch in less time than it took me to get the VIP frame reader doing what I wanted. My version also used a fraction of the logic elements. My version is also less capable in some ways, but it did everything I needed.  To answer your other questions:  Bit 1 is the second bit, bit 0 is the first.   You need to manually reset the interrupt in your ISR. For example: IOWR(BASE, 2, 2);

[Altera_Forum]

Here is a diff of the files I changed to get the interrupt to work: 

--- alt_vipvfr91_prc.v 2009-10-22 07:53:50.000000000 -0400 +++ c:/temp/alt_vipvfr91_prc.v 2010-01-20 19:29:06.430571800 -0500 @@ -3,7 +3,10 @@ ( clock, reset, - + + // frame complete flag + complete, + // Avalon-MM master interface master_av_clock, master_av_reset, @@ -70,6 +73,9 @@ input clock; input reset; +// frame complete flag +output complete; + // Avalon-MM master interface input master_av_clock; input master_av_reset; --- alt_vipvfr91_vfr.v 2009-10-22 07:53:50.000000000 -0400 +++ c:/temp/alt_vipvfr91_vfr.v 2010-01-20 19:29:01.785306100 -0500 @@ -121,6 +121,8 @@ prc( .clock(clock), .reset(reset), + + .complete(interrupts), //wire the master straight through, there's nothing to be done to that .master_av_clock(master_av_clock),

[Altera_Forum]

Ok, that is interesting. Indeed, there is no Interrupt connection from the cpu to the frame reader. How did you solve the issue that the "slave_av_irq" is not synthesized? 

 I have made the changes in the according source files in my quartus installation.

[Altera_Forum]

Changing the files in the Qartus folder and recompiling should work.  

 Perhaps you need to delete the db folder before recompiling?  Are you maybe using the MegaWizard to instantiate the frame reader? I found with the MegaWizard Quartus will copy the vfr files from the Quartus folder to your project folder. In this case you need to make sure the files in your project directory are also modified. When using SOPC builder, it will just reference the Quartus folder instead.

[Altera_Forum]

Here an update of my issue: I have tested your patch, kevin (only with service pack 1, which came out this week). The interrupt seems to fire now. Thank you for sharing this. 

 But still I do not get a valid picture (now the frame reader is ignored by the alpha blending mixer, nothing is shown). Did you do more changings to make the frame reader work? I will test the system by SignalTabs as soon as I am at work tomorrow morning and will give some more feedback.

[Altera_Forum]

The only change I made to the frame reader was the interrupt patch I posted. I installed 9.1 SP1 today and was a bit surprised to see they did not fix the interrupt issue. 

 Are you still trying to do interlaced video?   If yes, then the interlaced value you specify for the control packet would have to alternate every field. One tricky thing I ran into with the interrupt is that immediately after the interrupt fires, the frame reader will start the next frame, if it is still enabled, even before you service the interrupt and clear the flag. This makes it impossible to switch frames cleanly.  The solution is to clear the enable bit immediately after you set it. The frame read will finish the current frame even when you clear the enable bit. Then, when the interrupt fires, you can switch the frame setup cleanly. Reset the interrupt flag before momentarily re-enabling the frame reader.  To clear the interrupt: IOWR(ALT_VIP_VFR_0_BASE, 2, 2);  To kick off the next (or first) frame: IOWR(ALT_VIP_VFR_0_BASE, 0, 3); IOWR(ALT_VIP_VFR_0_BASE, 0, 2);

[Altera_Forum]

I put the frequency of the onchip memory to 27 MHz and changed my interrupt service routine to the recomendations considering to clear the enable bit before sending out the next frame (thank you kevin). Now the frame reader can read data from the onchip memory and stream valid avalon video and data packets to the alpha blending mixer. 

 Altera has following workaround for the issue with the interrupt which is not connected, which I have tested successfully with the frame reader design reading data from onchip memory:   to fix the interrupt connectivity issue, you have to change the file "c:\altera\91 \ip\altera\frame_reader\src_hdl\alt_vipvfr91_vfr.v" on line 279  From: .interrupts (interrupts), To: .interrupts (irq_FROM_controller_TO_slave),    In Quartus II service pack 2 the issue will be solved.   I have also a design in which the onchip memory is replaced by a DDR SDRAM and instead of reading from the onchip memory it reads its data from the DDR SDRAM. However, there seem to be bandwith problems yet, since I can show a valid picture, with a limited width only.

[Altera_Forum]

Guys, 

 Your posts are very useful because the frame_reader is very problematic w/o informations.  Though I still have one problem:  Switching off/on the layer of graphics at Alpha Blending it flickers - randomly. The reason is that this core (the Alpha Belnding) does not care about field syncronization. As Altera states that, btw. So some external logic will be necessary to avoid somehow this issue.

[Altera_Forum]

Hello! I was really happy to find this thread, because I also have problems to configure the FrameReader. Unfortunately I can't adapt your comments for my case. It would be really nice if you help me. 

 I am trying to output Data from an onchip_memory (32bit) to a clocked_video_output in 1024x768. I have written "85" into all memory blocks. Now while reading I am using the following instanciation for the FreamReader:   // Frame Select IOWR(ALT_VIP_VFR_0_BASE, 3, 0); // Frame 0 Base Address IOWR(ALT_VIP_VFR_0_BASE, 4, ONCHIP_MEMORY2_1_BASE); // Frame 0 Words IOWR(ALT_VIP_VFR_0_BASE, 5, 196608); //1024*768*1/(32/8) // Frame 0 Single Cycle Color Patterns IOWR(ALT_VIP_VFR_0_BASE, 6, 2359296); //1024*768*1Cycle // Frame 0 Reserved // IOWR(FRAMEREADER_BASE, 7, 0); // reserved for future use ? // Frame 0 Width IOWR(ALT_VIP_VFR_0_BASE, 8, 1024); // Frame 0 Height IOWR(ALT_VIP_VFR_0_BASE, 9, 768); // Frame 0 Interlaced IOWR(ALT_VIP_VFR_0_BASE, 10, 3); // See VIP Processing Suite - Table 4-4 // start Frame Reader IOWR(ALT_VIP_VFR_0_BASE, 0, 1); The goal later will be to write pixel data to the ram from different sources and then connect them from the ram to one tft, so that I can prebuilt the image in ram and pixelselect the sources. Therefore I want to use RGB in progressive and 1024x768.  My problems are the calculation of Words, Single Cycle Color Patterns and the right setting for Interlaced. It would be really nice if you could explain to me how I find out the number of cycles I need.  Thanks, Peter.

[Altera_Forum]

I really don't get it working. I was thinking that I have got the following conditions: 

 8bits per pixel per color plane 3 color planes in parallel Master port width 64Bit onchip_ram with 32bit width width 1024 height 768  SingleCycleColorPattern => 1024x768 = 786432 (like p. 5-28 right bottom) Words => width of a word is 64bit, one SingleCycleColorPattern has 24bit so i can get 2 SingleCycleColorPatterns in every word. => 786432 / 2 = 393216  Progressive Output means 0011 for 4bit InterlacingParameters (see Table 4-4) this means 3 (=>0011).  So now I tried:  // Frame Select IOWR(ALT_VIP_VFR_0_BASE, 3, 0); // Frame 0 Base Address IOWR(ALT_VIP_VFR_0_BASE, 4, ONCHIP_MEMORY2_1_BASE); // Frame 0 Words IOWR(ALT_VIP_VFR_0_BASE, 5, 393216); // Frame 0 Single Cycle Color Patterns IOWR(ALT_VIP_VFR_0_BASE, 6, 786432); // Frame 0 Reserved // IOWR(ALT_VIP_VFR_0_BASE, 7, 0); // Frame 0 Width IOWR(ALT_VIP_VFR_0_BASE, 8, 1024); // Frame 0 Height IOWR(ALT_VIP_VFR_0_BASE, 9, 768); // Frame 0 Interlaced IOWR(ALT_VIP_VFR_0_BASE, 10, 3); // start Frame Reader IOWR(ALT_VIP_VFR_0_BASE, 0, 1);  But it seems wrong. Does not work. Can anyone approve my calculations or has got a demo or example configuration? PLEASE.

[Altera_Forum]

Ok I have got it. I changed all MemoryMaster Width to 32Bit and then calculated: 

  // Frame 0 Single Cycle Color Patterns IOWR(ALT_VIP_VFR_0_BASE, 6, 786432); //1024x768 = 786432 (like p. 5-28 right bottom) // Frame 0 Single Cycle Color Patterns IOWR(ALT_VIP_VFR_0_BASE, 5, 786432); //width of a word is 32bit, one SingleCycleColorPattern has 24bit so i can get 1 complete SingleCycleColorPatterns in every word. => 786432 / 1 = 786432

[Altera_Forum]

your post is very usful. 

I use the frame reader in my design with the following settings: Bits per pixel per color plane: 8 Number of color planes in parallel: 3 Number of color planes in sequence: 1 Maximum Image width: 1920 Maximum Image height: 1080 Master port width: 64 Read master FIFO depth: 1024 Read master FIFO burst target: 64 Use separate clock for the Avalon MM master interface: unchecked  Kindly confirm if the following calculations for number the words and the Single Cycle Color Patterns are true or not 1- // Words IOWR(FRAMEREADER_BASE, 5, 259200); // (1920*1080*8*3)/48= 1036800  2- // Single Cycle Color Patterns IOWR(FRAMEREADER_BASE, 6, 2073600); // 1920*1080=2073600 number of pixels in the frame    I apreciate your reply ASAP