VIP handling of Interlaced video

[Altera_Forum]

I am struggling with the processing of interlaced NTSC video using the VIP 9.0 cores, and hope someone here can give me some advice. 

 My project needs to take two NTSC interlaced inputs from ADV7180 decoders, do some processing, and then send the output to a single ADV7171 encoder. I am using an alpha mixer on the output stage of my chain of VIP components, and while I can send a test pattern such as color bars (generated with the VIP test pattern component) through the alpha mixer, none of my interlaced source video comes through.  The test pattern is setup to generate interlaced 4:4:4 output, and I am sending the NTSC data to the mixer in that format too. I am starting to suspect issues with interlaced video in general with the mixer. Maybe someone can shed some light on it for me.  I have another project that works fine doign the same thing, with the difference being that I deinterlace the NTSC input video, and output to a progressive display. That works. In this case I need to output interlaced video though, and as there is no VIP "Interlacer" component, it seems that I have to keep the data in interlaced form throughout my VIP flow.  The current VIP chain looks something like this:  For both NTSC inputs:  CVI (8-bit)->CPR (16-bit)->Clipper (to 243 lines)->CRS (to 4:4:4)->Frame Buffer  Each of the two frame buffers, along with a test pattern generator, connect to an Alpha Blending Mixer component. The test pattern generator is set to produce color bars, interlaced, in 4:4:4 format.  The chain from the Alpha Blending Mixer to the output looks like this:  Mixer->CRS (back to 4:2:2)->Frame Buffer->CPR (8-bit)->CVO  The only thing I can get to come through is the background layer - the color bars.  Any advice is appreciated!  Jim

[Altera_Forum]

Yeah I'd be suspect of how the mixer is going to handle it. The video IP user guide indicates that the mixer will accept interlaced data but with the following disclaimer: 

   --- Quote Start ---  Interlaced/Progressive: Progressive. Interlaced data is accepted but the offset from the top edge for each foreground layer has to be adapted in consequence.  --- Quote End ---    Here is what I would do in order of increasing time commitment. 1 - double-check that you are correctly enabling the individual foreground channels (I assume you are using a NIOS for this). 2 - I would try a test pattern other than colorbars for the background. Something with some horizontal content. 3 - simplify the design so that you only have one video input for the sake of test. Stay with this until you get it working. 4 - Signaltap the inputs to the mixer. 5 - Simulate the design.  Jake

[Altera_Forum]

Jakob, 

 Thanks for the quick reply, and I am already somewhat down that path.  I don't have a NIOS in the design at the moment, although it is planned. Right now I am using a JTAG master to run TCL scripts to write to the control registers on the mixer.  When processing interlaced video, am I correct in thinking that all the processing is done on a FIELD basis (F0 or F1) rather than a frame basis, unless I deinterlace the video?  Do you know of any way to produce interlaced video using the VIP tools, once it is in progressive format in a frame buffer?   Thanks!  Jim

[Altera_Forum]

Yes you are correct in assuming that every field would be processed as if it were a frame. However, I've never run interlaced data through the mixer and don't know the behavior. 

 I believe Altera has a simple interlacer block which I think is due to be released with 9.1. However, if you ping the right guys maybe they can give you a pre-release version. I've written my own simple interlacer but all it does is drop every other line. I'm sure you could write something similar. If you do so, you'll have to process everything in progressive format and interlace at the end. I haven't had the need to write a true interlacer (with the frame buffer) yet but I will shortly.  Since you are using a JTAG master, I'd double-check to make sure those registers are getting set right. If it were me I'd throw in a NIOS really quick and some on-chip memory.  Jake

[Altera_Forum]

Any idea who I contact at Altera? ;-) I would be happy to beta test a new Interlacer core! ;-) I've got an immediate need for it, and am under maintenance on all my software at the moment - just paid for it about 2 months ago. 

 Thanks.  Jim  P.S. I just contacted the local FAE I deal with, and he is going to rattle the tree at Altera and see what he can find out for me....

[Altera_Forum]

Hi, 

 Your FAE should be asking for the reference design called V2. This reference design contains a beta version of the interlacer although this is not a full interlacer and it just drops every other line.  When processing interlaced video, the mixer, like many VIP cores, will do the processing on a field basis. In order to do the mixing, you probably will have to ensure that F0s are properly synchronized together because the mixer will just use whatever is coming in. The disclaimer note on the VIP guide means that when specifying (x,y) coordinates for the top-left pixels you have to divide y by 2.  The double buffer supports interlaced video but the triple buffer does not. If you turn on the drop or repeat checkboxes you may end up with a stream that contains successive F0s or successive F1s.  vgs

[Altera_Forum]

VGS, 

 Thanks, and your explanation of how the VIP handles interlaced video explains a lot. I've been selecting triple buffers, and have issues if I send more than one video source into the frame buffer. I'll try to process with double buffers and see if it makes a difference.  My last test on Friday involved going back to progressive video by using 2 deinterlacers, and that works, but of course the output is all over the place vertically on my monitor, which is expecting interlaced NTSC, not progressive.  I'll be talking to the FAE this morning about access to that core. Thanks again.  Jim

[Altera_Forum]

jfmorris, I'm very interested by the interlacer core too! 

Please let me know, by updating this post, if your FAE is able to give you the beta version of the interlacer core! Thanks

[Altera_Forum]

The Altera DSP specialist will be in my office later today. He's the one who acts as the front-end for the VIP. I'll ask him about it. 

 Jake

[Altera_Forum]

I have the V2 reference design, which includes new Beta versions of the following cores: 

  Clocked Video In  Clocked Video Out  Frame Buffer  Deinterlacer  Interlacer  All of them have various modifications to properly handle interlaced video. Unfortuntely, with Quartus 9.0 sp1, I cannot compile either the reference V2 design, or my own design. I bomb out with the following error (twice) when the VFB (Frame Buffer) is being elaborated:  Error: IP Generator Error: Variable msg_field_interlace_4 does not exist in the variable life table  At that point Quartus II gives up and we are done.  Unfortunately I don't think the regular frame buffer will work, as the beta one has a new checkbox for enabling "Interlaced video compatibility".  Jim

[Altera_Forum]

Did the reference design include some accompanying patches for the VIP cores? My understanding was that it would. 

 Jake

[Altera_Forum]

Ok, good catch. There is no documentation on it, but I found a patch installer buried in the folder. Installing it over my Quartus II now. Will let you know if that helps matters.... 

 Jim

[Altera_Forum]

Hi all! 

 Did anyone get the new Interlacer from Altera? Does it work? Jake, what is the downside of just dropping every other line to interlace?  Thank you!  Thiago

[Altera_Forum]

It's not necessarily a downside, it just depends on what type of interlacing you want to perform. Dropping every other line is more true to the way classic systems interlaced the video. Remember interlacing was created because video capture and display devices were not fast enough to perform at 60 progressive frames per second. 

The other alternative is to actually de-weave the data which requires a frame buffer. In this case you read out every other line, then go back and read the unread lines from the same frame. Interlacing using this method has some benefits. Primarily, if it is de-interlaced at the receiver using a weave algorithm, you get the original image back exactly. Also, it means that your progressive frame rate (ahead of the interlacer) does not have to be 2X the output frame rate. This can put less pressure on components ahead of the interlacer.  Jake

[Altera_Forum]

interlacing was created a lot earlier than video capture devices. ;) 

 i think it was created to compromise between flicker and bandwidth.

[Altera_Forum]

I was referring to cameras. A camera is a video capture device.  

 Early equipment could not transmit 60 frames per second. However, they could have chosen to transmit 30 frames per second rather than 60 fields per second and still used the same amount of bandwidth. With interlaced video, each line still only updates 30 times per second but your brain gets new video data 60 times per second. This results in a better perception of motion by the viewer. You gain better resolution in the time domain but sacrifice resolution in the image quality.  For instance, supposing you are watching NASCAR and the cars are running at 180mph (290kph) which translates to 264 feet per second (80 meters per second). So if you were watching progressive video at 30fps, you would see the car move 8.8 feet (2.7 meters) between each frame. With interlaced video at 60 fields per second, you see the car move 4.4 feet (1.35 meters) between fields. The motion appears smoother and less jerky. The tradeoff however is the quality of the image degrades because you're only getting half the picture each time. However, your brain tends to notice motion problems more than quality problems.  Jake