Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

Difference in h264 stream generated by IMSDK HW encoder and SW encoder

Ramashankar
New Contributor III
601 Views

Hi,

I am using sample_encode (version 6.0.0.49) application as reference and generating h264 stream by using HW encoder and SW encoder both. My source code, encoder parameters and test machine (win 8.1) all are same in both the cases. I am using another test machine which is running on Linux and having a h264 HW decoder using GStreamer framework and VAAPI, for decoding those encoded streams.

Now my issue is that the hw decoder on linux machine is able to decode sw generated h264 stream properly, but in case of hw generated stream it is giving corrupt output frame. When i am analysing both the encoded stream using a h264 stream analyser software, I am finding some differences, like, presence & frequency of pps packet, values of VUI parameters in sps packet etc. There could be some more differences which I am unable to recognise.

So I would like to know that:

1. Why there are differences in both stream when I am providing exactly same parameters in both cases?

2. What all parameters may differ in both cases and how I can control them?

3. How can I get stream with similar property from hw also as in sw?

~

Thanks,

0 Kudos
10 Replies
Ramashankar
New Contributor III
601 Views

Just to add further information for the reference:

- Media SDK version being used is: 6.0.0.349

- attaching system analyzer log also for my test machine (Win 8.1).

0 Kudos
Jiandong_Z_Intel
Employee
601 Views

Hi Ramashankar,

It looks like your MSDK version is not the most new version. Can you download and setup from Intel® Media SDK (https://software.intel.com/en-us/media-sdk) ? 

And, can you test again with the most new Intel® Media SDK ?

Thanks

Zachary

 

0 Kudos
Ramashankar
New Contributor III
601 Views

Hi Zachary,

I downloaded and setup the latest media sdk (Intel(R)_Media_SDK_2016.0.1) and latest samples (Intel Media Samples 6.0.0.68). I used the sample_encode module as it is (not a single line changed by me) and generated the h264 encoded stream by using HW Impl and SW impl both and by using same yuv420 dump. The stream difference is still there and its same as I mentioned earlier.

Some key differences are:

1. In SW, I/IDR frame is generated only once, that is the very first frame, then it was never repeated, while in HW stream this frame is getting repeated at some regular interval.

2. In SW, PPS is generated before every frame, but in HW it is also only once.

There are other differences also like SW: bitstream_restriction_flag is 0, cpb_size_scale = 2, while in HW: bitstream_restriction_flag is 1 cpb_size_scale = 3 etc.

Well, as i mentioned, my problem is that my decoder on linux machine (using VAAPI) is able to decode SW generated stream properly but corrupting with HW generated stream. I think probably when decoder encounters next I/IDR frame its output starts corrupting. So I was trying to simulate same I/IDR setting for HW stream as in SW, but unable to do it. I am able to change PPS and bitstream_restriction_flag setting in HW (using some extended header params), but unable to change cpb_size_scale and IDR frame setting.

While on other hand, i feel that there should be some fix/changes required at decoder side also coz the HW generated stream is successfully decode on my windows machine (using intel media SDK HW decoder impl).

So can you guide me further on following two things:

1. How to generate same h264 stream from HW as in SW?

2. What could be the potential issue in Linux side decoder?

~ Thanks.

0 Kudos
Ramashankar
New Contributor III
601 Views

Hi,

just to add further information based on my today's investigation:

I observed that my stream on Linux side h264 decoder is getting corrupted whenever it encounters an IDR frame (except the very first IDR). It is not corrupting on I-frame, but on IDR frame only.

As my SW encoder generated h264 stream contains one and only one IDR frame but HW encoder generated stream contains several IDR that is why it is getting corrupted. Just to confirm it further, if i delete and re-create decoder on linux machine everytime i receive an IDR frame then decoding is proper. 

So can you guide me further on following two things:

1. How to generate one and only IDR using HW encoder also?, OR

2. What could be the potential issue in Linux side decoder and how it can be fixed?

~Thanks,

0 Kudos
Jiandong_Z_Intel
Employee
601 Views

Hi Ramashankar,

About your HW encode, have you tried the parameter mfxInfoMFX.IdrInterval and  mfxExtCodingOption2.RepeatPPS ? you can find them in mediasdk-man.pdf

For Linux side, I notice you are using "GStreamer framework and VAAPI", which backend of VAAPI be used ?

what is the output of following command line ?

$ vainfo 

Can you decode the h264 stream via MSDK sample_decode_drm ?

 

Thanks,

Zachary

 

0 Kudos
Ramashankar
New Contributor III
601 Views

Hi Zachary,

About your HW encode, have you tried the parameter mfxInfoMFX.IdrInterval and  mfxExtCodingOption2.RepeatPPS ?

Yes i had tried both of above. I am able to stop PPS through above parameter. I can configure IDR interval to generate IDR frame after every Nth I-Frame. But I want to stop the IDR frame generation at all after the very first time.

If I set IDRInterval to 50, it delays the IDR generation till 50th I frame (duration wise it was around 12 minutes stream when i did not set any other GOP parameter). Corruption did not occur until 12 minute on decoder side. So should I set it to some very big valid no like any INFINITE value, in order to stop it permanently?  

Also, what parameter/setting is being passed in SW encoder due to which it is not repeating IDR? Note: By default I am not setting any IDRInterval or other parameter explicitly for SW or HW encoder pipeline.

what is the output of following command line ? $ vainfo 

Please find the output of this command in attachment.

Can you decode the h264 stream via MSDK sample_decode_drm

I am able to decode this stream using sample_player on windows machine. On Linux, I am able to decode the file dump of this stream using GST-Launch.

Thanks,

 

0 Kudos
Jiandong_Z_Intel
Employee
601 Views

Hi Ramashankar,

How about set mfxInfoMFX.IdrInterval=0, then every I-frame is an IDR-frame ?

For Linux side you are using Intel OTC driver from following output of vainfo

vainfo: Driver version: Intel i965 driver - 1.2.2
 

The OTC driver is from 01.org. Maybe following URL can help you :

https://01.org/linuxgraphics/forum

 

Thanks,

Zachary

0 Kudos
Ramashankar
New Contributor III
601 Views

Hi Zachary,

Yes, setting mfxInfoMFX.IdrInterval to 0 makes all I-frame as IDR frame. So it is not what I am looking now.

Well, i debugged further using tracer.exe and making a call to MFXVideoENCODE_GetVideoParam(), then I found that by default HW encoder impl is setting IdrInterval=0 and GopPicSize=256, while SW encoder impl is setting IdrInterval=32767 and GopPicSize=32767. That means SW encoder may repeat IDR after 32767x32767 frames which is around 414 days stream @30 fps.

So I think by setting this value [IdrInterval=32767 and GopPicSize=32767] explicitly in HW encoder also, my current issue will get resolved for the time being (though its not perfect solution, but will work well in current scenario).

What is your opinion on this?

Thanks,

 

0 Kudos
Ramashankar
New Contributor III
601 Views

Hi Zachary,

So I think by setting this value [IdrInterval=32767 and GopPicSize=32767] explicitly in HW encoder also, my current issue will get resolved for the time being (though its not perfect solution, but will work well in current scenario).

What is your opinion on this?

Can you please share your input/suggestion on this? What could be the side effect of this, if any?

As my decoder will always get the stream from encoder side from very first frame itself, so there will be no case of missing the first IDR frame and SPS PPS.

Thanks,

0 Kudos
Jiandong_Z_Intel
Employee
601 Views

Hi Ramashankar,

Sorry for my late response.

I think you can try  [IdrInterval=32767 and GopPicSize=32767],  because your gop size is big, you may got only one gop if the frame is not too many, so you can got the sequence that you want.

But I don't think big gop is always good, and we need balance video size, quality, calculation.

 

Thanks,

Zachary

 

0 Kudos
Reply