Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

Intel Media SDK Video SyncOperation MFX_WRN_IN_EXECUTION

Ashim_Prasad
Beginner
1,470 Views

After running decoder and VPP (For color conversion and Scaling), SyncOperation for Color conversion starts Failing with MFX_WRN_IN_EXECUTION. I had a timeout of 200 ms, which I increased to 1000 ms, but still same result.

Since the application is a real-time application, can't wait and need to drop frame in case it can't be processed. How to handle MFX_WRN_IN_EXECUTION? How to to flush the buffer that's queued for processing, because after sometime application runs out of allocated buffers.

OS: Ubuntu

Media Server Studio 2017 is used.

0 Kudos
7 Replies
Dmitry_E_Intel
Employee
1,470 Views

There is no way to flush already queued tasks. Application is expected call SyncOperation until it successfully returns MFX_ERR_NONE.

200 ms is completely unexpected. With HW MSDK implementation color conversion with scaling for one frame usually takes up to 10 ms. 

A few questions to better understand your case:

Do you use HW or SW MSDK implementation ? Which decoder?  What are input and output VPP color formats ?  Resolution? 

What is AsyncDepth value used in initialization of both decoder and VPP? Do decoder and VPP belong to the same MSDK session? 

What is the memory type between decoder and VPP and from VPP: system, video, opaque? 

If you call SyncOperation after both decoder and VPP, where do you see the glitch? In decoder or VPP?

If you set the timeout e.g. in 10 sec, when SyncOperation returns control? 

Can you reproduce a similar issue with sample_decode? sample_decode has VPP support for scaling.

Regards,

Dmitry

0 Kudos
Ashim_Prasad
Beginner
1,470 Views

One correction it's VPP and Encoder and not Decoder/VPP

To Answer your questions

Overall application is following

Capture from camera 1920x1080 @ 30 fps using V4L2 APIs in UYVY format. One session is created to convert UYVY to BGRA. In and out memory are video memory. Then BGRA frame is mapped to CPU space and some pre-processing over it is done (like drawing, text, various cv analysis etc.). After this another media SDK session is given to convert from BGRA to NV12 and Encode to h.264 (This color-conv and encode happens in same session).

Both the sessions are using HW implementation.

Session 1: Color Conversion from UYVY to BGRA - uses AsyncDepth of 65

Session 2: Color Conversion from BGRA to NV12 and Encoder uses AsyncDepth of 5

SyncOperation fails for Color Conversion of Session 1 - which is doing only UYVY to BGRA. This fails after sometime around 40-60 minutes of operation (Roughly after application has processed about 70000 to 90000 frames).

Not able to reproduce it on sample code.

0 Kudos
Dmitry_E_Intel
Employee
1,470 Views

One recommendation: if your app does live-streaming, it's better  to use AsyncDepth=1 because bigger AsyncDepth values may improve performance but also will increase latency. 

Now I understand you pipeline much better. However to tell you the truth it's very hard make any conclusion without a real reproducer.  The fact that the issue happens after 40-60 minutes makes me think that it might a low level issue (e.g. in KMD). 

Therefore a few more questions: 

on Ubuntu, do you have 4.4 kernel with patches from MSS package? 

When you say that you use V4L2 and send in VPP1 surfaces in video memory, do you mean that that create vaSurfaceID's from V4L2 buffers (without memory copy) or you lock/unlock VPP input video memory and manually load data from camera?

Do you join session1 and session2 via MFXJoinSession() call? 

 

 

0 Kudos
Ashim_Prasad
Beginner
1,470 Views

It's 4.4 Kernel with kernel patches applied which came along with media server package.

Currently V4L2 buffer is copied to a vaSurface. Though I was looking for a way to avoid this copy - Do you have any suggestion on this as well (Any sample would help).

Session 1 and Session 2 are not joined. They are used independent of each other.

0 Kudos
Ashim_Prasad
Beginner
1,470 Views

There was a typo - AsyncDepth is 5 and not 65 for session 1.

>> Session 1: Color Conversion from UYVY to BGRA - uses AsyncDepth of 65

Few more observations:

1. I changed AsyncDepth to 1 for both the sessions. Now this error comes after 20+ hrs.

2. If I capture at 10 FPS with 1280x720 resolution - This error didn't occur for more than 48+ hrs. So possibly issue happens after number of frames being processed.

0 Kudos
ray_c_
Beginner
1,470 Views

Hi Ashim Prasad

0 Kudos
Ashim_Prasad
Beginner
1,470 Views

Hi Ray,

- Reducing AsyncDepth to 1 helps and makes it run for longer

- Also I moved to yocto from ubuntu and had made it run much longer (More than 48+ hrs with Full HD @ 30 FPS). Looks like the kernel patches for generic kernel possibly have some issues. In intel's yocto build these patches are already included.

0 Kudos
Reply