Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

howto copy opaque surface to real surface

koby_s_
Beginner
505 Views

Hi !

I am looking for a simple way to copy opaque surface to a real surface (and save the output to a raw (n/yv12) file).

The sdk docs says (under "Surface Type Neutral Transcoding" section) : "It is possible to copy an opaque surface to a “real” surface through a VPP operation."

Is there any special configuration that I need to configure the VPP?

Should I create the VPP out surfaces using my frame allocator ?

I found this : https://software.intel.com/en-us/node/562475?wapkw=opaque  but it seems to be relevant only for user plugins.

Right now i configured my app to work exactly as the sdk doc describes (DECODE -> Opaque Surfaces -> VPP -> Opaque Surfaces -> ENCODE)  but the output video is jumpy and not smooth and I am trying to isolate the problem.

 

Thank you

koby

0 Kudos
6 Replies
Jiandong_Z_Intel
Employee
505 Views

Hi koby,

Maybe you can refer to https://software.intel.com/en-us/forums/intel-media-sdk/topic/600634

You are right, Opaque memory can be used as usual in MSDK plugin. and can't be read content in application.

 

Thanks,

Zachary

 

0 Kudos
koby_s_
Beginner
505 Views

Hi Zachary,
Thank you for your answer but it didn't help much.

I am not trying to read the content of the opaque surface.
I am trying to copy its content to a "real" surface as the reference manual says ("It is possible to copy an opaque surface to a “real” surface through a
VPP operation", reference manual, page 29)

I am trying to: decode -> opaque surface -> VPP (resize) -> D3D surface -> save as YV12  file.

The decoder is using opaque surfaces as output.
The VPP component was initialized in a different session that has it's own allocator (D3D9) and also has mfxExtOpaqueSurfaceAlloc as one of its external parameters.

The VPP's mfxExtOpaqueSurfaceAlloc.In was copied from the Decoder's mfxExtOpaqueSurfaceAlloc.Out and the VPP's mfxExtOpaqueSurfaceAlloc.Out is empty (I zeroed the structure with memset ).

I am allocating frames using the external allocator as the VPP->QueryIOSurf responds and it seems that calling to VPP->RunFrameVPPAsync works fine (i am getting MFX_ERR_NONE).
BUT - When I am trying to call SyncOperation on the output sync point I am getting MFX_ERR_UNSUPPORTED.
I tried to call SyncOperation on the local mfxVideoSession and on the parent mfxVideoSession. both failed and returned MFX_ERR_UNSUPPORTED.

To conclude:
1. I have 2 sessions. 
2. decoder session that uses opaque surfaces as output
3. vpp session that use uses opaque surfaces as input(the decoder output) and D3D9 surfaces as output with a D3DFrameAllocator
4. The VPP session (child) is joining the Decoding session (parent)
5. VPP_FrameAsync return MFX_ERR_NONE
6. SyncOperation after the VPP_FrameAsync return MFX_ERR_UNSUPPORTED

What should I do in order to copy the content from an opaque surface in to a real surface ?

Thank you
koby

0 Kudos
Jiandong_Z_Intel
Employee
505 Views

Hi Koby,

From you pipeline "decode -> opaque surface -> VPP (resize) -> D3D surface -> save as YV12  file",  can you confirm "D3D surface" be allocated via  "mfxExtOpaqueSurfaceAlloc" or "D3DFrameAllocator" ? I think it should D3DFrameAllocator because this surface will be read content.

I don't know if your pipeline can work without using "opaque surface" (use D3D surface instead opaque surface).

BTW, Which OS are you using ? Windows or Linux ? it looks like Windows, because D3D surface is used.

Can you share log from system_analyzer tool located at (mediasdk/tools/analyzer)? This will help provide us your system details. 

 

Can you check your code with following in page 30 in mediasdk-man.pdf ?

If the application shares opaque surfaces among different SDK sessions, the application must
join the sessions before SDK component initialization and ensure that all joined sessions have
the same hardware acceleration device handle. Setting device handle is optional only if all
components in pipeline belong to the same session.

 

Thanks

Zachary

0 Kudos
koby_s_
Beginner
505 Views

Hi Zachary

1. D3D Surfaces are being allocated using GeneralAllocator(which holds D3DFrameAllocator).
   I can confirm (by stepping in with the debugger) that the GeneralAllocator instance is using D3DFrameAllocator.

2. My Pipeline Is working (almost) flawlessly with D3D surfaces and external allocator.
3. For developing I am using windows 10.
4. I will upload logs soon
5. I am joining 2 sessions. A decode only session + a VPP only session.
   Since I am working with live streams I must have the decoder session active, then, I am adding VPP sessions according to the user request.
   Both of the sessions are using the same HW device handle.
   I am joining the child session right after the creation, then initialize it with the requested settings.

Thank you
Koby

0 Kudos
Jiandong_Z_Intel
Employee
505 Views

Hi Koby,

From your pipeline, and the error like below.

6. SyncOperation after the VPP_FrameAsync return MFX_ERR_UNSUPPORTED

we can see, VPP (resize) with opaque surface as input , and with  D3D surface (video memory) as output.

Is my understanding correct ?

we can find following in page 29  in mediasdk-man.pdf

"With opaque surface, the SDK will map
the surface type to either system memory buffer or video memory surface at runtime."

I think only with "video memory surface" can make VPP(resize) work, because VPP output is  video memory.

And "It is possible to copy an opaque surface to a “real” surface through a VPP operation.", I think the meaning maybe is need create VPP plugin.

 

Thanks

Zachary

 

 

 

0 Kudos
koby_s_
Beginner
505 Views

Hi Zachary,

Yes, you understand correctly.

The VPP is working correctly (I can tell that by connecting encoder to the VPP output and check the encoder output).

Thank you for the clarification of how to copy (with a VPP plugin..)

The problem as I wrote in my first post is that the output video is jumpy and I am trying to find who to blame (decoder / vpp / encoder).

It looks like the decoder output is broken and the output frames are not reordered. I am still checking my code and will report back soon.

Thank you

Koby

0 Kudos
Reply