MFX/MSDK and OpenCL interop on Linux ?

[Robby_S]

Hi, I am planning to optimize one of my modules with OpenCL.

I am using CentOS 7.1 and MSDK 2015R6. I looked up the example/sample codes provided by Intel. Two examples that deal with MSDK and OpenCL interop are all for Windows, as they both depend on the function clEnqueueAcquireDX9MediaSurfacesKHR() to use media surfaces as OpenCL memory objects, and that function only works with the DX9 adapter API.

Two other examples (motion estimation) that do work on Linux appear to be just OpenCL. I don't see MSDK and OpenCL interop in them.

Is MFX/MSDK and OpenCL interop currently supported on Linux ?

Thanks,

Robby

[Jiandong_Z_Intel]

Hi Robby,

For run OpenCL sample on Linux. You can try following command line.

$ ./sample_encode_drm h264  -hw -i input.yuv -o output.h264 -w 720 -h 480 -angle 180 -opencl

or

$ ./sample_multi_transcode_drm -hw -i::h264 input.h264 -o::h264 output.h264  -angle 180 -opencl

Thanks,

Zachary

[Robby_S]

Thanks Zachary. It looks like MSDK - OpenCL interop is supported on CentOS, though I have been unable to run the samples. pVPPPlugin->LoadDLL is giving me errors although both libsample_rotate_plugin.so and libsample_plugin_opencl.so are in the same directory. That's not important yet. Don't worry about it for now.

The real issue is, these two examples need the library sample_plugin_opencl to work. The source code does not show exactly how MSDK and OpenCL inter-operate. What I really need is an example like ocl_media_sdk_interop for the Windows version, which shows how my OpenCL code can get hold of the frame surface buffer from MSDK, manipulate the frame buffer, and return it to MSDK.

Do we have something like that for CentOS? Or maybe the source code for libsample_plugin_opencl.so (the source code may or may not be exactly what I am looking for)?

Thanks,

Robby

[Jiandong_Z_Intel]

Hi Robby,

You can refer to the source code of libsample_plugin_opencl.so, under MSDK samples\sample_user_modules\rotate_opencl folder.

OpenCLFilterVA::InitSurfaceSharingExtension

OpenCLFilterVA::PrepareSharedSurfaces

OpenCLFilterVA::ProcessSurface

Thanks

Zachary

[Robby_S]

Hi Zachary, 

Thanks for pointing me to the source code.

One more question: Is it possible to use opaque memory when sharing a frame surface between MFX/MSDK and OpenCL? I am doubtful, but would like to get a confirmation.

Thanks again,

Robby

[Jiandong_Z_Intel]

Hi Robby,

Thanks for you asked this question.

First, we should understand opaque memory, following can be found in page 30 in <MSDK root>/doc/mediasdk-man.pdf.

Since the SDK manages the association of opaque surface to “real” surface types internally, the
application cannot read the content of opaque surfaces. Also the application does not get any
opaque-type surface allocation requests if the application specifies an external frame allocator.

Then, we can found following in page 11 in <MSDK root>/doc/mediasdkusr-man.pdf

Accessing Opaque Surfaces
If plug-in function works with opaque surfaces at input/output, the function needs to retrieve
the corresponding native surface by calling the GetRealSurface function. Then this real surface
can be used as usual. For example, to get access to surface data plug-in function should call
Lock function from FrameAllocator exposed by core interface.

So, Opaque memory can be used as usual in MSDK plugin. and can't be read content in application. I think the answer is Yes, but It only available in MSDK plugin, and usually external frame allocator can be better control the surface.

Thanks,

Zachary

[Robby_S]

Thanks Zachary for the explanation. That does not seem to be an easy task. Is there any example code demonstrating how it can be done?

-Robby

[Jiandong_Z_Intel]

Hi Robby,

Actually I want to suggest you change opaque memory to external frame allocator. 

no need create a MSDK plugin for read content of opaque memory  if the application specifies an external frame allocator.

In MSDK sample code vaapiFrameAllocator(<MSDK samples>\sample_common\src\vaapi_allocator.cpp) is an external frame allocator.

mfxStatus CDecodingPipeline::CreateAllocator() {
        .............

        // create VAAPI allocator
        m_pMFXAllocator = new vaapiFrameAllocator;
        MSDK_CHECK_POINTER(m_pMFXAllocator, MFX_ERR_MEMORY_ALLOC);
        vaapiAllocatorParams *p_vaapiAllocParams = new vaapiAllocatorParams;
        MSDK_CHECK_POINTER(p_vaapiAllocParams, MFX_ERR_MEMORY_ALLOC);

        p_vaapiAllocParams->m_dpy = va_dpy;
        m_pmfxAllocatorParams = p_vaapiAllocParams;

        /* In case of video memory we must provide MediaSDK with external allocator
        thus we demonstrate "external allocator" usage model.
        Call SetAllocator to pass allocator to mediasdk */
        sts = m_mfxSession.SetFrameAllocator(m_pMFXAllocator);
        MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

        ............
}

Thanks,

Zachary

[Robby_S]

Hi Zachary,

Correct me if I am wrong, but my understanding is that the advantage of the opaque memory is that an application developer does not need to worry about frame allocators.

Per our discussions so far, I see little point in trying to get MSDK and OpenCL to inter-operate using the opaque memory. In my case anyway, I need MSDK and OpenCL inter-operation so as to run OpenCL on the GPU. It probably makes more sense to simply use the video memory instead. Am I right ?

(Actually I suspect that's the most common user scenarios.)

Thanks,

Robby

[Jiandong_Z_Intel · Accepted solution]

Hi Robby,

Yes, opaque has it's advantage.  but it also has shortcoming, for better control  and extend for software. my personal point is external frame allocator can bring more benefit.  Actually I thinking we are discussing the software designation, we always balance on software designation, more powerful or more simply. Sometimes it depends on software complexion and  it is hard to make a decision. 

Thanks

Zachary