Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

H.264 hardware accelerated encoder stops working after a while

Javier_A_
Beginner
1,006 Views

Hello:

I have developed an application to encode the video coming from a video acquisition card to H.264. The video card gives high resolution images in YUV format at 50 fr/sec.
The application is using MS Media Foundation with the Intel Quick Sync Video Encoder.
My code takes the images, puts them into the Color Converter module, then into the H264 encoder and finally send them to a file.
The application runs pretty well but after some time (aleatory but usually between 30 and 90 minutes) the video encoder sends a MEError event and stops giving more output.

The computer has an i7-3610QE with an HD4000. Windows 7 SP1 64 bits.Intel Media SDK 2014 R2 for Clients.

Can someone give some help to find what is causing the error?

Thank you all

Javier Andrés

here follows the Invoke function of the Media Foundation Asynchronous process

I have also attached the C++ files

STDMETHODIMP MpegEncoder::Invoke(IMFAsyncResult* pAsyncResult)
{
    HRESULT hr;
    HRESULT hStatus;
    IMFMediaEventPtr pEvent;
    MediaEventType meType;

    try
    {
        TESTHR(hr = mpH264EncoderEventGenerator->EndGetEvent(pAsyncResult, &pEvent));
        TESTHR(hr = pEvent->GetType(&meType));
        TESTHR(hr = pEvent->GetStatus(&hStatus));
        if (hStatus == S_OK)
        {
            if (meType == METransformNeedInput)
            {
                HRESULT hr;
                BYTE *pbBuffer;
                DWORD status;
                IMFSamplePtr pYUVSample, pNV12Sample;
                IMFMediaBufferPtr pYUVBuffer, pNV12Buffer;
                MFT_OUTPUT_STREAM_INFO streaminfo;

                mTrace->Trace(1, L"New METransformNeedInput event");

                WaitForSingleObject(mEventHaveInput, INFINITE);

                if (image != nullptr)
                {
                    TESTHR(hr = MFCreateMemoryBuffer(size, &pYUVBuffer));
                    TESTHR(hr = pYUVBuffer->Lock(&pbBuffer, NULL, NULL));
                    TESTHR(hr = MFCopyImage(pbBuffer, mWidth * 2, image, mWidth * 2, mWidth * 2, mHeight));
                    TESTHR(hr = pYUVBuffer->SetCurrentLength(size));
                    TESTHR(hr = pYUVBuffer->Unlock());
                    TESTHR(hr = MFCreateSample(&pYUVSample));
                    TESTHR(hr = pYUVSample->AddBuffer(pYUVBuffer));
                    TESTHR(hr = pYUVSample->SetSampleDuration(duration));
                    TESTHR(hr = pYUVSample->SetSampleTime(timestamp));
                    TESTHR(hr = mpColorConverter->ProcessInput(0, pYUVSample, 0));

                    MFT_OUTPUT_DATA_BUFFER nv12OutputDataBuffer;
                    ZeroMemory(&nv12OutputDataBuffer, sizeof(nv12OutputDataBuffer));
                    TESTHR(hr = mpColorConverter->GetOutputStreamInfo(0, &streaminfo));
                    TESTHR(hr = MFCreateSample(&pNV12Sample));
                    TESTHR(hr = MFCreateMemoryBuffer(streaminfo.cbSize, &pNV12Buffer));
                    TESTHR(hr = pNV12Sample->AddBuffer(pNV12Buffer));
                    nv12OutputDataBuffer.pSample = pNV12Sample;
                    TESTHR(hr = mpColorConverter->ProcessOutput(0, 1, &nv12OutputDataBuffer, &status));

                    if (newFile)
                    {
                        mTrace->Trace(1, L"Set MFSampleExtension_Discontinuity");
                        TESTHR(hr = nv12OutputDataBuffer.pSample->SetUINT32(MFSampleExtension_Discontinuity, TRUE));
                        newFile = false;
                    }
                    TESTHR(hr = mpH264Encoder->ProcessInput(0, nv12OutputDataBuffer.pSample, 0));
                }
                SetEvent(mEventNeedInput);
            }
            else if (meType == METransformHaveOutput)
            {
                DWORD status;
                MFT_OUTPUT_DATA_BUFFER h264OutputDataBuffer;

                mTrace->Trace(1, L"New METransformHaveOutput event");

                ZeroMemory(&h264OutputDataBuffer, sizeof(h264OutputDataBuffer));
                hr = mpH264Encoder->ProcessOutput(0, 1, &h264OutputDataBuffer, &status);
                if (hr == MF_E_TRANSFORM_STREAM_CHANGE)
                {
                    mTrace->Trace(1, L"New MF_E_TRANSFORM_STREAM_CHANGE event");
                    if (h264OutputDataBuffer.dwStatus & MFT_OUTPUT_DATA_BUFFER_FORMAT_CHANGE)
                    {
                        mTrace->Trace(1, L"New MFT_OUTPUT_DATA_BUFFER_FORMAT_CHANGE event");
                        IMFMediaTypePtr pType;
                        TESTHR(hr = mpH264Encoder->GetOutputAvailableType(0, 0, &pType));
                        TESTHR(hr = mpH264Encoder->SetOutputType(0, pType, 0));
                    }
                }
                else if (hr == S_OK)
                {
                    if (mpWriter == NULL)
                    {
                        IMFMediaTypePtr pType;
                        TESTHR(hr = mpH264Encoder->GetOutputAvailableType(0, 0, &pType));

                        IMFByteStreamPtr pByteStream;
                        IMFMediaSinkPtr pMediaSink;
                        TESTHR(hr = MFCreateFile(MF_ACCESSMODE_READWRITE, MF_OPENMODE_DELETE_IF_EXIST, MF_FILEFLAGS_NONE, mFilename.c_str(), &pByteStream));
                        TESTHR(hr = MFCreateMPEG4MediaSink(pByteStream, pType, NULL, &pMediaSink));
                        TESTHR(hr = MFCreateSinkWriterFromMediaSink(pMediaSink, NULL, &mpWriter));
                        TESTHR(hr = mpWriter->BeginWriting());
                    }
                    TESTHR(hr = mpWriter->WriteSample(0, h264OutputDataBuffer.pSample));
                    h264OutputDataBuffer.pSample->Release();
                    if (h264OutputDataBuffer.pEvents != NULL)
                        h264OutputDataBuffer.pEvents->Release();
                }
                else
                    TESTHR(hr);
            }
            else if (meType == METransformDrainComplete)
            {
                mTrace->Trace(1, L"New METransformDrainComplete event");
                TESTHR(hr = mpH264Encoder->ProcessMessage(MFT_MESSAGE_COMMAND_FLUSH, 0));
                SetEvent(mEventDrainComplete);
            }
            else if (meType == MEError)
            {
                PROPVARIANT pValue;
                TESTHR(hr = pEvent->GetValue(&pValue));
                mTrace->Trace(1, L"MEError, value: %u", pValue.vt);
                error = true;
                SetEvent(mEventNeedInput);
            }
            else
            {
                PROPVARIANT pValue;
                TESTHR(hr = pEvent->GetValue(&pValue));
                mTrace->Trace(1, L"Unknown event type: %lu, Value: %u", meType, pValue.vt);
            }
            TESTHR(hr = mpH264EncoderEventGenerator->BeginGetEvent(this, NULL));
        }
    }
    catch(com_error ex)
    {
        mTrace->Trace(1, L"Exception in %s(%d): 0x%08X", ex.GetFilename(), ex.GetLinenum(), ex.Error());
    }

    return S_OK;
}

 

0 Kudos
17 Replies
Harshdeep_B_Intel
1,006 Views

Hi,

I see that you have developed your own application and running into error after 30-40 minutes. So before I go ahead and look into the issue, I need more details hence suggest to follow this format . Your application is using a MS Media foundation and we have a working sample of MS media foundation. I highly suggest to take a look into the sample .

For questions such as these, we recommend you try to use existing samples and/or tutorials to run the input in question and see if it works. If it works fine, then there is a very high chance your application has some issues. Also if you send us the logs from these tools, it will help us in understanding your set up environment. 

Thanks,

0 Kudos
Javier_A_
Beginner
1,006 Views

Hello,

Is it not easy to translate my implementation to the sample because the internal design of the application is too different. However I have made a small modification on the sample to make it repeat the translation of the source file continuously, and after some minutes it triggers an exception. The exception seems similar to what happens in my application, that is, the application cannot open the hardware encoder any more.

Attached to this message you can find:

  • Full sample application with the small modification. The only modified file is MainViewModel.cs, function TranscodeCommand.
  • A sample file to be used as source to the sample application.
  • The output of the Media SDK System Analyzer.
  • The output of the Media SDK Tracer utility taken while the modified sample application was running.

The problem can be reproduced by starting the sample application, selecting the source file and clicking the Transcode button. The application will continuously transcode the file until an exception occurs. The application has been compiled using VS2012.

I can also send my application modified so that the problem can be reproduced without the need of an acquisition card. I suppose  you prefer starting  with your own sample, but let me know if you want me to send my application.

About the priority of the problem, it is high for me as the application is fully finished but I cannot deliver it because of this problem.

Thank you for your help.

Javier Andrés

0 Kudos
Harshdeep_B_Intel
1,006 Views

Hi,

Thank you for sending in the logs. I will look into the issue and get back with further information.

0 Kudos
Harshdeep_B_Intel
1,006 Views

Hi,

I see from Analyser.log that you are using a Old driver build - 3621.Can please update your driver to the new build -3969 and check out your application with the new driver. Also the tracer log which was attached was not complete, please resend  the complete tracer log.

Thanks,

0 Kudos
Javier_A_
Beginner
1,006 Views

Hello,

Thank you for spending your time on this issue.

I have updated the driver to version 10.18.10.3945 and tested the sample application again. I see no difference, after translating the source file a few times the exception is triggered.

I am attaching a new analyzer.log and a new tracer.log. I tried to make sure the tracer.log is complete this time.

I have also added the file Javier_tracer.log corresponding to the execution of my own application until it got the error.

Javier A.

0 Kudos
Harshdeep_B_Intel
1,006 Views

Hi,

 We will look into the logs and update with further information about the issue.

Thanks.

0 Kudos
Harshdeep_B_Intel
1,006 Views

Hi,

So I have few updates. We were not able to reproduce the issue using the tracer log on our side. We looked into  "MF Sample Application - Modified.zip"  package you had provided and did see exception been raised when the input test clip was "Fichero_20141016_200300.mp4 " also when it was loaded as input to the sample, we saw that Audio decoder was not able to identify the input .

So we tired a different test .mp4 clip with both audio/video and your modified sample app did not raise any exception. I think exception was previously raised because Audio decoder was not able to recognize the input. Can you please give a try with a different input sample with both audio and video. Let us know if this helps.

Thanks,

 

0 Kudos
Javier_A_
Beginner
1,006 Views

Hello,

About the tracer utility, I am not been able to start the modified sample nor my application compiled for AnyCpu or 64 bits when the tracer utility is started. All the captured traces I have sent have been generated with the applications compiled for 32 bits (but running on the same 64 bits OS). However, without using the tracer utility both compilations (32 and 64 bits) give the same results, that is, the same exceptions.

About the audio, my application does not manages any audio stream and that is why I made my testing with a file without audio. However, I have recorder a new file with audio and video at 1920x1080 50fps and tested the modified sample with it. It gives the same exception on my testing computer. 

Attached to this message you can find the new tracer.log file generated while the modified sample was transcoding the new audio-video file, and the audio-video file itself.

 

0 Kudos
Harshdeep_B_Intel
1,006 Views

Hi,

So here are few updates from the logs and your Modified Code sample. So I do see that your modified sample raising an exception when an input sample is transcoded. So, I looked into the code modifications you have implemented in MF sample in MainViewModel.cs. So, you have implemented a  dispatcherTimer_Tick function in which a dispatcher timer is started for our HW which is truly asynchronous. When we step through your modified code, I see an exception being raised from the Dispatcher function which starts and stops and raises an exception before initializing encoder plugins. I suggest to look into your modifications which is causing these exceptions before initialization for the encoder in the code. Hope this helps!

Thanks,

0 Kudos
Javier_A_
Beginner
1,006 Views

Hello

    "I suggest to look into your modifications which is causing these exceptions before initialization for the encoder in the code"

Sorry but it is not my modification who is causing the exceptions. The modification is only simulating the transcode button being pressed immediately again after it finish each transcode operation. I made it to avoid the need to press the button 20 o more times until the exception triggers.

However I made it myself this morning with the original sample without modification and at the 26th button press the exception was raised. Attached you can find a file with all the messages thrown by the sample application in this case. I suggest you also try it. Maybe you need 20, 30 or more button presses, I don't know, but I am sure it will finally fail. I have to point that I always tested with 1920x1080 50fps, I don't know if it fails with another resolution or frame speed.

I am pretty sure there is something inside the API, the driver or elsewhere, but not in the application, that is not behaving correctly. It doesn't matter if the file has or not audio, the video encoder should work correctly. And it doesn't matter either if the operation is triggered by a human pressing a button or a timer.

I am sorry to say so crudely, but I think we are looking for the problem at the incorrect place.

Thank you for your help.

Regards

Javier Andrés

0 Kudos
Harshdeep_B_Intel
1,006 Views

Hi,

Thank you for the log. We were not able to reproduce the issue earlier, hence we tried to triage the issue from all the angles to help you and replicate the same on our side. We have replicated the issue from your last log and I will be speaking to my team on the issue and reach back with further updates and information. 

Thanks,

0 Kudos
Javier_A_
Beginner
1,006 Views

Thanks again for your help.

Regards

Javier

0 Kudos
Javier_A_
Beginner
1,006 Views

Are there any news on this.

Thank you

Javier

0 Kudos
Harshdeep_B_Intel
1,006 Views

Hi,

We understand the desire for a fix. Our engineers have been working on the issue.  The issue continues to be 'escalated' and it is reasonable to expect a fix available soon.

Thanks,

0 Kudos
Javier_A_
Beginner
1,006 Views

Hello again

Can you, please, give me the status of this issue?

Thank you

Javier

0 Kudos
Harshdeep_B_Intel
1,006 Views

Hi,

We have escalated the issue. Our engineers are working on the issue and are expected to arrive with a fix soon. I will update once I get any information on the same.

Thanks,

0 Kudos
Harshdeep_B_Intel
1,006 Views

Hi,

Can you please send us logs from MFTrace.exe (tool from Windows Kits). 

Link: http://msdn.microsoft.com/library/windows/desktop/ff684915(v=vs.85).aspx.

These logs can further help our team for further required analysis. 

Thanks,

0 Kudos
Reply