I am getting some random errors encoding 3840 x 2160 video using the HEVC encoder.
Sometimes it works, but many times I get a crash in mfxplugin32_hevce_sw.dll here:
144E1450 movdqa xmm0,xmmword ptr [eax]
144E1454 movdqa xmmword ptr [esi],xmm0 /* here, esi non-zero, but garbage */
144E1458 movdqa xmm0,xmmword ptr [eax+10h]
144E145D movdqa xmmword ptr [esi+10h],xmm0
144E1462 movdqa xmm0,xmmword ptr [eax+20h]
144E1467 movdqa xmmword ptr [esi+20h],xmm0
144E146C movdqa xmm0,xmmword ptr [eax+30h]
144E1471 movdqa xmmword ptr [esi+30h],xmm0
It looks to me that the plugin is trying to allocate an internal buffer into which the uncompressed data is copied. I believe the allocation fails, causing the destination pointer to become garbage. The destination buffer is not checked, so there is an access violation (caught with a try/catch) by the plugin, which causes pEncoder->m_mfxVideoSession.SyncOperation to fail with MFX_ERR_ABORTED (not a good error, by-the-way).
I am using this in a DirectShow encoder and at the time this exception is thrown, the memory allocated by GraphEdit is about 1.2GB. I am using v188.8.131.52 of mfxplugin32_hevce_sw.dll (release version), build 6/9/2014. Is there a newer version of the plugin that will solve this issue?
When it fails, it fails around frame 15-20. It seems that if it passes that point, it will continue until the end.
Please check the encoding with one of our samples using sample_encode for H.265 4k video. As for as on H.265 encoder, we did not come across any memory leak error. In regarding to DirectShow encoder, there has been no new support.