H264 encode fails on Windows 10

[Tetrick__Cary]

I'm using a windows build of ffmpeg to encode with, and it works well on Windows 7, but periodically fails on Windows 10.

The symptoms are dependent on the Intel graphics driver version.
Intel(R) HD Graphics 4600 20.19.15.4352 Active
Intel(R) HD Graphics 4600 20.19.15.4463 Active -- returns error
You'll either see ffmpeg appear to lock up, or return an error "encoder failed".

My debugging shows me that it get's stuck in a loop calling MFXVideoENCODE_EncodeFrameAsync()
which returns MFX_WRN_DEVICE_BUSY continuously with the older driver.
The newer driver returns MFX_ERR_DEVICE_FAILED.

This occurs on Windows 10, but not Windows 7. it happens about 40% - 50% of the time at the start of an encode.

It seems to occur more often at higher resolutions like 1920x1080.

Could this be caused by incorrect init parameters?

Any insight into why this might happen would be greatly appreciated.

I'm attaching mfx tracer logs - one successful, one failed.

[Jiandong_Z_Intel]

Hi There,

Can you produce this issue with MSDK sample_encode ?

Thanks,

Zachary

[Tetrick__Cary]

Hi Zachary,

I tried the sample_encode using this commandline:

sample_encode.exe h264 -i out.raw -o out.264 -w 1920 -h 1080 -nv12 -hw -b 8000 -u speed -num_slice 1 -f 24 -la -lad 40

This was the closest I could get to what the ffmpeg encoder does on a simple encode.

I still see differences in the tracer logs, but this leads me to believe that the problem is somewhere in the ffmpeg implementation.

I extracted raw nv12 from an mp4 file with

ffmpeg -loglevel verbose -i EX-1.mp4 -c:v rawvideo -video_size 1920x1080 -framerate 24000/1001 -pix_fmt nv12 -f rawvideo out.raw

attached:

out.raw - nv12 frames extracted.

out.264 - encoded output from sample_encoder

tracer log from sample encoder

[Tetrick__Cary]

quick note - there is a case posted on the ffmpeg bug tracker. I've added a few notes there.

https://trac.ffmpeg.org/ticket/5708

[Surbhi_M_Intel]

Hi, 

Looking at the cmd line you have provided of the sample_encode and ffmpeg. I have few questions to debug this - 

1. Through sample_encode, were you able to reproduce the problem?
2. If yes, the rate control method(Look ahead) which you have used in sample_encode is not available through ffmpeg. 
3. While extracting file from mp4 seems like you are decoding the content to extra raw frames. To avoid that you can use sample_multi_transcode which will be much close to your scenario. 
4. As you have already seen, there is a handle in the sample that if there is a MFX_ERR_device_FAILURE, then encoder needs to be close and reset. Also, explained in the manual , section "Hardware Device Error Handling" on Pg33. Possible to include this handle in your application

If you can provide us the reproducer we should be able to try this at our end and further debug.

Thanks,
Surbhi

[Tetrick__Cary]

Hello Surbhi, 

Thanks for getting back to me on this.

1. I have not been able to reproduce through sample_encode. 
2. The use of look ahead in sample_encode was intended to emulate the settings i had in ffmpeg. Recent versions of ffmpeg do appear to support look ahead, and even prefers it. Below is some of the output form ffmpeg:
[graph 0 input from stream 0:0 @ 000001A2B218BA40] w:1920 h:1080 pixfmt:nv12 tb:1001/24000 fr:24000/1001 sar:0/1 sws_param:flags=2
[h264_qsv @ 000001A2B21B5960] Initialized an internal MFX session using hardware accelerated implementation
[h264_qsv @ 000001A2B21B5960] Using the VBR with lookahead (LA) ratecontrol method
[h264_qsv @ 000001A2B21B5960] profile: high; level: 40
[h264_qsv @ 000001A2B21B5960] GopPicSize: 250; GopRefDist: 4; GopOptFlag: closed ; IdrInterval: 0
[h264_qsv @ 000001A2B21B5960] TargetUsage: 4; RateControlMethod: LA
[h264_qsv @ 000001A2B21B5960] TargetKbps: 8000; LookAheadDepth: 40
[h264_qsv @ 000001A2B21B5960] NumSlice: 1; NumRefFrame: 2
[h264_qsv @ 000001A2B21B5960] RateDistortionOpt: OFF
[h264_qsv @ 000001A2B21B5960] RecoveryPointSEI: OFF IntRefType: 0; IntRefCycleSize: 0; IntRefQPDelta: 0
[h264_qsv @ 000001A2B21B5960] MaxFrameSize: 62464; MaxSliceSize: 0;
[h264_qsv @ 000001A2B21B5960] BitrateLimit: ON; MBBRC: OFF; ExtBRC: unknown
[h264_qsv @ 000001A2B21B5960] Trellis: auto
[h264_qsv @ 000001A2B21B5960] RepeatPPS: ON; NumMbPerSlice: 0; LookAheadDS: unknown
[h264_qsv @ 000001A2B21B5960] AdaptiveI: unknown; AdaptiveB: unknown; BRefType: off
[h264_qsv @ 000001A2B21B5960] MinQPI: 0; MaxQPI: 0; MinQPP: 0; MaxQPP: 0; MinQPB: 0; MaxQPB: 0
[h264_qsv @ 000001A2B21B5960] Entropy coding: CABAC; MaxDecFrameBuffering: 2
[h264_qsv @ 000001A2B21B5960] NalHrdConformance: OFF; SingleSeiNalUnit: ON; VuiVclHrdParameters: OFF VuiNalHrdParameters: OFF

I have tried other rate control methods in both ffmpeg and sample_encode with the same results.

3. I extracted raw frames partly because that is the requirement of sample_encode, but also because in our use case we actually do pipe raw video into ffmpeg to be encoded. It also reduces the reproduction to (mostly) the encode.

4. I did notice the reset in sample_encode, and made an attempt to do a reset in ffmpeg. It did not work, but I'll have to revisit this as I'm not sure I got it right.

To reproduce - I'll attach my build of ffmpeg. But I was able to also reproduce using the latest build from https://ffmpeg.zeranoe.com/builds/
Here is the command line using the already attached files:
ffmpeg -y -loglevel verbose -f rawvideo -video_size 1920x1080 -framerate 24000/1001 -pix_fmt nv12 -i out.raw -c:v h264_qsv -b:v 8000k out.mp4

[Jiandong_Z_Intel]

Hi,

I try to reproduce this is on SKL with 20.19.15.4463  on Win10 , but your command line work well for me currently.

jiandon@JIANDON-MOBL1 /c/ffmpeg/bin
$ ffmpeg -y -loglevel verbose -f rawvideo -video_size 1920x1080 -framerate 24000/1001 -pix_fmt nv12
 -i out.raw -c:v h264_qsv -b:v 8000k out.mp4
ffmpeg version N-80924-gb450b82 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (GCC)
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b
--enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55. 28.100 / 55. 28.100
  libavcodec     57. 48.103 / 57. 48.103
  libavformat    57. 41.100 / 57. 41.100
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 47.100 /  6. 47.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  1.100 /  2.  1.100
  libpostproc    54.  0.100 / 54.  0.100
[rawvideo @ 00000000026dd840] Estimating duration from bitrate, this may be inaccurate
Input #0, rawvideo, from 'out.raw':
  Duration: 00:00:13.22, start: 0.000000, bitrate: 596600 kb/s
    Stream #0:0: Video: rawvideo, 1 reference frame (NV12 / 0x3231564E), nv12, 1920x1080, 596600 kb/s, 23.98 tbr, 23.98 tbn, 23.98 tbc
[graph 0 input from stream 0:0 @ 0000000000f5bd80] w:1920 h:1080 pixfmt:nv12 tb:1001/24000 fr:24000/1001 sar:0/1 sws_param:flags=2
[h264_qsv @ 0000000000f59800] Initialized an internal MFX session using hardware accelerated implementation
[h264_qsv @ 0000000000f59800] Using the VBR with lookahead (LA) ratecontrol method
[h264_qsv @ 0000000000f59800] profile: high; level: 40
[h264_qsv @ 0000000000f59800] GopPicSize: 250; GopRefDist: 4; GopOptFlag: closed ; IdrInterval: 0
[h264_qsv @ 0000000000f59800] TargetUsage: 4; RateControlMethod: LA
[h264_qsv @ 0000000000f59800] TargetKbps: 8000; LookAheadDepth: 40
[h264_qsv @ 0000000000f59800] NumSlice: 1; NumRefFrame: 2
[h264_qsv @ 0000000000f59800] RateDistortionOpt: OFF
[h264_qsv @ 0000000000f59800] RecoveryPointSEI: OFF IntRefType: 0; IntRefCycleSize: 0; IntRefQPDelta: 0
[h264_qsv @ 0000000000f59800] MaxFrameSize: 783360; MaxSliceSize: 0;
[h264_qsv @ 0000000000f59800] BitrateLimit: ON; MBBRC: OFF; ExtBRC: unknown
[h264_qsv @ 0000000000f59800] Trellis: auto
[h264_qsv @ 0000000000f59800] RepeatPPS: ON; NumMbPerSlice: 0; LookAheadDS: unknown
[h264_qsv @ 0000000000f59800] AdaptiveI: unknown; AdaptiveB: unknown; BRefType: off
[h264_qsv @ 0000000000f59800] MinQPI: 0; MaxQPI: 0; MinQPP: 0; MaxQPP: 0; MinQPB: 0; MaxQPB: 0
[h264_qsv @ 0000000000f59800] Entropy coding: CABAC; MaxDecFrameBuffering: 2
[h264_qsv @ 0000000000f59800] NalHrdConformance: OFF; SingleSeiNalUnit: ON; VuiVclHrdParameters: OFF VuiNalHrdParameters: OFF
[mp4 @ 0000000000f57fe0] Using AVStream.codec to pass codec parameters to muxers is deprecated, use
AVStream.codecpar instead.
Output #0, mp4, to 'out.mp4':
  Metadata:
    encoder         : Lavf57.41.100
    Stream #0:0: Video: h264 (h264_qsv), 1 reference frame ([33][0][0][0] / 0x0021), nv12, 1920x1080, q=2-31, 8000 kb/s, 23.98 fps, 24k tbn, 23.98 tbc
    Metadata:
      encoder         : Lavc57.48.103 h264_qsv
    Side data:
      cpb: bitrate max/min/avg: 0/0/8000000 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_qsv))
Press  to stop, [?] for help
No more output streams to write to, finishing.e=00:00:10.84 bitrate=8180.8kbits/s speed=5.38x
frame=  317 fps=142 q=-0.0 Lsize=   12992kB time=00:00:13.13 bitrate=8101.0kbits/s speed= 5.9x
video:12988kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.028872%
Input file #0 (out.raw):
  Input stream #0:0 (video): 317 packets read (985996800 bytes); 317 frames decoded;
  Total: 317 packets (985996800 bytes) demuxed
Output file #0 (out.mp4):
  Output stream #0:0 (video): 317 frames encoded; 317 packets muxed (13300140 bytes);
  Total: 317 packets (13300140 bytes) muxed

Graphics Devices:
        Name                                         Version             State
        Intel(R) HD Graphics 520                     20.19.15.4463       Active

System info:
        CPU:    Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz
        OS:     Microsoft Windows 10 Enterprise
        Arch:   64-bit

[Tetrick__Cary]

Hi Jiandong, 

Did you try it several times? It is intermittant.

[Salvaterra__Davide]

Hi everone,

I know this is an old topic, but as of today I'm still experiencing the very same issues you describe.

Did anybody manage to make it work correctly?

Thank you!!