Intel® Integrated Performance Primitives
Deliberate problems developing high-performance vision, signal, security, and storage applications.

Problem with muxing MPEG2 video and audio


Hi Intel Gurus,

I have a trouble to mux YUV420 video with audio to MPEG2. It's no issue to mux just the video. Works like a charm. But when I try to use PutAudioData I receive Unhandled exception error: Access violation reading location 0x0000000.


This is extraction from my code:

 // Video Info
 UMC::VideoStreamInfo VideoInfo; 
 memset(&VideoInfo, 0, sizeof(VideoInfo));
 VideoInfo.streamPID    = 0;     
 VideoInfo.stream_type   = UMC::MPEG2_VIDEO; 
 VideoInfo.color_format   = UMC::YUV420;
 VideoInfo.interlace_type  = UMC::PROGRESSIVE;
 VideoInfo.aspect_ratio_height = 9;
 VideoInfo.aspect_ratio_width = 16;
 VideoInfo.clip_info.height  = 1080;
 VideoInfo.clip_info.width  = 1920;
 VideoInfo.bitrate    = 10000000;
 VideoInfo.framerate    = 24;
 VideoInfo.disp_clip_info.height = 1080;
 VideoInfo.disp_clip_info.width = 1920;

 // Audio Info
UMC::AudioStreamInfo AudioInfo;
memset(&AudioInfo, 0, sizeof(AudioInfo));
AudioInfo.streamPID   = 1;
AudioInfo.stream_type  = UMC::AAC_AUDIO;  
AudioInfo.channels   = 2;
AudioInfo.bitPerSample  = 16;
AudioInfo.bitrate   = 24;
AudioInfo.sample_frequency = 44100;

 UMC::MPEG2MuxerParams MuxerParams;
 MuxerParams.m_lpDataWriter   = &Writer;
 MuxerParams.m_SystemType   = UMC::MPEG2_PURE_VIDEO_STREAM;
MuxerParams.m_nNumberOfTracks = 2; 

 MuxerParams.pTrackParams   = new UMC::TrackParams[MuxerParams.m_nNumberOfTracks];
 // Track 1
 MuxerParams.pTrackParams[0].type  = UMC::VIDEO_TRACK;  
 MuxerParams.pTrackParams[0] = &VideoInfo;  
 MuxerParams.pTrackParams[0].bufferParams.m_prefInputBufferSize = 2000000;  
 // Track 2
 MuxerParams.pTrackParams[1].type  = UMC::AUDIO_TRACK; 
 MuxerParams.pTrackParams[1] = &AudioInfo;
 MuxerParams.pTrackParams[1].bufferParams.m_prefInputBufferSize = 20000000;   
 MuxerParams.pTrackParams[1].bufferParams.m_prefOutputBufferSize = 20000000;  

 UMC::MPEG2Muxer Muxer;

 // MPEG2 Encoder - video
 UMC::MPEG2EncoderParams videoParams;  = 1080  = 1920;    = 1000000;   = UMC::YUV420;    = 24;   = UMC::MPEG2_VIDEO; = 9;  = 16;
 videoParams.numThreads     = 1; 

 UMC::MPEG2VideoEncoder Mpeg2Encoder;

 // AAC Encoder - audio
  UMC::AACEncoderParams audioParams;
  audioParams.ns_mode      = 1;  
  audioParams.stereo_mode     = UMC_AAC_JOINT_STEREO; 
  audioParams.m_info_in.channels   = 2;
  audioParams.m_info_in.sample_frequency = 44100;
  audioParams.m_info_in.bitrate   = 24000;
  audioParams.audioObjectType   = UMC::AOT_AAC_LC;
  audioParams.outputFormat    = UMC::UMC_AAC_ADTS; 
  audioParams.m_info_out.channels   = 2;
  audioParams.m_info_out.bitrate   = 24000;
  audioParams.m_info_out.sample_frequency = 44100;

 UMC::AACEncoder AacEncoder;

 UMC::VideoData DataVideoIn; 
 DataVideoIn.Init( 1920, 1080, UMC::YUV420,8);

 Ipp8u* pOutputVideoData = ippsMalloc_8u(3110400);
 Ipp8u* pOutputAudioData  = ippsMalloc_8u(4096);

 UMC::MediaData DataVideoOut;
 UMC::MediaData MuxVideoData;
 UMC::AudioData DataAudioIn;
 UMC::AudioData DataAudioOut;
 UMC::MediaData MuxAudioData;

 DWORD dwEvent;
 int nEncodedVideoFrames = 0;
 int nEncodedAudioFrames = 0;


Now I use WaitForMultipleObjects to catch the video and buffers


// Audio Part


 UMC::Status status;

 DataAudioIn.SetBufferPointer(pAudioFrameBuffer, nAudioBufferSize);

  DataAudioOut.SetBufferPointer(pOutputAudioData, 100000000);

  status = AacEncoder.GetFrame(&DataAudioIn, &DataAudioOut);
  if (status == UMC::UMC_OK)

      int nDataSize = DataAudioOut.GetDataSize();
      MuxAudioData.SetBufferPointer(pOutputAudioData, nDataSize);
      memcpy(MuxAudioData.GetDataPointer(), DataAudioOut.GetDataPointer(), nDataSize);

         status = Muxer.PutAudioData(&MuxAudioData);       // I receive Unhandled exception right here
         if (UMC::UMC_ERR_NOT_ENOUGH_BUFFER == status)  

      }while (UMC::UMC_ERR_NOT_ENOUGH_BUFFER == status);  


Audio Part works fine with MPEG4 muxing. The difference only that during initialization I used parameters specific to MPEG4.


Thank you in advance

Sergey Tsybikov


0 Kudos
3 Replies
One more comment: In the previous sample I defined UMC::MPEG2MuxerParams MuxerParams; MuxerParams.m_SystemType = UMC::MPEG2_PURE_VIDEO_STREAM; This setting allow me to mux only video and I received the crash for audio. If I defined MuxerParams.m_SystemType = UMC::MPEG2_PURE_AUDIO_STREAM the mux of audio is fine, but the crash for video mux. Does anyone know what parameter I have to use to combine audio and video together. I tried almost avery value from SystemStreamType but no success. Thank you Sergey Tsybikov
0 Kudos
It probably won't be as simple as a parameter change. Many problems have been discovered with using audio and video together in UMC's muxers. As samples, their original purpose was to facilitate viewing output from the sample video codecs since many players only work with container formats. While it may be possible to use them as a base (probably with a lot of work to get started, and then later to harden them) they aren't an ideal starting point for anything beyond what they were originally written for. This has been discussed at great length internally, but for now there is not much more to say than that it is unlikely additional work to bring the muxers up to "turnkey" level will be prioritized. This article may be of interest as an alternative, since the method described could be applied to a wide variety of video codecs: Best regards, Jeff
0 Kudos
Hi Jeff, Thank you for your reply. The alternative that you suggested doesn't work for me. We have our own medical camera and the camera process is feeding the memory buffers with video and audio data. This is not a file, buffers with single video and audio frames. So my idea is to mux the data to the mpeg file based on user request (mpeg1/2/4). MPEG4 muxer works great but MPEG2 does not. I have to find the solution to mux video and audio data to mpeg2. Thank you. Sergey Tsybikov Smith&Nephew Endoscopy
0 Kudos