When H264 stream Pic width is 360(0x168)

[atvan]

When H264 stream PIC WIDTH is 360(0x168), the IPP will parse it as 0x170. It works perfect if use IPP decoder. However, when I record the stream as .mp4, VLC doesn't play the .mp4 file until I manually change the pic_width from 0x170 to 0x168.

Do you have any idea how to solve this?

[Victor_C_Intel1]

Quoting - ezcom

When H264 stream PIC WIDTH is 360(0x168), the IPP will parse it as 0x170. It works perfect if use IPP decoder. However, when I record the stream as .mp4, VLC doesn't play the .mp4 file until I manually change the pic_width from 0x170 to 0x168.

Do you have any idea how to solve this?

Good day,
The stream we are talking about is not regular. It use cropping feature of the standard. Thepoint is that the h264 decoder can handle data only on macroblock basis. It requires frame dimensions aligned to 16 pixels. To comporess a frame of arbitrary size the cropping is used. It is just 4 values in sequence header, describingindent from each side of the frame. But in fact the stream itself has dimension aligned to 16.

I believe this is not anissueof the decoder, but of the splitter. What tools do you use to pack h264 stream into .mp4 container?

[atvan]

Hey Victor,

I wrote the writer to pack

Quoting - Victor Cherepanov (Intel)

Good day,
The stream we are talking about is not regular. It use cropping feature of the standard. Thepoint is that the h264 decoder can handle data only on macroblock basis. It requires frame dimensions aligned to 16 pixels. To comporess a frame of arbitrary size the cropping is used. It is just 4 values in sequence header, describingindent from each side of the frame. But in fact the stream itself has dimension aligned to 16.

I believe this is not anissueof the decoder, but of the splitter. What tools do you use to pack h264 stream into .mp4 container?

Hey Victor,

Thanks for your information. I parsed the sps and get
sps.frame_cropping_rect_bottom_offset = 4
sps.frame_cropping_rect_top_offset = 0
sps.frame_cropping_rect_left_offset = 0
sps.frame_cropping_rect_right_offset = 0

Because thepic_height of theencoder is set to 360(0x168), I simply add
if(sps.frame_cropping_flag)
{
if(sps.frame_cropping_rect_bottom_offset || sps.frame_cropping_rect_top_offset) pic_height -= 8;
if(sps.frame_cropping_rect_left_offset || sps.frame_cropping_rect_right_offset) pic_width -= 8;
}

It seems work now.But shouldn'tit be 0x16c instead of 0x168?


Have a good weekend!

[Victor_C_Intel1]

Quoting - ezcom

Thanks for your information. I parsed the sps and get
sps.frame_cropping_rect_bottom_offset = 4
sps.frame_cropping_rect_top_offset = 0
sps.frame_cropping_rect_left_offset = 0
sps.frame_cropping_rect_right_offset = 0

Because thepic_height of theencoder is set to 360(0x168), I simply add
if(sps.frame_cropping_flag)
{
if(sps.frame_cropping_rect_bottom_offset || sps.frame_cropping_rect_top_offset) pic_height -= 8;
if(sps.frame_cropping_rect_left_offset || sps.frame_cropping_rect_right_offset) pic_width -= 8;
}

Hi again,
it looks like few code lines are not quite correct. Let's look at the UMC code.


Ipp32u cropLeft, cropRight, cropTop, cropBottom;

// set initial cropping values
cropLeft = pSeqParamSet->frame_crop_left_offset;
cropRight = pSeqParamSet->frame_crop_right_offset;
cropTop = pSeqParamSet->frame_crop_top_offset * (2 - pSeqParamSet->frame_mbs_only_flag);
cropBottom = pSeqParamSet->frame_crop_bottom_offset * (2 - pSeqParamSet->frame_mbs_only_flag);

// adjust cropping values depending on chroma format
switch (chroma_format_idc)
{
case CHROMA_FORMAT_420:
cropLeft *= 2;
cropRight *= 2;
cropTop *= 2;
cropBottom *= 2;
break;

case CHROMA_FORMAT_422:
cropLeft *= 2;
cropRight *= 2;
break;

default:
break;
}

And the final values cropLeft, cropRight, cropTop, cropBottom should be substructed form frame dimensions.

Victor

[atvan]

Hey Victor,

I found the code wherethe UMC splitter calculatesh264picturedimentions. I don't see how chroma format affects picture dimentions from this piece of code. But I guess that should work in my splitter too.

Thank for your information. It's very helpful.

Regards,
ezcom


void H264ParsingCore::GetInfo(VideoStreamInfo& info)
{
const Slice& sh = m_last.IsReady() ? m_last : m_prev;
if (m_prev.IsReady())
{
const Sps& sps = sh.GetPps().GetSps();
info.stream_type = H264_VIDEO;
info.profile = sps.profile_idc;
info.level = sps.level_idc;
info.clip_info.width = 16 * sps.frame_width_in_mbs;
info.clip_info.width -= 2 * (sps.frame_cropping_rect_left_offset + sps.frame_cropping_rect_right_offset);
info.clip_info.height = 16 * sps.frame_height_in_mbs;
info.clip_info.height -= 2 * (sps.frame_cropping_rect_top_offset + sps.frame_cropping_rect_bottom_offset);
info.clip_info.height *= (2 - sps.frame_mbs_only_flag);

[Victor_C_Intel1]

Quoting - ezcom

Hey Victor,

I found the code wherethe UMC splitter calculatesh264picturedimentions. I don't see how chroma format affects picture dimentions from this piece of code. But I guess that should work in my splitter too.

Thank for your information. It's very helpful.

Regards,
ezcom


void H264ParsingCore::GetInfo(VideoStreamInfo& info)
{
const Slice& sh = m_last.IsReady() ? m_last : m_prev;
if (m_prev.IsReady())
{
const Sps& sps = sh.GetPps().GetSps();
info.stream_type = H264_VIDEO;
info.profile = sps.profile_idc;
info.level = sps.level_idc;
info.clip_info.width = 16 * sps.frame_width_in_mbs;
info.clip_info.width -= 2 * (sps.frame_cropping_rect_left_offset + sps.frame_cropping_rect_right_offset);
info.clip_info.height = 16 * sps.frame_height_in_mbs;
info.clip_info.height -= 2 * (sps.frame_cropping_rect_top_offset + sps.frame_cropping_rect_bottom_offset);
info.clip_info.height *= (2 - sps.frame_mbs_only_flag);

It looks like ParsingCore supports 4:2:0 chroma format only.

[atvan]

Quoting - Victor Cherepanov (Intel)

It looks like ParsingCore supports 4:2:0 chroma format only.

Hum, I'm able totrack the decoding process to here,

UMC::Status
TimedColorConverter::GetFrame(UMC::MediaData *in, UMC::MediaData *out)
{
...

It seemssimple_player decodes everything into YUV420 first, and use TimedColorConverter to convert the YUV420 output to requested color space.Could it affect performance?