Problem with h.264 decoding CIF and 2CIF

[bcisrd]

Hello,

I have developed an RTSP application that displays video from cameras on a local network.

My development platform is:
Windows 7 (32 bit)
Visual Studio 2008
IPP vers 7.0.4.196

I am displaying video successfully for multiple stream formats excluding CIF (352288) and 2CIF (704x240).

The image produced from the decoder for 352x288 is blocky and the color is off (see CIF_252x288.png).

The image produced from the decoder for 704x240 is squatty. The image is displayed by the ratio defined instead of 704x480 (2CIF_704x240.png).

As stated the application displays other streams successfully. For example 1280x720 streams great. Because of this I am handling SPS and PPS properly (at least some of the time). Is there Pre or Post processing of the image the application should be doing before or after h.264 decoder or is the h.264 decoder supposes to handle these images?

Im hoping someone will be kind and direct me on my error.

2CIF (704X240)


CIF (352x288

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//Below is how I am initiating decoder. I removed failure lines to consolidate code

// pData -- byte array containing SPS and FPS parameters
//nSize -- Size of pData array
// nThreadCount -- always 1
// nWidth -- return video width to calling function
// nHeight -- returns video height to calling function

bool Init (BYTE *pData, unsigned long nSize, int nThreadCount, int &nWidth, int &nHeight)
{
bool bRetVal = false;
UMC::Status nStatus;
nWidth = 0;
nHeight = 0;
UMC::VideoDecoderParams VidDecodeParams;
m_pvidDecoder = new UMC::H264VideoDecoder();
if (m_pvidDecoder)
{
m_inVid.SetBufferPointer (pData, nSize);
m_inVid.SetDataSize (nSize);

VidDecodeParams.pPostProcessing = NULL;
VidDecodeParams.info.stream_type = UMC::H264_VIDEO;
VidDecodeParams.numThreads = nThreadCount;
VidDecodeParams.lFlags = 0;
VidDecodeParams.m_pData = &m_inVid;
nStatus = m_pvidDecoder->Init(&VidDecodeParams);
if(nStatus == UMC::UMC_OK)
{
UMC::H264VideoDecoderParams params;
nStatus = m_pvidDecoder->GetInfo (&params);
if(nStatus == UMC::UMC_OK)
{
nWidth = params.info.clip_info.width;
nHeight = params.info.clip_info.height;
nStatus = m_outVid.Init (nWidth, nHeight, UMC::YUY2, 8);
if(nStatus == UMC::UMC_OK)
{
nStatus = m_outVid.Alloc ();
if(nStatus == UMC::UMC_OK)
{
bRetVal = true;
}
}
}
}
}

return (bRetVal);
}

[Chao_Y_Intel]

Hello,

Do you have raw video stream, and if you play with these raw video with simpleplayer application, and see it work?

Also I see you implicitly to cover the the video from YUV420 to YUY2 format, if you just decoded as YUV420, how does the video work?

thanks,
Chao

[bcisrd]

I do not have raw video to use with sample player. With a little research I could create a compatible file.

My video display code currently only works with YUY2. So I will have to do some work to get an answer regarding YUV420 question.

Reviewing the SDP I receive from the camera I found that I wasnt doing anything with profile-level-id data.

v=0
o=- 0 0 IN IP4 10.1.12.237
s=LIVE VIEW
t=0 0
c=IN IP4 0.0.0.0
m=video 0 RTP/AVP 35
a=rtpmap:35 H264/90000
a=control:rtsp://10.1.12.237/video
a=fmtp:35 packetization-mode=1;profile-level-id=42001e;sprop-parameter-sets=Z0IAHtoCwfP/AAUACykC,aN4FYg==

Currently my application converts sprop-parameter-sets data and passes it decoder Init function (pData array). But Im not using profile-level-id

profile-level-id=42001e
profile_idc 66
profile_iop 0
level_idc 30

My question is how do I assign these values to the decoder? I have tried below without any changes. Is this the wrong approach? Is this info supose to be included in the pData array along with 'sprop-parameter-sets' ?

bool Init (BYTE *pData, unsigned long nSize, int nThreadCount, int &nWidth, int &nHeight, CString strProfileLevelId)
{

UMC::VideoDecoderParams VidDecodeParams;


BYTE nProfileIdc = 0;
BYTE nProfileIop = 0;
BYTE nLevel_idc = 0;
if(!strProfileLevelId.IsEmpty ())
{
if(strProfileLevelId.GetLength () >= 2)
{
nProfileIdc = HexToDec(strProfileLevelId.Mid (0, 2));
if(strProfileLevelId.GetLength () >= 4)
{
nProfileIop = HexToDec(strProfileLevelId.Mid (2, 2));
if(strProfileLevelId.GetLength () >= 6)
nLevel_idc = HexToDec(strProfileLevelId.Mid (4, 2));
}
}

...
VidDecodeParams.pPostProcessing = NULL;
VidDecodeParams.info.stream_type = UMC::H264_VIDEO;

// profile_idc
VidDecodeParams.info.profile = nProfileIdc;

//profile_iop
VidDecodeParams.info. ???

// level_idc
VidDecodeParams.info.level = nLevel_idc;

VidDecodeParams.numThreads = nThreadCount;
VidDecodeParams.lFlags = 0;
VidDecodeParams.m_pData = &m_inVid;

nStatus = m_pvidDecoder->Init(&VidDecodeParams);

}

[Chao_Y_Intel]

Hello,

Profile and level id are not needed forinputparamters for decoder. They are the output from the decoder.

Thanks,
Chao

[bcisrd]

The simple player does not play the 2 streams.

See next post.

[bcisrd]

Okay, I think I have found my problem. It appears the RTSP program is not handling STAP-A packets properly.

I will address this problem and update this post.

Thanks for all the help.

[bcisrd]

Most of my problem was addressed by processing the RTP STAP-A type properly. But, I still have an issue on displaying the 2Cif image (704x240). It is becoming apparent that there is pre/post processing required!?

Will someone explain the proper procedure to display a 2CIF video? As stated in earlier post the image produced by the encoder is 704x240 instead of expected 704x480.

Also, if you review the attachment you will see the quality of the image distorts and clears up towards the end of video stream. Any direction on howcorrect this would be helpful.

Thanks for all the help.

[Pavel_V_Intel · Accepted solution]

Good day.

You need to correct image size to match DAR.
For simple_player it would be something like this:

avsync.cpp, line 253:
[cpp] m_DecodedFrameSize = m_pVideoInfo->clip_info; // correct DAR if(m_pVideoInfo->aspect_ratio_width != m_pVideoInfo->aspect_ratio_height) { Ipp32f fScale = ((Ipp32f)m_pVideoInfo->aspect_ratio_width/m_DecodedFrameSize.width)/((Ipp32f)m_pVideoInfo->aspect_ratio_height/m_DecodedFrameSize.height); if(fScale > 1) m_DecodedFrameSize.width = m_DecodedFrameSize.width*fScale; else m_DecodedFrameSize.height = m_DecodedFrameSize.height/fScale; }[/cpp]

Also, if you review the attachment you will see the quality of the image distorts and clears up towards the end of video stream. Any direction on howcorrect this would be helpful.

It seems that there are data losses in the stream, codec had reconstructed that it could.

[Steve_Browne]

Just as a side note you might want to verify the pixel aspect ratio before using it as shown above. I've seen some video streams report ratios such as 0x1 and 0x0 (this case would be fine in the code above), etc. Also I've found its not always reliable (especially regarding 2CIF) as some streams don't properly report the pixel aspect ratio as 1x2 or a similar value. As a general rule if its valid and the aspect width or aspect height are greater than 1 then use it. In other cases you may want to explicitly list known 2CIF resolutions (or other known resolutions with issues) and adjust for the proper pixel aspect ratios.