topic Re: JPEG/MJPEG encoder line duplication problem in Intel® Integrated Performance Primitives

JPEG/MJPEG encoder line duplication problem

mojonez — Wed, 10 Sep 2008 22:10:38 GMT

Hi there,

I'm using the IPP 5.3 JPEG encoder to create an MJPEG encoder. Everything works fine except that an occasional frame will have about a 10 line duplication within the image. I have attached an avi file which illustrates this (e.g. frames 14,27,36,39). Also attached areencoder/decoder logs that list the byte counts. It occurs whether using the Intel MJPEG decoder (using the Simple Player sample ) or else other third party decoders via windows media player.

Things I'vetested which have lead me to isolate the encoder::

Source image data seems OK (recording withother encoders or just raw output does not display this artifact).
Other decoders also display this artifact, seeming to indicate its not in the decoder.
The byte increasefor the erroneous frame seems to be approximately the amount of bytes for the number of duplicate lines.
Same problem with different data sources (cameras - color/grayscale, etc)

System specs:

Intel Core2 Quad
Windows XP
Intel IPP 5.3.3.082
Statically linked Intel Libs
Statically linked Intel audio/video libs ( built usingstatic c rtuntime (/MT) )
MFC Test app (static C runtime (/MT) )
CPU usage : ~ 25%

Here's the encode routine.

pSrc is an simple rgb array
width640
height480

int MJPEGEncoder::EncodeFrame(unsigned char* pSrc, int nSrcBytes, unsigned char* pDest, int nDestBytes, int* pnDestBytesUsed)
{
int iResult = 0;
JERRCODE jerr;
IppiSize imgSize;
CJPEGEncoder encoder;
CMemBuffOutput jpegDest;
JMODE mode = JPEG_BASELINE;
JCOLOR color = JC_YCBCR;
JSS sampling = JS_422;
int step = (m_iWidth * 3) + DIB_PAD_BYTES(m_iWidth, 3);
int quality = 85;
int nSourceChannels = 3;
imgSize.width = m_iWidth;
imgSize.height = m_iHeight;
MyTimer timer;
timer.Start();
jpegDest.Open(pDest, nDestBytes);
jerr = encoder.SetSource((Ipp8u*)pSrc, step, imgSize, nSourceChannels, JC_RGB);
if(jerr != JPEG_OK)
{
if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Error setting source [err code = %d].
", jerr);
return jerr;
}
jerr = encoder.SetParams(mode, color, sampling, 0, 0, quality);
if(jerr != JPEG_OK)
{
if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Error setting params [
err code = %d].
", jerr);
return jerr;
}
jerr = jpegDest.Open(pDest, nDestBytes);
if(jerr != JPEG_OK)
{
if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Error opening destination [err code = %d].
", jerr);
return jerr;
}
jerr = encoder.SetDestination(&jpegDest);
if(jerr != JPEG_OK)
{
if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Error setting destination [err code = %d].
", jerr);
return jerr;
}
jerr = encoder.WriteHeader();
if(jerr != JPEG_OK)
{
if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Error writing header [err code = %d].
", jerr);
return jerr;
}
jerr = encoder.WriteData();
if(jerr != JPEG_OK)
{
if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Error writing data [err code = %d].
", jerr);
return jerr;
}
timer.Stop();
double dMillisecs = timer.Duration() * 1000.0f; // millisecs*pnDestBytesUsed = encoder.NumOfBytes();
if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Frame Encoded. [Time(msecs):%3.2f	nInBytes:%d	nOutBytes:%d	nBytesAvailable:%d
", dMillisecs, nSrcBytes, *pnDestBytesUsed, nDestBytes);
return iResult;
}
Again, it works fine, except 1 in every n frames or so will have a duplicated segement, where n tends to vary.
If anyone has any thoughts or suggestions I really appreciate it.
Morgan

Re: JPEG/MJPEG encoder line duplication problem

Vladimir_Dudnik — Thu, 11 Sep 2008 04:51:38 GMT

Hello Morgan,

Thanks for detailed info you provided. I really see no any major issues in code you provide, except you calljpegDest.Open(pDest, nDestBytes) twice, but that should not lead to an issue in this particular case. I would recommend you to store each frame as a separate JPEG file to verify that wrong data comes from JPEG encoder and not corrupted somehow at AVI creation time.

Note, you may want to call get_num_threads() or set_num_threads()functions (defined in jpegbase.h) to see if that behaviour is related to threading implemented in encoder.

Regards,
Vladimir

Re: JPEG/MJPEG encoder line duplication problem

sahil_malhotra — Fri, 12 Sep 2008 07:49:22 GMT

If you could provide a tarball of your source along with Makefiles and a screenshot of your core usage while the code is running, I can have a look at it. actually i'm struggling with something like this myself in my robotics projects.

Re: JPEG/MJPEG encoder line duplication problem

mojonez — Mon, 15 Sep 2008 22:04:45 GMT

Thanks for the reply, Vladimir.

Same problem occurs when saving each frame as a JPEG. Attached image shows the problem. Around the chin area the lines are duplicated. It looks like exactly 8 lines are duplicated, which i think is the same as the encoder's block size.

Calling get_num_threads returns 4.

However, changingthenum threads to 1 -the problem seems to have gone away, with the exception that i now get an occasional 4x4 pixel flickering light grey block in the top-right corner of the image, however thisi can work around.

testing with different threads count:

numthreads 1 - OK (no line duplicates)
numthreads 2 - OK (no line duplicates)
numthreads 3 - BAD (line duplicates)
numthreads 4 - BAD (line duplicates)

So, I'll leave her at 2 threads and call it good.

Thanks so much for the help. I don't think i would have discovered that on my own.

Morgan

Re: JPEG/MJPEG encoder line duplication problem

mojonez — Mon, 15 Sep 2008 22:10:55 GMT

Hi Sahil,

Attached is a screenshot of the CPU usage.

I'm not able tosend out app source code.

Are you getting line duplication as well? If so, is it 8 pixels high?

Re: JPEG/MJPEG encoder line duplication problem

sahil_malhotra — Tue, 16 Sep 2008 05:03:37 GMT

My line duplication problem was more of a corner pixels problem... turned out about stack space overlaps .... my corner pixels were becoming part of multiple threads stack space when split the image into four data structures on my quad.... But yeah a duplication problem it was.... reading your output... when you go over 2 hardware threads , one or more of your threads are reading the image wrong and writing the final pixels right.... means they are duplicating outputs from other threads and putting them in the places they were supposed to at splitting time.... and looking at your problem youve split them int horizontal segments....

Re: JPEG/MJPEG encoder line duplication problem

bassraf — Tue, 25 Nov 2008 13:03:42 GMT

Hi guys,

I'm using IPP 6.0 (final), and i'm still getting artifacts, shifted lines, flashes etc.when encoding (decoding is fine) to JPEG using OPENMP and MSVC2008 Pro. Setting numthreads to 2 on a quadcore (intel) CPU fixes this.

Anyone has a resolution for this, encoding is most important for us.

Best regards,

Bas

Re: JPEG/MJPEG encoder line duplication problem

Vladimir_Dudnik — Tue, 25 Nov 2008 13:08:30 GMT

Which of several IPP JPEG codecs doyou use, JPEG codec from UIC or JPEGView sample or UMC MJPEG codec?

Vladimir

Re: JPEG/MJPEG encoder line duplication problem

bassraf — Wed, 26 Nov 2008 01:31:04 GMT

Hi Vladimir, thanks for your quick response.

I'm using the UMC JPEG code.

Looking at your question I assume I should use a different codebase (what/why are they different anyway?) ?

Bas

Re: JPEG/MJPEG encoder line duplication problem

Vladimir_Dudnik — Wed, 26 Nov 2008 06:39:10 GMT

Yes, some differences are possible between JPEG codec in audio-video-codecs sample and image-codecs sample. Usually, most up to date codec is kept in image-codecs sample and then it might be ported to MJPEG codec.

Regards,
Vladimir

Re: JPEG/MJPEG encoder line duplication problem

bassraf — Wed, 26 Nov 2008 11:00:32 GMT

So should I be using the code in image-codecs/UIC/src/codec/image/jpegor image-codecs/jpegview/jpeg ?

(i'm now using the latter,and encoding seems to be fine, but decoding now has artifacts??)

Regards,

Bas

Re: JPEG/MJPEG encoder line duplication problem

bassraf — Wed, 26 Nov 2008 11:09:06 GMT

Ok, looking at stills from the decoder (compiled using OPENMP MSVC) there are multiple line duplications of 8 lines taking place. Weird thing is, the encoder used to have artifacts, but with this codebase (from image-codecs/jpegview/jpeg) the decoder has the duplication problem when using multiple cores (everything > 1 core has the duplication problem)

Bas

Re: JPEG/MJPEG encoder line duplication problem

Vladimir_Dudnik — Thu, 27 Nov 2008 15:48:53 GMT

Are you saying that UIC JPEG codec have artifacts at encoder and line duplication problem with decoder? We do not see that behaviour in our internal testing.

Vladimir

Re: JPEG/MJPEG encoder line duplication problem

bassraf — Sun, 30 Nov 2008 02:06:53 GMT

Do methods like ippiYCbCrToRGB_8u_P3C4R also make use of OPENMP ?

I'm starting to wonder if these calls might be involved.

I've actually tried all codebases now,i'm starting to get confused.. setting numthreads to 2 works..

Since i've tried so much, i'm not sure anymore what works and what doesn't, but looking at my own previous post.. the decoder from UIC seems to be fine now.. but i'm almost 100% sure the encoder is duplicating lines still..

Btw i'm not destroying/creating the encoder class each frame as I have seen done by others, might this be the culprit? (i'll try)

Bas

Re: JPEG/MJPEG encoder line duplication problem

Vladimir_Dudnik — Tue, 02 Dec 2008 12:27:52 GMT

You may link with non threaded IPP static libraries to exclude any possibilityof infuence of IPP internal threading.

If you can attach piece of code which reproduce UIC JPEG encoder line duplication issue that will help us to investigate the problem.

Regards,
Vladimir