JPEG/MJPEG encoder line duplication problem

[mojonez]

Hi there,

I'm using the IPP 5.3 JPEG encoder to create an MJPEG encoder. Everything works fine except that an occasional frame will have about a 10 line duplication within the image. I have attached an avi file which illustrates this (e.g. frames 14,27,36,39). Also attached areencoder/decoder logs that list the byte counts. It occurs whether using the Intel MJPEG decoder (using the Simple Player sample ) or else other third party decoders via windows media player.

Things I'vetested which have lead me to isolate the encoder::

• Source image data seems OK (recording withother encoders or just raw output does not display this artifact).
• Other decoders also display this artifact, seeming to indicate its not in the decoder.
• The byte increasefor the erroneous frame seems to be approximately the amount of bytes for the number of duplicate lines.
• Same problem with different data sources (cameras - color/grayscale, etc)

System specs:

Intel Core2 Quad
Windows XP
Intel IPP 5.3.3.082
Statically linked Intel Libs
Statically linked Intel audio/video libs ( built usingstatic c rtuntime (/MT) )
MFC Test app (static C runtime (/MT) )
CPU usage : ~ 25%

Here's the encode routine.

pSrc is an simple rgb array
width640
height480

int MJPEGEncoder::EncodeFrame(unsigned char* pSrc, int nSrcBytes, unsigned char* pDest, int nDestBytes, int* pnDestBytesUsed)
{int iResult = 0;
JERRCODE jerr;
IppiSize imgSize;
CJPEGEncoder encoder;
CMemBuffOutput jpegDest;
JMODE mode = JPEG_BASELINE;
JCOLOR color = JC_YCBCR;
JSS sampling = JS_422;int step = (m_iWidth * 3) + DIB_PAD_BYTES(m_iWidth, 3);int quality = 85;int nSourceChannels = 3;
imgSize.width = m_iWidth;
imgSize.height = m_iHeight;
MyTimer timer;
timer.Start();
jpegDest.Open(pDest, nDestBytes);
jerr = encoder.SetSource((Ipp8u*)pSrc, step, imgSize, nSourceChannels, JC_RGB);if(jerr != JPEG_OK)
{if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Error setting source [err code = %d].
", jerr);return jerr;
}
jerr = encoder.SetParams(mode, color, sampling, 0, 0, quality);if(jerr != JPEG_OK)
{if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Error setting params [
err code = %d].
", jerr);return jerr;
}
jerr = jpegDest.Open(pDest, nDestBytes);if(jerr != JPEG_OK)
{if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Error opening destination [err code = %d].
", jerr);return jerr;
}
jerr = encoder.SetDestination(&jpegDest);if(jerr != JPEG_OK)
{if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Error setting destination [err code = %d].
", jerr);return jerr;
}
jerr = encoder.WriteHeader();if(jerr != JPEG_OK)
{if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Error writing header [err code = %d].
", jerr);return jerr;
}
jerr = encoder.WriteData();if(jerr != JPEG_OK)
{if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Error writing data [err code = %d].
", jerr);return jerr;
}
timer.Stop();double dMillisecs = timer.Duration() * 1000.0f; // millisecs
*pnDestBytesUsed = encoder.NumOfBytes();if(m_bWriteTraceFile)
fprintf(m_fpTrace, "[MJPEG] Frame Encoded. [Time(msecs):%3.2f	nInBytes:%d	nOutBytes:%d	nBytesAvailable:%d
", dMillisecs, nSrcBytes, *pnDestBytesUsed, nDestBytes);return iResult;
}
Again, it works fine, except 1 in every n frames or so will have a duplicated segement, where n tends to vary.
If anyone has any thoughts or suggestions I really appreciate it.
Morgan

[Vladimir_Dudnik]

Hello Morgan,

Thanks for detailed info you provided. I really see no any major issues in code you provide, except you calljpegDest.Open(pDest, nDestBytes) twice, but that should not lead to an issue in this particular case. I would recommend you to store each frame as a separate JPEG file to verify that wrong data comes from JPEG encoder and not corrupted somehow at AVI creation time.

Note, you may want to call get_num_threads() or set_num_threads()functions (defined in jpegbase.h) to see if that behaviour is related to threading implemented in encoder.

Regards,
Vladimir

[sahil_malhotra]

If you could provide a tarball of your source along with Makefiles and a screenshot of your core usage while the code is running, I can have a look at it. actually i'm struggling with something like this myself in my robotics projects.

[mojonez]

Thanks for the reply, Vladimir.

Same problem occurs when saving each frame as a JPEG. Attached image shows the problem. Around the chin area the lines are duplicated. It looks like exactly 8 lines are duplicated, which i think is the same as the encoder's block size.

Calling get_num_threads returns 4.

However, changingthenum threads to 1 -the problem seems to have gone away, with the exception that i now get an occasional 4x4 pixel flickering light grey block in the top-right corner of the image, however thisi can work around.

testing with different threads count:

numthreads 1 - OK (no line duplicates)
numthreads 2 - OK (no line duplicates)
numthreads 3 - BAD (line duplicates)
numthreads 4 - BAD (line duplicates)

So, I'll leave her at 2 threads and call it good.

Thanks so much for the help. I don't think i would have discovered that on my own.

Morgan

[mojonez]

Hi Sahil,

Attached is a screenshot of the CPU usage.

I'm not able tosend out app source code.

Are you getting line duplication as well? If so, is it 8 pixels high?

[sahil_malhotra]

My line duplication problem was more of a corner pixels problem... turned out about stack space overlaps .... my corner pixels were becoming part of multiple threads stack space when split the image into four data structures on my quad.... But yeah a duplication problem it was.... reading your output... when you go over 2 hardware threads , one or more of your threads are reading the image wrong and writing the final pixels right.... means they are duplicating outputs from other threads and putting them in the places they were supposed to at splitting time.... and looking at your problem youve split them int horizontal segments....

[bassraf]

Hi guys,

I'm using IPP 6.0 (final), and i'm still getting artifacts, shifted lines, flashes etc.when encoding (decoding is fine) to JPEG using OPENMP and MSVC2008 Pro. Setting numthreads to 2 on a quadcore (intel) CPU fixes this.

Anyone has a resolution for this, encoding is most important for us.

Best regards,

Bas

[Vladimir_Dudnik]

Which of several IPP JPEG codecs doyou use, JPEG codec from UIC or JPEGView sample or UMC MJPEG codec?

Vladimir

[bassraf]

Hi Vladimir, thanks for your quick response.

I'm using the UMC JPEG code.

Looking at your question I assume I should use a different codebase (what/why are they different anyway?) ?

Bas

[Vladimir_Dudnik]

Yes, some differences are possible between JPEG codec in audio-video-codecs sample and image-codecs sample. Usually, most up to date codec is kept in image-codecs sample and then it might be ported to MJPEG codec.

Regards,
Vladimir

[bassraf]

So should I be using the code in image-codecs/UIC/src/codec/image/jpegor image-codecs/jpegview/jpeg ?

(i'm now using the latter,and encoding seems to be fine, but decoding now has artifacts??)

Regards,

Bas

[bassraf]

Ok, looking at stills from the decoder (compiled using OPENMP MSVC) there are multiple line duplications of 8 lines taking place. Weird thing is, the encoder used to have artifacts, but with this codebase (from image-codecs/jpegview/jpeg) the decoder has the duplication problem when using multiple cores (everything > 1 core has the duplication problem)

Bas

[Vladimir_Dudnik]

Are you saying that UIC JPEG codec have artifacts at encoder and line duplication problem with decoder? We do not see that behaviour in our internal testing.

Vladimir

[bassraf]

Do methods like ippiYCbCrToRGB_8u_P3C4R also make use of OPENMP ?

I'm starting to wonder if these calls might be involved.

I've actually tried all codebases now,i'm starting to get confused.. setting numthreads to 2 works..

Since i've tried so much, i'm not sure anymore what works and what doesn't, but looking at my own previous post.. the decoder from UIC seems to be fine now.. but i'm almost 100% sure the encoder is duplicating lines still..

Btw i'm not destroying/creating the encoder class each frame as I have seen done by others, might this be the culprit? (i'll try)

Bas

[Vladimir_Dudnik]

You may link with non threaded IPP static libraries to exclude any possibilityof infuence of IPP internal threading.

If you can attach piece of code which reproduce UIC JPEG encoder line duplication issue that will help us to investigate the problem.

Regards,
Vladimir