IPP ZLIB sample issue

[andrei_m]

Hello!

Our windows application uses libpng to decode PNG files, libpng in turn uses zlib library for decoding. Currently we are evaluating the IPP library to test if there will be any speed improvements. So I have downloaded your 5.3 IPP_ZLIB sample and replaced our original zlib with the optimized one and it seems to be failing on decoding. libpng retuns "Not enough image data" error. Further brief investigation showed that there is an issue with the inflate() function in zlib. Project was compiled on VS 2005 without openmp flag.

Unfortunately I haven't done any deep investigation, maybe somebody have expirienced the same issue and have a solution?

Thanks,
Andrei.

[Vladimir_Dudnik]

You may need tocheck modifications required to functions from jdatasrc.c and jdatadst.c files which are usually part of application rather than libjpeg library. I would refer you to Intel IJG and Image Artefactsarticle I found on the WEB which explain this a bit.

There also is known limitation of modified libjpeg, namely it does not support suspended operations because of bits prefertching in IPP huffman function. If your application use suspended mode you may turn off only IPP huffman functions by undefining IPPJ_HUFF macro in jconfig.h file. Doing this you still be able to use IPP optimized IDCT and color conversion routins which should maintain performance advantage over original non-optimized libjpeg code.

Regards,
Vladimir

[xiazhou_cnd]

Thanks for the reply. Ihave a question aboutthe suspended operation issue:it's valid for bothI/O or only for I or only forO?

Quoting - Vladimir Dudnik

You may need tocheck modifications required to functions from jdatasrc.c and jdatadst.c files which are usually part of application rather than libjpeg library. I would refer you to Intel IJG and Image Artefactsarticle I found on the WEB which explain this a bit.

There also is known limitation of modified libjpeg, namely it does not support suspended operations because of bits prefertching in IPP huffman function. If your application use suspended mode you may turn off only IPP huffman functions by undefining IPPJ_HUFF macro in jconfig.h file. Doing this you still be able to use IPP optimized IDCT and color conversion routins which should maintain performance advantage over original non-optimized libjpeg code.

Regards,
Vladimir

[Vladimir_Dudnik]

Both input and output operations affected.

Vladimir

[xiazhou_cnd]

It seems that the code doesn't work if IPPJ_HUFF is turned off. I have tested a small jpeg.It is working OK with IPPJ_HUFF on, butwhen IPPJ_HUFF is off, IPP's libjpeg returns error saying that there are extra bytes beforethe end of image and yields a corrupted image.

However, If IPPJ_HUFF is on,large images are not handled correctly due to that, I guess, in my testing program, the suspended mode is requiredfor streaming.

Another thing I feel very uncomfortable is that, though I have offered IPP's libjpeg the whole image, it is stillcalling back of fill_input_buffer function again and again with some data left in the buffer. The only that I can do is just returning the call. This behavior doesn't yield wrong results, but very annoying and cpu wasting.

I don't how to attach files here. Otherwise, I would provide the test image and the processing log details.

[Vladimir_Dudnik]

Thanks for reporting on this. The quick instructions on how to attach files you may find at the second top thread on this forum.

Regards,
Vladimir

[xiazhou_cnd]

I am still unable to upload images. When I click Edit, there is no "upload" but "UpDate". Click UpDate, the server seems trying to do something, but nothing happens eventually.

Here is my test case.

The image is processed by a loop, first decoded (read_scanlines) and then re-encoded (write_scanlines) one scanline a time. The quantization tables are generated with quality=70. With IPPJ_HUFF is on, the image is processed correctly. But IPP outputs a corrupted image when IPPJ_HUFF is off. The following is from the log file:

The total size of the original 60x45 image (original.jpg) is 4817 bytes. The input buffer (next_input_byte) is initially fed with 1972 bytes. After 15th scanline is processed, IPP calls fill_input_buffer with 70 bytes left in the buffer. Another 871 bytes are appended (now bytes_in_buffer = 941). After 23rd scanline is processed, IPP calls fill_input_buffer again with 24 bytes left in the buffer. Another 1950 bytes are appended (now bytes_in_buffer = 1974). Then IPP finishes all scanlines and output a corrupted image (processedNcorrupted.jpg) with 1445 bytes. Please note that there are still 24 bytes needed to be read. Surprisingly enough, IPP calls fill_input_buffer again (after finishing output the whole image!) with bytes_in_buffer = 0. The last 24 bytes are fed and then IPP complains "Corrupt JPEG data: 158 extraneous bytes before marker 0xd9".

[xiazhou_cnd]

Ok. I finally figured out how to upload. Here are the images that I was talking about.

[xiazhou_cnd]

A question about IPP Zlib: Does the same limitation of the bits prefetching in IPP huffman function apply to IPP Zlib, i.e., does IPP Zlibsupport streaming?

Quoting - Vladimir Dudnik (Intel)

You may need tocheck modifications required to functions from jdatasrc.c and jdatadst.c files which are usually part of application rather than libjpeg library. I would refer you to Intel IJG and Image Artefactsarticle I found on the WEB which explain this a bit.

There also is known limitation of modified libjpeg, namely it does not support suspended operations because of bits prefertching in IPP huffman function. If your application use suspended mode you may turn off only IPP huffman functions by undefining IPPJ_HUFF macro in jconfig.h file. Doing this you still be able to use IPP optimized IDCT and color conversion routins which should maintain performance advantage over original non-optimized libjpeg code.

Regards,
Vladimir

[Vladimir_Dudnik]

In IPP ZLIB different huffman function is used, so it does not have such feature as prefetching of bits

Vladimir

[nemorino]

Quoting - xiazhou_cnd

Ok. I finally figured out how to upload. Here are the images that I was talking about.

I have same problem.

I'm using l_ipp_ia32_p_6.1.1.042 and l_ipp-samples_p_6.1.1.050.

In

#ifndef IPPJ_HUFF
src->pub.fill_input_buffer = fill_input_buffer;
#else
src->pub.fill_input_buffer = fill_input_buffer_intellib;
#endif
...


#ifdef IPPJ_HUFF
METHODDEF(boolean)
fill_input_buffer_intellib (j_decompress_ptr cinfo)
{
my_src_ptr src = (my_src_ptr) cinfo->src;
size_t bytes_left = src->pub.bytes_in_buffer;
size_t bytes_to_read = INPUT_BUF_SIZE - bytes_left;
size_t nbytes;

if(src->start_of_file || cinfo->progressive_mode)
{
WARNMS(cinfo, JWRN_JPEG_EOF2);
return fill_input_buffer(cinfo);
}

MEMCOPY(src->buffer,src->pub.next_input_byte,bytes_left);
.....


But, file is corrupted and
$convert -resize 400x300 aaa.jpg bbb.jpg
convert: Premature end of JPEG file
...

What's wrong?

[Vladimir_Dudnik]

Can you check this with IPP IJG samples? There is ijg_timing and cjpeg/djpeg applications which you can try to ensure correct JPEG files generated.

Regards,
Vladimir