IPP JP2 encoder can't generate the lossless bitstream for 16bit image

[kjhinfinitt_com]

I'm using IPP 6.1.5.

Using uic, I compressed a 16bit gray image with WT53/MCT losslessly and decompressed the bitstream using uic too.

But I found that the decompressed image is slight different from the original image.

It can't be differentiated by mere eyes, but failed with binary compare.

In my case, about 900 pixels are different by pixel value 1 from the original pixels, among 512x512 pixels.

When I decompressed the bitstream using other tool, that false image was produced, too.

So I'm sure it's an encoder issue.

Any idea?

[Thomas_Jensen1]

Maybe a side-track, but I never understood why UIC allows MCT (Multi Component Transform) to be enabled for grayscale images (single component).

[Vladimir_Dudnik]

Hello,

First of all, let's differentiate issues related to sample application from issues related to UIC JPEG2000 codec. You have intentionally used MCT (which is basically color convertion step) then why do you expect bitwise parity? Note, MCT is enabled in UIC picnic application because JPEG2000 standard does not put any limitrations on it use regardless of lossy or lossless compression mode.

Regards,
Vladimir

[kjhinfinitt_com]

Hi, all.

I don't put the MCT flag intentionally, it was a mistake.

But it's not related with the MCT flag, because it's almost same even without the MCT flag.

When I compressed the image with no-MCT and WT53, the bitstream is not lossless, either.

[Vladimir_Dudnik]

We did not see such issue with JPEG2000 codec before. Could you please describe in details what steps you did to detect that issue?

Regards,
Vladimir

[Vladimir_Dudnik]

Hello,

do you still have this issue with the latest version of IPP 7.0? We still not reproduce this problem on our side.
If you could share your source image for our investigation it may help to understand the root cause of issue (if there are any).

Regards,
Vladimir

[Thomas_Jensen1]

I didn't report this issue at first, but I now tested it carefully, and I found that IPP 7 does have a severe show-stopper bug in the UIC JPEG2000 Lossless sample for 16 bit decoder (not encoder as it seems).

When using 16u images and decoding a JP2, LL file, it decodes without error messages to a faulty image.
Condition = Pixels has high values (values in the upper part of 64K).
Faulty image = bright area has become fully white.

When using 16u images and decoding a JP2, LL file, it decodes without error messages to a proper image.
Condition = Pixels has only low values (values in the lower part of 64K).
Proper image = pixel-for-pixel exact.

IPP 6.1 can decode the JP2 properly.

My test:
IPP = w_ipp_7.0.1.127_ia32
Test program = w_ipp-uic_p_7.0.1.046.zip = ipp-samples\image-codecs\uic\bin\ia32\picnic.exe
(used as-is, not recompiled)

1. Run picnic.exe
2. Load 16u_C1 uncompressed tiff with high values (displayed properly).
3. Save to an JP2 LL MCT=off file (file is proper).
4. Load JP2 file (image is faulty).
5. Load JP2 file using IPP 6.1 (image is proper).

You can reproduce using Photoshop:
1. Load any image.
2. Set Mode = Grayscale, 16 bit.
3. Auto-Levels (to ensure image has high pixel values).
4. Save As Tiff.

Use this 16 bit grayscale image with high pixel values when testing.

[Vladimir_Dudnik]

Hi Thomas,

thanks for reporting that issue, we will check and if possible update UIC JPEG2000 codec in the nearest IPP release (IPP 7.0.2 expected early next year).

Regards,
Vladimir

[Sergey_Ryadno]

Hi Thomas,

i reproduced this bug on my side, found root case of it and fixed it. I hope, the fix for this issue wil be available in next IPP release.

[Thomas_Jensen1]

May I ask, is your fix in the samples domain or in the IPP domain?
If in sample domain, can I see your fix?

[Vladimir_Dudnik]

Hi Thomas,

the fix is on sample level, you should modify jpeg2k.cpp file in UIC applications (transcoder and picnic) as following (basically change Ipp16s type to Ipp16u type):

line 470:im_err = IppConvert(imagePn.Buffer().DataPtr()[0].p32s, dataOrderPn.LineStep()[0], (Ipp16u*)ptr, image.Step(),size,1);

line 525:Ipp16u* ptr = image;

line 541:ptr = (Ipp16u*)((Ipp8u*)ptr + image.Step());

Regards,
Vladimir

[Thomas_Jensen1]

I understand the fix. I haven't gotten to test it yet.

However, I see more sample code for jpeg2000 that seems to suffer from 16s types, that possibly should be 16u types:

"im_err = IppThreshold(0, (1 << (image.Precision())) - 1, tmpRow, sz.width*3);"

The function IppThreshold only exists in 16s and 32s overloads, so tmpRow will be 16s, which is wrong, since we are working on 16u image pixels.

The same goes for IppConvert()

[Gennady_F_Intel]

Hi Thomas,

IPP 7.0 Update 2 is available now,the issue should be fixed in the vesion. please refer to bleow information to get more information

Version 7.0.2 of the Intel Integrated Performance Primitives (Intel IPP) library is now available. The library is available as astand-alone productor as a component inIntel Parallel Studio 2011,Intel Parallel Studio XE 2011,Intel C++ Studio XE 2011,Intel Composer XE 2011, andIntel C++ Composer XE 2011. Please visit theIntel Software Evaluation Centerto download evaluation versions of these and other Intel software products.

What's New in 7.0.2

• Additional optimizations for the256-bit Intel AVXSIMD instruction set.
• FurtherIntel Atom Processor optimizationshave been incorporated.
• Data compression gains of ~2x via a redesign of the "Inflate for Fixed Huffman" algorithm.
• Cryptography improvements include 1.4x faster "sign verify" and a 2x faster ippsMontForm() function.
• The ipp_zlib library is now based on the zlib 1.2.5 distribution.
• Review a complete listof changes, bug fixes and new features on-line.

--Gennady

[dexylitol]

Hi, all,

I was the original issuer (changed my login id. sorry)

I tested the above thing using the latest IPP 7.0.4 and uic samples.And the problem is still there.

Using the attached file (CT.org.pnm - I made it with the 16 bit pixel data out of CT image), I did the followings.

uic_transcode_con.exe -i CT.org.pnm -o CT.jp2 -l 1

uic_transcode_con.exe -i CT.jp2 -o CT.rec.pnm

Then, I compared the pixel data of the two files; CT.org.pnm and CT.rec.pnm.

There are lots of pixels having different pixel values. (Interestingly, all the pixel differences are 1)

I'm not sure where the problem is (decoder-side or encoder-side).

Actually., I reported this issue in premier.inte.com last year and they closed the issue saying that the issue was fixed in IPP 7.0.1. But it still exists in 7.0.4.