topic Re: valgrind reports errors with zlib in Intel® Integrated Performance Primitives

valgrind reports errors with zlib

mwrighton — Mon, 14 Dec 2009 23:01:53 GMT

Hi,
I've replaced our zlib code with the intel version. We now see several classes of valgrind errors:

==28737== Memcheck, a memory error detector
==28737== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==28737== Using Valgrind-3.5.0 and LibVEX; rerun with -h for copyright info
==28737== Command: /home/mwrighton/work/t3/output/linux-x86_64-64/bin-gcc-4.3.3-debug/stylusdbg doit.tcl
==28737== Parent PID: 6128

==28737== Conditional jump or move depends on uninitialised value(s)
==28737== at 0x4E975B8: ippGetCpuFeatures (in /home/mwrighton/work/t3/output/linux-x86_64-64/bin-gcc-4.3.3-debug/...)
==28737== by 0x4E96BC1: ippStaticInit (in /home/mwrighton/work/t3/output/linux-x86_64-64/bin-gcc-4.3.3-debug/...)

==28737== Thread 2:
==28737== Conditional jump or move depends on uninitialised value(s)
==28737== at 0x4E74247: u8_ippsInflate_8u (in /home/mwrighton/work/t3/output/linux-x86_64-64/bin-gcc-4.3.3-debug/...)
==28737== by 0x4E64DE3: tzl123_inflate (in /home/mwrighton/work/t3/output/linux-x86_64-64/bin-gcc-4.3.3-debug/...)
==28737== by 0x4D4E61B: Infra_DsrcZlib::UncompressMore() (infradsrczlib.cpp:128)
==28737== by 0x4D4E7CE: Infra_DsrcZlib::GetBytes(void*, unsigned long) (infradsrczlib.cpp:163)

==28737== Source and destination overlap in memcpy(0xf229e20, 0xf231e1c, 32768)
==28737== at 0x81754EA: memcpy (mc_replace_strmem.c:482)
==28737== by 0x4E6091E: tzl123_deflate (in /home/mwrighton/work/t3/output/linux-x86_64-64/bin-gcc-4.3.3-debug/...)
==28737== by 0x4D4D7F6: Infra_DsinkZlib::CompressBytes(int) (infradsinkzlib.cpp:174)
==28737== by 0x4D4D936: Infra_DsinkZlib::PutBytes(void const*, unsigned long) (infradsinkzlib.cpp:155)
==28737== by 0x4D74929: Dgen_DataOutput::Flush() (dgendataoutput.cpp:146)

We had the 'inflate' seems analogous to one that has to be suppressed in the 'official' zlib release, but not the memcpy one. Has anyone else seen this?

Re: valgrind reports errors with zlib

Alexander_Naduev__In — Tue, 15 Dec 2009 09:42:34 GMT

Hello mwrighton,
you are the first person who notified us about such problem. Nevertheless, I see the memcpy issue was caused by Infra_DsinkZlib class - it isn't a part of the ipp_zlib sample. Please, test it first of all. Thank you.

Re: valgrind reports errors with zlib

mwrighton — Tue, 15 Dec 2009 20:49:52 GMT

Quoting - Alexander Naduev (Intel)

No, please read it more carefully. infra_dsinklib merely calls zlib, which then does the overlapping call.

I also test ipp_minigzip, it exhibits the same sympton:

[mwrighton@sina bin]$ valgrind ./ipp_minigzip junkfile
==1136== Memcheck, a memory error detector
==1136== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==1136== Using Valgrind-3.5.0 and LibVEX; rerun with -h for copyright info
==1136== Command: ./ipp_minigzip junkfile
==1136==
==1136== Conditional jump or move depends on uninitialised value(s)
==1136== at 0x43BE88: ippGetCpuFeatures (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/ipp_minigzip)
==1136== by 0x43B491: ippStaticInit (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/ipp_minigzip)
==1136== by 0x4461B5: ??? (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/ipp_minigzip)
==1136== by 0x40110A: ??? (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/ipp_minigzip)
==1136== by 0x44614F: ??? (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/ipp_minigzip)
==1136== by 0x446110: __libc_csu_init (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/ipp_minigzip)
==1136== by 0x386841C39E: (below main) (in /lib64/tls/libc-2.3.4.so)
==1136==
==1136== Source and destination overlap in memcpy(0x4a3fb90, 0x4a47a8c, 33028)
==1136== at 0x49074EA: memcpy (mc_replace_strmem.c:482)
==1136== by 0x4050A0: tzl123_deflate (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/ipp_minigzip)
==1136== by 0x4020DA: tzl123_gzwrite (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/ipp_minigzip)
==1136== by 0x401734: gz_compress (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/ipp_minigzip)
==1136== by 0x401838: file_compress (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/ipp_minigzip)
==1136== by 0x401A44: main (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/ipp_minigzip)
==1136==
==1136==
==1136== HEAP SUMMARY:
==1136== in use at exit: 0 bytes in 0 blocks
==1136== total heap usage: 10 allocs, 10 frees, 417,145 bytes allocated
==1136==
==1136== All heap blocks were freed -- no leaks are possible
==1136==
==1136== For counts of detected and suppressed errors, rerun with: -v
==1136== Use --track-origins=yes to see where uninitialised values come from
==1136== ERROR SUMMARY: 3 errors from 2 contexts (suppressed: 7 from 7)

Re: valgrind reports errors with zlib

Alexander_Naduev__In — Wed, 16 Dec 2009 11:46:48 GMT

Hello mwrighton,
Please, could you provide the following information:
1. ipp_zlib version;
2. exact string number in deflate.c file, where the memcpy's call, which leads to the memory overlap,is situated. I couldn't find out it from your previous listings.
Thank you.

Re: valgrind reports errors with zlib

mwrighton — Wed, 16 Dec 2009 18:10:12 GMT

Quoting - Alexander Naduev (Intel)

IPP Version: 6.1.2.051
l_ipp-samples_p_6.1.2.056 ( we modified to have the tzl123 prefix)

I recompiled it with debug symbols:
[mwrighton@sina linux-x86_64]$ valgrind ./ipp_minigzip ./junkfile
==32727== Memcheck, a memory error detector
==32727== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==32727== Using Valgrind-3.5.0 and LibVEX; rerun with -h for copyright info
==32727== Command: ./ipp_minigzip ./junkfile
==32727==
==32727== Conditional jump or move depends on uninitialised value(s)
==32727== at 0x441B68: ippGetCpuFeatures (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/linux-x86_64/ipp_minigzip)
==32727== by 0x441171: ippStaticInit (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/linux-x86_64/ipp_minigzip)
==32727== by 0x40FAAC: IPP_ZLIB_STATIC_INIT::IPP_ZLIB_STATIC_INIT(int) (ipp_static.cpp:28)
==32727== by 0x40FA78: __static_initialization_and_destruction_0(int, int) (ipp_static.cpp:31)
==32727== by 0x40FA8D: global constructors keyed to link_to_ipp_tzl123_lib (ipp_static.cpp:31)
==32727== by 0x44BE95: ??? (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/linux-x86_64/ipp_minigzip)
==32727== by 0x401142: ??? (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/linux-x86_64/ipp_minigzip)
==32727== by 0x44BE2F: ??? (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/linux-x86_64/ipp_minigzip)
==32727== by 0x44BDF0: __libc_csu_init (in /home/mwrighton/work/t3/infra_other/libtzl123/ipp_zlib/bin/linux-x86_64/ipp_minigzip)
==32727== by 0x386841C39E: (below main) (in /lib64/tls/libc-2.3.4.so)
==32727==
==32727== Source and destination overlap in memcpy(0x4a3fb90, 0x4a47a96, 33018)
==32727== at 0x49074EA: memcpy (mc_replace_strmem.c:482)
==32727== by 0x408292: deflate_common (deflate.c:1833)
==32727== by 0x4053D9: tzl123_deflate (deflate.c:788)
==32727== by 0x402D66: tzl123_gzwrite (gzio.c:628)
==32727== by 0x40154B: gz_compress (minigzip.c:120)
==32727== by 0x401738: file_compress (minigzip.c:214)
==32727== by 0x401A6D: main (minigzip.c:317)
==32727==
==32727==
==32727== HEAP SUMMARY:
==32727== in use at exit: 0 bytes in 0 blocks
==32727== total heap usage: 10 allocs, 10 frees, 417,147 bytes allocated
==32727==
==32727== All heap blocks were freed -- no leaks are possible
==32727==
==32727== For counts of detected and suppressed errors, rerun with: -v
==32727== Use --track-origins=yes to see where uninitialised values come from
==32727== ERROR SUMMARY: 51 errors from 2 contexts (suppressed: 7 from 7)

1833 is UPDATE_WINDOW below:

while( (s->lookahead > 0) || (s->strm->avail_in > 0) ) {
if( s->strm->avail_in <= wsize ) {
UPDATE_WINDOW(s->strm->avail_in);
{ /* block */
int read_length = MIN(s->strm->avail_in, s->window_size - s->strstart - s->lookahead);
s->lookahead += read_buf( s->strm, s->window + s->strstart + s->lookahead,

Re: valgrind reports errors with zlib

Alexander_Naduev__In — Thu, 17 Dec 2009 07:48:01 GMT

OK, I understand the main reason of the issue - src and dstpointto the same buffer, but I think it isn't real issue, because difference between src and dst pointers is enough large in this case. Nevertheless, I'm goingto use ippsMove_8u instead ofzmemcpy in further IPP versions for avoiding such issues. Thank you.