Pointer Checker linker errors and robustness

[rwfirstpr_com_au]

I am attempting to use the Pointer Checker with with a trial version of Intel C++ Studio XE for Linux (installed from cpp_studio_xe_2013_sp1_update3.tgz).  I have the compiler integrated under Eclipse 4.3 with CDT as noted in my recent two messages.  This is a Debian 8 (Testing) system and I have removed the GCC 4.9 compiler and libraries so that the Intel compiler can work with the libraries from GCC 4.8.3.  

After reading the documentation I concluded that a minimal use of this facility would involve adding this option to the compiler command line:
-check-pointers=rw
and that a full use of it would involve adding:
-check-pointers=rw
-check-pointers-dangling=all
-check-pointers-undimensioned         

As far as I know, I don't need to add any code to handle this, though I could in order to customize what happens if a problem is detected.

I want to try this on a real program, but first I am trying just the first option with a Hello World program.  This causes about a hundred link errors, starting with:
./src/Hello-World-Intel-C++-1.o: In function `main':
../src/Hello-World-Intel-C++-1.cpp:(.text+0x5d): undefined reference to `__chkp_map_rsi_for_store'
../src/Hello-World-Intel-C++-1.cpp:(.text+0x82): undefined reference to `__chkp_map_rdi_for_store'
makefile:45: recipe for target 'Hello-World-Intel-C++-1' failed
../src/Hello-World-Intel-C++-1.cpp:(.text+0xca): undefined reference to `__chkp_map_rax_for_load'
../src/Hello-World-Intel-C++-1.cpp:(.text+0x10c): undefined reference to `__chkp_cpstore'
../src/Hello-World-Intel-C++-1.cpp:(.text+0x137): undefined reference to `__chkp_map_for_load'

It did not help for me to add:
#include<chkp.h>
or to configure the linker to search the .so libraries, including libchkp.so, in /opt/intel/cpp-studio-xe/composer_xe_2013_sp1.3.174/compiler/lib/intel64 .

Googling reveals no pages where anyone else has such trouble.  Googling "__chkp_map_rsi_for_store" leads to a solitary Intel page: https://software.intel.com/sites/default/files/icl_log.txt which made no sense to me.

Googling: Intel C++ "pointer checker" leads to about 1,630 results.  These seem to be mainly Intel documentation and marketing material, with a few discussions and little evidence of people actually using the Pointer Checker.  This discussion http://blog.regehr.org/archives/939 might be of general interest regarding checking pointers, with some alternative approaches.

I couldn't understand this message, regarding an earlier update of the compiler: http://www.tuicool.com/articles/2EjaM3 .  The only two messages concerning  Pointer Checker in this forum report false positives, and have not been answered.

I want to roughly quantify the performance penalty of Pointer Checker, but can't get past first base with Hello World.  The documentation mentions that the performance hit is too high for production code, but does not quantify it.  I found an Intel presentation, in various forms, such as:
    
    https://software.intel.com/en-us/articles/pointer-checker-to-detect-buffer-problems-and-dangling-pointers-part-2
    
which quantifies the performance penalty: "Runtime cost is high, about 2X-5X the execution time (based on some open source application runs - work in progress and not yet published), and code size increases from 20 percent to 100 percent or more depending on the application."

This is probably sufficient to rule it out for our intended application.  G++'s -D_GLIBCXX_DEBUG facility may be sufficient and my impression is that the performance degradation is minimal, such as less than 10%.  In one test, a fully optimized (-O3 -static) program ran about 5% faster - I don't know why.

If I can get the Intel Pointer Checker operating, I will be able to find out the performance penalty on our code.  However, the dearth of reports of people using it makes me wonder if this is a bleeding edge new development with little history of robust industrial use.

[KitturGanesh]

Hi,
I don't have a Debian 8 system to try out (BTW, Debian 6/7 is what's officially supported for the version you're using). Nevertheless,  it's recommended that you use icc to link, as this will enable other features of compiler to work.

Note that if the -check-pointers:rw switch is given to icc at link-time, the correct libraries will be included automatically.

With manual linking using ld, this should work as below as well:

    -L/opt/intel/composerxe/lib/intel64 -lchkp -lchkpwrap

Yes, there aren't any published performance benchmarks and use of pointer checker feature is considered a debug feature that you
can use for debugging buffer overflows before releasing the product.

Let me know if the above solution resolves the issue?

_Kittur

[rwfirstpr_com_au]

Hi Kittur,

Thanks for your reply, but I am still unable to get the Pointer Checker to work.  

I am using the Intel compiler via the Eclipse integration and this uses the compiler (icpc) to link.  I recall reading that this results in ld being called to do the actual linking, with the compiler potentially doing some other things beforehand, including actions regarding Inter Procedural Optimization between files.

The Eclipse integration automatically propagates my "O3" selection from the compiler section of the project properties settings to the linker section.  I have the "-static" option set in the linker section too.  The resulting commmand line shows that icpc is being called by Eclipse to do the linking.

I first tried adding your suggested "-check-pointers:rw" to the linker command line via project properties > C/C++ Build > Settings > Tool Settings > Intel Intel(R) 64 C++ Linker (v14.0.0) > Miscellaneous > Additional Options.  I got the same results with "-check-pointers=rw".  The command line was:

  icpc -static -O3 -check-pointers=rw -o "(executable file name)" (long list of .o object files)

This generated errors:

  ld: /opt/intel/cpp-studio-xe/composer_xe_2013_sp1.3.174/compiler/lib/intel64/libchkpwrap.a(chkpwrap.o): undefined reference to symbol '__tls_get_addr@@GLIBC_2.3'

   makefile:46: recipe for target '(executable file name))' failed
//lib/x86_64-linux-gnu/ld-linux-x86-64.so.2: error adding symbols: DSO missing from command line

I did not try to figure out what might be behind this.  Instead I tried your other suggestion of adding "-L/opt/intel/composerxe/lib/intel64 -lchkp -lchkpwrap" to the ld command line.  Since I am not driving ld directly, I added a form of this in place of the text mentioned above in the linker ... Additional Options:

  -L/opt/intel/composerxe/lib/intel64 -lchkp -lchkpwrap

For my system, after looking for the correct directory and updating your library names to something which matched the files I found there:

  libchkp.so            30690 Apr 25 23:19
  libchkpwrap.a        116468 Apr 25 23:19
  libchkpwrap_h.a       63110 Apr 25 23:19
  libchkpwrap_h_w.a     44680 Apr 25 23:19
  libchkpwrap_w.a       84334 Apr 25 23:19

I added:

  -L/opt/intel/cpp-studio-xe/lib/intel64 -libchkp -libchkpwrap

This resulted in errors:

  ld: cannot find -libchkp
  ld: cannot find -libchkpwrap

Removing these two items from ... Additional Options generated the long error message which I described initially.

I probably won't pursue this further since I was primarily interested in quantifying the performance slowdown to see if it might be acceptable for production code, but since starting this investigation I read of the 2X to 5X estimate, which I think rules it out.

I had no trouble with the G++ compiler (4.8.3) using the -D_GLIBCXX_DEBUG bounds checking option, as described here:

  https://gcc.gnu.org/onlinedocs/libstdc++/manual/debug_mode.html

With one test program (all this is with O3) which primarily involved vectors and deques, there was a marginal slowdown (5% or so), but a lot of the program's work involved I/O, so this doesn't show the real performance impact for a program which is CPU rather than I/O bound.  With a second program which mainly involved string manipulation, and less I/O, this option caused the executables to run a few percent faster.  This puzzling outcome is explained in bug from 2008: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=38132 which was deemed WONTFIX and apparently applies to libstdc++ before 2008 and to the present day.  String functions are normally not inlined with O3, but turning the bounds checking on enables them to be inlined.

It remains to be seen what slowdown occurs with the numeric and vector based core of the program, which we haven't yet written.  As long as it is 20 to maybe 40% slower, I think this will be a price worth paying for substantial bounds checking.  However, this G++ approach doesn't report line and file numbers (maybe it can be made to do so) and it is for "checking interactions among standard iterators, containers, and algorithms", whereas the Intel Compiler Checker handles plain C-style arrays and perhaps other things due to its operation at the very low level of pointers.

I suggest that the documentation of Pointer Checker could be improved by the addition of a simple example program, with full instructions on compiler and linker switches, an explanation of what libraries need to be linked, what error messages the Pointer Checker generates (without the user writing code to generate these, or with such code if this is mandatory) and which options to select in the Eclipse integration.

Also, kcc wrote in August 2013 of false positives with the Pointer Checker:  

  https://software.intel.com/en-us/forums/topic/413959
  https://software.intel.com/en-us/forums/topic/413960

and there were no replies.

 - Robin

[KitturGanesh]

Hi Robin,

Thanks for pointing out the missing information in our documentation for Pointer Checker. We will fix this in the first upcoming release that we are able to do so. As far as the forum posts that are mentioned below, we acknowledge that these are artifacts of the Pointer Checker design, but do not have these specific improvements planned at this time. 

Here is the information that is missing (including the information that we have provided already):

Pointer Checker cannot be used with the -static flag, which forces all libraries to be linked statically. The reason is that the Pointer Checker library “libchkp.so” must be shared by all executables and libraries in a process without duplication by design. While it is certainly possible to ensure, manually, that only a single library copy is linked, we found in testing that it was quite easy for users to accidentally include multiple copies of the library, causing strange runtime errors. For this reason, we do not allow use of -static with Pointer Checker.

If the system linker “ld” is used to link programs with Pointer Checker, the libchkp.so and libchkpwrap.a library must be included. They are typically found in the /opt/intel/composerxe/lib/intel64 directory. Or, if the “icc” or “icpc” compiler drivers are used to link, these libraries will be automatically included if the “-check-pointers:X” switch is used. Example:

icc -check-pointers:rw a.o b.o c.o 

or

ld a.o b.o c.o -L/opt/intel/composerxe/lib/intel64 -lchkp -lchkpwrap

It is strongly suggested that any program code that manipulates pointers, be compiled with Pointer Checker. If Pointer Checker compiled code makes calls to a function that changes pointer data, and that function is not instrumented with Pointer Checker, the compiler will make conservative assumptions and inaccurate results may occur. Typical C and C++ library calls such as memcpy(), new, etc. are converted by the compiler to Pointer Checker compatible versions automatically.

Hope the above clarifies your issues. Again, thanks for your feedback (on the doc) which I've passed on to the product team.

_Kittur