Isolating failure point

[David_M_17]

Previously I collected data on KNL system with Advisor XE (survey).  I want to change the input data set and rerun.  I also want to collect different data.   I am running this the same way - but it now fails to run.  I get a message like hthis:  advixe: Collection stopped.
advixe: Warning: The application returned a non-zero exit value.

Then more error messages.   

My run is something like this: mpirun -np 1 advixe-cl -collect survey -project-dir /<path>/RUNFILES/dirname /<path>/RUNFILES/FILES/Craft.exe  : -np 127 /path/RUNFILES/Craft.exe

Is there a flag I can add to trace what is going on with mpi and advisor xe?   If I understood why it was failing maybe I could modify something.  If I drop advixe and just enter:mpirun -np 1 /<path>/RUNFILES/FILES/Craft.exe  : -np 127 /path/RUNFILES/Craft.exe

the application runs just fine.   How do I trace what is happening so I can debug it?

[David_M_17]

Since my application failed I decided to try one of the intel provided examples.  I followed the directions in README and tried to run the intel  provided example mpi_sample (this is from <path>advisor_2017/samples/en/C++/mpi_sample.tgz).

I cut and pasted commands from the intel provided README file:

make 1_mpi_sample_serial

then

mpirun -n 4 advixe-cl -collect survey --project-dir ./survey -- ./1_mpi_sample_serial 100000000

Intel(R) Advisor Command Line Tool

The results are shown below.  It creates directories - but they are meaningless.  Continuing with instructions of the README provides more Error: messages.

So why does Advisor XE fail on my system?  How do I get it working?

Copyright (C) 2009-2017 Intel Corporation. All rights reserved.
Intel(R) Advisor Command Line Tool
Copyright (C) 2009-2017 Intel Corporation. All rights reserved.
Intel(R) Advisor Command Line Tool
Copyright (C) 2009-2017 Intel Corporation. All rights reserved.
Intel(R) Advisor Command Line Tool
Copyright (C) 2009-2017 Intel Corporation. All rights reserved.
advixe: Collection started. To stop the collection, either press CTRL-C or enter from another console window: advixe-cl -r /work1/people/drmackay/advitest/mpi_sample/try2/rank.1/hs001 -command stop.
advixe: Collection started. To stop the collection, either press CTRL-C or enter from another console window: advixe-cl -r /work1/people/drmackay/advitest/mpi_sample/try2/rank.3/hs001 -command stop.
advixe: Collection started. To stop the collection, either press CTRL-C or enter from another console window: advixe-cl -r /work1/people/drmackay/advitest/mpi_sample/try2/rank.2/hs001 -command stop.
advixe: Collection started. To stop the collection, either press CTRL-C or enter from another console window: advixe-cl -r /work1/people/drmackay/advitest/mpi_sample/try2/rank.0/hs001 -command stop.
advixe: Collection stopped.
advixe: Warning: The application returned a non-zero exit value.
advixe: Opening result  0 % Clearing the database                              advixe: Collection stopped.
advixe: Collection stopped.
advixe: Warning: The application returned a non-zero exit value.
advixe: Warning: The application returned a non-zero exit value.
advixe: Opening result 31 % Processing profile metrics and debug information
advixe: Error: Error 0x4000002a (Database interface error) -- Cannot run data transformation `Add Fake Loop Data'.
advixe: Opening result 99 % done
advixe: Opening result 100 % done
advixe: Error: Data loading failed.
advixe: Error: Unknown error during finalization
advixe: Opening result 31 % Resolving call target names for dynamic code
advixe: Error: Error 0x4000002a (Database interface error) -- Cannot run data transformation `Add Fake Loop Data'.
advixe: Opening result 31 % Processing profile metrics and debug information
advixe: Error: Error 0x4000002a (Database interface error) -- Cannot run data transformation `Add Fake Loop Data'.
advixe: Opening result 99 % done
advixe: Opening result 100 % done
advixe: Error: Data loading failed.
advixe: Error: Unknown error during finalization
advixe: Opening result 99 % done
advixe: Opening result 100 % done
advixe: Error: Data loading failed.
advixe: Error: Unknown error during finalization