How to attach debugger to third-party .EXE to test a DLL?

[anthonyrichards]

I have IVF composer XE 2011 with VS2005.
I have a DLL which is used by a third-party app. I copy the .DLL to a specified folder where the app looks for it.
The app finds the DLL OK.
I want to debug the DLL - how do I start the app with the debug DLL and get the VS debugger attached so it can break into the DLL where I want it to?

I have tried Project properties..configuration properties...debugging and set the command line to start the APP and added the required argument to start it. This appears to do something,
When I 'Start debugging' the app starts but when it gets into my DLL, the VS debugger appears not to be activated, no break points are sensed, no debug windows open etc.

What can I do to link the App+my DLL with the VS debugger?

Previously compiling the DLL with the Compaq VF, I would 'Select executable for debug session' under project properties...debug and I would have no problem starting the app and getting into the DevStudio debugger

[Wendy_Doerner__Intel]

I would recommend you build your DLL with the debug switches and link to the debug versions of the libraires. You coulddo this from a project in Microsoft Visual Studio.

Then create a project inside Microsoft Visual studio where youcan put the 3rd party app.Make sure your DLL is in the same directory as the third party app. And make the 3rd party app project dependent on the DLL. You can either set breakpoints in the DLL or step into the third party app until you get your DLL with source. Please See the Microsoft Visual Studio Debugger for more details on how to do this.

Our documentation has more details about how to set debug switches.
I have excerpted the relevant section below on debugging.

Hope these suggestions help,

------

Wendy

Attaching or including files in a post

Preparing for Debugging using Microsoft Visual Studio*

The following applies to Windows* operating systems only.

To prepare your program for debugging when using the integrated development environment (IDE):

1. Start the IDE (select the appropriate version of Microsoft Visual Studio in the program folder).

2. Open the appropriate solution (using the Solution menu, either Open Solution or Recent Projects).

3. Open the Solution Explorer View.

4. To view the source file to be debugged, double-click on the file name. The screen resembles the following:

   
   <>

5. In the Build menu, select Configuration Manager and select the Debug configuration.

6. To check your project settings for compiling and linking, select the project name in the Solution Explorer. Now, in the Project menu, select Properties, then click the Fortran folder in the left pane. Similarly, to check the debug options set for your project (such as command arguments or working directory), click the Debugging folder in the Property Pages dialog box.

7. To build your application, select Build>Build Solution.

8. Eliminate any compiler diagnostic messages using the text editor to resolve problems detected in the source code and recompile if needed.

9. Set breakpoints in the source file and debug the program.

[anthonyrichards]

Thanks for your suggestion, but I did not perhaps make clear that the third-part app is an .EXE which I can just run. Like MS EXCEL say.
It is straightforward using CVF to debug a DLL that EXCEL calls by just stipulating that EXCEL.EXE be started to start the debug session. By directing EXCEL to the Debug version of a DLL using the standartd Visual Basic Declare statements will allow the debugger to be attached and to go into action as soon as EXCEL calls a routine in the DLL that contains a breakpoint.

In the present case, I copy the debug DLL to the app's folder so that it is found and loaded when the app starts.
I just want the IVF debugger to be attached when I run the APP (execute its .EXE file) and when it calls a routine in my DLL that contains a break point.

[Andrew_Smith]

Your approach is essentially the same as mine but it works for me.

After you start your app, the output window should show it has loaded your dll so you can confirm the correct one was loaded.

You could compile your dll directly to your app folder and avoid the copy step.

You can also use the Debug: attach dialog if you want to start your app outside VS.

[Andrew_Smith]

I mean the output window in Visual Studio

[Wendy_Doerner__Intel]

My instructions were for putting the third party app .exe in a project in Microsoft Visual Studio. That is the way we have done it, but it sounds like you are getting some other guidance closer to what you want to do from others on this thread.

------

Wendy

Attaching or including files in a post

[bmchenry]

one other option (which may be in one of the prior responses) is the following:
Install the 3rd party exe to thedebug folder for your 'startup project' dll in your solution
(for example I have a 5 dll solution which are all'run' by a 3rd party EXE)

then in Configuration->Debugging
set 'Command' to the exe (with path) of the'external' exe program

This works efficiently for me
I would guess you can also set it to the EXE installation folder but then if you havethe dlls create debug output it becomes a nightmare to figure out where the output will end up

(To debugI simply start debug which launches the 'external exe' and when my routine(s) get called by the 'external exe' i have traps to catch/stop/do whatever

Be sure to also set the 'Output Directory' to $(SolutionDir)$(ConfigurationName) for all your dlls

the reason i do it this way is then ALL output (when you write output to check, etc) is in the debug directory

[anthonyrichards]

I am having no luck with this.
It appears to matter where the debug version of the DLL is kept.
When using Visual Studio, suppose that the Fortran code is in folder /mydll/ and the created debug DLL mydll.dll and its object files and the debug information (mydll.PDB ) is in folder /mydll/debug/.

I have to copy mydll.dll to the third-party application's folder /thirdparty/umr/, as that is where the application expects to find it and to load it from. I cannot point the application to an alternative dll folder.

So when I start the third-party application, it loads and runs with the copy of my dll that I copied to /thirdparty/umr/ loaded.

In such a situation, how can the Visual Studio debugger break in to the mydll.dll copy that is in the /thirdpaty/umr/ folder and start stepping through the Fortran code that only exists in the /mydll/ folder?

I have tried copying the third-party application into the mydll/debug/ folder and starting the debug session by executing it. The debugger does not break in to the dll code when expected, even though all the debug info is available in the /mydll/debug/ folder. I think that this is because the third-party application loads the copy that is in its /thirdparty/umr/ folder and teh Visual Studio debugger 'knows' nothing about it and cannot connect to it.

[bmchenry]

it is not 'copy' the third party application, it is 'install' the 3rd party application to the debug forlder for your dll.
try that and it should work.
Have a separate test system orwhen ready torun the release,re-install the 3rd party application to the expected folder, include your needed dlls, and voila!
release ready!
(use http://www.dependencywalker.com/to check for what you need to supply to 3rd party)

[IanH]

You are really-really-sure that it is your most-recently-built debug DLL that's being loaded? (Did you followed Andrew's suggestion of examining the output window?) If you can "break" into the program after it has loaded your DLL (perhaps engineer something in your DLL's code, such as a long delay, to make that easier) what does VS2005 say under Debug > Windows > Modules for your DLL? If you then right click on your DLL, what does it say for the Symbol Load Information? Can it find the symbols?

The DLL will (or should) have in it a hard coded path to the PDB with the debug information that was generated by the linker at the time the DLL was built. When the application loads this (or any) DLL the VS 2005 debugger is notified and goes looking for the PDB file - if the DLL that the application has loaded matches one that it has in hand for your project then I would expect that a-DLL-debugging-you-will-go. The DLL and the PDB file must correspond exactly - there are checksums built in to make sure that they were both produced at the same time from the same build. Perhaps this is tripping up in some way.

[anthonyrichards]

Thanks for the advice about the .PDB file.
I gave up trying to use the third party application as a test program in the face of problems getting the copied DLL to enter debug mode when required.

I created a test program project to accompany the DLL project in the same solution.
It is in its own seperate folder and creates its own /debug/ folder.
I added the .LIB file created with the .DLL to the test program project so that the test program knew to link and import from the DLL.

I set up the DLL project Debugging properties to start the test program (giving the full path) and set the working directory as the test program's (giving the full path), where the required input data files for the DLL routines were located.

I copied the debug DLL AND its .PDB file from its /debug/ folder to the /debug/ folder of the test project.
Then I selected the DLL project for Debug, started the debugger, the test program started and when the test program called the routine in the DLL the debugger found the break point in the DLL code and debugging of the DLL code became available. Relief!

As a result, I have been able to find a couple of bugs in the code that CVF missed and which eventually caused the IVF code to object and crash in the third-party app. Also, as a result I have found weak points in a recently added (third-party) routine for solving polynomial roots which failed to explicitly deallocate allocatable arrays, relying on them being automatically deallocated on returning. /Qauto compilation led to them being automatically saved and causing a crash when it came time to allocate them in the code. There was no test for allocation. I changed to /Qauto_scalar and that particular cause of failure was removed, but I shall change the code to specifically deallocate all the allocated arrays on exit.

Finally, I have copied the debug DLL to the third party application and have successfully got it to run without crashing the app. The bugs I found were apparently the cause of previous falures.

Thanks all for all of your advice.

I think that the failure to copy the .PDB file along with the .DLL it applies to was the main problem.