How to debug code called by another program?

[Anthony_Richards]

I have written code that interacts with an optical analysis program via DDE. I got this working fine when I was running on a 32-bit platform.
I now have to cope with the optical analysis program now running on a 64-bit platform.

The back and forth DDE communication part works OK - when I start the optical analysis program, it talks to my code OK (diagnostic message boxes put up by my code confirms this), but the process now falls over when my code tries to open a modal Windows dialog, so I want to debug my code.

However, in my VS2012, I cannot see/find how to specify that debugging has to start by executing the optical
analysis program rather than my own. Any suggestions how to do this?

Also, I have to copy my debug code to a specific folder, for the optical analysis program to access it, so the code that has to be debugged will not be in the /debug/ folder created and populated by the compiler.

So how do I get the VS debugger to 'latch onto' the code that needs debugging?

 Do I also have to accompany my code with the PDB file that is created along with it?

[IanH]

An easy way is to have your code pause at a known point (perhaps by displaying a message box displayed or similar) and then use Debug > Attach to process to commence debugging. 

There's a particular search order for PDB files, the details of which escape me, but I'm pretty sure that if the PDB file is sitting in the same directory as the executable for the process being debugged then it will be found.

[Steven_L_Intel1]

The answer to your direct question is that you change the project property Debugging > Command to name the executable you want to start. I don't know if this will help you, though, as I tend to doubt the debugger will see your breakpoints. Ian's approach is likely to be more successful.

[Anthony_Richards]

Thanks,Ian, for that. It worked!

I copied the .PDB file along with the .EXE to the required folder, started the optical application, waited for the first response from my application (a messagebox ) which was started via a call to a function in it,  then selected 'Attach to process' and attached to the process that owned the response then started debugging and it found my break point after I continued my application.

However, I have ground to a halt because now my first Windows dialog will not display and my application crashes at the point after executing DlgModal.

Thanks again though.

[andrew_4619]

set a callback for the id of the dialog itself as this will be called with cbtype == dlg_INIT as the next item after dlgmodal. You can at least then set a breakpoint and look at the dlg structure to see if anything looks wrong.

[Anthony_Richards]

Yes, I do that as I populate a dialog edit box with some text before displaying the dialog. As I exit the call back, the app crashes with an access violation.
I will examine the DLG structure as you recommend.

[bmchenry]

In the event the other solution isn't working for you you might try....

Per Steve "change the project property Debugging > Command to name the executable you want to start" has worked fine for me for years in VS. Visual Studio starts the process and then any breakpoints you set will occur if and when the EXE calls your program.

One note: I keep the calling EXE and DLLs and support files all in the same directory for debugging  I install the EXE program to the 'working directory' $(SolutionDir)

And then use .$(SolutionDir)$(ConfigurationName) for the target directories

I have 4 to 6 separate DLLS which are called by a 3rd party exe (with many other Dlls) for which i have no code and break points work in any and all of my DLLs (of course if and when they are called by the EXE or one of my DLLs which was called by the EXE)

[Steven_L_Intel1]

Yes, I know this works for EXEs that call DLLs. I don't know how well it works when one EXE somehow invokes a second EXE.