Invalid generation of PDB file

Roman_Z_ · ‎09-03-2017

Finally I've got the smallest possible program to file reproduce a nasty bug.

#include <stdio.h>
#include <functional>

void TestFunc1(const std::function<void()>& func)
{
	printf("TestFunc1\n");
	func();
}

int main()
{	//  set breakpoint to this line in Release mode
	TestFunc1([=]
	{
	});
    return 0;
}

Steps to reproduce:

Compile in Release mode.
Set breakpoint to line #11.
Run the program using VS debugger.
Wait for breakpoint.
Optionally, create minidump at this point using Task Manager. DON'T use Visual Studio Debug\Save dump as... because it will remove breakpoints from memory.
Continue execution of the program.
An access violation will occur.

It's cool, because without this breakpoint everything works as expected. This problem exists even in Debug mode, but I can't make a small program to reproduce this bug.

There is a way to actually see this bug in disassembly of minidump:

Environment: Windows 10, Visual Studio 2017, x86, ICC 17.0.4.210 (I'm sure that at least 15.x and 16.x compilers are affected too on windows). I saw this bug in Visual Studio 2013, 2015 many years ago.

jimdempseyatthecove · ‎09-05-2017

Roman,

First, I must commend you on your detective work. This type of problem is very difficult to detect. I haven't attempted to reproduced your problem as of yet. However, it reminds me of a similar nasty bug I experienced with MS VS IDE. In my case, the IDE got into a situation where it had a list of locations stating where to insert break points, however, at least one of these break points did NOT appear in the list of break points. IOW, using the IDE I could not see the break point. To make matters worse, the particular misplaced break point was placed into a byte offset into an instruction sequence. IOW changing the instruction. Fortunately, this unfortunate condition resulted in an illegal instruction (it changed the SIB byte), thus causing a fault. It could have just as well produced an incorrect instruction that completed execution (with incorrect results).

The hard part in diagnosing the problem was that after trap into the debugger, the offending break point was removed. Just as what you experienced. So suspiciously, you bug may be related to my bug (but it could be something else). After a lot of experimentation, the fix in my case was to use the IDE Debugger "Break Points" window, which did not show the offending break point, and select the big red X to remove all break points. This apparently performed an init or dtor/ctor of the tables.

I suggest you try that on your failing test program.

Jim Dempsey

jimdempseyatthecove · ‎09-05-2017

In looking at the right half of your screenshot, you find that location 00EF1000 contains CC (int 3) break point instruction.

Location 00EF100C, the second byte of the instruction starting at 00EF100B, also contains CC, showing that the original instruction sequence

89 6C 24 04 was changed to
89 CC 24 04

IOW non-1st byte of instruction sequence containing INT 03 break instruction. This was exactly the situation I experienced years ago. I suspect that performing the

Delete All Breakpoints

Will correct your symptom. (Note, to not delete individual break points, nor select each/all break points then delete selected).

Please reply as to if this corrects your problem.

Jim Dempsey

Roman_Z_ · ‎09-05-2017

Hello Jim.

Yes the problem is related with invalid address of INT 03 instruction. Delete All Breakpoints corrects the symptom as expected, but debugging without breakpoints is very difficult.

I think that it is a bug in ICC. From my point I can't trust breakpoints now, because they are unreliable and simple breakpoints can break correctly built software. Everyone knows that debugging takes a lot of time, this bug adds another layer of complexity. Not every senior c++ developer could understand root cause of this problem, so this bug should be finally fixed.

jimdempseyatthecove · ‎09-06-2017

>>Yes the problem is related with invalid address of INT 03 instruction. Delete All Breakpoints corrects the symptom as expected, but debugging without breakpoints is very difficult.

I thought this was implicit in my post....

After removing all break points, you can then set new break points. The "Delete All" gets rid of the invisible/fractured break point.

The trick is: Delete All via red X (even when no break points are listed).

Jim Dempsey

Roman_Z_ · ‎09-07-2017

jimdempseyatthecove wrote:

After removing all break points, you can then set new break points...

And what next? If I set this breakpoint again, Visual Studio sets INT 03 back to invalid location. Even if you start with new empty console project and write the same code, set the same breakpoint, you will trigger this bug. It isn't related to any kind of breakpoint location caching.

jimdempseyatthecove · ‎09-07-2017

Roman,

I think I see what is happening. First I will tell you the fix, then explain:

#include <stdio.h>
#include <functional>

void TestFunc1(const std::function<void()>& func)
{
 printf("TestFunc1\n");
 func();
}

int main()
{ //  set breakpoint to this line in Release mode
 TestFunc1(
 [=] // place Lambda on different line than breakpoint
 {
 });
    return 0;
}

If that does not work then insert a dummy statement to break on in front of TestFunc1. Example: Sleep(0); or whatever you want to use as a benign non-optimizable-out statement.

By formerly placing it on "TestFunc1([]" you effectively instructed the debugger to place a break point in two locations. One on the function call, and a second on the entry to the lambda function. While these two locations appear on the same source line, they reside in different binary locations. In Release mode, apparently the compiler optimized some code that made the placement of the break point on the lambda entry point ambiguous.

Jim Dempsey

Roman_Z_ · ‎09-07-2017

jimdempseyatthecove wrote:

By formerly placing it on "TestFunc1([]" you effectively instructed the debugger to place a break point in two locations....

Jim,

This is explanation for developer. As a user of Visual Studio I just want to break at this location (for any reason) and continue execution until the end. That's all.

#include <stdio.h>
#include <functional>
#include <windows.h>

void TestFunc1(const std::function<void()>& func)
{
	printf("TestFunc1\n");
	func();
}

int main()
{	//  breakpoint here still brakes the application
	Sleep(0);
	TestFunc1([=]
	{
		printf("B\n");
	});
    return 0;
}

edit: typo

jimdempseyatthecove · ‎09-08-2017

int main()
{
 //  breakpoint here still brakes the application...
 Sleep(0);  // Break OK on Sleep, step over OK...
// ... when trying to step into TestFunc1 (even with extra line) failure
 TestFunc1(
 [=]
 {
  printf("B\n"); // also unable to insert breakpoint here
 });
 return 0;
}

I am afraid with optimizations enabled you will not be able to establish a meaningful break point.

This inconvenience may or may not be fixable.

Jim Dempsey

jimdempseyatthecove · ‎09-08-2017

Hmm!!!

I even tried this:

#include <stdio.h>
#include <functional>
#include <windows.h>

__declspec(noinline) void TestFunc1(const std::function<void()>& func)
{
 printf("TestFunc1\n");
 func();
}

#define USE_BREAKPOINT_HELPER
#if defined(USE_BREAKPOINT_HELPER)
__declspec(noinline) void __breakpoint_helper()
{
 static int boink = 0;
 ++boink; // break here
}
#define BREAKPOINT_HELPER __breakpoint_helper();
#else
#define BREAKPOINT_HELPER
#endif

int main()
{
 Sleep(0);
 //  ** enable breakpoint in __breakpoint_helper
 TestFunc1(
 [=]
 {
  BREAKPOINT_HELPER
  printf("B\n");
 });
 return 0;
}

With break on ++boink. Stepping out of __breakpoint_helper would not return to printf in lambda function in Release mode, it does in debug mode.

Also, when at break on ++boink, setting stack focus up one level, goes to }); line in main() and not to the printf line.

Additionally, by using:

#pragma intel optimization_level 0
int main()

I could not step out of __breakpoint_helper to the printf

Neither would using:

 TestFunc1(
#pragma intel optimization_level 0
 [=]
 {
  BREAKPOINT_HELPER
  printf("B\n");
 });

So, other than compiling in Debug mode (or at least the file containing the lambda function of interest), I'm afraid you are waylaid.

Jim Dempsey

Roman_Z_ · ‎09-10-2017

Jim,

It's not a problem for me to stop at the beginning of the main function. I could just use __debugbreak() compiler intrinsic function. I just wanted to show small application that reproduces the bug with invalid location of breakpoints.

jimdempseyatthecove · ‎09-11-2017

Roman,

I am surprised that someone from Intel has not responded to this with one of those please file a support ticket at ... (some link I do not have at hand).

It is a good reproducer. My responses were attempts at a work around.

Jim Dempsey