Window x64 17.0.1 compiler, Visual Studio 2015, and std::list checked iterators issue

[AndrewC]

I have encountered a number of mysterious crashes on Windows x64 using 17.0.1 in _DEBUG mode only. In _DEBUG mode on Windows, various STL functions are validated using what are called "checked iterators"

Here is my sample test program. To reproduce, create a Visual C++ 2015 console project, and run in x64|Debug mode.

#include "stdafx.h"
#include <list>
#include <iostream>
using namespace std;
int main()
{
	std::list<int> a;
	a.clear();
	for(auto & i : a){// we have an iterator loop here, this forces creation of some sticky iterator data structures in the list
		cout << i << endl;// some dummy code
	}
	a.clear();// Intel Compiler and Visual Studio 2015 compiler behave differently on second clear()
    return 0;
}

/// THE FOLLOWING IS PART OF <list>, list::clear() calls _Orphan_ptr(nullptr) in _DEBUG mode
 #if _ITERATOR_DEBUG_LEVEL == 2
	void _Orphan_ptr(_Nodeptr _Ptr)
		{	// orphan iterators with specified node pointers
		_Lockit _Lock(_LOCK_DEBUG);
		const_iterator **_Pnext = (const_iterator **)this->_Getpfirst();
		if (_Pnext != 0)
			while (*_Pnext != 0)
				{	// test an iterator
				if ((*_Pnext)->_Ptr == this->_Myhead()
					|| (_Ptr != nullptr_t{} && (*_Pnext)->_Ptr != _Ptr))
					_Pnext = (const_iterator **)(*_Pnext)->_Getpnext();
				else
					{	// orphan the iterator
					(*_Pnext)->_Clrcont();
					*_Pnext = *(const_iterator **)(*_Pnext)->_Getpnext();
					}
				}
		}
 #endif /* _ITERATOR_DEBUG_LEVEL == 2 */

when I step into the function list::_Orphan_ptr  I see different behavior between Visual Studio 2015 native compiled and Intel 17.0.1 compiled in _DEBUG mode, that is with _ITERATOR_DEBUG_LEVEL=2, 

In simple language, if the list is empty, the while() loop should NEVER be entered. 

Visual Studio 2015 -> _Pnext!=0, *_PNext=0 , while loop is skipped ( as expected)

Intel Compiler 17.0.1 -> _Pnext!=0, *_PNext=<some random number>, while loop is entered, often causing an access violation.

This may be a bug in the checked iterator code of MS STL or it may be a bug in the Intel Compiler - I am not sure 100% at the moment.

[AndrewC]

I have discovered what the issue is, at it appears to be a bug in Intel Compiler 17.0.1

The destructor of the "std::list::const_iterator" created implicitly in the loop (for auto &i :a) is never called.

This can be tested by putting a break point in the function below - file <xutility> at line 140. When compiled with Visual C++ ( Visual Studio 2015)  this destructor is called. When compiled with Intel Compiler 17.0.1, it is not called.

~_Iterator_base12()

Changing the loop to be as below "fixes" the problem.

	for (auto it = a.begin(); it != a.end();++it) {

[Yuan_C_Intel]

Hi, Vasci_

I cannot reproduce any error with your reproducer. I created a Visual C++ project with your test code, see attached. 

Please let me know if anything is missing.

Thanks.

[AndrewC]

I can reproduce with your project

• Build your example
• Put breakpoints at lines 114 and 144 in c:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include\xutility
  • These lines are in the constructor and destructor of _Iterator_base12

• 	_Iterator_base12()
  		: _Myproxy(0), _Mynextiter(0)
  		{	// construct orphaned iterator
  		}
  
  ...
  
  ~_Iterator_base12() _NOEXCEPT
  		{	// destroy the iterator
   #if _ITERATOR_DEBUG_LEVEL == 2
  		_Lockit _Lock(_LOCK_DEBUG);
  		_Orphan_me(); /// BREAKPOINT HERE
   #endif /* _ITERATOR_DEBUG_LEVEL == 2 */
  		}
  

• Run
  • Code does stop at constructor
  • Code does not stop at destructor breakpoint
• Switch compiler to Visual C++ compiler
• Build
• Run
  • Code will stop at both break points.

This is the root cause of the issue. The 'implicit' iterator  is not being destroyed when compiled with Intel Compiler.

Microsoft Visual Studio Professional 2015
Version 14.0.25431.01 Update 3
Microsoft .NET Framework
Version 4.6.01055

Intel® Parallel Studio XE 2017 Update 1 Composer Edition for C++ Windows*   Package ID: w_comp_lib_2017.1.143

Intel® Parallel Studio XE 2017 Update 1 Composer Edition for C++ Windows* Integration for Microsoft* Visual Studio* 2015, Version 17.0.71.14,

[AndrewC]

"******Intel Premier Support is currently down for unexpected maintenance.*****"

So ... Premier Support is down ( is it ever coming back up) - what is the next step here.

[Judith_W_Intel]

Yolanda, I'm pretty sure this is already fixed in 17.0  (see cq #415213 - this was a 17.0 update 1 regression with destructors not called on temporaries created in range-based for loops. You may have to use MSVC++ 2015 and compile with /Od to see the problem...

Judy

[Yuan_C_Intel · Accepted solution]

Thank you, Judy and Vasci_ for the information and detail steps.

Yes, I have reproduced the issue now and found the interator destructor of the range based for loop is not called by icc.

As Judy said, this issue is a regression in 17.0 Update 1. The fix is targeting to be available in the next update of 17.0.

I verified the case with 17.0 initial release and it works as expected.

I have entered this test case in cq#415213 and will let you know when I have more update on this issue.

Thanks.

[AndrewC]

Great, glad this could be reproduced. This can really cause a lot of problems on Windows in _DEBUG mode just because of the nature of checked iterators.

Looking forward to the next update.

[Judith_W_Intel]

cq #415213 is already fixed and closed. So the fix will be in the next 17.0 update (update 2).

Sorry for the inconvenience.

Judy

[simmse]

Hello Intel Developer Support,

Approximately when is the C++ 17.0, Update 2 scheduled for release?  I am currently troubleshooting a vector<int> push_back access violation in Debug x64 mode.  The access violation occurs when adding a second element.  The _Container_base12::_Orphan_all() method attempts to access an unset pointer called _Myproxy.  Immediately prior to that the contents of the **_Pnext is checked against zero (*_Pnext != 0) within a for loop condition.  The *_Pnext does not equal zero after the first traversal within the loop.  The assignment obtains 0xcccccccccccccccc from *_Pnext = (*_Pnext)->_Mynextiter in the for loop.  In other words at line 214 of the Visual Studio 2015, 14.0.25431.01, Update 3, xutility header, the whole method is:

inline void _Container_base12::_Orphan_all()
{ // orphan all iterators
 #if _ITERATOR_DEBUG_LEVEL == 2
 if (_Myproxy != 0)
  { // proxy allocated, drain it
  _Lockit _Lock(_LOCK_DEBUG);

  for (_Iterator_base12 **_Pnext = &_Myproxy->_Myfirstiter;
   *_Pnext != 0; *_Pnext = (*_Pnext)->_Mynextiter)
   (*_Pnext)->_Myproxy = 0;

  // 
  // This line is after the for loop body.
  // Comment by simmse.
  //
  _Myproxy->_Myfirstiter = 0;
  }
 #endif /* _ITERATOR_DEBUG_LEVEL == 2 */
}

Since *_Pnext is set to 0xcccccccccccccccc, the attempt to dereference *_Pnext to the invalid _Myproxy fails with an access violation, debugger caught exception on a pointer set to 0xFFFFFFFFFFFFFFFF.

Thank You

[Yuan_C_Intel]

Hi. Simmse

This original is a regression in 17.0 Update 1. To verify if you are encountering the same issue, you can try install 17.0 initial release and see if it works correctly?

Thanks.

[Todd_W_]

I hit what seems to be the same issue as simmse's reported earlier this week, also in 17.0 update 1.  Depending on how the code's called infinite loops can occur since _Pnext never becomes null or the list destructor AVs.

I've worked around by changing the affected for loops to clunkier syntax but where is the 17.0 initial installer?  The Parallel Studio installer path I have seems locked to update 1.  And https://downloadcenter.intel.com/ just says "There are currently no downloads available for Development Software".

[AndrewC]

Rather than changing my code I worked around the issue by never building in _DEBUG mode. I just created a new configuration I call Release-Debug which is "Release" mode with all optimizations turned off.  I tried turning off checked iterators in _DEBUG mode, but that causes problems when linking with any C++ compiled with checked iterators.

Update 2 can't come soon enough!

[jase439]

I encountered this issue earlier this month. Glad to see a fix in the queue.

I worked around the issue by compiling with _ITERATOR_DEBUG_LEVEL=1 for my Intel debug targets.

[simmse]

Hello Yolanda,

Sorry for the delay.  The test with the initial 17.0 release is not a good idea.  The first time I update my system to 17.0 Update 1, I have to uninstall and reinstall 17.0, Update 1 three times and then run Visual Studio 2015 Update 3 repair.  The reason being is that the Update 1 installer directly affects configuration settings within Visual Studio, which cause all C++ file extensions to no longer display file content in the Visual Studio text editor window.  I would rather not spend hours living through and resolving that issue again.  My Technical Director submits the editor problem for resolution approximately two months ago.  Hopefully Update 2 does not adversely affect the Visual Studio editing settings.

Regarding not using the _DEBUG macro, the fundamental issue seems to be unpredictable behavior with debug builds.  How is it guaranteed that a release build does not have the issues moved to other memory locations where they are nearly impossible to detect?

[AndrewC]

Update 2 cannot come soon enough.

Can we please have a scheduled release date?

[Yuan_C_Intel]

Hi, Vasci_ and Simmse

Sorry for the inconvenience.

I can't tell the exact release date. We are working on the 2017 Update 2 release right now. It should be available for downloading very soon.

Thanks.

[simmse]

Hello Yolanda,

Last Friday, we receive information that Update 2 is released.  I receive an Intel e-mail over the weekend saying the same.  A couple of the team have already upgraded.  The empty C++ file types within the Visual Studio 2015, Update 3 editor continue though.  I will likely install Update 2 later today.  Hopefully the push_back problem is resolved.

Sincerely,

simmse

[AndrewC]

Hi

The original issue that started this thread with checked iterators in _DEBUG mode is resolved.

Thanks

[simmse]

Hello All,

I and others have installed Update 2 the last three business days.  So far, testing shows the push_back problem on a debug build configuration is resolved.  However, the Visual Studio 2015, Update 3 repair has to occur to resolve the navy edit window on C++ files within Visual Studio, after installing Intel C++ 17.0, Update 2.

Sincerely,

simmse