Preprocessor bug in Intel C++ 19.0 for Windows

[eldiener]

I am the current maintainer, and a past contributor, of the Boost preprocessor library, known as Boost PP for short. In running the tests for Boost PP with the Intel C++ compiler for Windows 19.0, and seeing a single test failure, I discovered a preprocessor bug in the Intel compiler. Before I show the bug let me explain that Boost PP, while supporting the C++ standard preprocessors of compilers such as gcc and clang, also supports through a number of workarounds in the Boost PP code the non-standard preprocessor of Visual C++. Since Intel C++ on Windows emulates the preprocessing of Visual C++ the Boost PP library also supports Intel C++ for Windows by treating it with the exact same workarounds as Visual C++ in the Boost PP code. Therefore the bug I am about to show, which is a reduced example of the single failed Boost PP library test using Intel C++ for Windows, needs to be compared to the equivalent Visual C++ compiler output to most easily see that it is a bug in the Intel C++ for Windows preprocessor. This reduced example uses none of the code in the Boost PP library but can be run on its own.

The code is:

#define TRANSFORM_A(m,e) m(LPAREN() COMMA() TRANSFORM_A_ID)
#define TRANSFORM_B(m,e) m(RPAREN() TRANSFORM_B_ID)
#define TRANSFORM_A_ID() TRANSFORM_A
#define TRANSFORM_B_ID() TRANSFORM_B
#define TRANSFORM_REM(data) data
#define EAT(...)
#define LPAREN() (
#define RPAREN() )
#define COMMA() ,

TRANSFORM_A (TRANSFORM_REM, )() (EAT, ?) NIL TRANSFORM_B (TRANSFORM_REM, )() (EAT, ?)

When the above line becomes expanded using Intel C++ for Windows 19.0 with vc14.1 ( VS2017 ) compatibility, the result is incorrectly:

( NIL )

Using Visual C++ 14.1 ( VS2017 ) the expanded code is correctly:

( , NIL )

Both gcc and clang, as well as Intel C++ for Linux, expand the code correctly.

My command line switches when running the Intel C++ for Windows 19.0 compiler are:

-TP /Z7 /Od /Ob0 /W3 /GR /MDd /Zc:forScope /Zc:wchar_t /nologo /Qwn5 /Qwd985 -Qoption,c,--arg_dep_lookup /Qvc14.1 -Qpchi- /wd4675 /EHs -c

The incorrect preprocessor output also occurs in Intel 17.0 and Intel 18.0, for Windows. The Intel C++ for Linux 19.0 compiler, which does intend to implement a C++ standard preprocessor, runs the Boost PP tests without any errors and produces the correct output in the above case.

Let me reiterate that while the code above seems artificial it does occur in very similar form in the Boost PP library when implementing one of the pieces of functionality in that library, and is the reason for the single instance of a failing test when running the Boost PP tests with the Intel C++ compiler for Windows. For those who are really interested the failing case in the Boost PP tests for Intel C++ for Windows occurs when running 'BOOST_PP_SEQ_TO_LIST(())', where the output with Intel C++ for Windows is '( BOOST_PP_NIL )' instead of the correct '( , BOOST_PP_NIL )'.

If you need a full source file showing the bug using the code above I can supply a .cpp file, but you should be able to set one up for yourself from the code above. At what point the expansion of the macros fails to expand correctly I can not tell. If Intel C++ for Windows had a program/tool which was able to show its own step by step expansion of a macro I could have found out exactly where the preprocessor was failing, but unfortunately Intel C++, along with every other compiler, offers no such tool.

[eldiener]

This preprocessor bug, which evidently has never even been looked at by Intel by the lack of response to this bug report, still exists in Update 5 of Parallel Studio XE 2019 for Windows. I guess the way to ignore bugs in your product is just to ignore bug reports in this forum.

[Viet_H_Intel]

We have a list of bugs to address and need to prioritize them. If you can show me how this is going to affect your application (i.e. fail to compile, give a wrong results or crash at runtime) then I can escalate it. 

[eldiener]

Viet Hoang (Intel) wrote:

We have a list of bugs to address and need to prioritize them. If you can show me how this is going to affect your application (i.e. fail to compile, give a wrong results or crash at runtime) then I can escalate it. 

The bug gives wrong results. I discovered it testing the Boost Preprocessor library, of which I am the maintainer. A programmer using the Boost Preprocessor library to create and use preprocessor macros could trigger the bug. Even a programmer just writing his own preprocessor macros could trigger the bug. Since the bug produces incorrect preprocessor output, who knows if the causes of the bug do not produce incorrrect preprocessor macro output in other cases. The bug does not exist in the Intel C++ compiler for Linux, but only in the Intel C++ compiler for Windows. If this is not enough for you to escalate the issue and fix the bug then I do not have any further information that can get you to do so. If you need to know the exact Boost Preprocessor library test which shows the bug I can give that to you, but I did much work to discover the reason, as far as I was able, that the test fails and presented it to you in my OP.

[Viet_H_Intel]

Please see the attached image for result of our next release version.

[eldiener]

Viet Hoang (Intel) wrote:

Please see the attached image for result of our next release version.

The image does not show the t.c source you are compiling. But if the image is proof that the bug I have specified in this thread has been fixed in your next release version I will be glad if that is the case.

[Viet_H_Intel]

t.c is a copy of the one you provided

C:\Temp>notepad t.c

#define TRANSFORM_A(m,e) m(LPAREN() COMMA() TRANSFORM_A_ID)
#define TRANSFORM_B(m,e) m(RPAREN() TRANSFORM_B_ID)
#define TRANSFORM_A_ID() TRANSFORM_A
#define TRANSFORM_B_ID() TRANSFORM_B
#define TRANSFORM_REM(data) data
#define EAT(...)
#define LPAREN() (
#define RPAREN() )
#define COMMA() ,

TRANSFORM_A (TRANSFORM_REM, )() (EAT, ?) NIL TRANSFORM_B (TRANSFORM_REM, )() (EAT, ?)

[eldiener]

This bug still exists in Intel 19.1 ( Parallel Studio XE 2020 for Windows ), despite your assertion that it has been fixed for the next version when I reported it for Intel 19.0 ( Parallel Studio XE 2019 for Windows ).

[Viet_H_Intel]

Can you look at my output?

[eldiener]

Viet Hoang (Intel) wrote:

Can you look at my output?

Did you try the above with Intel C++ 19.1 ( Intel Parallel Studio XE 2020 ) on Windows ? My test shows the result to be '( NIL )'' and not the correct '( , NIL )'.

[eldiener]

Viet Hoang (Intel) wrote:

Can you look at my output?

What I discovered is this. Given the program:

#define TRANSFORM_A(m,e) m(LPAREN() COMMA() TRANSFORM_A_ID)
#define TRANSFORM_B(m,e) m(RPAREN() TRANSFORM_B_ID)
#define TRANSFORM_A_ID() TRANSFORM_A
#define TRANSFORM_B_ID() TRANSFORM_B
#define TRANSFORM_REM(data) data
#define EAT(...)
#define LPAREN() (
#define RPAREN() )
#define COMMA() ,

#define STRINGIZE(...) STRINGIZE_I(__VA_ARGS__)
#define STRINGIZE_I(...) #__VA_ARGS__

int main()
{

  TRANSFORM_A (TRANSFORM_REM, )() (EAT, ?) NIL TRANSFORM_B (TRANSFORM_REM, )() (EAT, ?);

  STRINGIZE(TRANSFORM_A (TRANSFORM_REM, )() (EAT, ?) NIL TRANSFORM_B (TRANSFORM_REM, )() (EAT, ?));
  
  return 0;
}

The preprocessor output is:

test_show_macro_output_bug.cpp
#line 1 "test_show_macro_output_bug.cpp"
#line 10 "test_show_macro_output_bug.cpp"

int main()
{

  ( ,  NIL ) ;

  "( NIL )";

  return 0;
}

This makes no sense but is what Intel C++ 19.1 on Windows produces. I admit I never thought when testing this bug that the actual problem was what I have showed above and not that Intel C++ 19.1 was expanding the macro incorrectly. That Intel C++ 19.1 can not expand the simple STRINGIZE macro correctly is mind-boggling to me. BTW in the in-line code editor, the __VA_ARGS__ shows incorrectly as VA ARGS.

[Viet_H_Intel]

Thank for the new test case. I've reported it (CMPLRIL0-32402) to our Developer.