Intel® C++ Compiler
Community support and assistance for creating C++ code that runs on platforms based on Intel® processors.

Visual Studio built in native unit tests aren't discovered when built with Intel Compiler

Mike_C1
Beginner
2,729 Views

Does the Intel compiler support compiling native unit test projects using Microsoft's built in unit testing framework? I can't find it listed as an issue under the Microsoft compatibility documentation.

I've tried the following steps in both Visual Studio 2013 and 2015 and with Intel C++ Compiler 16 and 15. Building with older versions of the Microsoft Compiler seems to work fine.

  • Create a new unit test project - New Project/Test/Native Unit Test
  • Rebuild All
  • View Test Explorer Window  - the single sample test is listed
  • Change the properties of the project to build with Intel Compiler 15 or 16 (Intel compiler 15 only builds with Visual Studio 2013)
  • Rebuild All
  • View Test Explorer Window  - no tests are listed.

If I run dumpbin on the DLLs they export the same list of exported functions with the same names in the same order.

I'm not sure how to diagnose this problem further. I can't find any documentation on how the Microsoft Unit Test frame work discovers the tests. The apparent entry points for finding test information don't get called when built with the Intel compiler.

0 Kudos
14 Replies
Mike_C1
Beginner
2,729 Views

One extra piece of information. The intel build has a warning that doesn't occur in the microsoft compiler builds:

1>unittest1.obj : warning LNK4078: multiple 'testvers' sections found with different attributes (40301040)

The references to this section that I can see are all in CppUnitTest.h below.

Is there a problem here because the testvers section is declared as read only but the s_version variable is not declared as const? Would the Intel compiler generate different code from the Microsoft compiler? Could this be enough to change the behaviour of the code? The other __declspec(allocate declared variables are all const and don't generate linker warnings.

Modifying the header to add const to the variable does not fix the warning, interestingly.

#pragma section("testvers$", read, shared)

#define ALLOCATE_TESTDATA_SECTION_VERSION __declspec(allocate("testvers$"))

static EXPORT_METHOD const ::Microsoft::VisualStudio::CppUnitTestFramework::TestDataVersion* CALLING_CONVENTION __GetTestVersion() 
{
    ALLOCATE_TESTDATA_SECTION_VERSION
    static ::Microsoft::VisualStudio::CppUnitTestFramework::TestDataVersion s_version = { __CPPUNITTEST_VERSION__ };

    return &s_version;
}

 

0 Kudos
Mike_C1
Beginner
2,729 Views

Anyone care to comment on this before I report it as a bug on Intel Premier Support? 

Unless this issue is specific to my machine configuration, it appears that native unit testing has never worked with the Intel compiler Visual Studio integration (Visual Studio native unit tests were only added in Visual Studio 2013 or perhaps 2012, which I don't have installed) 

I've switched to using Google Test with the third party Google Test test adapter and this integrates absolutely fine with the Visual Studio tests window with both the Microsoft or Intel compilers.

I haven't managed to break on or step through the test discovery entry points. There doesn't appear to be any detailed spec for how a DLL should be built to integrate with Microsoft native unit testing though the DLLs I'm building appear to have identically named entry points so I'm at a dead end on how to diganose this further.

 

0 Kudos
Kittur_G_Intel
Employee
2,729 Views

Hi Mike,
Pardon the delay in response to your thread. Well, my peer is investigating your issue with the IDE team and will update you accordingly. Appreciate your patience till then.
Regards,
Kittur

0 Kudos
Henk-Jan_L_
Beginner
2,729 Views

Hi Kittur,

Any updates on this? Or should I assume that the Intel compiler (16.0) does not support native unit test projects.

Regards HJ!

0 Kudos
Cedric_W_
Beginner
2,729 Views

Hello,

I can confirm this bug still exists as of today with compiler version 17.

Is there any update on this ?

Regards

CW

 

0 Kudos
Mike_C1
Beginner
2,729 Views

Any updates on this?

I can confirm the bug is still exactly the same with Visual Studio 2017 with latest updates and Intel Compiler 18.0 update 1. The results are the same in debug/release and win32/x64

I've tried switching on the new 2017 linker option "Generate Debug Information optimized for sharing and publishing (/DEBUG:FULL)" but it makes no difference to the behaviour:

I still have no ideas how to diagnose this further as the details of how Visual Studio 2017 discovers unit tests for its own test framework appear to be undocumented. I've pasted the diagnostic level output from the output/tests window below. Can't see anything obvious. Anyone have any ideas?

Thanks,

 

[27/11/2017 22:08:33 Diagnostic] Pausing Queue..., current operation is: no current operation
[27/11/2017 22:08:33 Diagnostic] About to Enqueue operation 'WaitForBuildOperation', hashcode:16423033 
[27/11/2017 22:08:33 Diagnostic] Enqueue operation 'WaitForBuildOperation', hashcode:16423033 
[27/11/2017 22:08:33 Diagnostic] Operation left in the the queue: 1
[27/11/2017 22:08:33 Diagnostic]  'WaitForBuildOperation', hashcode:16423033
[27/11/2017 22:08:33 Diagnostic] 

[27/11/2017 22:08:33 Diagnostic] Processing Queue .....
[27/11/2017 22:08:33 Diagnostic] Operation Dequeue : 'WaitForBuildOperation'
[27/11/2017 22:08:33 Diagnostic] VirtualReadOnlyTestDataStore.OperationStateChanged State=OperationSetStarted, operationInProgress=False
[27/11/2017 22:08:33 Diagnostic] TestDiscoveryStats.OperationStateChanged State=OperationSetStarted, InProgress=False
[27/11/2017 22:08:35 Diagnostic] Firing BuildCompleted event...
[27/11/2017 22:08:35 Diagnostic] Container Discoverer 'Microsoft.VisualStudio.TestWindow.Extensions.CPSTestContainerDiscoverer' has container changes
[27/11/2017 22:08:35 Diagnostic] Firing BuildCompleted event...
[27/11/2017 22:08:35 Diagnostic] Container Discoverer 'Microsoft.VisualStudio.TestWindow.VsAdapters.VsProjectOutputContainerDiscoverer' has container changes
[27/11/2017 22:08:35 Diagnostic] VirtualReadOnlyTestDataStore.OperationStateChanged State=OperationSetFinished, operationInProgress=False
[27/11/2017 22:08:35 Diagnostic] TestDiscoveryStats.OperationStateChanged State=OperationSetFinished, InProgress=False
[27/11/2017 22:08:35 Diagnostic] test container discoverer executor://cpsprojectoutputtestcontainerdiscoverer/v2, discovered 0 containers
[27/11/2017 22:08:35 Diagnostic] No containers found from 'Microsoft.VisualStudio.TestWindow.Extensions.CPSTestContainerDiscoverer' :
[27/11/2017 22:08:35 Diagnostic] Container discoverer 'Microsoft.VisualStudio.TestWindow.Extensions.CPSTestContainerDiscoverer' has no containers
[27/11/2017 22:08:35 Diagnostic] test container discoverer executor://vsprojectoutputcontainerdiscoverer/v1, discovered 1 containers
[27/11/2017 22:08:35 Diagnostic] Containers from 'Microsoft.VisualStudio.TestWindow.VsAdapters.VsProjectOutputContainerDiscoverer' :
[27/11/2017 22:08:35 Diagnostic]  C:\Source\misc\Intel Compiler Unit Test Failure 2017\Intel Compiler Unit Test Failure\x64\Debug\Intel Compiler Unit Test Failure.dll:executor://vsprojectoutputcontainerdiscoverer/v1
[27/11/2017 22:08:35 Diagnostic] About to Enqueue operation 'DiscoverSelectedOperation', hashcode:21436414 
[27/11/2017 22:08:35 Diagnostic] Enqueue operation 'DiscoverSelectedOperation', hashcode:21436414 
[27/11/2017 22:08:35 Diagnostic] Operation left in the the queue: 1
[27/11/2017 22:08:35 Diagnostic]  'DiscoverSelectedOperation', hashcode:21436414
[27/11/2017 22:08:35 Diagnostic] 

[27/11/2017 22:08:35 Diagnostic] Processing Queue .....
[27/11/2017 22:08:35 Diagnostic] Operation Dequeue : 'DiscoverSelectedOperation'
[27/11/2017 22:08:35 Diagnostic] VirtualReadOnlyTestDataStore.OperationStateChanged State=OperationSetStarted, operationInProgress=False
[27/11/2017 22:08:35 Diagnostic] TestDiscoveryStats.OperationStateChanged State=OperationSetStarted, InProgress=False
[27/11/2017 22:08:35 Diagnostic] VirtualReadOnlyTestDataStore.OperationStateChanged State=ChangeDetectionStarting, operationInProgress=True
[27/11/2017 22:08:35 Diagnostic] TestDiscoveryStats.OperationStateChanged State=ChangeDetectionStarting, InProgress=False
[27/11/2017 22:08:35 Diagnostic] DiscoveryOperation<DiscoverSelectedOperation> FinishedChangedCotainers, changed container count is 1
[27/11/2017 22:08:35 Diagnostic] VirtualReadOnlyTestDataStore.OperationStateChanged State=ChangeDetectionFinished, operationInProgress=False
[27/11/2017 22:08:35 Diagnostic] TestDiscoveryStats.OperationStateChanged State=ChangeDetectionFinished, InProgress=False
[27/11/2017 22:08:35 Diagnostic] Discovering the following containers :
[27/11/2017 22:08:35 Diagnostic]  C:\Source\misc\Intel Compiler Unit Test Failure 2017\Intel Compiler Unit Test Failure\x64\Debug\Intel Compiler Unit Test Failure.dll
[27/11/2017 22:08:35 Diagnostic] VirtualReadOnlyTestDataStore.OperationStateChanged State=DiscoveryStarting, operationInProgress=False
[27/11/2017 22:08:35 Diagnostic] TestDiscoveryStats.OperationStateChanged State=DiscoveryStarting, InProgress=True
[27/11/2017 22:08:35 Informational] ------ Discover test started ------
[27/11/2017 22:08:35 Diagnostic] ***    Discover TestCase using 'InMemoryUnitTestWriter' ***
[27/11/2017 22:08:35 Warning] Test run will use DLL(s) built for framework Framework45 and platform X86. Following DLL(s) will not be part of run: 
Intel Compiler Unit Test Failure.dll is built for Framework None and Platform X64.
 Go to http://go.microsoft.com/fwlink/?LinkID=236877&clcid=0x409 for more details on managing these settings.
[27/11/2017 22:08:35 Diagnostic] RunSettings Content:
<RunSettings>
  <RunConfiguration>
    <ResultsDirectory>C:\Source\misc\Intel Compiler Unit Test Failure 2017\Intel Compiler Unit Test Failure\TestResults</ResultsDirectory>
    <SolutionDirectory>C:\Source\misc\Intel Compiler Unit Test Failure 2017\Intel Compiler Unit Test Failure\</SolutionDirectory>
    <TargetPlatform>X86</TargetPlatform>
    <TargetFrameworkVersion>Framework45</TargetFrameworkVersion>
  </RunConfiguration>
  <GoogleTestAdapterSettings xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <SolutionSettings>
      <Settings>
        <PrintTestOutput>true</PrintTestOutput>
        <TestDiscoveryRegex />
        <TestDiscoveryTimeoutInSeconds>30</TestDiscoveryTimeoutInSeconds>
        <WorkingDir>$(ExecutableDir)</WorkingDir>
        <PathExtension />
        <CatchExceptions>false</CatchExceptions>
        <BreakOnFailure>false</BreakOnFailure>
        <RunDisabledTests>false</RunDisabledTests>
        <NrOfTestRepetitions>1</NrOfTestRepetitions>
        <ShuffleTests>false</ShuffleTests>
        <ShuffleTestsSeed>0</ShuffleTestsSeed>
        <TraitsRegexesBefore />
        <TraitsRegexesAfter />
        <TestNameSeparator />
        <DebugMode>false</DebugMode>
        <TimestampOutput>false</TimestampOutput>
        <ShowReleaseNotes>true</ShowReleaseNotes>
        <ParseSymbolInformation>true</ParseSymbolInformation>
        <AdditionalTestExecutionParam />
        <ParallelTestExecution>false</ParallelTestExecution>
        <MaxNrOfThreads>0</MaxNrOfThreads>
        <BatchForTestSetup />
        <BatchForTestTeardown />
        <KillProcessesOnCancel>false</KillProcessesOnCancel>
        <UseNewTestExecutionFramework>true</UseNewTestExecutionFramework>
        <DebuggingNamedPipeId>ac17c89e-22da-45f9-9109-09ddc9535b2a</DebuggingNamedPipeId>
      </Settings>
    </SolutionSettings>
    <ProjectSettings />
  </GoogleTestAdapterSettings>
</RunSettings>
[27/11/2017 22:08:35 Diagnostic] VirtualReadOnlyTestDataStore.OperationStateChanged State=DiscoveryStarted, operationInProgress=False
[27/11/2017 22:08:35 Diagnostic] TestDiscoveryStats.OperationStateChanged State=DiscoveryStarted, InProgress=True
[27/11/2017 22:08:35 Informational] ========== Discover test finished: 0 found (0:00:00.0470005) ==========
[27/11/2017 22:08:35 Diagnostic] VirtualReadOnlyTestDataStore.OperationStateChanged State=DiscoveryFinished, operationInProgress=False
[27/11/2017 22:08:35 Diagnostic] TestDiscoveryStats.OperationStateChanged State=DiscoveryFinished, InProgress=False
[27/11/2017 22:08:35 Diagnostic]  Provider 'GroupByClassProvider' found 0 groups.
[27/11/2017 22:08:35 Diagnostic] VirtualReadOnlyTestDataStore.OperationStateChanged State=OperationSetFinished, operationInProgress=False
[27/11/2017 22:08:35 Diagnostic] TestDiscoveryStats.OperationStateChanged State=OperationSetFinished, InProgress=False

 

0 Kudos
Mike_C1
Beginner
2,729 Views

 

This line in the log suggests that the unit test runner is trying to run the Intel compiler built DLL as a managed unit test rather than a native unit test. Haven't yet found a way to change this.


[27/11/2017 22:08:35 Warning] Test run will use DLL(s) built for framework Framework45 and platform X86. Following DLL(s) will not be part of run:  46  Intel Compiler Unit Test Failure.dll is built for Framework None and Platform X64. 

 

0 Kudos
Wai_P_
Novice
2,729 Views

Any updates on this? We would like to be to use intel build for our unit tests.

0 Kudos
Surau__William
Beginner
2,729 Views

I have the same problem with using Visual Studio 2012. When I open a native Unit Test project everything works fine, but after changing to x64 architecture and change the compiler to Intel C++ Compiler XE 14.0 everything gets build but no test discovery occurs.

 

How to fix this problem? I'm searching since days to find a solution.

Regards William

0 Kudos
Mike_C1
Beginner
2,729 Views

I've changed the title of this thread to reflect that the same problem happens across multiple Visual Studio and Intel Compiler versions.

This problem appears to have got worse with 2019 versions.

Presumably Intel doesn't consider this a priority to fix as I haven't found any official documentation of this compatibility limit. I'm already reducing my usage of the Intel compiler to CPU intensive code rather than whole projects. This limitation makes unit testing those Intel code islands a lot less convenient than it could be - my current workaround is to use the Google Test framework instead.

 

If I follow the steps above with Visual Studio 2019 16.4.1 and Intel C++ Compiler 2019 update 5 the empty unit test project won't even build with the Intel Compiler selected with the following build errors (slightly edited for better readability) :

 

pch.cpp
UnitTestVS2019.cpp
MSVC\14.24.28314\include\utility(92): error : expected an identifier
          !_Is_implicitly_default_constructible<_Uty1>::value || !_Is_implicitly_default_constructible<_Uty2>::value)
          ^

MSVC\14.24.28314\include\utility(91): error : not a valid member class or function template declaration
      constexpr explicit(
      ^

MSVC\14.24.28314\include\utility(91): error : "explicit" is not allowed
      constexpr explicit(
      ^

MSVC\14.24.28314\include\utility(93): error : expected a ";"
          pair() noexcept(
          ^

MSVC\14.24.28314\include\utility(175): error : expected an identifier
      constexpr explicit(!is_convertible<const _Other1&, _Ty1>::value // TRANSITION, VSO#946746
                         ^

MSVC\14.24.28314\include\utility(175): error : not a valid member class or function template declaration
      constexpr explicit(!is_convertible<const _Other1&, _Ty1>::value // TRANSITION, VSO#946746
      ^

MSVC\14.24.28314\include\utility(175): error : "explicit" is not allowed
      constexpr explicit(!is_convertible<const _Other1&, _Ty1>::value // TRANSITION, VSO#946746
      ^

MSVC\14.24.28314\include\utility(177): error : expected a ";"
          pair(const pair<_Other1, _Other2>& _Right) noexcept(is_nothrow_constructible_v<_Ty1, const _Other1&>&&
          ^


0 Kudos
Viet_H_Intel
Moderator
2,729 Views

Hi Mike,

Visual Studio 2019 16.4.1 just released last week, and Intel 19.0 update 5 released back in July. Thus, we couldn't be able to test out Visual Studio 2019 16.4.1 back then.

 

0 Kudos
Mike_C1
Beginner
2,729 Views

That's a fair point. However native unit tests appear never to have worked with Intel compiler in the testing I've done so I'm guessing it's something that Intel is not interested in supporting though Intel goes to great lengths to be compatible with Visual Studio in many other areas. Some sort of official response on this would be helpful.

0 Kudos
Mike_C1
Beginner
2,729 Views

Another update with vs2019 16.4.1 and the new Intel 2020 parallel studio compiler 19.1.

The unit test projects now build again but the tests still aren't discovered as per previous versions. I've listed my various diagnostic attempts in previous posts. I'm not sure how to investigate further without more information on how Microsoft builds and enumerates the tests in its native unit testing DLLs and I haven't managed to find any documentation on this previously.

 

0 Kudos
Mike_C1
Beginner
2,729 Views

Cool, it seems to be fixed! Intel compiled native tests now seem to work with my recently updated tools setup:

 

Microsoft Visual Studio Professional 2019 Version 16.5.2
Intel® C++ Compiler   Package ID: w_comp_lib_2020.1.216
Intel® C++ Compiler – extension version 19.1.0.16, Package ID: w_comp_lib_2020.1.216, 

 

FWIW Microsoft unit tests can also be run from the console using vstest.console.exe to help debug any problems.

 

I'm having apparently unrelated issues running Intel Inspector on Microsoft Native Unit Tests compiled with both Microsoft and Intel compilers but that's a discussion for a different post.

0 Kudos
Reply