pybind11 linking error with ICC 18.0 and 19.0 but not 17.0

[Basic__Boris]

I report this here and not on the pybind11 issues as it is a problem related to ICC on Windows. The code compiles and links with ICC 17.0 but not with ICC 18.0 (I checked with updates 2 and 4) nor 19.0 (initial release and update 1). It also compiles with MSVC and MinGW-w64.

The problem is that the linker will fail if the the main pybind11 module cpp file calls a function defined in another pybind11 wrapper cpp file. The following code can be used to reproduce the issue, using pybind11 2.2.4.

Main pybind11 module:

#include <pybind11/pybind11.h>
#include <iostream>

namespace py = pybind11;

void my_function() 
{
    std::cout << "my_function()\n";
}

void init_other(py::module& m);

PYBIND11_MODULE(test, m) 
{
    m.def("my_function", &my_function);
    init_other(m);
}

Other pybind11 wrapper file:

#include <pybind11/pybind11.h>
#include <iostream>

namespace py = pybind11;

void my_other_function() 
{
    std::cout << "my_other_function()\n";
}

void init_other(py::module& m) 
{
    m.def("my_other_function", &my_other_function);
}

This produces the following errors:

1>other.obj : : error LNK2005: "public: static void __cdecl `struct pybind11::detail::internals & __cdecl pybind11::detail::get_internals(void)'::`7'::_FUNC<lambda_2>(class std::exception_ptr)" (?_FUNC<lambda_2>@?6??get_internals@detail@pybind11@@YAAEAUinternals@23@XZ@SAXVexception_ptr@std@@@Z) already defined in module.obj
1>other.obj : : error LNK2005: "public: static void __cdecl `struct pybind11::detail::internals & __cdecl pybind11::detail::get_internals(void)'::`9'::_FUNC<lambda_3>(class std::exception_ptr)" (?_FUNC<lambda_3>@?8??get_internals@detail@pybind11@@YAAEAUinternals@23@XZ@SAXVexception_ptr@std@@@Z) already defined in module.obj
1>other.obj : : error LNK2005: "public: static void __cdecl `public: __cdecl pybind11::capsule::capsule(void const *,void (__cdecl*)(void *))'::`2'::_FUNC<lambda_0>(struct _object *)" (?_FUNC<lambda_0>@?1???0capsule@pybind11@@QEAA@PEBXP6AXPEAX@Z@Z@SAXPEAU_object@@@Z) already defined in module.obj
1>other.obj : : error LNK2005: "public: static void __cdecl `protected: void __cdecl pybind11::cpp_function::initialize_generic(struct pybind11::detail::function_record *,char const *,class type_info const * const *,unsigned __int64)'::`80'::_FUNC<lambda_4>(void *)" (?_FUNC<lambda_4>@?FA@??initialize_generic@cpp_function@pybind11@@IEAAXPEAUfunction_record@detail@3@PEBDPEBQEBVtype_info@@_K@Z@SAXPEAX@Z) already defined in module.obj
1>other.obj : : error LNK2005: "public: static class pybind11::handle __cdecl `protected: void __cdecl pybind11::cpp_function::initialize<class `struct std::pair<class std::_List_iterator<class std::_List_val<struct std::_List_simple_types<struct std::pair<struct _typeobject * const,class std::vector<struct pybind11::detail::type_info *,class std::allocator<struct pybind11::detail::type_info *> > > > > >,bool> __cdecl pybind11::detail::all_type_info_get_cache(struct _typeobject *)'::`5'::<lambda_8>,void,class pybind11::handle>(class `struct std::pair<class std::_List_iterator<class std::_List_val<struct std::_List_simple_types<struct std::pair<struct _typeobject * const,class std::vector<struct pybind11::detail::type_info *,class std::allocator<struct pybind11::detail::type_info *> > > > > >,bool> __cdecl pybind11::detail::all_type_info_get_cache(struct _typeobject *)'::`5'::<lambda_8> &&,void (__cdecl*)(class pybind11::handle))'::`2'::_FUNC<lambda_17>(struct pybind11::detail::function_call &)" (?_FUNC<lambda_17>@?1???$initialize@V<lambda_8>@?4??all_type_info_get_cache@detail@pybind11@@YA?AU?$pair@V?$_List_iterator@V?$_List_val@U?$_List_simple_types@U?$pair@QEAU_typeobject@@V?$vector@PEAUtype_info@detail@pybind11@@V?$allocator@PEAUtype_info@detail@pybind11@@@std@@@std@@@std@@@std@@@std@@@std@@_N@std@@PEAU_typeobject@@@Z@XVhandle@4@$$V@cpp_function@pybind11@@IEAAX$$QEAV<lambda_8>@?4??all_type_info_get_cache@detail@2@YA?AU?$pair@V?$_List_iterator@V?$_List_val@U?$_List_simple_types@U?$pair@QEAU_typeobject@@V?$vector@PEAUtype_info@detail@pybind11@@V?$allocator@PEAUtype_info@detail@pybind11@@@std@@@std@@@std@@@std@@@std@@@std@@_N@std@@PEAU_typeobject@@@Z@P6AXVhandle@2@@Z@Z@SA?AV92@AEAUfunction_call@52@@Z) already defined in module.obj
1>other.obj : : error LNK2005: "public: static void __cdecl `protected: void __cdecl pybind11::cpp_function::initialize<class `struct std::pair<class std::_List_iterator<class std::_List_val<struct std::_List_simple_types<struct std::pair<struct _typeobject * const,class std::vector<struct pybind11::detail::type_info *,class std::allocator<struct pybind11::detail::type_info *> > > > > >,bool> __cdecl pybind11::detail::all_type_info_get_cache(struct _typeobject *)'::`5'::<lambda_8>,void,class pybind11::handle>(class `struct std::pair<class std::_List_iterator<class std::_List_val<struct std::_List_simple_types<struct std::pair<struct _typeobject * const,class std::vector<struct pybind11::detail::type_info *,class std::allocator<struct pybind11::detail::type_info *> > > > > >,bool> __cdecl pybind11::detail::all_type_info_get_cache(struct _typeobject *)'::`5'::<lambda_8> &&,void (__cdecl*)(class pybind11::handle))'::`7'::_FUNC<lambda_15>(struct pybind11::detail::function_record *)" (?_FUNC<lambda_15>@?6???$initialize@V<lambda_8>@?4??all_type_info_get_cache@detail@pybind11@@YA?AU?$pair@V?$_List_iterator@V?$_List_val@U?$_List_simple_types@U?$pair@QEAU_typeobject@@V?$vector@PEAUtype_info@detail@pybind11@@V?$allocator@PEAUtype_info@detail@pybind11@@@std@@@std@@@std@@@std@@@std@@@std@@_N@std@@PEAU_typeobject@@@Z@XVhandle@4@$$V@cpp_function@pybind11@@IEAAX$$QEAV<lambda_8>@?4??all_type_info_get_cache@detail@2@YA?AU?$pair@V?$_List_iterator@V?$_List_val@U?$_List_simple_types@U?$pair@QEAU_typeobject@@V?$vector@PEAUtype_info@detail@pybind11@@V?$allocator@PEAUtype_info@detail@pybind11@@@std@@@std@@@std@@@std@@@std@@@std@@_N@std@@PEAU_typeobject@@@Z@P6AXVhandle@2@@Z@Z@SAXPEAUfunction_record@52@@Z) already defined in module.obj
1>other.obj : : error LNK2005: "public: static void __cdecl `protected: void __cdecl pybind11::cpp_function::initialize<class `struct std::pair<class std::_List_iterator<class std::_List_val<struct std::_List_simple_types<struct std::pair<struct _typeobject * const,class std::vector<struct pybind11::detail::type_info *,class std::allocator<struct pybind11::detail::type_info *> > > > > >,bool> __cdecl pybind11::detail::all_type_info_get_cache(struct _typeobject *)'::`5'::<lambda_8>,void,class pybind11::handle>(class `struct std::pair<class std::_List_iterator<class std::_List_val<struct std::_List_simple_types<struct std::pair<struct _typeobject * const,class std::vector<struct pybind11::detail::type_info *,class std::allocator<struct pybind11::detail::type_info *> > > > > >,bool> __cdecl pybind11::detail::all_type_info_get_cache(struct _typeobject *)'::`5'::<lambda_8> &&,void (__cdecl*)(class pybind11::handle))'::`9'::_FUNC<lambda_16>(struct pybind11::detail::function_record *)" (?_FUNC<lambda_16>@?8???$initialize@V<lambda_8>@?4??all_type_info_get_cache@detail@pybind11@@YA?AU?$pair@V?$_List_iterator@V?$_List_val@U?$_List_simple_types@U?$pair@QEAU_typeobject@@V?$vector@PEAUtype_info@detail@pybind11@@V?$allocator@PEAUtype_info@detail@pybind11@@@std@@@std@@@std@@@std@@@std@@@std@@_N@std@@PEAU_typeobject@@@Z@XVhandle@4@$$V@cpp_function@pybind11@@IEAAX$$QEAV<lambda_8>@?4??all_type_info_get_cache@detail@2@YA?AU?$pair@V?$_List_iterator@V?$_List_val@U?$_List_simple_types@U?$pair@QEAU_typeobject@@V?$vector@PEAUtype_info@detail@pybind11@@V?$allocator@PEAUtype_info@detail@pybind11@@@std@@@std@@@std@@@std@@@std@@@std@@_N@std@@PEAU_typeobject@@@Z@P6AXVhandle@2@@Z@Z@SAXPEAUfunction_record@52@@Z) already defined in module.obj
1>other.obj : : error LNK2005: "public: static class pybind11::handle __cdecl `protected: void __cdecl pybind11::cpp_function::initialize<void (__cdecl*&)(void),void,struct pybind11::name,struct pybind11::scope,struct pybind11::sibling>(void (__cdecl*&)(void),void (__cdecl*)(void),struct pybind11::name const &,struct pybind11::scope const &,struct pybind11::sibling const &)'::`2'::_FUNC<lambda_20>(struct pybind11::detail::function_call &)" (?_FUNC<lambda_20>@?1???$initialize@AEAP6AXXZXUname@pybind11@@Uscope@2@Usibling@2@@cpp_function@pybind11@@IEAAXAEAP6AXXZP6AXXZAEBUname@2@AEBUscope@2@AEBUsibling@2@@Z@SA?AVhandle@2@AEAUfunction_call@detail@2@@Z) already defined in module.obj
1>other.obj : : error LNK2005: "public: static void __cdecl `protected: void __cdecl pybind11::cpp_function::initialize<void (__cdecl*&)(void),void,struct pybind11::name,struct pybind11::scope,struct pybind11::sibling>(void (__cdecl*&)(void),void (__cdecl*)(void),struct pybind11::name const &,struct pybind11::scope const &,struct pybind11::sibling const &)'::`7'::_FUNC<lambda_18>(struct pybind11::detail::function_record *)" (?_FUNC<lambda_18>@?6???$initialize@AEAP6AXXZXUname@pybind11@@Uscope@2@Usibling@2@@cpp_function@pybind11@@IEAAXAEAP6AXXZP6AXXZAEBUname@2@AEBUscope@2@AEBUsibling@2@@Z@SAXPEAUfunction_record@detail@2@@Z) already defined in module.obj
1>other.obj : : error LNK2005: "public: static void __cdecl `protected: void __cdecl pybind11::cpp_function::initialize<void (__cdecl*&)(void),void,struct pybind11::name,struct pybind11::scope,struct pybind11::sibling>(void (__cdecl*&)(void),void (__cdecl*)(void),struct pybind11::name const &,struct pybind11::scope const &,struct pybind11::sibling const &)'::`9'::_FUNC<lambda_19>(struct pybind11::detail::function_record *)" (?_FUNC<lambda_19>@?8???$initialize@AEAP6AXXZXUname@pybind11@@Uscope@2@Usibling@2@@cpp_function@pybind11@@IEAAXAEAP6AXXZP6AXXZAEBUname@2@AEBUscope@2@AEBUsibling@2@@Z@SAXPEAUfunction_record@detail@2@@Z) already defined in module.obj

[Viet_H_Intel]

I dont have pybind11 avail currently. Can you create a preprocess file (-E) for module.cpp and other.cpp and attach to this thread?

I am hoping to compile from your preprocess file without having to have pybind11 installed. 

[Basic__Boris]

Here they are.

[Viet_H_Intel]

Can I have the command line options?

Thanks,

Viet

[Basic__Boris]

Compiler options: /MP /GS /TP /W3 /Zc:wchar_t /D "_WINDLL" /D "_MBCS" /Zc:forScope /GR /MD /Fa"Release/" /EHsc /nologo

Linker options: /OUT:".\pybind11_test.pyd" /MANIFEST /NXCOMPAT /DYNAMICBASE "C:\anaconda3\libs\Python36.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib" /IMPLIB:"./pybind11_test.lib" /DLL /MACHINE:X64 /INCREMENTAL:NO /SUBSYSTEM:CONSOLE /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /NOLOGO /TLBID:1

Thanks,

Boris

[Viet_H_Intel]

Thanks, Since there are a few dependencies for the linker, I don't think I will be able to link to see the error, unless I have the same setup.

[Basic__Boris]

Outside Windows system libraries, the only external one is the python library. I attached it to this post.

[Viet_H_Intel]

Thanks, with that I think I will able to reproduce it. 

[Viet_H_Intel]

I've reported this problem to our Compiler Developer. The case number is: CMPLRIL0-30801