Build Native Libraries on windows

[jasno]

Hi,

    I am trying to build a library (static and shared) for the MIC on windows. I have compiled my src using the /Qmic flag but now I don't get how to generate a library. For example I compile a src file as:

ifort /Qmic -nologo -auto -fp -align array64byte -fp-model precise -mp1 -mkl -openmp -fpp -c -o performance_parameters_sb1.o c:/modules/performance_parameters_sb1.f90

then have tried these three methods to turn it into being a static library . . .

xilink -lib /OUT:c:/BUILD_win64-MIC_MKL_64/engine/LIBS/mkl/perflibs/performance_parameters_sb1.a c:/BUILD_win64-MIC_MKL_64/engine/OBJECTS/performance_parameters_sb1.o
xilib: executing 'k1om-mpss-linux-ar.exe'
xilib: executing 'lib'
Microsoft (R) Library Manager Version 10.00.30319.01
Copyright (C) Microsoft Corporation.  All rights reserved.

/OUT:c:/BUILD_win64-MIC_MKL_64/engine/LIBS/mkl/perflibs/performance_parameters_sb1.a
c:/BUILD_win64-MIC_MKL_64/engine/OBJECTS/performance_parameters_sb1.o
c:/BUILD_win64-MIC_MKL_64/engine/OBJECTS/performance_parameters_sb1.o :
fatal error LNK1107: invalid or corrupt file: cannot read at 0x2F88

and I also tried

$ xilib /OUT:c:/BUILD_win64-MIC_MKL_64/engine/LIBS/mkl/perflibs/performance_parameters_sb1.a c:/BUILD_win64-MIC_MKL_64/engine/OBJECTS/performance_parameters_sb1.o

xilib: executing 'k1om-mpss-linux-ar.exe'
xilib: executing 'lib'
Microsoft (R) Library Manager Version 10.00.30319.01
Copyright (C) Microsoft Corporation.  All rights reserved.

/OUT:c:/BUILD_win64-MIC_MKL_64/engine/LIBS/mkl/perflibs/performance_parameters_sb1.a
c:/BUILD_win64-MIC_MKL_64/engine/OBJECTS/performance_parameters_sb1.o
c:/BUILD_win64-MIC_MKL_64/engine/OBJECTS/performance_parameters_sb1.o :
fatal error LNK1107: invalid or corrupt file: cannot read at 0x2F88

so I went looking for the linux tools and tried them directly:

k1om-mpss-linux-ar.exe ar c:/BUILD_win64-MIC_MKL_64/engine/LIBS/mkl/perflibs/performance_parameters_sb1.a c:/BUILD_win64-MIC_MKL_64/engine/OBJECTS/performance_parameters_sb1.o
C:\Program Files\Intel\MPSS\x86_64-mpsssdk-linux\usr\bin\k1om-mpss-linux\k1om-mpss-linux-ar.exe:
c:/BUILD_win64-MIC_MKL_64/engine/OBJECTS/performance_parameters_sb1.o: File format not recognized

What is the correct way to do this ?

[Kevin_D_Intel]

Ok. The location of x86_64-k1om-linux-ar.exe that I noted was MPSS 2.1 specific; the location of yours is for MPSS 3.x. Thanks for noting that.

Nothing is needed for handling the .o files. Those are handled invisibly by the xilib and other tools to create the corresponding MIC side static library at the same time the CPU-side static lib is created.

Your trace shows a routine is called in the offload section that has not been resolved in the offload image. That can be due to lacking the target(mic) decoration as you suspected; however, it appears the xilib command you cited from earlier thread is likely the cause. The earlier suggestions to include “..\” in the -out: argument is now ill-advised and leads to unwanted mangling of the MIC-side static library name and placement of it. Perhaps something else changed since the time I offered that advice to the release of newer Composer XE SP1 releases.

For now do not use the “..\” path component in the xilib command and just allow xilib to create both the CPU-side and MIC-side static libraries in the same directory as the .obj files. (I will edit that earlier post and remove the suggestion of placing the resulting .lib with a path component on -out: and just suggest it be created in the current folder)

You might check your build for similar name mangling and unexpected placement of the <file>MIC.a library based the details in my example below.

Here’s the mangling of the MIC-side library that I see trying to use the earlier advice. The name of the MIC-side library includes “..” where the backslash was dropped and note the library is created in the same directory as the .obj files, whereas the CPU-side library was placed in the parent folder; thus ..\ was handled properly.

C:\DPD200256240\StaticLib_only\MyStaticLib\x64\Debug> xilib -out:..\MyStaticLib.lib *.obj
xilib: executing 'x86_64-k1om-linux-ar.exe'
xilib: executing 'lib'
Microsoft (R) Library Manager Version 11.00.60315.1
Copyright (C) Microsoft Corporation.  All rights reserved.

-out:..\MyStaticLib.lib
tbo_sort_lib.obj

C:\DPD200256240\StaticLib_only\MyStaticLib\x64\Debug> dir *MyStati*
 Volume in drive C has no label.

 Directory of C:\DPD200256240\StaticLib_only\MyStaticLib\x64\Debug

07/31/2014  08:09 AM            49,800 ..MyStaticLibMIC.a
               1 File(s)         49,800 bytes
               0 Dir(s)  86,699,732,992 bytes free

C:\DPD200256240\StaticLib_only\MyStaticLib\x64\Debug> dir ..\MyStat*
 Volume in drive C has no label.

 Directory of C:\DPD200256240\StaticLib_only\MyStaticLib\x64

07/31/2014  08:09 AM            41,684 MyStaticLib.lib
               1 File(s)         41,684 bytes
               0 Dir(s)  86,699,732,992 bytes free

[PKM]

Thanks Kevin

My files also had the two dangling dots and your suggestion resolved that problem. However, even after relinking i still have the exact same problem in the program where i want to use the static library ... It still seems that the MIC routines are not being linked at all.

One thing that could potentially explain this is a well hidden "no such file or directory" in the build log as seen below - but I have no idea what the message is relating to. As you can see from the command line i link both the main library paralution.lib and paralutionMIC.a explicitly, since i am not sure whether the MIC file is passed to the MIC linker implicitly. Removing the explicit link to the MIC library have no influence on the outcome.

Any further suggestions?

Link /OUT:"x64\Debug\Console1.exe" /INCREMENTAL:NO /NOLOGO /LIBPATH:"p:\casper.kirkegaard\ftn\steen test project\Console1\Console1\paralution bins" /qoffload-ldopts="-lifcoremt c:\temp\ParalutionMIC.a" /MANIFEST /MANIFESTFILE:"P:\casper.kirkegaard\ftn\steen test project\Console1\Console1\x64\Debug\Console1.exe.intermediate.manifest" /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /DEBUG /PDB:"P:\casper.kirkegaard\ftn\steen test project\Console1\Console1\x64\Debug\Console1.pdb" /SUBSYSTEM:CONSOLE /IMPLIB:"P:\casper.kirkegaard\ftn\steen test project\Console1\Console1\x64\Debug\Console1.lib" paralution_fortran.obj paralution.lib "x64\Debug\iters.obj" "x64\Debug\blassm.obj" "x64\Debug\itaux.obj" "x64\Debug\unary.obj" "x64\Debug\inout.obj" "x64\Debug\ilut.obj" "x64\Debug\formats.obj" "x64\Debug\qsort.obj" "x64\Debug\mError.obj" "x64\Debug\mTicToc.obj" "x64\Debug\matvec.obj" "x64\Debug\GaussDirectSol.obj" "x64\Debug\mArrays.obj" "x64\Debug\mSparseMatrix.obj" "x64\Debug\Console1.obj"

No such file or directory

Link: executing 'link'

paralution_fortran.obj : warning LNK4204: 'P:\casper.kirkegaard\ftn\steen test project\Console1\Console1\x64\Debug\vc110.pdb' is missing debugging information for referencing module; linking object as if no debug info

[Kevin_D_Intel]

I have not been able to recreate the error yet so I do not know what is causing that. I thought maybe a path with embedded spaces but that works for me. I believe the error is from the MS linker side.

When specifying the static library (that contains the offload code) for linking within another project, you only specify the Windows (i.e. CPU-side) .lib name. It will be passed accordingly to the MIC-side link in the form of the .a name.

As noted earlier, you must undo the project dependencies (Project > Project Dependencies) and then for the console application, under Properties > Linker > Additional Dependencies, add only a reference to the CPU-side name. For my example discussed earlier I only specified the following under this property setting:  $(SolutionDir)\MyStaticLib\$(OutDir)\MyStaticLib.lib

Also make sure that if you relocate your static library w/offload somewhere that you relocate both the .lib and MIC.a so as to keep them side-by-side always. Our next 15.0 release merges the .lib/.a into a single .lib image so the handling is more natural to Windows. The same applies to a dynamic library where the.dll/.so are merged into a single .dll.

[PKM]

Thanks Kevin

15.0 is due later this month right? In that case I think it will be better if I postpone further work on the project until after the release ..

C

[Kevin_D_Intel]

Yes, that's our target and we appear to be on track.

[Kevin_D_Intel]

I confirmed the defects (DPD200250854, DPD200256240) mentioned in replies #18 and #20 are both fixed in the Intel® Composer XE 2013 SP1 Update 4 release (2013.1.4.211 - Linux) --AND-- the Intel® Parallel Studio XE 2015 Initial Release (2015.0.090 - Linux).