Translating Visual Studio project to command line

[rorban]

For various reasons, I need to use the compiler at command line to compile a project that I had previously been compiling successfully in Visual Studio 2012. 

The Solution properties pages show several different command lines, one for "Fortran," one for "Linker," one for "Resources," and one for MIDL. Can I combine these somehow as as argument to the ifort command so that one call to ifort builds the entire solution (equivalent to "Rebuild Solution" in Visual Studio)? If so, how? Just concatenate them, or something else?

Also, the documentation says that the source files need to be put in the correct order so that dependencies are resolved correctly. If I Rebuild Solution in Visual Studio, does the order of compilation as shown in the Output Window correspond to the appropriate ordering of the files on the command line?

Thanks for any insight.

[Lorri_M_Intel]

Do a rebuild-all, and look at the build log.

That will display all the command line switches passed to ifort and the linker.

You should be able to use that log file to create a .bat file to build from the command line.

Although, it may require a little bit of tweaking of directory names.

              --Lorri

[mecej4]

If your project uses Fortran source files with many modules and USE statements, a batch file to recreate the EXE/DLL/LIB-s from scratch may not be of much use during development, unless the compilation time is insignificant. Instead, a makefile is needed that has enough information to enable recompiling only those sources that have been changed and their dependents. There are a number of third party Fortran module-aware tools that will scan your source, analyze dependencies and output a makefile. Search the Web with combinations of "make", "depend" and "Fortran".

[rorban]

Lorri Menard (Intel) wrote:

Do a rebuild-all, and look at the build log.

That will display all the command line switches passed to ifort and the linker.

You should be able to use that log file to create a .bat file to build from the command line.

Although, it may require a little bit of tweaking of directory names.

              --Lorri

I have more questions.

First, the purpose of converting to command line is to be able to automate the building of dozens of variations of the output dll depending on preprocessor definitions and target architecture (SSE4.1 or AVX2). My goal is to make a big batch file to generate the necessary variants without intervention, where the batch file simply repeats the required command lines over and over with the necessary modifications in each one. Rebuilding each variant is OK because a rebuild takes less than a minute, so the whole build process with multiple builds should takes less than two hours.  

There are about 70 sources files. The project uses whole-program optimization.The build log is quite complicated, with 11 invocations of ifort. In several places, it also says

Creating temporary file "RSP1.rsp" with content...

So for each section of the batch file, do I just insert 11 invocations of ifort and one invocation of Link? If so, how do the temporary files RSP1.rsp get handled? If each section of the batch file just consists of 12 lines (11 invocations of ifort and one invocation of link), will the compiler create the RSP1.rsp files automatically such that the next invocation of ifort finds them?]

Why are the temporary RSP1.rsp files needed? 

Where the build log says

Creating command line "ifort @"C:\Project_Path\MyProject\Release\RSP1.rsp""

does that translate into another line in the batch file:

ifort @"C:\Project_Path\MyProject\Release\RSP1.rsp" ?

Finally, the build log shows no invocation of ifort before the first RSP1.rsp file is created. How is creation of the first RSP1.rsp achieved in the batch file? Assume that I will have pre-compiled the resource files, as there are several variants that must be linked depending on other build parameters, so it will not be necessary to invoke the resource compiler.

Boiled down (with multiple source files removed from the command lines, illustratively replaced by "more_source_files.f90"... ), the build log looks like this:

Build Log
 	 	
 Build started: Project: NativeAP, Configuration: Release|Win32 

Output
 	 	
Deleting intermediate files and output files for project 'MyProject', configuration 'Release|Win32'.
Compiling resources...
Rc /fo "Release/Resource1.res" "C:\Project_Path\Resource1.rc"
Microsoft (R) Windows (R) Resource Compiler Version 6.3.9600.17336
Copyright (C) Microsoft Corporation.  All rights reserved.

Compiling with Intel(R) Visual Fortran Compiler 16.0.1.146 [IA-32]...
Creating temporary file "RSP1.rsp" with contents
[
 /nologo /MP /O2 /QxSSE4.1 /Qipo /I"C:\Project_Path" /I"C:\Project_Path\Include" /DisAM=0 /DisHD=0 /DisFM=0 /DnChannels=6 /extend_source:132 /standard-semantics /warn:declarations /warn:unused /warn:truncated_source /warn:uncalled /warn:interfaces /names:lowercase /iface:cref /module:"Release\\" /object:"Release\\" /Fd"Release\vc120.pdb" /gen-dep:"Release\MyProject.dep" /traceback /check:none /libs:dll /threads /c /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\\bin" "C:\Project_Path\source_files.f90" ...
]
Creating command line "ifort @"C:\Project_Path\MyProject\Release\RSP1.rsp""

ifort /nologo /MP /O2 /QxSSE4.1 /Qipo /I"C:\Project_Path" /I"C:\Project_Path\Include" /DisAM=0 /DisHD=0 /DisFM=0 /DnChannels=6 /extend_source:132 /standard-semantics /warn:declarations /warn:unused /warn:truncated_source /warn:uncalled /warn:interfaces /names:lowercase /iface:cref /module:"Release\\" /object:"Release\\" /Fd"Release\vc120.pdb" /gen-dep:"Release\MyProject.dep" /traceback /check:none /libs:dll /threads /c /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\\bin" "C:\Project_Path\more_source_files.f90"...
ifort /nologo /MP /O2 /QxSSE4.1 /Qipo /I"C:\Project_Path" /I"C:\Project_Path\Include" /DisAM=0 /DisHD=0 /DisFM=0 /DnChannels=6 /extend_source:132 /standard-semantics /warn:declarations /warn:unused /warn:truncated_source /warn:uncalled /warn:interfaces /names:lowercase /iface:cref /module:"Release\\" /object:"Release\\" /Fd"Release\vc120.pdb" /gen-dep:"Release\MyProject.dep" /traceback /check:none /libs:dll /threads /c /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\\bin" "C:\Project_Path\more_source_files.f90"...
ifort /nologo /MP /O2 /QxSSE4.1 /Qipo /I"C:\Project_Path\\" /I"C:\Project_Path" /I"C:\Project_Path\Include" /DisAM=0 /DisHD=0 /DisFM=0 /DnChannels=6 /extend_source:132 /standard-semantics /warn:declarations /warn:unused /warn:truncated_source /warn:uncalled /warn:interfaces /names:lowercase /iface:cref /module:"Release\\" /object:"Release\\" /Fd"Release\vc120.pdb" /traceback /check:none /libs:dll /threads /c /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\\bin" "C:\Project_Path\more_source_files.f90"...
ifort /nologo /MP /O2 /QxSSE4.1 /Qipo /I"C:\Project_Path" /I"C:\Project_Path\Include" /DisAM=0 /DisHD=0 /DisFM=0 /DnChannels=6 /extend_source:132 /standard-semantics /warn:declarations /warn:unused /warn:truncated_source /warn:uncalled /warn:interfaces /names:lowercase /iface:cref /module:"Release\\" /object:"Release\\" /Fd"Release\vc120.pdb" /gen-dep:"Release\MyProject.dep" /traceback /check:none /libs:dll /threads /c /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\\bin" "C:\Project_Path\more_source_files.f90"...
Creating temporary file "RSP1.rsp" with contents
[
 /nologo /MP /O2 /QxSSE4.1 /Qipo /I"C:\Project_Path" /I"C:\Project_Path\Include" /DisAM=0 /DisHD=0 /DisFM=0 /DnChannels=6 /extend_source:132 /standard-semantics /warn:declarations /warn:unused /warn:truncated_source /warn:uncalled /warn:interfaces /names:lowercase /iface:cref /module:"Release\\" /object:"Release\\" /Fd"Release\vc120.pdb" /gen-dep:"Release\MyProject.dep" /traceback /check:none /libs:dll /threads /c /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\\bin" "C:\Project_Path\more_source_files.f90"...
]
Creating command line "ifort @"C:\Project_Path\MyProject\Release\RSP1.rsp""

Creating temporary file "RSP1.rsp" with contents
[
 /nologo /MP /O2 /QxSSE4.1 /Qipo /I"C:\Project_Path" /I"C:\Project_Path\Include" /DisAM=0 /DisHD=0 /DisFM=0 /DnChannels=6 /extend_source:132 /standard-semantics /warn:declarations /warn:unused /warn:truncated_source /warn:uncalled /warn:interfaces /names:lowercase /iface:cref /module:"Release\\" /object:"Release\\" /Fd"Release\vc120.pdb" /gen-dep:"Release\MyProject.dep" /traceback /check:none /libs:dll /threads /c /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\\bin" "C:\Project_Path\more_source_files.f90"...
]
Creating command line "ifort @"C:\Project_Path\MyProject\Release\RSP1.rsp""

ifort /nologo /MP /O2 /QxSSE4.1 /Qipo /I"C:\Project_Path" /I"C:\Project_Path\Include" /DisAM=0 /DisHD=0 /DisFM=0 /DnChannels=6 /extend_source:132 /standard-semantics /warn:declarations /warn:unused /warn:truncated_source /warn:uncalled /warn:interfaces /names:lowercase /iface:cref /module:"Release\\" /object:"Release\\" /Fd"Release\vc120.pdb" /gen-dep:"Release\MyProject.dep" /traceback /check:none /libs:dll /threads /c /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\\bin" "C:\Project_Path\more_source_files.f90"...
ifort /nologo /MP /O2 /QxSSE4.1 /Qipo /I"C:\Project_Path" /I"C:\Project_Path\Include" /DisAM=0 /DisHD=0 /DisFM=0 /DnChannels=6 /extend_source:132 /standard-semantics /warn:declarations /warn:unused /warn:truncated_source /warn:uncalled /warn:interfaces /names:lowercase /iface:cref /module:"Release\\" /object:"Release\\" /Fd"Release\vc120.pdb" /gen-dep:"Release\MyProject.dep" /traceback /check:none /libs:dll /threads /c /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\\bin" "C:\Project_Path\more_source_files.f90"...
ifort /nologo /MP /O2 /QxSSE4.1 /Qipo /I"C:\Project_Path" /I"C:\Project_Path\Include" /DisAM=0 /DisHD=0 /DisFM=0 /DnChannels=6 /extend_source:132 /standard-semantics /warn:declarations /warn:unused /warn:truncated_source /warn:uncalled /warn:interfaces /names:lowercase /iface:cref /module:"Release\\" /object:"Release\\" /Fd"Release\vc120.pdb" /gen-dep:"Release\MyProject.dep" /traceback /check:none /libs:dll /threads /c /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\\bin" "C:\Project_Path\more_source_files.f90"...
ifort /nologo /MP /O2 /QxSSE4.1 /Qipo /I"C:\Project_Path" /I"C:\Project_Path\Include" /DisAM=0 /DisHD=0 /DisFM=0 /DnChannels=6 /extend_source:132 /standard-semantics /warn:declarations /warn:unused /warn:truncated_source /warn:uncalled /warn:interfaces /names:lowercase /iface:cref /module:"Release\\" /object:"Release\\" /Fd"Release\vc120.pdb" /gen-dep:"Release\MyProject.dep" /traceback /check:none /libs:dll /threads /c /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\\bin" "more_source_files.f90"...
Linking...
Creating temporary file "RSP1.rsp" with contents
[
 /OUT:"Release\MyProject.dll" /VERSION:0.9 /INCREMENTAL:NO /NOLOGO /MANIFEST /MANIFESTFILE:"Release\MyProject.dll.intermediate.manifest" /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /DEBUG /PDB:"C:\Project_Path\MyProject\Release\MyProject.pdb" /SUBSYSTEM:WINDOWS /OPT:REF /IMPLIB:"C:\Project_Path\MyProject\Release\MyProject.lib" /DLL "Release\more_source_files.obj"..."Release/Resource1.res"
]
Creating command line "Link @"C:\Project_Path\MyProject\Release\RSP1.rsp""

   Creating library C:\Project_Path\MyProject\Release\MyProject.lib and object C:\Project_Path\MyProject\Release\MyProject.exp

Embedding manifest...
mt.exe /nologo /outputresource:"C:\Project_Path\MyProject\Release\MyProject.dll;#2" /manifest "Release\MyProject.dll.intermediate.manifest"

MyProject - 0 error(s), 0 warning(s)

[Steven_L_Intel1]

You don't need the RSP files - that's just a method the VS build system uses if the command gets long.

Typically you compile each source file using "ifort /c" and any other options you need. Then you link the executable using ifort (not link and without /c), naming all the object files. Most people use makefiles for this.

Some of the options that get put into the build you don't need or can be shortened. For example, you might have shorter paths for /include and /module. You don't need /Qlocation,link.

[rorban]

Steve Lionel (Intel) wrote:

You don't need the RSP files - that's just a method the VS build system uses if the command gets long.

Typically you compile each source file using "ifort /c" and any other options you need. Then you link the executable using ifort (not link and without /c), naming all the object files. Most people use makefiles for this.

Some of the options that get put into the build you don't need or can be shortened. For example, you might have shorter paths for /include and /module. You don't need /Qlocation,link.

Are you saying that in my case, ifort should be invoked 70 times, one for each source file? (In the build log, it appears that the source files are grouped and ifort is invoked for the various groups.)

In the final invocation of ifort (to link the executable), does the order of the *.obj files matter? (Can I used the order shown in the build log?) And is the compiled resource file placed after all of the *.obj files, as it is in the build log?

How is whole-program optimization handled in this scenario?

Might I suggest that the compiler's documentation on the command line be updated to answer novice questions like the ones I have, as I dislike wasting people's time when I can find the answers quickly through documentation? The examples shown in the documentation are for very simple projects.

Also, I think it would be valuable if the Intel compiler included an makefile generator that could automatically translate a working Visual Studio Fortran project into a makefile. Makefile generation could just be an another build option in Visual Studio.

Thanks for everyone's help.

[rorban]

Steve Lionel (Intel) wrote:

You don't need the RSP files - that's just a method the VS build system uses if the command gets long.

Typically you compile each source file using "ifort /c" and any other options you need. Then you link the executable using ifort (not link and without /c), naming all the object files. Most people use makefiles for this.

Some of the options that get put into the build you don't need or can be shortened. For example, you might have shorter paths for /include and /module. You don't need /Qlocation,link.

Are you saying that in my case, ifort should be invoked 70 times, one for each source file? (In the build file, it appears that the source files are grouped and ifort is invoked for the various groups.)

In the final invocation of ifort (to link the executable), does the order of the *.obj files matter? (Can I used the order shown in the build log?) And is the compiled resource file placed after all of the *.obj files, as it is in the build log?

How is whole-program optimization handled in this scenario?

Might I suggest that the compiler's documentation on the command line be updated to answer novice questions like the ones I have, as I dislike wasting people's time when I can find the answers quickly through documentation? The examples shown in the documentation are for very simple projects.

Also, I think it would be valuable if the Intel compiler included an makefile generator that could automatically translate a working Visual Studio Fortran project into a makefile. Makefile generation could just be an another build option in Visual Studio.

[Steven_L_Intel1]

If you want to group the files, you can. You just need to make sure that you compile module sources before the sources that USE the modules. Note the use of /MP which compiles the sources in parallel, saving build time.

What I often do is take the series of ifort commands from the buildlog.htm and edit them into a batch file, removing the pieces of the command I don't want. Takes a few minutes in a text editor.

There are makefile generators for Fortran available. VS itself used to have one but Microsoft took it out. I will also comment that one can invoke devenv (VS) from the command line to build a project. This is an alternative to using a batch file or makefile and is a solution I like better.

We document how to use VS and how to use the command line. It isn't often that someone wants to convert from one to the other and trying to address all the possible issues and needs would be complex. Once you understand the basics of how command line builds work, it isn't difficult to make the translation.

[rorban]

Steve Lionel (Intel) wrote:

If you want to group the files, you can. You just need to make sure that you compile module sources before the sources that USE the modules. Note the use of /MP which compiles the sources in parallel, saving build time.

What I often do is take the series of ifort commands from the buildlog.htm and edit them into a batch file, removing the pieces of the command I don't want. Takes a few minutes in a text editor.

There are makefile generators for Fortran available. VS itself used to have one but Microsoft took it out. I will also comment that one can invoke devenv (VS) from the command line to build a project. This is an alternative to using a batch file or makefile and is a solution I like better.

We document how to use VS and how to use the command line. It isn't often that someone wants to convert from one to the other and trying to address all the possible issues and needs would be complex. Once you understand the basics of how command line builds work, it isn't difficult to make the translation.

After working through everything and finally getting it working, my only suggestions regarding the documentation would be (a) to add one more example that shows how to call the linker (including linker options) from an ifort command. The technique is explained but not illustrated, (b) to add a paragraph explaining how to edit the contents of BuildLog.htm to make a command line batch file, and (c) to explain how manifests are handled in the command line.

Regarding manifests, all the documentation says is:

"The Intel® Fortran Compiler supports manifests, a Visual Studio* feature. Manifests describe run-time dependencies of a built application. A manifest file can be embedded in the assembly, which is the default behavior, or can be a separate standalone file. You can use the Manifest Tool property pages, which are accessed through Project > Properties, to change project settings that affect the manifest."

I don't know if a manifest is needed to make the built dll work with the Intel redistributables, and there is nothing in the command line documentation about manifests. BuildLog.htm shows the following as the final step:

Embedding manifest...

mt.exe /nologo /outputresource:"output.dll;#2" /manifest "Release\output.dll.intermediate.manifest"

but I don't know if this happens automatically if I call the linker from ifort. I can say that the output of the command line build (as I have it so far) includes a file: output.dll.intermediate.manifest, but I want the dependency information to be built into the dll itself and I'm sure what I need to do in the command line: do I need to call mt.exe explicitly, or does calling the linker from ifort take care of this automatically?

[jimdempseyatthecove]

>>First, the purpose of converting to command line is to be able to automate the building of dozens of variations of the output dll depending on preprocessor definitions and target architecture (SSE4.1 or AVX2).

Try MS VS Batch Build: https://msdn.microsoft.com/en-us/library/jj651644.aspx

An alternate way to do this is to create multiple Projects within the Solution. Each project having different output folders but same source files, and different compiler options and #defines. Then add an additional project that is dependent on all your DLL build projects, and which "packages" the .DLL outputs.

Jim Dempsey

[rorban]

jimdempseyatthecove wrote:

>>First, the purpose of converting to command line is to be able to automate the building of dozens of variations of the output dll depending on preprocessor definitions and target architecture (SSE4.1 or AVX2).

Try MS VS Batch Build: https://msdn.microsoft.com/en-us/library/jj651644.aspx

An alternate way to do this is to create multiple Projects within the Solution. Each project having different output folders but same source files, and different compiler options and #defines. Then add an additional project that is dependent on all your DLL build projects, and which "packages" the .DLL outputs.

Jim Dempsey

Thanks for the tips! I didn't know about the Batch Build feature of Visual Studio, although now that I look, it's starting right at me in the Build menu :-)