lld-link: error: section larger than 4 GiB

[JulieMarieC]

Hello,

I was using ifort compiler, but i have seen the message that it will be deprecated, so I'm trying to use ifx instead, but I get this error:

lld-link: error: section larger than 4 GiB: .data

Do you have any suggestions?

Best regards,

Julie

[Steve_Lionel]

Do you really have a static variable (COMMON, most likely) larger than 4GB?

This isn't a compiler issue - add -mcmodel medium to the compiler options.

[JulieMarieC]

It seems that '-mcmodel medium' is not recognized by ifx compiler.

[Barbara_P_Intel]

On Linux the syntax is "-mcmodel=medium". There is no equivalent option for Windows. See this section in the Fortran Developer Guide and Reference.

[JulieMarieC]

Thanks for your answer. I will in the next few months, maybe there will be an update that will allow this option for Windows.

[Steve_Lionel]

No, there won't - Windows has a hard limit of 2GB static code and data, due to the design of the executable format. Use ALLOCATABLE arrays instead of very large static arrays.

[JulieMarieC]

Ok, thanks for this precision.

So I don't know why it works with ifort but not with ifx: maybe it is linked to the fact that with ifort, our code is compiled for win32, but we have to compile for x64 with ifx?

[Steve_Lionel]

It doesn't work with 64-bit ifort either. 

[OldeMan]

---

@Steve_Lionel wrote:

It doesn't work with 64-bit ifort either. 

---

I'm getting the same message when compiling with "ifx".  This is on code that we've been compiling for years on "ifort" as both 32 and 64 bit, and as executables and DLLs.  This is F77 code that compiles into 20~30MB executables and/or DLLs.  I assure you, there is no way anything in its static data structure approaches 4GB.  (Unfortunately, the code is export restricted... so I can't share.)

Interestingly, I only get this message when I use "ifx" to create a "DLL".  There are no complaints when I build various forms of executable.

How do I get information about the ".data" constituents?  I'd like to see what ballooned to over 4GB.

[jimdempseyatthecove]

>>How do I get information about the ".data" constituents?  I'd like to see what ballooned to over 4GB.

On Windows, assuming you are interested in the .data section, for each object file use dumpbin /section:.data yourfile.obj

C:\test\Console21\Console21\x64\Release>dumpbin /section:.data console21.obj
Microsoft (R) COFF/PE Dumper Version 14.38.33130.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file console21.obj

File Type: COFF OBJECT

SECTION HEADER #2
   .data name
       0 physical address
       0 virtual address
      E8 size of raw data
     6A3 file pointer to raw data (000006A3 to 0000078A)
       0 file pointer to relocation table
       0 file pointer to line numbers
       0 number of relocations
       0 number of line numbers
C0500040 flags
         Initialized Data
         16 byte align
         Read Write

  Summary

          E8 .data

C:\test\Console21\Console21\x64\Release>

Use the programmers calculator to convert hex to decimal.

Once you find the file with the copious data section, you can investigate further.

The issue may not be in the .data section. You can use dumpbin without /section:... to get a summary of all the sections.

C:\test\Console21\Console21\x64\Release>dumpbin console21.obj
Microsoft (R) COFF/PE Dumper Version 14.38.33130.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file console21.obj

File Type: COFF OBJECT

  Summary

           0 .bss
          E8 .data
          B9 .drectve
           C .pdata
           8 .rdata
         36F .text
           C .xdata

Jim Dempsey

[jimdempseyatthecove]

For the offending file, to get additional information, try:

dumpbin /all /rawdata:none YourFile.obj

Jim Dempsey

[JohnDrohan]

Hi Jim - thanks again for these additional diagnostics.

Unfortunately, in my case, none of my object files when processed through the dumpbin binary indicate any true size issues. I've examined both the .data and the .bss sections.

Some other data:

IFORT compiler: sum of size of all .obj files: 39.2 MB

IFX compiler: sum of size of all .obj files: 42.8 MB

IFORT compiler: .map file - looks like it gets completely written to disk.

IFX compiler: .map file - looks like it fails to get completely written to disk, failing in section 3 with what does appear to be a memory address beyond a 4GB limit: 

"0003:fdcb9690 GTDATA_7FA$ISIGN 000000027ec7c690 gtdata_7fa.obj"

Is there anything I could send that would help with further diagnosing this issue? In the meantime, I have some possible refactoring ideas I could implement, but they would take time, and I'd likely just be guessing.

Thanks,

John

[JohnDrohan]

Was there ever any solution suggested for this issue? I am in the exact same boat as @OldeMan - export controlled restricted code that has compiled successfully for years in 32/64-bit IFORT into a dll 30-ish megabytes in size.  

Everything compiles, but upon link get the exact same error: section larger than 4 GiB: .data

We are still working with 2024.2.0.635 compiler due to needing to support 32-bit binaries going forward.

[Steve_Lionel]

I'm going to take a guess and suggest that it's the same issue as I describe here - you have an include file that declares one or more COMMON blocks and you initialize the COMMON in that same include file. This is not allowed in Fortran, but it seems the ifort compiler handles it in a different way than ifx. The result is that with ifx, your initializations are repeated in every object file where the COMMON is included.

See my linked post for the solution to that.

[OldeMan]

@Steve_Lionel  - I'm guessing you didn't mean to post a link to a politico article describing staff cuts as NASA?

[OldeMan]

Steve, I was intrigued by your description of the potential (likely?) problem:  #includes that include declarations of COMMON(s) and initialization of same.  Nope .. I checked our several hundred #include'd header files .. and none are guilty of this.

Also ... we only get this weird "section larger than 4 GiB: .data" when compiling our code in DLL form.  When compiling (essentially) the same code as an EXE ... it works fine with "ifx".  I don't think that fits your guess at the cause?

[Steve_Lionel]

Link fixed - sigh.

The other thread also involves a DLL, but the original poster found a different solution. At this point I don't know what's going on.

[JohnDrohan]

I can also confirm that our code base has no common declarations in our include files. In fact, we converted 90+% of our .inc files to modules and have only 2 small ones remaining. From the other thread, I will examine our .map file and compare the IFORT/IFX versions of each to see if there is a similar difference/issue.

[jimdempseyatthecove]

This may be an issue previously addressed by @Steve_Lionel 

The .data segment is an appendable segment as used by the linker, where each object file's static variables (global or local, iow module, common, and procedure, are appended to the .data section.

Should (many of) your source files contain:

     include "someOtherFile.**bleep**"

or

     #include "someOtherFile.**bleep**"

.AND.

    Those files contain variables that are initialized

Then each instance of those INCLUDEs may (emphasis added) cause the data to be appended to the .data section.

As to if this is a linker bug or not, this would be for others to decide.

Steve's recommendation was to place those data items in a module and then replace the INCLUDE with a USE TheModuleYouMade.

Jim Dempsey

[jimdempseyatthecove]

Hmm, the forum is editing my post (changed triple x to **bleep**)

Jim

[JohnDrohan]

OK - I believe I may have fixed it using the above reply and some hints found in this thread as well: Re: Huge .bss section size difference with IFX vs IFORT? - Intel Community

When we transitioned from include files to modules, we kept much of the syntax the same.  And even worse....kept the common declarations inside the module, even though they were not necessary. I have commented out all the common statements in our biggest module and got everything to link correctly.  Now to test results.

I'll post more once I conduct further testing of the built dll,  and thanks to everyone for responding quickly to my questions.

John