Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.

Fortran “additional relocation overflows” error while reading big data file

Yeon__Jejoon
Novice
3,080 Views

I have following format of input file: 

    4400000
          1       4       4    7056    4203    7190    2856       0       0       0       0       0       0       0   0.942    0.93   0.944   0.925       0       0       0       0       0       0   3.846     0.0   1.627
          2       3       3    8553     481    7734       0       0       0       0       0       0       0       0    0.63   0.923   0.322       0       0       0       0       0       0       0    1.93     2.0  -0.792
          3       3       2    7007    3625       0       0       0       0       0       0       0       0       0   0.359    0.93       0       0       0       0       0       0       0       0   1.889     2.0  -0.728
          4       4       4    1611     854    7901    8306       0       0       0       0       0       0       0   0.923   0.944   0.936   0.915       0       0       0       0       0       0   3.867     0.0   1.488
          5       3       4    7657    1180    6659    4147       0       0       0       0       0       0       0   0.332    0.69   0.346   0.574       0       0       0       0       0       0   1.969     2.0  -0.634
          6       3       2    1978     523       0       0       0       0       0       0       0       0       0   0.931   0.917       0       0       0       0       0       0       0       0   1.849     2.0  -0.805
    ...

 

This file has total 27 columns. I tried to read this file using following command:  

    do l=1,num
     read (1,42) atindex(l),atype(l),nbonds(l),(conn(l,j),j=1,10),molnr(l),(bos(l,j),j=1,10),abo(l),nlp(l),charge(l)
    enddo

(num = number of lines) and 

    42  format (i7,13i7,11f7.3,f7.1,f7.3)

I compiled this using intel compiler. However, this creates following "additional relocation overflows" error: 

    ....
    /opt/shared/intel/2018u4/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin/libifcoremt.a(for_init.o): In function `for__signal_handler':
        for_init.c:(.text+0x906): additional relocation overflows omitted from the output

It looks like memory issue due to too much column. That error never happens when I read only upto integer column (column 14) with proper format. However, when I tried to read real numbers, then the error occur. 
  
This is research server cluster, not my laptop or desktop so I wouldn't be possible to modify installed setting of intel compiler. 

How can I escape from this issue? 

0 Kudos
1 Solution
Steve_Lionel
Honored Contributor III
3,080 Views

I am very skeptical that the I/O statement or format is directly related to the relocation error, other than that it might push the data size over the "small" limit. I have not evaluated the correctness of the format, etc., but don't see that as related to the link error.

Short answer - compile with "-mcmodel medium" and see what you get.

View solution in original post

0 Kudos
13 Replies
Steve_Lionel
Honored Contributor III
3,080 Views

Please show the complete and actual text of the message. The part you quoted is the least interesting part.

0 Kudos
mecej4
Honored Contributor III
3,080 Views

Sorry, I misdiagnosed the problem and I don't seem able to delete the post.

0 Kudos
Yeon__Jejoon
Novice
3,080 Views

Steve Lionel (Ret.) (Blackbelt) wrote:

Please show the complete and actual text of the message. The part you quoted is the least interesting part.

$ ifort test10d.f -o test10d
/tmp/ifortYNdLHJ.o: In function `MAIN__':
test10d.f:(.text+0x59c): relocation truncated to fit: R_X86_64_32S against `.bss'
test10d.f:(.text+0x736): relocation truncated to fit: R_X86_64_32S against `.bss'
/opt/shared/intel/2018u4/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin/libifcoremt.a(for_init.o): In function `for__process_start_time':
for_init.c:(.text+0x8): relocation truncated to fit: R_X86_64_PC32 against `for_process_start_time'
/opt/shared/intel/2018u4/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin/libifcoremt.a(for_init.o): In function `handle_flt_und':
for_init.c:(.text+0x34): relocation truncated to fit: R_X86_64_PC32 against symbol `for__l_fpe_mask' defined in .bss section in /opt/shared/intel/2018u4/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin/libifcoremt.a(for_init.o)
for_init.c:(.text+0x356): relocation truncated to fit: R_X86_64_PC32 against symbol `for__l_undcnt' defined in .bss section in /opt/shared/intel/2018u4/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin/libifcoremt.a(for_init.o)
for_init.c:(.text+0x3a8): relocation truncated to fit: R_X86_64_PC32 against symbol `for__l_undcnt' defined in .bss section in /opt/shared/intel/2018u4/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin/libifcoremt.a(for_init.o)
for_init.c:(.text+0x3b0): relocation truncated to fit: R_X86_64_PC32 against symbol `for__l_undcnt' defined in .bss section in /opt/shared/intel/2018u4/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin/libifcoremt.a(for_init.o)
for_init.c:(.text+0x3bc): relocation truncated to fit: R_X86_64_PC32 against symbol `for__l_excpt_info' defined in .bss section in /opt/shared/intel/2018u4/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin/libifcoremt.a(for_init.o)
for_init.c:(.text+0x3dd): relocation truncated to fit: R_X86_64_PC32 against symbol `for__l_excpt_info' defined in .bss section in /opt/shared/intel/2018u4/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin/libifcoremt.a(for_init.o)
for_init.c:(.text+0x430): relocation truncated to fit: R_X86_64_PC32 against symbol `for__l_undcnt' defined in .bss section in /opt/shared/intel/2018u4/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin/libifcoremt.a(for_init.o)
/opt/shared/intel/2018u4/compilers_and_libraries_2018.5.274/linux/compiler/lib/intel64_lin/libifcoremt.a(for_init.o): In function `for__signal_handler':

 

0 Kudos
Yeon__Jejoon
Novice
3,080 Views

mecej4 (Blackbelt) wrote:

The READ statement contains 26 elements in the I/O list, whereas the input data has 27 items per record, as does the format specification. Please investigate and correct this before doing anything else.

Hmm but even free format (2,*) creates the same error... I also checked with proper format but still the same error. 

0 Kudos
Steve_Lionel
Honored Contributor III
3,080 Views

See if this helps: https://software.intel.com/en-us/articles/avoiding-relocation-errors-when-building-applications-with-large-global-or-static-data-on-intel64/

0 Kudos
JohnNichols
Valued Contributor III
3,078 Views

Read as a charter array - then take the array apart 

0 Kudos
JohnNichols
Valued Contributor III
3,078 Views

character sorry

 

0 Kudos
Yeon__Jejoon
Novice
3,080 Views

Nichols, John wrote:

Read as a charter array - then take the array apart 

You mean, read all columns as characters, and print them in 2 separate files by 14 columns each, then analyze? 

0 Kudos
John_Campbell
New Contributor II
3,080 Views

I looked at the data you posted. It does not agree with the format statement you used. The columns of numbers are more than 7 characters wide. (This may simply be due to re-formating in the forum post.)

You are showing "read(1,42)...". Are you taking the data as terminal input, or are you opening the file on unit 1 ?

An alternative, as John N suggests, would be to try a simple test fortran program and a copy of the data file:

Use a data file and open the file using "open (unit=11, file='data_copy.txt')  "and then

read each line into "character input_line*256". ( trailing blanks are not a problem) using read (11,41)  input_line

you could then read 27 values into a real array(27), using either "read(input_line,42) array" or "read(input_line,*) array" . (read num first)

write out the results you read.

I have not suggested error tests on I/O statements, as you should use the Fortran I/O response.

Start small and expand the test. Hopefully this could identify where the error is not coming from.

There is a lot to be gained in a simple data validation program, which you can enhance as you learn more about the data quirks. eg max, min, avg, median of values, zeros, or bad values is very useful for recorded test results.

Probably not here as no "additional relocation overflows" evident in the approach so far.

Probably the error is in the way you have defined your arrays. Are you using auto allocate arrays in some unusual way?

0 Kudos
Steve_Lionel
Honored Contributor III
3,081 Views

I am very skeptical that the I/O statement or format is directly related to the relocation error, other than that it might push the data size over the "small" limit. I have not evaluated the correctness of the format, etc., but don't see that as related to the link error.

Short answer - compile with "-mcmodel medium" and see what you get.

0 Kudos
JohnNichols
Valued Contributor III
3,078 Views

Yeon, Jejoon wrote:

Quote:

Nichols, John wrote:

 

Read as a charter array - then take the array apart 

 

 

You mean, read all columns as characters, and print them in 2 separate files by 14 columns each, then analyze? 

Somewhere on this forum is a program by Jim Dempsey for taking apart CSV files which is similar to your problem -- the separator character is different is the only problem.  He provided an elegant solution to my brute force method.  

But I would try the other solution -- but reading files such as yours reading with read(1,*)..... is usually much better

 

0 Kudos
Yeon__Jejoon
Novice
3,080 Views

Steve Lionel (Ret.) (Blackbelt) wrote:

I am very skeptical that the I/O statement or format is directly related to the relocation error, other than that it might push the data size over the "small" limit. I have not evaluated the correctness of the format, etc., but don't see that as related to the link error.

Short answer - compile with "-mcmodel medium" and see what you get.

Thank you so much. This worked like a charm. 

0 Kudos
Yeon__Jejoon
Novice
3,080 Views

@John Campbell @Nichols, John Thank you so much for comment. 

0 Kudos
Reply