Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.

ifx removes object files

foxtran
New Contributor I
2,072 Views

Hello,

I have tried to compile some application which consists of a large number of files. Accidentally, I noticed that ifx started to remove files:

 

[ build_ifx_reldeb]$ tree > tree.initial
[ build_ifx_reldeb]$ /opt/intel/oneapi/compiler/2023.2.0/linux/bin/ifx  -O2 -g @CMakeFiles/drpa.rsp -o drpa
[ build_ifx_reldeb]$ tree > tree.after
[ build_ifx_reldeb]$ diff tree.initial tree.after -upN --color
--- tree.initial 2023-11-19 22:30:29.674035720 +0100
+++ tree.after 2023-11-19 22:30:42.785903961 +0100
@@ -1186,7 +1186,6 @@
 |   |   |   |   |-- h_g_tc_uc_rangesep.f-pp.f.d
-|   |   |   |   |-- h_h_tc_rangesep.f.o
 |   |   |   |   |-- h_h_tc_rangesep.f.o.ddi​



and more files were deleted (at this time, only 17 of them)
@CMakeFiles/drpa.rsp contains about 4400 lines (so, about 4400 .o files are linked together)

Fortunately, linking of drpa is successful, but other binaries can not be linked because ifx removed some of necessary files.

My shell supports such large number of arguments:

$ getconf ARG_MAX
76800000

Used ifx version:
$ ifx --version
ifx (IFX) 2023.2.0 20230622
Copyright (C) 1985-2023 Intel Corporation. All rights reserved.

 

1 Solution
Igor_V_Intel
Employee
468 Views

This bug is fixed in latest compiler version. I have verified that ifx 2024.2.1 works as expected.


View solution in original post

15 Replies
foxtran
New Contributor I
2,041 Views

Dear, 

I have the same behaviour with IFX 2024.0.0. It does not matter, do I use ninja nor make build scripts. 

Optimization options also has no effect.

0 Kudos
Igor_V_Intel
Employee
2,008 Views

What happens if you split this to 2 or 4 commands? It will be interesting to take a look on the output with -### option. Could you please run the linking with this option and attach a log? Will be nice to check if you actually exceeded 76800000 limit (it depends on how long are your lines, hope not more than 17k). The output from -### option should help to understand what is actually finally executed.


0 Kudos
foxtran
New Contributor I
2,001 Views

Dear Igor,

What happens if you split this to 2 or 4 commands?

I suppose it will work.

Fortunately, I created a script that reproduces observed behaviour:

#!/usr/bin/bash

#L1="a b c d e f g h i j k l m n o p q r s t v x y z"
L1="a b c d" # does not remove files with these directory names
L1="a b c d e"
L2=${L1}
L3=${L1}

OPT="-O2 -g"
LIBS="-L${CMPLR_ROOT}/opt/compiler/lib ${MKLROOT}/lib/libmkl_intel_ilp64.so  ${MKLROOT}/lib/libmkl_intel_thread.so  ${MKLROOT}/lib/libmkl_core.so  ${CMPLR_ROOT}/lib/libiomp5.so  -lpthread  -lm  -ldl  ${MKLROOT}/lib/libmkl_intel_ilp64.so  ${MKLROOT}/lib/libmkl_intel_thread.so ${MKLROOT}/lib/libmkl_core.so  ${CMPLR_ROOT}/lib/libiomp5.so  -lpthread  -lm  -ldl  -lpthread  -lm  -ldl  ${MKLROOT}/lib/libmkl_intel_ilp64.so  ${MKLROOT}/lib/libmkl_intel_thread.so  ${MKLROOT}/lib/libmkl_core.so  ${CMPLR_ROOT}/lib/libiomp5.so  ${MKLROOT}/lib/libmkl_intel_ilp64.so  ${MKLROOT}/lib/libmkl_intel_thread.so  ${MKLROOT}/lib/libmkl_core.so  ${CMPLR_ROOT}/lib/libiomp5.so  -lpthread  -lm  -ldl  -lirng"

mkdir -p tree
mkdir -p tree/mods

touch tree/null.f
echo "program main" > tree/main.f90

FILES_LIST=""

for i in ${L1}
do
  for j in ${L2}
  do
    echo "${i}${j}"
    for k in ${L3}
    do
      mkdir -p tree/${i}/${j}/${k}
      cat >tree/${i}/${j}/${k}/${i}${j}${k}_mod.f90 <<EOF
  subroutine print_${i}${j}${k}()
    print *, "call print_${i}${j}${k}()"
  end subroutine print_${i}${j}${k}
EOF
    ifx tree/${i}/${j}/${k}/${i}${j}${k}_mod.f90 ${OPT} -module tree/mods -c -o tree/${i}/${j}/${k}/${i}${j}${k}_mod.f90.o &
    FILES_LIST="$FILES_LIST $(pwd)/tree/${i}/${j}/${k}/${i}${j}${k}_mod.f90.o"
    cat >>tree/main.f90 <<EOF
    call print_${i}${j}${k}()
EOF
    done
  done
done
wait

echo "end program main" >> tree/main.f90

ifx tree/main.f90 ${OPT} -module tree/mods -c -o tree/main.f90.o
ifx tree/null.f -c -o tree/null.f.o
tree > tree.initial
echo ${FILES_LIST} > tree/files
echo ${LIBS} > tree/libs
echo "ifx ${OPT} @Tree/files $(pwd)/tree/main.f90.o $(pwd)/tree/null.f.o @Tree/libs -o tree/main"
ifx ${OPT} @Tree/files $(pwd)/tree/main.f90.o $(pwd)/tree/null.f.o @Tree/libs -o tree/main
tree > tree.after

diff tree.initial tree.after -upN --color

rm tree.initial tree.after
rm -rf tree

 

It expects that you have loaded oneAPI, so, MKLROOT, CMPLR_ROOT are defined and ifx is available.
At final after a few seconds, you will see the following log:

--- tree.initial	2023-11-22 19:24:32.198791170 +0100
+++ tree.after	2023-11-22 19:24:32.609787386 +0100
@@ -334,8 +334,7 @@
 |   |   |   |   |-- eab_mod.f90
 |   |   |   |   `-- eab_mod.f90.o
 |   |   |   |-- c
-|   |   |   |   |-- eac_mod.f90
-|   |   |   |   `-- eac_mod.f90.o
+|   |   |   |   `-- eac_mod.f90
 |   |   |   |-- d
 |   |   |   |   |-- ead_mod.f90
 |   |   |   |   `-- ead_mod.f90.o
@@ -406,11 +405,15 @@
 |   |       `-- e
 |   |           |-- eee_mod.f90
 |   |           `-- eee_mod.f90.o
+|   |-- files
+|   |-- libs
+|   |-- main
 |   |-- main.f90
 |   |-- main.f90.o
 |   |-- mods
 |   |-- null.f
 |   `-- null.f.o
+|-- tree.after
 `-- tree.initial

-157 directories, 256 files
+157 directories, 259 files


So, after linking (which was successful: see +main), some files were removed. Probably, the behaviour can be different on different machines, so, you can try to increase a number of created files.

From my experiments, with empty LIBS variable, the compilation is successful up to 26^3 files (I did not test more). As I said earlier, OPT flags does not play any role. Note, also there is an empty null.f. It also may play some role.

Best,
Igor


0 Kudos
Igor_V_Intel
Employee
1,978 Views

Thanks a lot for this reproducer and script. I was thinking to do the same but you saved a lot of time. Let me check if I can reproduce it and I will investigate it further.


0 Kudos
foxtran
New Contributor I
1,963 Views

@Igor_V_Intel, I just noticed that the script is not correct. Somewhat, after "at" symbol, the "tree" has large symbols while it should be small. Only lines 50 and 51 are affected.

0 Kudos
foxtran
New Contributor I
1,963 Views

After some experiments, 
- null.f.o has no effect
- minimal example when the problem is reproducible has the following LIBS variable:

LIBS="-lpthread  -lm  -ldl  -lpthread  -lm -ldl -lpthread  -lm  -ldl"

Then, reducing repeatable libs removes the problem 

0 Kudos
foxtran
New Contributor I
1,960 Views

Well... With -###

rm /tmp/ifx1715225509AnpRH2/ifxdashvXrkHm1
rm /tmp/ifx1715225509AnpRH2/ifxdummysjLn0Q.c
rm /tmp/ifx1715225509AnpRH2/ifxclangdashvDEKl9H
rm /tmp/ifx1715225509AnpRH2/ifxsearchdirs5kRXJB
rm /tmp/ifx1715225509AnpRH2/ifxgccdashvPPVbt8
rm /tmp/ifx1715225509AnpRH2/ifxldashvzSYfaD
rm /dev/shm/ger/tree/e/e/d/eed_mod.f90.o

The last deletion is not necessary

0 Kudos
Igor_V_Intel
Employee
1,880 Views

I modified the script and use this version now:

#!/usr/bin/bash

L1="a b c d" # does not remove files with these directory names
L1="a b c d e"
L2=${L1}
L3=${L1}

OPT="-O2 -g"
LIBS="-lpthread -lm -ldl"

mkdir -p tree
mkdir -p tree/mods

touch tree/null.f
echo "program main" > tree/main.f90

FILES_LIST=""

for i in ${L1}
do
  for j in ${L2}
  do
    echo "${i}${j}"
    for k in ${L3}
    do
      mkdir -p tree/${i}/${j}/${k}
      cat >tree/${i}/${j}/${k}/${i}${j}${k}_mod.f90 <<EOF
  subroutine print_${i}${j}${k}()
    print *, "call print_${i}${j}${k}()"
  end subroutine print_${i}${j}${k}
EOF
    ifx tree/${i}/${j}/${k}/${i}${j}${k}_mod.f90 ${OPT} -module tree/mods -c -o tree/${i}/${j}/${k}/${i}${j}${k}_mod.f90.o &
    FILES_LIST="$FILES_LIST $(pwd)/tree/${i}/${j}/${k}/${i}${j}${k}_mod.f90.o"
    cat >>tree/main.f90 <<EOF
    call print_${i}${j}${k}()
EOF
    done
  done
done
wait

echo "end program main" >> tree/main.f90

ifx tree/main.f90 ${OPT} -module tree/mods -c -o tree/main.f90.o
ifx tree/null.f -c -o tree/null.f.o
tree > tree.initial
echo ${FILES_LIST} > tree/files
echo ${LIBS} > tree/libs
echo "ifx ${OPT} @Tree/files $(pwd)/tree/main.f90.o $(pwd)/tree/null.f.o @Tree/libs -o tree/main"
ifx ${OPT} @Tree/files $(pwd)/tree/main.f90.o $(pwd)/tree/null.f.o @Tree/libs -o tree/main
tree > tree.after

diff tree.initial tree.after -upN --color

rm tree.initial tree.after
rm -rf tree

 

This is what I observe on my system:

 ./repro.sh
aa
ab
ac
ad
ae
ba
bb
bc
bd
be
ca
cb
cc
cd
ce
da
db
dc
dd
de
ea
eb
ec
ed
ee
tree/null.f: remark #5133: The input stream is empty
ifx -O2 -g @Tree/files /test/main.f90.o /test/tree/null.f.o @Tree/libs -o tree/main
--- tree.initial        2023-11-27 08:44:59.388925480 -0800
+++ tree.after  2023-11-27 08:44:59.655923016 -0800
@@ -406,11 +406,15 @@
 │   │       └── e
 │   │           ├── eee_mod.f90
 │   │           └── eee_mod.f90.o
+│   ├── files
+│   ├── libs
+│   ├── main
 │   ├── main.f90
 │   ├── main.f90.o
 │   ├── mods
 │   ├── null.f
 │   └── null.f.o
+├── tree.after
 └── tree.initial

-157 directories, 256 files
+157 directories, 260 files

 

0 Kudos
foxtran
New Contributor I
1,859 Views

@Igor_V_Intel,

Right. With your LIBS variable I see the same behaviour.

Please, try the following LIBS:
```
LIBS="-lpthread -lm -ldl -lpthread -lm -ldl -lpthread -lm -ldl"
```

For such case with your script, I see the problem:

 

 

ifx -O2 -g @tree/files /dev/shm/ger/tree/main.f90.o /dev/shm/ger/tree/null.f.o @tree/libs -o tree/main
--- tree.initial	2023-11-28 13:14:33.485538397 +0100
+++ tree.after	2023-11-28 13:14:33.646536858 +0100
@@ -422,16 +422,19 @@
 |   |       |   |-- eec_mod.f90
 |   |       |   `-- eec_mod.f90.o
 |   |       |-- d
-|   |       |   |-- eed_mod.f90
-|   |       |   `-- eed_mod.f90.o
+|   |       |   `-- eed_mod.f90
 |   |       `-- e
 |   |           |-- eee_mod.f90
 |   |           `-- eee_mod.f90.o
+|   |-- files
+|   |-- libs
+|   |-- main
 |   |-- main.f90
 |   |-- main.f90.o
 |   |-- mods
 |   |-- null.f
 |   `-- null.f.o
+|-- tree.after
 `-- tree.initial

-157 directories, 277 files
+157 directories, 280 files

 


I have repeatable link flags due to configuration with CMake. 

 

 

0 Kudos
foxtran
New Contributor I
1,787 Views
@Igor_V_Intel, was you able to reproduce an issue with repeatable libs in LIBS?
0 Kudos
foxtran
New Contributor I
1,693 Views

@Igor_V_Intel, ping

0 Kudos
Igor_V_Intel
Employee
1,685 Views

I have reproduced this problem with latest compiler build. Looks like a bug in the compiler driver (ifx). I escalated this to the development team to fix it in the upcoming versions.

Thank you for reporting this problem to us. A workaround may be to remove the duplicated libs entries (if possible).


0 Kudos
foxtran
New Contributor I
1,684 Views

Thank you for good news!

A workaround may be to remove the duplicated libs entries (if possible).

Unfortunately, CMake does not do that

0 Kudos
Igor_V_Intel
Employee
1,433 Views

The solution for this bug was found and implemented. Please expect the fix to be available in the upcoming updates.


0 Kudos
Igor_V_Intel
Employee
469 Views

This bug is fixed in latest compiler version. I have verified that ifx 2024.2.1 works as expected.


Reply