topic Re: Using static library: piculiar problem in Intel® Fortran Compiler

Using static library: piculiar problem

roddur — Sat, 24 Oct 2009 16:45:04 GMT

Dear friends,
In my code, I want to use two static libray(which generally do the same work, but from two different dir.) generated using makefile:

ar -rcs liblm_a.a lm-lib.o lm-lib-end.o $(lmobj) ...from LMTO_A
and
ar -rcs liblm_b.a lm-lib.o lm-lib-end.o $(lmobj) ...from LMTO_B

Now while "make"ing it, by
irun: $(FOBJ)
$(FC) $(FFLAGS) $(FPAR) -o $@ $(FOBJ) -L/home/rudra/LMTO-A -llm_a -L/home/rudra/LMTO-B/ -llm_b

i am facing a problem that the output comes like:

LM A called
Inside finits
/home/rudra/Recursion/ASR/CTRL_A
Hera I am
INITLG: redirect output to file: LM
INITLG: redirect error messages to file: ERR
lm-lib A Done
Starting LMTO_B
LM B called
Inside finits
/home/rudra/Recursion/ASR/CTRL_A
Hera I am
INITLG: redirect output to file: LM
INITLG: redirect error messages to file: ERR
OK
lm-lib B Done

like this.....which shows, the execution goes to LM_A and LM_B but as you can see, writes same directory:
/home/rudra/Recursion/ASR/CTRL_A
which should be actually /home/rudra/Recursion/ASR/CTRL_B in LM_B

what is peculiar is if I jus inverse the order of linking the library, like
irun: $(FOBJ)
$(FC) $(FFLAGS) $(FPAR) -o $@ $(FOBJ) -L/home/rudra/LMTO-B/ -llm_b -L/home/rudra/LMTO-A -llm_a

then the output comes:
LM A called
B-inits
/home/rudra/Recursion/ASR/CTRL_B
lm-lib A Done
Starting LMTO_B
LM B called
B-inits
/home/rudra/Recursion/ASR/CTRL_B
lm-lib B Done

I have no idea of what is going on.....can you plz suggest something? If necessary, my system is(testbench, tested on other also, with same result):
$ uname -a
Linux roddur 2.6.30.8-64.fc11.i686.PAE #1 SMP Fri Sep 25 04:56:58 EDT 2009 i686 i686 i386 GNU/Linux

and
$ ifort -v
Version 11.1

Re: Using static library: piculiar problem

Kevin_D_Intel — Sun, 25 Oct 2009 11:40:03 GMT

Looks like maybe you have a like named procedure present in both libraries and the one that is called depends on the link order of your two libraries.

I can't really guess from your output what the procedure name is, but it seems to be the one writing "Inside finits" (in the case lib liblm_a.a) and "B-inits" (in the case lib liblm_b.a).

If you expect each library to initialize differently when called then the cannot share the same initialization procedure name.

You could verify from which library the routine is linked from by generating a link map for the final executableusing the ifort -Wl option to pass the linker (ld) -Map option like this:

iforthello.f90 -Wl,-Map,hello.map

This command generates the link map named hello.map

Re: Using static library: piculiar problem

roddur — Mon, 26 Oct 2009 09:20:18 GMT

Quoting - Kevin Davis (Intel)

You could verify from which library the routine is linked from by generating a link map for the final executableusing the ifort -Wl option to pass the linker (ld) -Map option like this:

iforthello.f90 -Wl,-Map,hello.map

This command generates the link map named hello.map

This is a bit problematic as the libraries discussed here is of 55k
lines. in the main code i have called them as:
!the lmto library function !

!Function for atom A
call lm_a(oidxdn,opnu,opold,oqnu,oqold, &
clabl_1,lmx,w_oidn,w_opp,w_oqnu,w_oqold, &
d_ialpha,d_ifmt3d,d_itrans,d_jbasdn,d_ldn,d_lmaxw, &
d_ltmax,d_mdn,d_mmixat,d_mmixpq,d_nbas,d_nclass,d_ndimin, &
d_ngen,d_nit,d_nitat,d_niter,d_nkdmx,d_nkxyz,d_nl,d_nopts, &
d_norder,d_npts,d_nrxc,d_nrxyz,d_nsp, &
d_as,d_beta,d_dele,d_deltr,d_efermi,d_kap2,d_kfit, &
d_emax,d_emaxc,d_emin,d_eminc,d_eps,d_facvol,d_gamma,d_ommax1,
&
d_ommax2,d_range,d_rmaxes,d_rmaxs,d_rmaxs2,d_rmines,d_rms2, &
d_rmsdel,d_tolef,d_toleto,d_tolews,d_vmtz,d_wc,d_width,sw13)

write(*,*) "Starting LMTO_B"
!Function for atom B
call lm_b(oidxdn,opnu,opold,oqnu,oqold, &
clabl_1,lmx,w_oidn,w_opp,w_oqnu,w_oqold, &
d_ialpha,d_ifmt3d,d_itrans,d_jbasdn,d_ldn,d_lmaxw, &
d_ltmax,d_mdn,d_mmixat,d_mmixpq,d_nbas,d_nclass,d_ndimin, &
d_ngen,d_nit,d_nitat,d_niter,d_nkdmx,d_nkxyz,d_nl,d_nopts, &
d_norder,d_npts,d_nrxc,d_nrxyz,d_nsp, &
d_as,d_beta,d_dele,d_deltr,d_efermi,d_kap2,d_kfit, &
d_emax,d_emaxc,d_emin,d_eminc,d_eps,d_facvol,d_gamma,d_ommax1,
&
d_ommax2,d_range,d_rmaxes,d_rmaxs,d_rmaxs2,d_rmines,d_rms2, &
d_rmsdel,d_tolef,d_toleto,d_tolews,d_vmtz,d_wc,d_width,sw13)

!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~!
and filenames are hard coded as:

write(*,*)"Inside finits"
lmdir="/home/rudra/Recursion/ASR/"
atspec="A"
cband=lmdir//'BAND_'//atspec; cbnds=lmdir//'BNDS_'//atspec;
cbak =lmdir//'CBAK_'//atspec; cctrl=lmdir//'CTRL_'//atspec;
cdos =lmdir//'DOS_'//atspec; ceign=lmdir//'EIGN_'//atspec;
celf =lmdir//'ELF_'//atspec; celfc=lmdir//'ELFC_'//atspec;
celfv=lmdir//'ELFV_'//atspec; cinit=lmdir//'INIT_'//atspec;
clmdm=lmdir//'LMDM_'//atspec; clmfs=lmdir//'LMFS_'//atspec;
cmixm=lmdir//'MIXM_'//atspec; cmoms=lmdir//'MOMS_'//atspec;
cnalp=lmdir//'NALP_'//atspec; cnilp=lmdir//'NILP_'//atspec;
cnspt=lmdir//'NSPT_'//atspec; cpot =lmdir//'POT_'//atspec;
cpoti=lmdir//'POTI_'//atspec; crho =lmdir//'RHO_'//atspec;
crhof=lmdir//'RHOF_'//atspec; crhoc=lmdir//'RHOC_'//atspec;
crhos=lmdir//'RHOS_'//atspec; crhov=lmdir//'RHOV_'//atspec;
cstr =lmdir//'STR_'//atspec; cstr0=lmdir//'STR0_'//atspec;
cstr1=lmdir//'STR1_'//atspec; ctmp =lmdir//'TMP_'//atspec;
write(*,*) cctrl
C --- Set up the file logical units ---
ifi=fadd(cband,4)
ifi=fadd(cbnds,0)
ifi=fadd(cbak, 0)
ifi=fadd(cctrl,0)
ifi=fadd(cdos, 4)
ifi=fadd(ceign,4)
ifi=fadd(celf, 0)
ifi=fadd(celfc,0)
ifi=fadd(celfv,0)
ifi=fadd(cinit,0)
ifi=fadd(clmdm,4)
ifi=fadd(clmfs,0)
ifi=fadd(cmixm,4)
ifi=fadd(cmoms,4)
which gives the name of the files.
in ifort, i have tried to compile with
irun: $(FOBJ)
$(FC) $(FFLAGS) $(FPAR) -o $@ $(FOBJ) -L/home/rudra/LMTO-A -L/home/rudra/LMTO-B -llm_b -llm_a -Wl, -Map, hello.map
and hello.map shows result like:
$ grep liblm_b hello.map |wc -l
1691
and
$ grep liblm_a hello.map |wc -l
9

which should be same.
how to get pass this?

Re: Using static library: piculiar problem

Kevin_D_Intel — Mon, 26 Oct 2009 10:16:46 GMT

The snippets show calling different named routines so the link map idea isn't useful in hunting down your initial concern. If you believe the number of instances of the library name in the link map should be equal (when they clearly are not) then you should look at the details of link map and determine why more routines are resolved from liblm_b vs. liblm_a. All I can guess is that you have like named routines residing in these libraries and the linker resolves external reference from the first instance of the name found when searching libraries to resolve external references. Given that liblm_b is specified first, like named routines between these libraries will be resolved from liblm_b.

As for the source snippet shown, do you perhaps assign atspec in both procedures lm_a and lm_b as:

atspec="A"

Curious also, your comment refers to them as "Function for", but invocation is via CALL. I assume both are declared as SUBROUTINE and not FUNCTION.

Re: Using static library: piculiar problem

roddur — Mon, 26 Oct 2009 11:12:54 GMT

[cpp]As for the source snippet shown, do you perhaps assign atspec in both procedures lm_a and lm_b as:

atspec="A"
[/cpp]

yes...i have done that, as:

[cpp] write(*,*)"Inside finits" 
        lmdir="/home/rudra/Recursion/ASR/"
        atspec="A"  
        cband=lmdir//'BAND_'//atspec; cbnds=lmdir//'BNDS_'//atspec;
[/cpp]

is it wrong? how to fix that?

Curious also, your comment refers to them as "Function for", but invocation is via CALL. I assume both are declared as SUBROUTINE and not FUNCTION.

its my mistake; they are subroutines

Re: Using static library: piculiar problem

Kevin_D_Intel — Mon, 26 Oct 2009 12:08:27 GMT

There's nothing incorrect with the assignment and you can ignore my question. It doesn't make sense what I asked because if both libraries assigned "A" to atspec then the program would never print "CTRL_B" for the path as you showed earlier.

I'm guessing in liblm_b, you assign atspec to "B", correct?

Let's go back to the same named routine theory, because in re-reading your post I think what you showed is the entry call into each library (which is lm_a and lm_b) but that's probably not theinitialization routine that shares the same name.

It appears the program only prints "Inside finits" from the initialization routine in liblm_a, is that correct?
It appears this same initialization routine from liblm_b prints "B-init", is that correct?

So what's the name of the routine from each library that prints these unique statements?

Re: Using static library: piculiar problem

roddur — Mon, 26 Oct 2009 12:42:16 GMT

I'm guessing in liblm_b, you assign atspec to "B", correct?

yes

So what's the name of the routine from each library that prints these unique statements?
it is IOLIB/finits.f
for liblm_a, it is:

[cpp]
     subroutine finits
C- Machine and compiler-dependent initialization
C-----------------------------------------------------------------------
      implicit none
C Local variables:
      integer fadd,ifi,ifree
      character*55 cmd
      character(26)::lmdir
      character(1)::atspec
        
        character(32)::cband,cbnds,cbak,cctrl,cdos,ceign,celf,celfc, 
     .  celfv,cinit,clmdm,clmfs,cmixm,cmoms,cnalp,cnilp,cnspt,cpot,
     .  cpoti,crho,crhof,crhoc,crhos,crhov,cstr,cstr0,cstr1,ctmp

        write(*,*)"Inside finits" 
        lmdir="/home/rudra/Recursion/ASR/"
        atspec="A"  
        cband=lmdir//'BAND_'//atspec; cbnds=lmdir//'BNDS_'//atspec;
        cbak =lmdir//'CBAK_'//atspec; cctrl=lmdir//'CTRL_'//atspec;
        cdos =lmdir//'DOS_'//atspec;  ceign=lmdir//'EIGN_'//atspec;
        celf =lmdir//'ELF_'//atspec;  celfc=lmdir//'ELFC_'//atspec;
        celfv=lmdir//'ELFV_'//atspec; cinit=lmdir//'INIT_'//atspec;
        clmdm=lmdir//'LMDM_'//atspec; clmfs=lmdir//'LMFS_'//atspec;
        cmixm=lmdir//'MIXM_'//atspec; cmoms=lmdir//'MOMS_'//atspec;
        cnalp=lmdir//'NALP_'//atspec; cnilp=lmdir//'NILP_'//atspec;
        cnspt=lmdir//'NSPT_'//atspec; cpot =lmdir//'POT_'//atspec;
        cpoti=lmdir//'POTI_'//atspec; crho =lmdir//'RHO_'//atspec;
        crhof=lmdir//'RHOF_'//atspec; crhoc=lmdir//'RHOC_'//atspec;
        crhos=lmdir//'RHOS_'//atspec; crhov=lmdir//'RHOV_'//atspec;
        cstr =lmdir//'STR_'//atspec; cstr0=lmdir//'STR0_'//atspec;
        cstr1=lmdir//'STR1_'//atspec; ctmp =lmdir//'TMP_'//atspec;
        write(*,*) cctrl
C --- Set up the file logical units ---
      ifi=fadd(cband,4)
      ifi=fadd(cbnds,0)
      ifi=fadd(cbak, 0)
      ifi=fadd(cctrl,0)
      ifi=fadd(cdos, 4)
      ifi=fadd(ceign,4)
      ifi=fadd(celf, 0)
      ifi=fadd(celfc,0)
      ifi=fadd(celfv,0)
      ifi=fadd(cinit,0)
      ifi=fadd(clmdm,4)
      ifi=fadd(clmfs,0)
      ifi=fadd(cmixm,4)
      ifi=fadd(cmoms,4)
      ifi=fadd(cnalp,0)
      ifi=fadd(cnilp,0)
      ifi=fadd(cnspt,0)
      ifi=fadd(cpot, 0)
      ifi=fadd(cpoti,0)
      ifi=fadd(crho, 0)
      ifi=fadd(crhof,0)
      ifi=fadd(crhoc,0)
      ifi=fadd(crhos,0)
      ifi=fadd(crhov,0)
      ifi=fadd(cstr, 4)
      ifi=fadd(cstr0,4)
      ifi=fadd(cstr1,4)
      ifi=fadd(ctmp, 4)

        write(*,*) "Hera I am"
300   format(' FINITS: free disc-space in KB: ',i10)
      end[/cpp]

and for liblm_b, it is

[cpp] subroutine finits
C- Machine and compiler-dependent initialization
C-----------------------------------------------------------------------
      implicit none
C Local variables:
      integer fadd,ifi,ifree
      character*55 cmd
      character(1)::atspec
      character(26)::lmdir
        
      character(32)::cband,cbnds,cbak,cctrl,cdos,ceign,celf,celfc, 
     .  celfv,cinit,clmdm,clmfs,cmixm,cmoms,cnalp,cnilp,cnspt,cpot,
     .  cpoti,crho,crhof,crhoc,crhos,crhov,cstr,cstr0,cstr1,ctmp
      external fadd,lsequ

        write(*,*)"B-inits"
        lmdir="/home/rudra/Recursion/ASR/"
        atspec="B"  
        cband=lmdir//'BAND_'//atspec; cbnds=lmdir//'BNDS_'//atspec;
        cbak =lmdir//'CBAK_'//atspec; cctrl=lmdir//'CTRL_'//atspec;
        cdos =lmdir//'DOS_'//atspec;  ceign=lmdir//'EIGN_'//atspec;
        celf =lmdir//'ELF_'//atspec;  celfc=lmdir//'ELFC_'//atspec;
        celfv=lmdir//'ELFV_'//atspec; cinit=lmdir//'INIT_'//atspec;
        clmdm=lmdir//'LMDM_'//atspec; clmfs=lmdir//'LMFS_'//atspec;
        cmixm=lmdir//'MIXM_'//atspec; cmoms=lmdir//'MOMS_'//atspec;
        cnalp=lmdir//'NALP_'//atspec; cnilp=lmdir//'NILP_'//atspec;
        cnspt=lmdir//'NSPT_'//atspec; cpot =lmdir//'POT_'//atspec;
        cpoti=lmdir//'POTI_'//atspec; crho =lmdir//'RHO_'//atspec;
        crhof=lmdir//'RHOF_'//atspec; crhoc=lmdir//'RHOC_'//atspec;
        crhos=lmdir//'RHOS_'//atspec; crhov=lmdir//'RHOV_'//atspec;
        cstr =lmdir//'STR_'//atspec; cstr0=lmdir//'STR0_'//atspec;
        cstr1=lmdir//'STR1_'//atspec; ctmp =lmdir//'TMP_'//atspec;
         
C --- Set up the file logical units ---
      ifi=fadd(cband,4)
      ifi=fadd(cbnds,0)
      ifi=fadd(cbak, 0)
      ifi=fadd(cctrl,0)
      ifi=fadd(cdos, 4)
      ifi=fadd(ceign,4)
      ifi=fadd(celf, 0)
      ifi=fadd(celfc,0)
      ifi=fadd(celfv,0)
      ifi=fadd(cinit,0)
      ifi=fadd(clmdm,4)
      ifi=fadd(clmfs,0)
      ifi=fadd(cmixm,4)
      ifi=fadd(cmoms,4)
      ifi=fadd(cnalp,0)
      ifi=fadd(cnilp,0)
      ifi=fadd(cnspt,0)
      ifi=fadd(cpot, 0)
      ifi=fadd(cpoti,0)
      ifi=fadd(crho, 0)
      ifi=fadd(crhof,0)
      ifi=fadd(crhoc,0)
      ifi=fadd(crhos,0)
      ifi=fadd(crhov,0)
      ifi=fadd(cstr, 4)
      ifi=fadd(cstr0,4)
      ifi=fadd(cstr1,4)
      ifi=fadd(ctmp, 4)

        write(*,*)cctrl
300   format(' FINITS: free disc-space in KB: ',i10)
      end[/cpp]

Re: Using static library: piculiar problem

roddur — Mon, 26 Oct 2009 12:48:08 GMT

And as you suggest, I think adding the hello.map will be usefull....this is attached.

Re: Using static library: piculiar problem

Kevin_D_Intel — Mon, 26 Oct 2009 13:31:21 GMT

The link map isn't really needed. Your previous posts shows why your program behaved in the manner it did. You have the same named routine, finits, defined in each library. The linker will only link in one instance of this routine, not both,and the one that's linked in will be the first one seen based on the link order of your libraries. That's why you saw the output you did when you rearranged the libraries for the link step.

If you have other routines in these libraries that share the same names, and I suspect you do based on grep counts shown earlier, then the instance of the routine used will be the one that is seen first during the link step.

The only way to accomplishwhat I think you are seekingis to rename the same named routines to be unique to the library, perhaps adding a suffix of "A" or "B", which is likely a lot of work.

Re: Using static library: piculiar problem

roddur — Mon, 26 Oct 2009 13:43:44 GMT

Ok,
thanks!!
But I am confused!! As you can see from the make file, these routines are in different folder(LMTO-A and LMTO-B), but yes...under these folder everything is same.......it do not take into consideration that main folder is different? why this is so?

Re: Using static library: piculiar problem

Kevin_D_Intel — Mon, 26 Oct 2009 15:55:05 GMT

Quoting - roddur

The folder location is not preserved by the compiler. It also doesn't matter that the same named routines are placed into physically different static libs. The source is compiled into an object form (.o). The object contains only the routine name(s) as declared within the source (and following OS specific upper/lower case convention).

This is just the way things are designed. It would be a nightmare to maintain source code if folder names were somehow folded into the procedure's name in the object.

Think about your case. You would not be able to simply have:

CALL lm_a()

You'd have to modify your source to be:

CALL /home/rudra/LMTO-A/lm_a( )

Or something similar, right?

And what happens when you move the source containing procedure lm_a? You have to modify your source that calls the routine.

I'm not sure of your needs, but if you have procedures that are in common between the two paths through your program, then consider creating a third library, call it liblm_c.a, that contains only these common routines. This would then only require you change the source to any common procedure in one place to pick up a change applicable to either execution path.

Splitting out the common routines also makes liblm_a and liblm_b truly unique to the "a" and "b" paths in your program and probably makes them much smaller since each only contains routines unique to each path. You could even go a bit further perhaps if the uniqueness of these routines to the "a" and "b" paths includes procedure names, in which case all these unique procedures could actually live in one library as well since there would be no name clashes.

Anyway, hope this helps.

Re: Using static library: piculiar problem

roddur — Mon, 26 Oct 2009 21:45:46 GMT

Yes, it really helped!! thanks a lot!!
first i was in the impression that changing the file name inside the folder will serve it.....but this is not!!! I have to change the toutines, just like finits_a() and finits_b(), right?

Or, for one suggestion! what this two library do is take to elements(A and B), it creates different folder, do some calculation (and writes the output in the file created by finits).
So, if I call the same library twice with atspec as a variable argument, will there be any such clash? I think no. But just want to ensure before I move,,,,,,

Re: Using static library: piculiar problem

Kevin_D_Intel — Tue, 27 Oct 2009 08:10:04 GMT

Glad to hear that helped.

Yes, you are correct about needing to rename the routines finits_a/finits_b or follow your second ideaand only include one instance of finits in one of the libraries and use a variable argument for atspec. I think for overall clarity it would be best to have a single library containing the shared common routines only, but that's not required to get you past your immediate need.

Good luck.