Does "-assume protect_constants" switch fail its task ?

[e745200]

Managing complex lecagy applications, it is very important that some extent of backward compatibility is provided by the new compilers, so that a current-standard-violating application can be compiled and run in the new environment, while its source is cleaned up and made compliant for "the next release" and a safer future life.

One of the cases I came across is calling a procedure using a constant value in place of an argument which is INTENT [IN]OUT in the semantics of the called procedure.

This violation of the current standard used to work in the past because a temporary copy of the constant was passed to the procedure, and any change done to it in the called procedure was then discarded.

This behavior is still available in several compilers, and in some of them it is still the default or even the unique one.

Other compilers have different default behaviors, possibly changeable using some compiler option.

In some cases the passed constant really "changes" its value, and its new (undesired) value, leads to difficult to find errors. (One can hardly understand that 1 is no longer 1 ....)

In other cases the constants are stored in "read-only" memory locations, and the case produces an "Access violation" or "Segmentation fault". This happens with the Intel Fortran compiler ( see here a post by Steve Lionel ), and (IMHO) is a rude way to tell you that your code is out of date.

In the Intel Fortran compiler, the "permissive" behavior should be safely provided by the "-assume protect_constants" switch, the task of which should be to tell

the compiler to pass constants in a stack temporary so that the called procedure can safely store to it, with the changes being discarded on return. ( see here )

Unfortunately, it seems not to work, at least in Linux version 10 and 11, and the "Segmentation fault" occurs even when compiling using that switch.

This means that old code might require a lot of fixes to work fine ...

Has anyone a different experience ? Did I miss something or make something wrong (apart using non-compliant code) ?

This is my simple test:

$ cat pca.f

PRINT *,'MAIN - start up 1=',1
CALL SUB1(1)
PRINT *,'MAIN - After call 1=',1
END

SUBROUTINE SUB1(A)
INTEGER A
PRINT *,'Entered SUB1 - A=', A
A=A*10
PRINT *,'Leaving SUB1 - A=', A
RETURN

END

$ ifort -o pca pca.f && pca
MAIN - start up 1= 1
Entered SUB1 - A= 1
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pca 08049DB4 Unknown Unknown Unknown
pca 08049CC1 Unknown Unknown Unknown
libc.so.6 0050FDEC Unknown Unknown Unknown
pca 08049BB1 Unknown Unknown Unknown

$ ifort -assume protect_constants -o pca pca.f && pca
MAIN - start up 1= 1
Entered SUB1 - A= 1
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
pca 08049DB4 Unknown Unknown Unknown
pca 08049CC1 Unknown Unknown Unknown
libc.so.6 00126DEC Unknown Unknown Unknown
pca 08049BB1 Unknown Unknown Unknown

[Steven_L_Intel1]

The documentation for this switch is confusing. What you want is "-assume noprotect_constants". Protect in this context means "prevent modification of the argument", in other words, let the segv happen. noprotect_constants allows the modification to happen to a temp.

I'll work with the writers to see if we can make this clearer.

[Kevin_D_Intel]

The options are confusing. The option needed is: -assume noprotect_constants

$ ifort -V -assume noprotect_constants -o pca pca.f && pca

Intel Fortran Intel 64 Compiler Professional for applications running on Intel 64, Version 11.0 Build 20081105 Package ID: l_cprof_p_11.0.074

Copyright (C) 1985-2008 Intel Corporation. All rights reserved.

Intel Fortran 11.0-1558

GNU ld version 2.16.91.0.5 20051219 (SUSE Linux)

MAIN - start up 1= 1

Entered SUB1 - A= 1

Leaving SUB1 - A= 10

MAIN - After call 1= 1

[e745200]

Quoting - Steve Lionel (Intel)

The documentation for this switch is confusing. What you want is "-assume noprotect_constants". Protect in this context means "prevent modification of the argument", in other words, let the segv happen. noprotect_constants allows the modification to happen to a temp.

Quoting - Kevin Davis (Intel)

The options are confusing. The option needed is: -assume noprotect_constants

$ ifort -V -assume noprotect_constants -o pca pca.f && pca

Thanks for your attention and answers.

If I had read thoroughly the documentation of -assume, actually I would have found the right answer without bothering forum & support. The documentation is confusing as you say, nevertheless the information is somehow clear, but not retrievable at a glance.

I would try to explain why I was not able to understand what I needed, not to justify my blindness, but just in case it could help the technical writers to release a more helpful product.

The first reason is that when compulsing a reference manual, one would like to find all the needed specific information in just a few lines concentrated in a single spot, otherwise one can stop reading too early and miss the relevant piece of information, as it happened to me.

The explanation of the compilation flags (let us focus on the -assume [no]protect_constants switch, but the pattern is the same for all of them) is split in three places, instead, and not so close to each other :

1) a general explanation :
[no]protect_constants Determines whether a constant actual argument or a copy of it is passed to a called routine.

The purpose of which seems just to loosely define the scope of the switch. In fact, in this, as in many other cases where the doc gives a compound two-fold "explanation", one can hardly guess which one of the two altertanives determines the desired action. We can say that data are there but the chance is wasted of bringing immediately usable information.

2) the default specification: (about 60 lines later)
protect_constants A constant actual argument is passed to a called routine. Any attempt to modify it results in an error.

3) a further description : (about other 60 lines later)
assume noprotect_constants
Tells the compiler to pass a copy of a constant actual argument. This copy can be modified by the called routine, even though the Fortran standard prohibits such modification. The calling routine does not see any modification to the constant.

I can see that the information is clear, but the way it is delivered requires a reading of all the page to retrieve the dispersed pieces of information.

Another reason is that my eyes and mind were biased by the behavior of other compilers I use, in which the "protection" has a subtly different meaning. For instance, the Lahey Compiler never prevents the modification of the constant argument in the called routine; then, this might result in a dirty and dangerous alteration of the constant which propagates in the rest of the program (a "1" read in the code can actually become any other value), unless you "protect" it using the --pca (protect constant argument) compilation flag, in this case the constant argument, as seen in the caller routine, will retain its original value.

The two standpoints seem similar but are completely different.

Both are legitimate, of course, but I do not like any of them, for the following reasons, simply based on common sense:

1) If I use a constant when calling a routine, it is absolutely certain that I do not want it is changed, and that I am not interested, in that specific call, in any possible different value returned in that argument. Then it would ALWAYS be correct that a temporary variable, which is given on input the value of the constant, is passed to the routine. For example, this is the default and (afaik) unique possible behavior in XLF compiler. Using this approach no error might ever occurr, regardless of any INTENT declared and/or implemented for the argument in the called routine.

2) The fact that to obtain the above mentioned behavior I have to use specific flags (when, as said, the intention of the programmer is absolutely clear) means for sure that not using them I get some error, which can also difficult to find. I am not a compilers theorist or writer, but I cannot see any advantage in having this "flexibility".

3) The fact that the flags to obtain the same behavior are named in opposite ways in the Lahey and Intel compilers
[ --pca (protect constant argument) lahey , -assume NOprotect_constants intel ], is then the icing on the cake ...

I am sorry for the excess of verbosity, and thank you again.

PS

@ Steve Lionel,

I do not know if it is still possible, but, after your clarification, I think you should edit your own post "Don't touch me there" and replace /assume:protect_constants with /assume:noprotect_constants in the closing note.

PPS.

However, to become compiler-option-independent as far as possible (who knows what future compilers will need...), I started a campaign of removal of as many constants as possible from any call statement in my code (which is really a crazy task !)

[Steven_L_Intel1]

I edited the post - thanks. See, even I get confused!

[e745200]

Quoting - Steve Lionel (Intel)

I edited the post - thanks. See, even I get confused!

The real problem is that that flags, for the reasons I said, seems really useless !

Or perhaps, having a weak rationale behind, it turns out to be non-intuitive or even counter-intuitive, and this can generate misunderstanding and confusion.

I would like to go more in depth with the rationale behind this flag.

Using a constant as an actual argument for an INTENT OUT or INOUT dummy argument is said to be a violation of the current standard. Which is the paragraph where this issue is specifically dealt with in the standard ? Has the default behavior of the Intel compiler been designed to guard this issue or for other reasons ?

[Steven_L_Intel1]

Of course, a program that modified a dummy argument associated with a constant or expression, or that is declared INTENT(IN) violates the language rules. In the case of INTENT(IN), assuming the actual routine had that attribute visible the compiler will disallow any assignments at compile time. And the compiler will complain if you pass a constant to an INTENT(OUT). But the language rules do not permit an error for INTENT(INOUT), as this just says that the argument MAY be modified.

But in the real world, people DO pass constants and expressions to routines that modify the argument. For efficiency, most compilers do "constant pooling" so that there is one memory copy of a constant (say, 3), per routine. If you pass 3 as an actual argument, the compiler passes the address of this shared copy. Should the called routine do something like ARG=ARG+1, then 3 will become 4 when used subsequently.

The most efficient way to deal with this is to put the constant pool in a read-only image section - typically the same one that compiled code uses. There's no overhead for this, but if the called routine tries to modify the value an error will be raised by the OS.

As noted in my article you cited, the first version of Digital Visual Fortran did not do this, so you could modify the value of 3. When we added the read-only protection in version 6, many users complained that their programs stopped working, as they had been inadvertently modifying constant arguments. So we added the switch to do what the compiler already did if you passed an expression - it made a stack copy of the constant and passed the address of the copy. That way any (non-standard) writes to the value would be discarded on exit. This costs some extra instructions, so is not the default.

When we became Intel Fortran in 2003, it turned out that the Intel code generator didn't know how to do constants in read-only memory, so we had to add that back in.

In the real world, it is not always a solution to tell a customer that their code is broken. They'll say that it works with compilerx X, Y and Z and what's wrong with Intel? This is why we sometimes offer a switch, such as -assume noprotect_constants, to allow their bad code to continue working.

The only behavior change in the Intel compiler was to support read-only constants, and that was several versions ago.

[e745200]

Steve, thanks for discussing this issue with me ...

Quoting - Steve Lionel (Intel)

Of course, a program that modified a dummy argument associated with a constant or expression, or that is declared INTENT(IN) violates the language rules. In the case of INTENT(IN), assuming the actual routine had that attribute visible the compiler will disallow any assignments at compile time. And the compiler will complain if you pass a constant to an INTENT(OUT). But the language rules do not permit an error for INTENT(INOUT), as this just says that the argument MAY be modified.

It is clear that, when writing new code, the good programmer should adopt the syntax provided by the more recent standards, in order to get a more robust and reliable product. But when dealing with legacy code in FORTRAN 77, the INTENT can only be derived by the semantic of the routines code, a job I do not think compilers do or could do, being the calling and the caller procedure compiled independently.

Quoting - Steve Lionel (Intel)

But in the real world, people DO pass constants and expressions to routines that modify the argument.

This is only because the old compilers had by default the behavior now determined (in ifort) by -assume noprotect_constants, i.e. using a temporary memory location for them. Otherwise those programs would have never worked.

Quoting - Steve Lionel (Intel)

For efficiency, most compilers do "constant pooling" so that there is one memory copy of a constant (say, 3), per routine. If you pass 3 as an actual argument, the compiler passes the address of this shared copy. Should the called routine do something like ARG=ARG+1, then 3 will become 4 when used subsequently.

I came across this behavior the first time I ported my code to the Lahey Compiler, as I said. This kind of problem and the mechanism of the modification of the constants became then clear to me. And at that time I learned that the concept of "protection" of the constant argument against the modification done by the called procedure was implemented using a copy of it and, ALAS !, only using a specific switch ( --pca : protect constant arguments ).

Quoting - Steve Lionel (Intel)

The most efficient way to deal with this is to put the constant pool in a read-only image section - typically the same one that compiled code uses. There's no overhead for this, but if the called routine tries to modify the value an error will be raised by the OS.

As noted in my article you cited, the first version of Digital Visual Fortran did not do this, so you could modify the value of 3. When we added the read-only protection in version 6, many users complained that their programs stopped working, as they had been inadvertently modifying constant arguments. So we added the switch to do what the compiler already did if you passed an expression - it made a stack copy of the constant and passed the address of the copy. That way any (non-standard) writes to the value would be discarded on exit. This costs some extra instructions, so is not the default.

...

In the real world, it is not always a solution to tell a customer that their code is broken. They'll say that it works with compilerx X, Y and Z and what's wrong with Intel? This is why we sometimes offer a switch, such as -assume noprotect_constants, to allow their bad code to continue working.

This is what I would disagree upon, being on the user's side of the barricade.

1. Passing a constant argument to a subroutine which MAY modify that argument is not "inadvertently modifying constant arguments": it is just relying on a certain behavior of the compiler, i.e. working on a copy of the argument and discard it when returning. It was done consciously at that time and using the contemporary compilers.
2. To me, a constant is just the most simple possible form of expression, and there is no reasonable motivation for a compiler not to deal with it as with more complex expressions, when used as an argument. I can understand "constant pooling", but why not limit it to the constants not used as arguments ?
3. The "non-standard" writes would never occur if spending those extra instructions just when they are anyway needed to allow the program keep on working.
4. Bad programs ? if you mean "out of date", ok, but they were not bad when were written. And legacy programs ar a big reality.
5. Then, I really cannot understand why this was not assumed as the default, so that users would not have their program broken, nor spend time searching for errors caused by different behaviors of the compilers. Does that gain of efficency (???) worth the users' troubles ?
6. Finally, the name of the switch was perhaps not the best choice, referring to "constants" in general and not to the "constant arguments" - but in this, of course, I am biased by the previous experience with the Lahey compiler ... , and is also a very secondary issue (which however mislead you too ....)

I repeat: this is just a user's point of view. Perhaps the compilers developers have different main targets.

I am just saying that I would have preferred -assume noprotect_constants being the default (or even the unique possible) behavior of Intel Compiler, as I would have preferred --pca being the default (or even the unique possible) behavior of Lahey Compiler : because I really cannot see any use in the opposite choice !
In case the opposite choice could allow a superior efficency of the compilation process, it might have been a "compiling-optimization flag, potentially changing the semantic of the program".

Thanks for your attention.

[Steven_L_Intel1]

I know of many compilers over the years which allowed constants to be "modified" and the changes NOT discarded. As I mentioned, the default behavior works for correct programs and is higher performance.

I do frequently encounter user programs that "happened to work" on implementation X and the programmer therefore assumed it would work everywhere. That is not in itself a reason to slow down all applications because some users have written incorrect programs.

To my mind, it is a service to bring the error to the programmer;s attention, as we do, and then offer an alternative should they choose not to correct their program.

[Hirchert__Kurt_W]

Quoting - Steve Lionel (Intel)

But the language rules do not permit an error for INTENT(INOUT), as this just says that the argument MAY be modified.

While it is true that there is no guarantee that an INTENT(INOUT) argument will be modified, the language rules do require that the corresponding argument be definable, so an error message for an actual argument that is a constant or an expression is definitely permitted. Indeed, I suspect it is required when conformance reports are requested.

The case where a compile-time error is not possible (or, at least, possible only when compile-time analysis can be used to predict a run-time error) is for the case of unspecified intent (i.e., when no INTENT attribute is specified). Many people wrongly believe that unspecified intent is the same as INTENT(INOUT). They are similar, but the restrictions that apply to them are subltly different. An unspecified intent argument is required to be definable only in this cases where the argument is actually modified (i.e., a run-time restriction), whereas INTENT(INOUT) is required to be definable whether or not the argument is modified in this particular execution of the procedure.

[For those who do not speak standardese, "definable" is the standard's word for a variable which can be modified.]

-Kurt

[Hirchert__Kurt_W]

Quoting - e745200

This is what I would disagree upon, being on the user's side of the barricade.

1. Passing a constant argument to a subroutine which MAY modify that argument is not "inadvertently modifying constant arguments": it is just relying on a certain behavior of the compiler, i.e. working on a copy of the argument and discard it when returning. It was done consciously at that time and using the contemporary compilers.

Passing a constant (or indeed, any expression) to a dummy argument that is modified has been against the Fortran language rules for more than fifty years, and for all of those years, there have been compilers that would give wrong answers if you broke that rule. You may not have been aware of it, but this was an unwise programming practice when it was originally written, just as it is now. Perhaps "ignorantly" is more accurate than "inadvertently", but people in user support like Steve try to be polite and pretend you forgot the rules rather than that you never bothered to learn them.

Quoting - e745200

1. To me, a constant is just the most simple possible form of expression, and there is no reasonable motivation for a compiler not to deal with it as with more complex expressions, when used as an argument. I can understand "constant pooling", but why not limit it to the constants not used as arguments ?

If your compiler does "common subexpression elimination" (as most good optimizing compilers do), modifying a dummy argument associated with an arbitrary expression can be just as bad as modifying one associated with a constant.

Quoting - e745200

1. The "non-standard" writes would never occur if spending those extra instructions just when they are anyway needed to allow the program keep on working.

I haven't slightest idea what this sentence is trying to say.

Quoting - e745200

1. Bad programs ? if you mean "out of date", ok, but they were not bad when were written. And legacy programs ar a big reality.

Not "out of date", just "bad". Such programs were never correct; they may just accidentally have worked on some compilers, Whenever this program was written, there were contemporary compilers that would have given bad results or run-time errors from mistakes like this, so this is a "legacy program" issue only in the sense that in the days when you might use one machine for a decade, people were more likely to depend on the idiosyncracies of the compiler on their current machine rather than learning to write programs that might work on the full range of machines available.

Quoting - e745200

1. Then, I really cannot understand why this was not assumed as the default, so that users would not have their program broken, nor spend time searching for errors caused by different behaviors of the compilers. Does that gain of efficency (???) worth the users' troubles ?

Historically, people who chose to program in Fortran did so because it produced code (especially for numerical computations) that ran faster than comparable codes in other languages. Such programmers tended to be unsympathetic to the idea of slowing down correct programs to protect against incorrect programs. The cost of copying scalar constants is usually, but not always, negligible. If I were a developer, I would rather occasionally explain why a person's incorrect program produced wrong answers than to explain why a correct program ran slowly. [If many people depended on the incorrect behavior your program depends on, the tradeoff might be different, but the truth is that your program is relatively unusual in that regard.]

Quoting - e745200

1. Finally, the name of the switch was perhaps not the best choice, referring to "constants" in general and not to the "constant arguments" - but in this, of course, I am biased by the previous experience with the Lahey compiler ... , and is also a very secondary issue (which however mislead you too ....)

This is a bit of a no-win situation because the same words can be applied to both sides of the option. In both cases the constants are "protected". In one case, they are "protected" by compiler-generated code that copies them so the original constant is not modified. In the other, the memory system "protects" the constants (which have been placed in read-only memory), so the compiler doesn't need to generate code to copy them. The difference in perspective comes from whether you expect to directly tell the compiler what to do or to tell the compiler what conditions are and allow that to determine what code to generate.

Finally, allow me to make a suggestion that you may not appreciate hearing. If you expect to continue running this program in the future on other machines, using other compilers, or even just using newer versions of this compiler, there is a good chance that you would be better off in the long run fixing your source code to correctly say what you mean rather than searching for compiler options to interpret what you've said differently. If you need to copy a constant or expression value to someplace modifiable, do so explicitly with an assignment statement rather than depending on the compiler to do it behind your back. That's probably more work now, but the work you save in the future will almost certainly outweigh it.

[I apologize for your points being misnumbered above. The editor did that automatically when I broke up your points to respond to them individually.]

-Kurt

For the record, I am not an employee of Intel or any other compiler developer. For more than thirty years, my job included helping people with their Fortran problems, including the problems of porting programs to a new machine or new compiler, so I tend to be sensitive to the problems of undetected errors on one compiler manifesting themselves more seriously on another. Pretending your errors aren't really errors and looking for compiler support to "fix" the problem can be a quick solution to the immediate problem of getting the program working on a new platform, but it also tends to guarantee you will go through the same problems again in some future port.

[e745200]

Thanks to all for your precious observations (and reproaches !?!).

Quoting - hirchert

Passing a constant (or indeed, any expression) to a dummy argument that is modified has been against the Fortran language rules for more than fifty years, and for all of those years, there have been compilers that would give wrong answers if you broke that rule. You may not have been aware of it, but this was an unwise programming practice when it was originally written, just as it is now. Perhaps "ignorantly" is more accurate than "inadvertently", but people in user support like Steve try to be polite and pretend you forgot the rules rather than that you never bothered to learn them.

Perhaps I was not clear in pointing out my standpoint, and I'm sorry for not mastering english language (and FORTRAN too ....).

I would not appear as a happy violator of the standards !

I just have, among others, the task of mantaining this legacy code and porting it to new platforms and hence to new compilers, to make it available to the users as long as possible.

Doing this, for sure, I am trying to make the code as independent from the compilers as possible, that means more and more compliant to the standard.

When I said that the code was not "bad" but just "out of date" I did not mean that it was compliant to the standard and then "universally correct". I just meant that it had been working fine for several years on different platform and compilers, apparently relying on some non-standard extensions (or loosening ?). I also guess that the original developers did not imagine it would have lasted for such a long time, and then did not care too much about using non-standard extensions (alas!).

I would also say that application developers usually study the compilers manuals, more than the languages standards, then often deviate from the standards, protected by the misleading umbrella of their current compiler.

Now, it is not a matter of demanding the new compilers to allow non standard behaviors for ever, but just how to help the mantainers in improving code compliance smoothly.

Compilers users and compilers developers have the same strategic goal (to focus on the most qualifing aspects of their own jobs) which leads to different tactical goals: for the users, try not to touch the existing and working code, to have time for new developments, for the compiler developers, providing the most efficient compilers for the widest audience of users.

Sometimes these differences might lead to different preferences, that's all.

For instance, I would prefer removing standard violations after having studied the warning messages issued on request, more than looking for the more or less hidden cause of a segmentation fault (Unfortunately, -stand does not help in this case of passing constant arguments). But I can try to adapt my behavior to the context ...

Quoting - hirchert

Quoting - e745200

1. The "non-standard" writes would never occur if spending those extra instructions just when they are anyway needed to allow the program keep on working.

I haven't slightest idea what this sentence is trying to say.

Steve said that -assume noprotect_constants was not the default because it costs extra instructions, adding efficiency reasons to standard compliance motivations.

I was just trying to say that, to make the program work, extra instructions are anyway needed, because the constant value must anyway be assigned to a variable. And, done by the compiler (using the appropriate flag) or because explicitely coded, perhaps it would make a limited difference on the efficiency of the resulting code.
But actually I do not have enough information to know if this is correct or not. It was just a guess.

On the other hand, I definitively agree with you that removing the non standard usage of constant arguments is the ultimate solution, and, as I said in one of my first post in this discussion, this is what I am doing.

Thanks again for your time.