Slow x64 operation when optimized

[groupw_bench]

When compiled for the x64 platform, I'm finding some puzzling effects of optimization -- small problems run much more slowly when optimized, and large problems run a bit faster.

The program does some fairly heavy complex number calculation to fill some matrices which are solved in an equation using MKL procedures. Following the solution, additional calculations are done on the results. Virtually every part of the calculation is affected except the MKL procedures. Some operations are slowed by a factor of more than 10 when /O2 optimization is specified. This is occurring only when compiled for the x64 platform, and the slowdown occurs if any optimization (/O1 - /O3) is specified. It happens when the program is compiled for either single precision or double precision default real and complex variables. The strange thing is that when I run a problem with a dozen or so data points, it's terribly slow compared to the non-optimized program. When there are a couple of thousand data points, the optimization improves the speed. So I would like to take advantage of the optimization if possible.

I've tried changing just about every other program property that might possibly have an effect, with no change. I've also added NOOPTIMIZE compiler directives in some routines which are the only ones invoked during one step of the calculation, and see no difference at all in that part of the calculation. It's as though the problem isn't in optimization of the code itself but in the way the program interfaces with something external.

The program is a console type application although no console ever appears -- it gets its input from a file and sends its output to a file. It's running under 64 bit Windows 7 and does call a few Windows API functions, although disabling them for testing makes no difference. The program compiles and links with no warnings and no errors, and in brief tests returns correct results. The compiler is Composer XE 2011 SP1.

I realize there's not enough information here for a specific diagnosis. But I'm hoping I can get some suggestions about where I might start looking or what I might try in order to isolate the problem.

Thanks!

[Steven_L_Intel1]

I would recommend using a performance analysis tool, such as Intel VTune Amplifier XE. You can download a free 30-day trial of that if you like. Build the program with and without optimization, run it through hotspot analysis, and see where each is spending its time.

[jimdempseyatthecove]

First, check to see that you do not have runtime checks enabled for array bounds and/or uninitialized variables. Sometimes I set these in release mode during testing. Its an easy oversight.

Second, run VTune or other profiler (as Steve suggests). You may find an inordinant number of conversions from float to double (or double to float).

Third, if you can run the Debug builds side-by-side, open up the data (scalars and arrays) and try to observe if the variables are of the same size.

Forth, check the LOC(arrayX) of the arrays you may be processing to see if the position affects SSE/AVX performance.

And verify both compiled versions use the same extensions (IA32/FPU, SSE, AVX).

Jim Dempsey

[groupw_bench]

Thanks, folks.

I found it. This is my first time compiling for x64, and I got burned by a Windows API integer variable size. I had changed the fixed KIND = 4 integers to the c-type designations in the API function argument lists to allow for them to be 8 bytes on x64 systems. But somewhere along the line I had suppressed the mismatched data type compilation warning, so didn't catch the fact that one of those variables was used as a program function parameter which was defined in the calling procedure as KIND = 4. Surprisingly, everything worked when not optimized -- I'm glad optimization caused the error.

The end result of the variable mismatch was to cause the function to fail to send a message via a pipe to the calling program. The calling program was waiting around for the message to appear, and that caused a fixed delay in program execution which proportionately slowed runs with little data more than runs with a large amount of data.

Roy

[Steven_L_Intel1]

Glad to hear it. Next time you'll leave that error checking enabled, right?

[groupw_bench]

Absolutely! The problem is that I have two "projects" in one "solution" and am continually baffled and surprised by the Active Solution Configuration and Project Contexts in the Configuration Manager, along with the choices when right-clicking a project in the Solution Explorer. Sometimes when I change one, another changes unpredicably. (I'm sure there's some rational relationship among them but I haven't yet figured out what it is.) And more often than not, when I try to change the project properties, I discover I'm not actually changing the properties of the project, build, or platform I think I've selected (I have single and double precision, debug and release, and Win32 and x64 versions of both projects). But I've learned to stick with one "project" at a time per "solution". I fiddle with things until I get the properties of what I'm trying to build, and check the output dialog after building the program to see what I've just built. Then I open the Properties window and check to make sure the properties are what I thought they were.

Keeping only one project in the solution sure helps when debugging, too. With two projects, after carefully setting the Configuration Manager to build the right program, checking and setting the project properties for that program, building it, verifying that the one just built was the one I thought, I start the debugger, and. . . find that a different program is running! I forgot to right-click the project and choose Set as Startup Project!

Do other people have trouble with this, or am I just particularly dense?

In either case, I appreciate the help.