MXCSR state in threads

[Kat__Swat]

I was under the assumption that:

1) while floating-point settings are captured from the main thread, each thread can independently change the MXCSR's contents.

2) changing the MXCSR's settings inside a function called inside a thread don't affect the caller's register.

My experiments have left me confused (performed with the latest release (tbb2019_20181203oss_win) on an Intel Xeon E5-1650, Win 10):

using namespace tbb;

void printMXCSR(int n) {
    std::cout << "MXCSR in thread " << n << " before FTZ is " << _MM_GET_FLUSH_ZERO_MODE() << std::endl;
    _MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON);
    std::cout << "MXCSR in thread " << n << " after FTZ is " << _MM_GET_FLUSH_ZERO_MODE() << std::endl;
    std::cout << "In thread " << n << " setting MXCSR to default" << std::endl;
    _MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_OFF);
    std::cout << "MXCSR in thread " << n << " after default is " << _MM_GET_FLUSH_ZERO_MODE() << std::endl;
}

int main() {
    std::cout << "MXCSR in main thread before FTZ is " << _MM_GET_FLUSH_ZERO_MODE() << std::endl;
    _MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON);
    std::cout << "MXCSR in main thread after FTZ is " << _MM_GET_FLUSH_ZERO_MODE() << std::endl;
    task_group g;
    g.run([&]{printMXCSR(1);});
    g.run([&]{printMXCSR(2);});
    g.wait();
    std::cout << "MXCSR in main thread at end is " << _MM_GET_FLUSH_ZERO_MODE() << std::endl;
    return 0;
}

The snippet above produces the output:

MXCSR in main thread before FTZ is 0
MXCSR in main thread after FTZ is 32768
MXCSR in thread 2 before FTZ is 32768
MXCSR in thread 2 after FTZ is 32768
In thread 2 setting MXCSR to default
MXCSR in thread 2 after default is 0
MXCSR in thread 1 before FTZ is 32768
MXCSR in thread 1 after FTZ is 32768
In thread 1 setting MXCSR to default
MXCSR in thread 1 after default is 0
MXCSR in main thread at end is 0

My understanding from what's happening is that threads should not alter the MXCSR state, because they do affect the main thread; is this correct?

[Alexei_K_Intel · Accepted solution]

TBB propagates MXCSR settings to a thread that executes a particular task but does not restore MXCSR if the state is changed by this task. You may want to read the section in TBB documentation about provided floating point settings guarantees. Pay attention that all guarantees are provided if user does not change (or restores) MXCSR settings inside the tasks:

CAUTION
The above guarantees only apply in the following conditions:
The user code inside a task either does not change floating-point settings, or reverts any modifications and restores previous settings before the end of the task.
Intel TBB task scheduler observers are not used to set or modify floating point settings.
Otherwise, the stated guarantees are not valid and the behavior related to floating-point settings is undefined.

P.S. the main thread is allowed to execute tasks (when the wait method is called) so you can observe some side effects on the main thread.

Regards,
Alex