- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am trying to set up an OpenMP fortran code with a task thread that listens for input from the user and when received sets a variable that the main thread uses to determine when to stop running. I created the following code that seems like it should work. There is a module level logical that is set to .false. by default, but the task thread sets it to true when receiving a specific string from the user. This code works as expected in Debug, however, in release, the main thread doesn't ever see that the variable has changed.
module threads implicit none logical :: killThread integer :: testVariable contains subroutine taskThread() character(2) :: killme do while (.true.) ! waiting for user to input read(*,*) killme print *, killme ! if user inputs exact string 'OK' then exits infinite loop and ! sets module variables if (killme == 'OK') then killThread = .true. testVariable = 1 exit end if end do print *, 'exiting thread 0, testVariable = ', testVariable, ', address = ', loc(testVariable) end subroutine end module program threadtest use omp_lib use threads implicit none integer :: thread integer :: flushStart, flushStop, ticksPerSecond ! these should be shared module level variables that can be written and read by all threads killThread = .false. testVariable = 0 call SYSTEM_CLOCK(flushStart, ticksPerSecond) ! create two threads call omp_set_num_threads(2) !$omp parallel !$omp single thread = omp_get_thread_num() !$omp task call taskThread() print *, 'exited, thread 0, testVariable = ', testVariable, ', address = ', loc(testVariable) !$omp end task do while (.true.) ! infinite loop that just prints to the screen once per second until the module variable ! "killThread" is set to true call SYSTEM_CLOCK(flushstop, ticksPerSecond) if (real(flushStop-FlushStart) / ticksPerSecond > 1.0) then print *, 'tick thread 1, testVariable = ', testVariable, ', address = ', loc(testVariable) call SYSTEM_CLOCK(flushStart, ticksPerSecond) end if if (killThread) then exit end if end do !$omp end single !$omp end parallel print *, 'exiting program' end program
The issue is that the variable killThread is being set to true in the task thread, but in the main thread, it never sees it and never stops running. Everything I have seen with OpenMP and fortran is that module variables are supposed to be shared unless set to THREADPRIVATE (which these are not). I also set a test integer to see if it is getting changed. The value does get changed in the task thread, but the main thread still sees it as 0. The address of the variable reports the same in both cases, so I am confused.
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You need to add the VOLATILE attribute to the module variable. Otherwise the compiler sees there is nothing in the loop that sets the variable and therefore doesn't check it again.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks Steve. That worked and I think I understand why.
I actually had thought it was something like this (in Release due to optimization). I had tried to put in a condition in the loop that would change the value to true after the 500th iteration, but it seems like that did not work.
if (count > 500) then killThread = .true. end if
why would that not tell the compiler it can't optimize that check away?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'd have to see the whole modified code to analyze, since as you originally posted there is no count.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
OK, here you go. Just an integer counter that is initialized outside the infinite loop that increments at every tick. Like I said, your original fix worked fine, I'm just trying to understand.
module threads implicit none logical :: killThread integer :: testVariable contains subroutine taskThread() character(2) :: killme do while (.true.) ! waiting for user to input read(*,*) killme print *, killme ! if user inputs exact string 'OK' then exits infinite loop and ! sets module variables if (killme == 'OK') then killThread = .true. testVariable = 1 exit end if end do print *, 'exiting thread 0, testVariable = ', testVariable, ', address = ', loc(testVariable) end subroutine end module program threadtest use omp_lib use threads implicit none integer :: thread, count integer :: flushStart, flushStop, ticksPerSecond ! these should be shared module level variables that can be written and read by all threads killThread = .false. testVariable = 0 call SYSTEM_CLOCK(flushStart, ticksPerSecond) ! create two threads call omp_set_num_threads(2) !$omp parallel !$omp single thread = omp_get_thread_num() !$omp task call taskThread() print *, 'exited, thread 0, testVariable = ', testVariable, ', address = ', loc(testVariable) !$omp end task count = 0 do while (.true.) ! infinite loop that just prints to the screen once per second until the module variable ! "killThread" is set to true call SYSTEM_CLOCK(flushstop, ticksPerSecond) if (real(flushStop-FlushStart) / ticksPerSecond > 1.0) then print *, 'tick thread 1, testVariable = ', testVariable, ', address = ', loc(testVariable) count = count + 1 print *, count call SYSTEM_CLOCK(flushStart, ticksPerSecond) end if if (count > 500) then killThread = .true. end if if (killThread) then exit end if end do !$omp end single !$omp end parallel print *, 'exiting program' end program
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
And you waited 500 seconds to see if it exited? I would have expected that to work, but haven't tried it myself.
By the way, you can replace "do while (.true.)" with just "do".
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I changed 500 to 10 (why wait 500 sec?) and added the line
if (killThread) exit
before the read statement in taskThread. Then after 10 seconds it is necessary to type something and hit Enter to kill the thread, because it is always waiting for console input. The added line can be before or after the read statement.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I tried the program and, while I don't really understand OpenMP well, it appeared to do what it was supposed to. I added some more print statements to make sure I knew what was happening. Here's where I modified it:
myloop: do while (.true.) ! infinite loop that just prints to the screen once per second until the module variable ! "killThread" is set to true call SYSTEM_CLOCK(flushstop, ticksPerSecond) if (real(flushStop-FlushStart) / ticksPerSecond > 1.0) then print *, 'tick thread 1, testVariable = ', testVariable, ', address = ', loc(testVariable) count = count + 1 print *, count call SYSTEM_CLOCK(flushStart, ticksPerSecond) end if if (count > 10) then killThread = .true. print *, "Setting killthread true" end if if (killThread) then print *, "killThread is true" exit myloop end if end do myloop print *, "Outside loop"
When I run this I see:
tick thread 1, testVariable = 0 , address = 140694945976440 11 Setting killthread true killThread is true Outside loop OK OK exiting thread 0, testVariable = 1 , address = 140694945976440 exited, thread 0, testVariable = 1 , address = 140694945976440 exiting program
Isn't this what you want?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks, Steve.
The point of this program is to:
- execute some function on a main thread (in this case an infinite loop that ticks once per second) - while simultaneously...
- kick off a second thread that will wait for user input and exit everything if the user enters a certain string
If the user enters "OK" at any time then the program exits.
In your implementation, I think the main loop is just stopping when the counter reaches 11. I made it 500 because I don't want it to stop on a timer, rather I ONLY want it to stop when the user enters the string "OK" and presses enter. I only put that counter and extra exit condition in there because you were saying that the reason that if check wasn't done is the compiler saw that killthread could never change in the loop and optimizes that check away. Like I said, in its original form it works in debug mode. So if I had the counter and extra exit condition in the code, even if I never expected to let it go that long, why was volatile still required on the variable?
what I want is this...
tick thread 1, testVariable = 0 , address = 140695989965200 1 tick thread 1, testVariable = 0 , address = 140695989965200 2 tick thread 1, testVariable = 0 , address = 140695989965200 3 OK tick thread 1, testVariable = 0 , address = 140695989965200 4 OK exiting thread 0, testVariable = 1 , address = 140695989965200 killThread is true exited, thread 0, testVariable = 1 , address = 140695989965200 Outside loop exiting program
Note, that as soon as I press OK the code exits. But what the code (in Release) does is this...
tick thread 1, testVariable = 0 , address = 140696553034056 1 tick thread 1, testVariable = 0 , address = 140696553034056 2 tick thread 1, testVariable = 0 , address = 140696553034056 3 tick thread 1, testVariable = 0 , address = 140696553034056 4 OK OK exiting thread 0, testVariable = 1 , address = 140696553034056 exited, thread 0, testVariable = 1 , address = 140696553034056 tick thread 1, testVariable = 0 , address = 140696553034056 5 tick thread 1, testVariable = 0 , address = 140696553034056 6 tick thread 1, testVariable = 0 , address = 140696553034056 7
Note, after pressing OK, I can see that the taskthread exited (exited, thread 0, testVariable =...), but the main loop keeps ticking. Making killThread volatile fixes this behavior, but I'm not sure why it is required since there is a clear condition in the code that CAN change it. I am ok with making the variable volatile. I'm just trying to understand why it is necessary.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If I understand the assembly correctly, the compiler created a shadow variable (called r14b in my assembly listing) and it tested that. The code that set killThread based on count set both the module variable and the shadow variable. Since the test only looks at the shadow variable, that's why it doesn't trigger when you enter OK. Adding VOLATILE alerts the compiler that it really needs to check the module variable.
Here's the assembly when count>10:
mov DWORD PTR [THREADS_mp_KILLTHREAD], -1 ;75.17 mov r14b, -1 ;75.17
and here's the test:
test r14b, 1 ;79.17 je .B1.40 ; Prob 80% ;79.17
If I make killThread VOLATILE, I get this instead:
mov DWORD PTR [THREADS_mp_KILLTHREAD], -1 ;76.17
mov r8d, DWORD PTR [THREADS_mp_KILLTHREAD] ;80.13 test r8d, 1 ;80.17 je .B1.41 ; Prob 80% ;80.17
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Steve, r14b is the MASM style name for the lowest 8 bits of register R14 in X64; see, for example, https://software.intel.com/en-us/articles/introduction-to-x64-assembly . As with X86, there are no instructions that involve purely memory operands; i.e., with no registers involved.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ah, I didn't know that. Makes sense that it is using a register for the shadow. I never studied x64 assembly. Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I read a book on Assembly in around 1990, so I won't pretend I understand all of what you wrote. :)
So is the bottom line reason the original variable is not checked in lieu of the shadow variable is because it is being set in a separate thread. Similar to a situation where something is coming from the operating system or something like that?
This sort of seems like a bug in the OpenMP implementation. It is my understanding that module variables are supposed to be shared by all threads in OpenMP. So any module variable I set in any thread seems like it should be interpreted correctly by any other thread. But I am satisfied with using the volatile attribute to make it work correctly.
Thanks again for the help.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It's not a bug. Without VOLATILE, the compiler is allowed to assume that a variable isn't modified in ways not visible to it, such as other threads. This permits better optimization. Situations such as this are exactly what VOLATILE was designed for. Fortran 2018 expanded the use of ASYNCHRONOUS to be a "smaller hammer", however. I tried it with your example and it worked there too.
The difference is that with ASYNCHRONOUS, the compiler is allowed to use a register after it fetches the value for a reference, with VOLATILE it always has to go to memory.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
That makes a lot of sense. Thanks for all your help.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
VOLATILE is probably overkill for this. What is missing is an OpenMP flush operation that ensures that the thread executing the infinite loop has a a consistent view of the program's memory. (There is an implicit flush when the task ends, but that just ensures that the temporary view of the thread running the task is consistent with memory, it doesn't affect the other thread running the loop.)
Add !$OMP FLUSH(killThread) prior to the reference to killThread inside the conditional expression of the if statement inside loop and see what happens. See the section of your chosen edition of the OpenMP specification on the "Memory Model", and also the documentation of the FLUSH construct.
(Note that writing to killThread from within the infinite loop also introduces a race condition.)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
IanH (Blackbelt) wrote:Add !$OMP FLUSH(killThread) prior to the reference to killThread inside the conditional expression of the if statement inside loop and see what happens.
That does it.
![](/skins/images/7FC17B7B85029576C25F1E43CE255B51/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page