There can be many reasons for disagreement between results compiled with different compiler options. If the differences are small, e.g., the differences are only in the least significant digit, such differences may be considered normal. Large differences, on the other hand, can be caused by bugs -- bugs in your code, bugs in the library, bugs in the optimizer part of the compiler, etc.
It is not reasonable to expect a simple answer to your general question. Please provide a test case (code, data, instructions to build and run).
