Intel OneAPI -- Issues with OpenMP OMPT target directives

[Reuter_Jan]

Heya,

I've been trying to run a small example to get to know the OMPT interface of OpenMP. I'm interested in the OpenMP Target side of OMPT in particular. After testing several compilers both Intel OneAPI and AMDs ROCm looked perfect.

I installed the Intel OneAPI on my local machine and wrote a small test program, just to see what parameters are passed to the functions. During the execution, I noticed that the given parameters seemed off.

Especially the optype during ompt_device_target_data_op calls and the requested_num_teams in ompt_device_target_submit were conspicuous. In each call, the number of this parameter increases by one instead of containing the actual information about the optype or requested_num_teams. Other parameters also seem to contain unusable information.

Unfortunately I don't have access to another system to check if those issues are related to my machine.

I then tried to use the ROCm compiler on a different machine where the output looks like I would expect it.

The source code is attached as main.c. Here's a bit more information about the machines I used and the output.

Local machine:

Fedora Linux 37

Intel Core i7-1260P, NVIDIA MX550

Intel(R) oneAPI DPC++/C++ Compiler 2023.0.0 (2023.0.0.20221201)

Output: Attached as Output_Intel_OneAPI.txt

Remote server:

Rocky Linux 8

AMD EPYC 7443, AMD MI250

AMD clang version 15.0.0 (https://github.com/RadeonOpenCompute/llvm-project roc-5.4.0 22465 d6f0fe8b22e3d8ce0f2cbd657ea14b16043018a5)

Output: Attached as Output_AMD_ROCm.txt

[NoorjahanSk_Intel]

Hi,

Thanks for reaching out to us.

We are also able to reproduce the issue at our end. We are working on your issue, we will get back to you soon.

Thanks & Regards,

Noorjahan Shaik

[NoorjahanSk_Intel]

Hi,

The test case tries to print out the operation type via "print_ompt_device_target_data_op". But the generated output is giving unknown values (10, 12). which is not in OMP5.2 spec.

We have provided your feedback to the relevant team. At this moment there is no visibility of when it will be implemented and available for use. Please let me know if we can go ahead and close this case.

Thanks & Regards,

Noorjahan.

[Reuter_Jan]

Heya,

thanks for the feedback.

You're right, the test case does print out unknown values for the optype. However, this issue is not due to the OpenMP specifications. 

There are several bugs inside that test case that seem to be caused by the current implementation of the OMPT interface inside of OneAPI:

Let's look at the print_ompt_device_target_data_op parameters:

The optype does not show the actual operation that takes place. Instead, the optype increments by one. This can be verified, as the first and second call of the same method should contain mostly the same information. This is not the case. The first time #pragma omp target map(A[:N]) is called, we have an optype of 2. During the second call of the same method, the optype is 9. 

In addition, the src_addr is incrementing for a fixed host_op_id with the addresses 0x1, 0x2 and 0x3. Those pointers should all be invalid. 

The bytes parameter during the last print_ompt_device_target_data_op should not be 16384 PiB but only 8 / 40 bytes. 

In src_device_num and dest_device_num, the numbers are frequently below zero. As there are only two possible devices in the system, the number should not be negative but show the actual device instead. 

The codeptr_ra show 0x28 and 0x8 in each call. This seems not to be a valid code pointer.

The print_ompt_device_target_submit also has an issue with the requested_num_teams. You can verify it, again, by comparing the first and second call of the target method. During the first call, the requested_num_teams is reported as 1. The second time, it is reported as 8. In the meantime, the optype got incremented to a value of 7. 

As I've said in the first post, the output of the ROCm execution is also attached. I would expect to see a similar result. I've also noticed that the host_op_id and target_id differ between ROCm and the Intel OneAPI but need to check if there is even a guideline on how those parameters need to be defined. 

[NoorjahanSk_Intel]

Hi,

Thank you for your feedback. We have provided your feedback to the relevant team. At this moment there is no visibility of when it will be implemented and available for use. Please let me know if we can go ahead and close this case.

Thanks & Regards,

Noorjahan.

[Reuter_Jan]

You can go ahead and close the case. 

Hopefully the issue will be fixed soon. It would help our development immensely. 

[NoorjahanSk_Intel]

Hi,

Thanks for the confirmation.

If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Thanks & Regards,

Noorjahan.