Intel® oneAPI Data Parallel C++
Support for Intel® oneAPI DPC++ Compiler, Intel® oneAPI DPC++ Library, Intel ICX Compiler , Intel® DPC++ Compatibility Tool, and GDB*

OneMKL sample takes forever to build

AyushM
Novice
5,513 Views

I am using the latest oneAPI Base Toolkit in my local machine.I tried running the examples provided as zip inside the onemkl folder.Had some inlclude errors in my VS.So I had to turn on the option to use onemkl in my VS 19 and also had to add the folder containing the "common_for_examples.hpp".Noww VS doesn't show any errors,but takes forever to build my .cpp file.Please look into this and help.Urgent help required.Thanks in advance!!

0 Kudos
36 Replies
AbhishekD_Intel
Moderator
3,311 Views

Hi Ayush,


Thanks for reaching out to us.

We tried running some examples as mentioned by from MKL directory and we can compile and run them successfully from our side. We tried on both command-line as well as VS.


Each of the examples has Makefile with them, you can try building that Makefile in your Windows environment which will directly give you the executable.

To build the Makefile in windows please follow the below steps.

  1. Open Native Tools Command prompt for VS and execute the Intel\oneAPI\setvars.bat to enable the environment.
  2. Change the current directory to the extracted directory from the same instance of the command prompt and execute nmake command to build the Makefile for the particular example.
  3. You will get the executable for that particular sample example.


If you are using Visual Studio to build the same examples, then you can create a new DPCPP project and can copy all the required sample files into your project directory. You need to configure the entire project correctly for the headers and have to add some flags related to the particular sample in order to compile and run the project.

Compiling those examples will take some time in VS(less than a minute) if the configuration is set properly.


Hope the provided details will help you to resolve your issue. Do let us know if you face any problems while following the above details.



Warm Regards,

Abhishek



AyushM
Novice
3,276 Views

Thanks a lot such a well detailed solution!!

I think I did something similar except a few specific steps.I will follow the exact same steps as you mentioned and let you know if the error gets resolved

 

0 Kudos
AyushM
Novice
3,268 Views

Okay now I've followed the exact same steps using nmake.But I've still got the same issue.The obj file is getting generated but the exe file never gets generated.The command just keeps running and my memory utilization is going to 97%(8GB RAM).I don't know why this is happening.Please help me with the same.

0 Kudos
AbhishekD_Intel
Moderator
3,181 Views

Hi Ayush,


The Makefile with the examples is generating executables and then running them and storing results in a directory. Maybe the reason for the hang is due to the running executable. Please try to comment the line(running executable) from the Makefile and try building the application.

You can also try compiling a single example from a particular domain manually and see if it still hangs. If the hangs still persist then please share your logs(nmake) with all the steps you followed.


Warm Regards,

Abhishek


0 Kudos
AyushM
Novice
3,156 Views

I've commented the line and now the obj files are getting created successfully.But I've got no leads on how to run them.Do I need to use VS or should i use clang++ from cmd?

0 Kudos
ziglar
Beginner
3,146 Views

I have a similar issue building any projects in VS2019 that require me to include

#include "oneapi/mkl.hpp"

and making any calls to

oneapi::mkl::blas::row_major::gemm(device_queuetransAtransB, M, N, K, alphaAsharedldaBsharedldbbetaCsharedldc);

The compilation  can take 40 minutes and uses over 10Gb RAM and generates thousands of .tlog entries.

ziglar_0-1616156642069.png

Any help would be appreciated.

0 Kudos
AbhishekD_Intel
Moderator
3,134 Views

Hi,

 

The default behavior is Makefile is,

  1. It will compile a particular sample from a domain example and
  2. It will execute that compiled sample and will store results in _results\ with as <sample>.res file.
  3. Steps 1 & 2 will continue for all the samples and each sample will get compiled and execute one after the other.

 

Depending on the MKL example the Makefile will continue to run till all samples get executed. And as the samples are also getting executed it's utilizing more memories.

 

So you can configure the usage of your Makefile by selecting the proper options. For more details regarding the usage of Makefile related to each example, please refer to the output of nmake help command.

 

If you want to build all the samples without executing them the please try commenting the execution step from the Makefile.

Replace :

$(list_all): $(@).cpp

@Echo.---- Building/Running $(@D)\$(@F) for SYCL Device=$(device) ------

if not exist $(RES_DIR)\$(@D) md "$(RES_DIR)\$(@D)"

-$(COMP) $(COPTS) $** -o $(RES_DIR)\$(@D)\$(@F).exe

$(RES_DIR)\$(@D)\$(@F).exe > $(RES_DIR)\$(@D)\$(@F).res

 

To :

$(list_all): $(@).cpp

@Echo.---- Building/Running $(@D)\$(@F) for SYCL Device=$(device) ------

if not exist $(RES_DIR)\$(@D) md "$(RES_DIR)\$(@D)"

-$(COMP) $(COPTS) $** -o $(RES_DIR)\$(@D)\$(@F).exe

#$(RES_DIR)\$(@D)\$(@F).exe > $(RES_DIR)\$(@D)\$(@F).res

 

And to run the executable you can directly run the <sample>.exe from _results in the same instance of the command prompt. For more details please refer to the attached screenshot.

 

 

Hope the provided details will help you to resolve your issues.

 

Warm Regards,

Abhishek

 

0 Kudos
ziglar
Beginner
3,100 Views

Hi, if I can give an easier simple example of my problem.

Run up VS2019 and create a new project from the oneAPI sample template. Select the matrix_multiply project. Building it takes a few seconds.

Now go into project properties and select 'Intel Libraries for oneAPI', set use oneMKL to Sequential. Now building the same project takes 10's of minutes and the 'SYCL post-link device code processing tool' uses > 10Gb memory.

Surely this can't be correct? All I have done is added oneMKL to the project.

0 Kudos
ziglar
Beginner
3,088 Views

Hi, just been looking through the oneAPI-samples github repo and see this in the oneMKL samples folder (https://github.com/oneapi-src/oneAPI-samples/tree/2021.1-gold/Libraries/oneMKL/matrix_mul_mkl/README.md) and wondered when this 'known compiler issue' is likely to be fixed?

On a Windows* System

Run nmake to build and run the sample. nmake clean removes temporary files.

Warning: On Windows, static linking with oneMKL currently takes a very long time, due to a known compiler issue. This will be addressed in an upcoming release.

0 Kudos
AbhishekD_Intel
Moderator
3,053 Views

Hi @ziglar ,

 

Thanks for pointing out the link. That issue will soon get fixed in the upcoming releases.

@AyushM please give us an update on your problem.

Do let us know if you have any other doubts related to the samples which you are trying to use.

 

 

Warm Regards,

Abhishek

 

0 Kudos
AbhishekD_Intel
Moderator
3,006 Views

Hi Ayush,


Please give us an update on the provided details.



Warm Regards,

Abhishek


0 Kudos
AyushM
Novice
2,977 Views

I still haven't been able to generate a .exe file yet.The 3D object file gets generated but the command promt runs forever.My memory and disk usage is at 100%.

I've commented the last line of the makefile as you suggested,but still the issue persists.

AyushM_0-1617874944790.png

My command promt is stuck at the above screen.Please let me know if this problem is due to the compiler as mentioned in the github readme,or am I going wrong somewhere.

 

Thanks and Regards,

Ayush.

 

0 Kudos
ziglar
Beginner
2,970 Views

Hi Ayush, this was the exact same problem I was having.

The sycl cl linker was taking most of my resources and over 40 minutes to link - if you have enough resources and patience it will probably succeed.

0 Kudos
AyushM
Novice
2,955 Views
Hi ziglar, can you let me know if you were able to compile it after 40 mins? If yes then what are your PC specs.

I have 8GB RAM with Core i5 8300H 4.0GHz. Not sure if processor matters here.I left it for an hour but no compilation yet. And during that time my PC was unusable because of less available RAM
0 Kudos
ziglar
Beginner
2,945 Views

Hi Ayush, I have an older 8 thread i7 running @ 4.2GHz with 32Gb RAM. The RAM was the limiting factor and sometimes I even ran out and the linking failed.

The DPC compiler is unusable from that point of view until they fix the bug which is a real pity since I was really interested in trying to gain an understanding of OneMKL on GPU

0 Kudos
AyushM
Novice
2,935 Views

Yes I agree,its a real pity we can't run it on Windows.I guess I will be swiching to Linux.Any idea if it runs fine on that?

0 Kudos
ziglar
Beginner
2,931 Views

not yet tried Linux so don't know.

On Windows the issue, according to the link I found and posted earlier, suggests that it only occurs with static linking; I did try to link dynamically but ran into problems and gave up.

0 Kudos
AbhishekD_Intel
Moderator
2,915 Views

Hi,

 

Apology for the inconvenience.

We will try to address all the issues/points encountered by you.

 

The issue which you are facing(command prompt runs forever) depends on the system to system and processing power and it is also pointed out as a known issue in the above-mentioned link. So we can't really give you the exact time it will need for the compilation on Windows 10 systems. For us, it's not taking a long time to compile and run with Win10 and we followed the same steps as mentioned above.

 

We haven't encountered this problem on the Linux system and as mentioned in the same link above link the samples are not taking more than 15mins to compile and execute. We also have tested the same samples on the Devcloud with Ubuntu and it's running as expected without any delay. The issue is only observed on the Windows system with a few users.

 

Updates:

Please try updating the Base toolkit from 2021.1 to 2021.2 (latest release) and please let us know if you are getting the same issue.

 

 

 

Warm Regards,

Abhishek

0 Kudos
ziglar
Beginner
2,897 Views

Hi Abhishek, thanks for the update.

I have just installed 2021.2.0 and it looks like it all works as expected with build times for my oneMKL test project down from over 40 minutes to a few seconds.

Cheers,

0 Kudos
AyushM
Novice
2,911 Views

Can you please tell which examples did you run?

I've downloaded the 2021.2.0 update.I can't find the examples_sycl.zip in the latest onemkl folder anymore.

0 Kudos
Reply