MKL_INT vs size_t

[Petros_Mamales]

Hi,
I understand that the answer is 99.99% going to be negative, but for whatever it's worth:
Is there a way to use size_t instead of MKL_INT ? (like the way one can overwrite the MKL_Complex8 by std::complex).
Any C/C++ structure that will describe some (sparse) matrix, will index with size_t rather than with int (and its variations). For non-sparse matrices where only the matrix dimension are the integers one uses, things are largely OK.
But for the sparse compressed format matrices, there is always a danger that the size of the integer array elements will be casted erroneously.
(Not to mention all the warning messages the compiler will emit because of the missmatch).
Any suggestions? ideas/workarounds? Maybe an option in a future mkl release? (yes I can imagine the binary layout issue, but, maybe another little flag ? ;-)) )
TIA,
Petros
ps: although looking for a portable solution, using msvc2010 on win7 with 64bit builds.

[Andrey_N_Intel]

Hello Petros,

Please, have a look at the article that describes use of Intel MKL in C++ applications.
In particular, it covers the topic ofMKL_INT and size_t data types. The article is available at http://software.intel.com/en-us/articles/use-of-intel-mkl-data-types-in-cc-applications/.

Please, let us know if this addresses your question.

Best,
Andrey

[barragan_villanueva_]

Hi,

Please read the corresponding MKL article: http://software.intel.com/en-us/articles/use-of-intel-mkl-data-types-in-cc-applications/?wapkw=mkl_int+size_t+software.intel.com

[Petros_Mamales]

Hi Andrey/Victor,
Yes it does !! (was right to "gamble the 0.01% ;-)))
Thank you very much,
Petros

[Petros_Mamales]

[cpp]#include using namespace std; #define MKL_INT int #include #include int main() { /* m: 5 X 5 1 * 1.1 * * * 2 * * * 3.1 * 3 3.14 * * * * 4 2.17 * * 5.1 * 5 */ MKL_INT ia[ 6 ] ; ia[0] = 1 ; ia[1] = 3 ; ia[2] = 4 ; ia[3] = 7 ; ia[4] = 9 ; ia[5] = 11 ; MKL_INT ja[ 10 ] ; ja[0] = 1 ; ja[1] = 3 ; ja[2] = 2 ; ja[3] = 1 ; ja[4] = 3 ; ja[5] = 4 ; ja[6] = 4 ; ja[7] = 5 ; ja[8] = 3 ; ja[9] = 5 ; double values[ 10 ] ; values[0] = 1E0 ; values[1] = 1.1E0 ; values[2] = 2E0 ; values[3] = 3.1E0 ; values[4] = 3E0 ; values[5] = 3.14E0 ; values[6] = 4E0 ; values[7] = 2.17E0 ; values[8] = 5.1E0 ; values[9] = 5E0 ; MKL_INT m = 5 ; MKL_INT n = 5 ; MKL_INT gelda = 5 ; MKL_INT job[6] ; job[0] = 1; job[1] = 0; job[2] = 1; job[3] = 2; job[4] = 10; job[5] = 1; double * gedata = new double[ gelda * n ]; MKL_INT info = 0; mkl_ddnscsr( job, &m, &n, gedata, &gelda, values, ja, ia, &info ) ; for ( size_t i = 0; i != 5; ++i ){ for ( size_t j = 0; j != 5; ++j ) cout << gedata[i*gelda + j] << "t"; cout << endl ; } return 1; }
[/cpp] Hi Victor/Andrey,
I am afraid that then there are a few problems here..
Take for example the mkl_?csrdia (or any function that contains diagonal sparse format).
The distance vector (counting the distance from the diagonal) can have either sgn.
However defining MKL_INT as size_t does not accomodate for it !! There should be a different type for the
distance vector elements, in order that MKL_INT can be defined as size_t.
Also, converting from csr to dense is less than perfect as the above example shows.
If instead of :
[cpp]#define MKL_INT int [/cpp]I use:
[cpp]#define MKL_INT size_t [/cpp]I get the (very!!!) erroneous output.

What to do ?
Any workarounds?
TIA,
Petros

PS: Using mkl10.3.8 with vs2010 on win7
The 2 outputs:
int
1 0 1.1 0 0
0 2 0 0 0
3.1 0 3 3.14 0
0 0 0 4 2.17
0 0 5.1 0 5

vs. (size_t)
0 -6.27744e+066 -6.27744e+066 -6.27744e+066 -6.27744e+066
1.1 1 -6.27744e+066 2 -6.27744e+066
0 0 0 -6.27744e+066 -6.27744e+066
3.1 1 0 2 -6.27744e+066
0 0 0 0 0

[mecej4]

As the article in #2 declares,

... the user can use types that can be "binary compatible", that is, having the same representation or memory layout.

An example of correct redefinition of MKL_INT for ILP64 if positive integer values less than 2^63 are used:

#define MKL_INT size_t

One point that you may have overlooked is that redefining MKL_INT with a preprocessor #define or using a -D compiler option affects only your code and not MKL, since the latter is already compiled. If you do anything to your code that violates the requirement of binary compatibility, you will probably obtain incorrect answers or program aborts.

By using negative values for a variable declared to be of type size_t, you broke the rules, and this warning/disclaimer in the article applies:

It is the user's responsibility to make a thorough analysis of such cases before redefining data types in Intel MKL.

[SergeyKostrov]

Hi Petros,

Quoting Petros Mamales

I get the (very!!!) erroneous output.

What to do ?
Any workarounds?
TIA,
Petros

PS: Using mkl10.3.8 with vs2010 on win7
The 2 outputs:
int
1 0 1.1 0 0
0 2 0 0 0
3.1 0 3 3.14 0
0 0 0 4 2.17
0 0 5.1 0 5

vs. (size_t)
0 -6.27744e+066 -6.27744e+066 -6.27744e+066 -6.27744e+066
1.1 1 -6.27744e+066 2 -6.27744e+066
0 0 0 -6.27744e+066 -6.27744e+066
3.1 1 0 2 -6.27744e+066
0 0 0 0 0

Did you try toverify sizes of MKL_INT ( not re-defined with your macro! ) and size_t:

...
printf( "MKL_INT: %ld\nsize_t: %ld\n", sizeof( MKL_INT ), sizeof( size_t ));
...

[Petros_Mamales]

Hi Mecej4,
Fair enough. But then, the suggestion of typedef-ing MKL_INT is really of no use, if not misleading.
There are other examples (csrmultcsr, is one of them), where no negative integers are used, and if one tries
to use size_t will get nonsense (wasted a few hours with it ).
I understand the binary layout issue - indicated it in my initial request, where I mentioned it was a far shot.
Thank you for your help,
Petros

[Petros_Mamales]

Hi Sergey,
Thank you for the suggestion. My editor, informs me of the definition of the types.
However, it is a good think to have in mind.
Thank you, again,
Petros

[SergeyKostrov]

Hi Petros,

Here areexamples of declarationsof 'size_t'and 'MKL_INT' for 32-bit and 64-bit platforms:

...
#ifdef _WIN64
typedef unsigned __int64 size_t;
#else
typedefunsigned int size_t;
#endif
...

...
#if defined ( MKL_ILP64 )
#define MKL_INT long long int /* 64 bit integer for large arrays. */
#else
#define MKL_INT int /* 32 bit integer for arrays < 2^31-1 elements. */
#endif
...

If by some reason a32-bittype'int' wasn't used inyour second test it is possible thatyou're dealing with overflowof somevalues and
that is whythe output is wrong.

What are results of your verifications for 'size_t' and 'MKL_INT' types?

Best regards,
Sergey

[Petros_Mamales]

Hi Sergey,
When defining MKL_INT as int they are 4 (MKL_INT) and 8 (size_t) .
When MKL_INT is defined as size_t they are both 8.

Btw, can you please confirm that ddnscsr delivers a dense matrix in C-format (i.e. row-storage) ?

Thank you for all your help,
Petros

ps: Since the compressed sparse format uses arrays of integers, having ones with different element size than MKL_INT is a bad idea. Given this, it is hard to imagine a situation, for those matrices, where defining MKL as size_t would ever be safe.

[Andrey_N_Intel]

Hi Petros,

MKL_INT data type is 8 byteswhen youuse Intel MKL inILP64 mode, and 4 bytes, otherwise (additional details about ILP64 and LP64 modes are available in Intel MKL User Guide, section "Using the ILP64 Interface vs. LP64 Interface").
size_t type is8 bytes for _WIN64, and4 bytes, otherwise(in my version of MSVS2008 the definition of the size_t is in crtdefs.h).

So, when mapping MKL_INTinto size_t on Intel 64 CPUs, please make sureto specify ILP64 modeby defining -DMKL_INT64 and link against Intel Math Kernel Library inILP64 mode (mkl_intel_ilp64.lib,... for static linking).

I took your example, modified it as described in the article

#define MKL_INT size_t
#include

...
and built it using the command
icl -DMKL_ILP64 testcase.cpp mkl_intel_ilp64.lib mkl_core.lib mkl_intel_thread.lib libiomp5md.lib

The output of the example is

1 0 1.1 0 0

0 2 0 0 0

3.1 0 3 3.14 0

0 0 0 4 2.17

0 0 5.1 0 5

I have the same outputwith native MKL_INT.
Please, let me knowif this helps to resolve theissues on your side.

Answering another your question: ddnscsr converter does not deliver a dense matrix in C-format; it delivers the matrix in Fortran format (column-major storage) using0-based indexing scheme.

Please, feel free to ask more questions if any.

Thanks,
Andrey

[Petros_Mamales]

Hi Andrey,
Thank you very much for your response.
In the example I enclosed, the indexing of gedata is done in the row-storage format (C-style) for the
dense matrix:
cout<i.e. consecutive rows (i's) are gelda apart (this is a square matrix).
This rather looks like that it is in C-format. Am I missing something ?

For the rest, I have to delve into the user's guide, some more. Since I am writting plugg-ins for Excel/Python, I am not sure I can go LP64. Have to investigate.
At any rate, this integer mess turns out to be much more involved than I anticipated - and very likely for a good reason.

Thank you again for all your help,
Petros

[Andrey_N_Intel]

Hi Petros,

I double-checkedyour question on the format with the engineer responsible for that functionality, andhe confirms that the converter can return a dense matrix in row-major (that is, C) format.

Can you please provide a bit more details, so we would be able to better understandthe reasons,why you can'tlink against ILP64 interface of Intel MKL (and which would help you to correctly re-define MKL_INT with size_t data type as described in myprevious post)?

Best,
Andrey

[Petros_Mamales]

Hi Andrey,
Thank you for the reconfirmation.
As far as the LP64 issue, I did not say it is not possible. All I mean to say is that it requires further investigationon my part, that is not of the moment - as I commented I was expected the issue to be more light-weight.
Thank you very much for all your help,
Petros

[Andrey_N_Intel]

Hi Petros,
Please, let us know the results of the investigations on your side when ready. It would help us to understand ifthe question of MKL_INT/size_t is properly addressed, or ourfurther helpis necessary.
Best,
Andrey