GNU Octave gives different results with MKL

[ArchismanPanigrahi]

I am using GNU Octave 4.2.2 in Linux Mint 19.3 (which is based on Ubuntu 18.04).

When I use Octave with MKL ( I downloaded the online installer, installed in /opt directory, and followed https://askubuntu.com/a/913029/124466), I get wrong results when working with big matrices.

This is my code.

for a = 1:500
for b = 1:500
c(a,b) = sin(a + b^2);
endfor
endfor

g = eig(c);
max(real(g))

Without MKL (using default openblas), the output is  "ans = 16.915"
With MKL, the output is "ans = 1.5922e+06" (but this answer keeps changing everytime the code is run!!!).

I also checked some of my other codes (calculating eigenvalues and eigenvectors of certain matrices), and MKL always gives wrong results when the system size is big. MKL seems to diagonalize sparce matrices much faster, but it has become useless.

[mecej4]

You are probably blaming MKL without justification. I have not used Octave in two decades, and do not have it installed. Here, however, is a simple Fortran program that when compiled and linked with MKL gives the expected result.

program archi
use lapack95
implicit none
integer i,j
double precision g(500,500)
double precision wr(500),wi(500)
do i=1,500
   do j=1,500
      g(i,j) = sin(dble(i)+j*j)
   end do
end do
call gees(g,wr,wi)
print *,maxval(wr)
end program

The output:

   16.9148864979302

[ArchismanPanigrahi]

It is still present with the `intel-mkl` packages from the default Ubuntu repositories in Xubuntu 20.04.

Is this related to this Debian Bug? https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=921207

@mecej4 By the way, I want to double check whether the bug is only present in Octave+MKL, or always with MKL. How do I compile your Fortran code with MKL in Ubuntu?

[mecej4]

Use the MKL link advisor at https://software.intel.com/content/www/us/en/develop/articles/intel-mkl-link-line-advisor.html  to find out the command line to compile and link a Fortran program. I have used the Lapack95 interface layer, and used Ifort 19.1U2 on Windows 10.

You can also write the equivalent C code to solve the same problem. Whether you use C, Fortran and whether you use the GNU or Intel compilers, the underlying computational routines of MKL will be the same, so the results should match.

I do not think that you will find any errors in a mature product such as MKL when you use direct calls in Fortran or C to MKL routines and the calls are correctly formulated and the arguments passed are correct.

Lots of problems can occur when a package such as Matlab or Octave is used with MKL, and those problems are best addressed in places such as an Octave users' forum. 

[ArchismanPanigrahi]

I have opened a bug report with Octave, and someone provided an workaround.

Enter the command

export MKL_THREADING_LAYER=gnu

and run "octave" from Terminal. Now it should work, and the speedup of MKL still remains!!

To make this fix permanent, add the "export" line to the .bashrc file.

However, this workaround should be at https://software.intel.com/content/www/us/en/develop/articles/using-intel-mkl-in-gnu-octave.html so that user's will know about it. This issue is absent for small matrices, so it might go unnoticed by the users, and they might get totally wrong results.

[mecej4]

So, the cause is an inconsistency/clash between two different threading models -- GNU Openmp vs. Intel Openmp. Thanks for reporting the workaround. 

I agree that users of MKL on Linux should be informed about this somewhere (e.g., in the installation instructions for MKL). Unfortunately, as the years go by such information may become out of date or invalid, and the links may cease to point to an existing page.

After looking at your post and the responses at savannah.gnu.org, I must say that the people there are very knowledgeable and helpful!

[ArchismanPanigrahi]

This test script has been updated with results from Python, Julia, Scilab, R and Octave  https://gist.github.com/N0rbert/cfda101b8f0aa326df1edb6beee0076d

It has been found that both Octave and R require the environment variable to be set, in order to produce correct results in the test script.

However, even after setting environment variable, Octave undergoes segmentation fault (see https://savannah.gnu.org/bugs/?58926#comment10) when __run_test_suite__ is executed.

It has been reported by Norbert (author of the gist) that scilab has more test fails under MKL than under OpenBLAS.

Softwares like Octave, Scilab and R give correct results with OpenBLAS, but they sometimes fail with MKL in Ubuntu and Debian.

I am not saying MKL has bugs. Most likely there is some issue in the interfacing (e.g. the clash between GNU Openmp vs. Intel Openmp, but Octave still shows segmentation fault in its test run after using the workaround).

Maybe only prebuilt binaries (installed from repositories) are affected, and maybe this issue won't arise when these softwares are built from source (I don't know, I am only speculating), but this is a very serious issue, and should be documented somewhere, because most users would tend to use the prebuilt binaries in distros like Ubuntu.

Now that R is also affected, I would like to request the MKL community to investigate the issue further, and document it in pages like https://software.intel.com/content/www/us/en/develop/articles/using-intel-mkl-in-gnu-octave.html so that more users can avoid this bug.