MKL - LLS Coefficients

[atseifu]

Hello,
I am trying to write a curve-fitting routines as a part of DAQ, using the MKL LLS Driver routine. The results (coefficients) obtained using the library are drastically different, when compared to a commercially available software. (You guessed it already). The commercial program claims to use 80bit math (???).
Any suggestions on what I can do to improve my results?
Below is some more information:
1. Platform: Windows XP SP3, VS 2008 Pro, MKL Trial, Intel C++ Compiler Trial, 16k data points, 6 coefficient equation
2. C++ program calling MKL LLS, also tried the Qpc fp compiler options.

Apologies in advance if this post belongs to another forum.

Thank you.

[ArturGuzik]

Quoting - atseifu

Hello,
I am trying to write a curve-fitting routines as a part of DAQ, using the MKL LLS Driver routine. The results (coefficients) obtained using the library are drastically different, when compared to a commercially available software. (You guessed it already). The commercial program claims to use 80bit math (???).
Any suggestions on what I can do to improve my results?
Below is some more information:
1. Platform: Windows XP SP3, VS 2008 Pro, MKL Trial, Intel C++ Compiler Trial, 16k data points, 6 coefficient equation
2. C++ program calling MKL LLS, also tried the Qpc fp compiler options.

Apologies in advance if this post belongs to another forum.

Thank you.

Hi,

(1) 80-bit math sounds like extended precision calculations, but for C this should be long double (???).

(2) what do you mean by drastically different?

A.

[atseifu]

Quoting - ArturGuzik

Hi,

(1) 80-bit math sounds like extended precision calculations, but for C this should be long double (???).

(2) what do you mean by drastically different?

A.

Hello,

1. Yes, it is (probably) 80 bit math. But how do I use MKL with 80 bit math?
2. The coefficients obtained using SVD for LLS from the two programs (for a given equation form) differ by more than 50%. The 80 bit calculation results in an excellent fit.

Any insight/advice is greatly appreciated.
Thank you.

[ArturGuzik]

Quoting - atseifu

Hello,

1. Yes, it is (probably) 80 bit math. But how do I use MKL with 80 bit math?
2. The coefficients obtained using SVD for LLS from the two programs (for a given equation form) differ by more than 50%. The 80 bit calculation results in an excellent fit.

Any insight/advice is greatly appreciated.
Thank you.

1. Probably you can't (but I might be wrong here, and someone from Intel will provide more reliable answer to this)
2. Can you elaborate more on that? Which routines from MKL you're using? dtrnlsp_**** or just call SVD and get the (Moore-Penrose) pseudo-inverse matrix yourself to solve OLS problem? Do you input exactly the same Jacobian for both (commercial code) and your MKL version (probably commercial code gets the Jacobian itself)?

A.

[atseifu]

Quoting - ArturGuzik

1. Probably you can't (but I might be wrong here, and someone from Intel will provide more reliable answer to this)
2. Can you elaborate more on that? Which routines from MKL you're using? dtrnlsp_**** or just call SVD and get the (Moore-Penrose) pseudo-inverse matrix yourself to solve OLS problem? Do you input exactly the same Jacobian for both (commercial code) and your MKL version (probably commercial code gets the Jacobian itself)?

A.

I am using the Lapack LLS Driver routine(s) DGELS and DGELSS, from which I get the coefficients directly. The commercial code takes data set as input and gives out the coefficients, i.e., I dont deal with internal calculations. All these are linear fits in 2 and 3 dimensions.

Thank you.

[ArturGuzik]

Quoting - atseifu

I am using the Lapack LLS Driver routine(s) DGELS and DGELSS, from which I get the coefficients directly.

OK then, so there is no issue with the algorithm. There are two possibilities:
(1) your Jacobian has a (minor) bug somewhere (say one derrivative is incorrect, has wrong sign etc.)
(2) you really need an extended precision. What are actuall values of the coefficients? Tiny? Any normalization possible?

I assume you already tested your code on (some smaller and with known solution) example.

A.

[atseifu]

Quoting - ArturGuzik

OK then, so there is no issue with the algorithm. There are two possibilities:
(1) your Jacobian has a (minor) bug somewhere (say one derrivative is incorrect, has wrong sign etc.)
(2) you really need an extended precision. What are actuall values of the coefficients? Tiny? Any normalization possible?

I assume you already tested your code on (some smaller and with known solution) example.

A.

Thanks for the prompt response.
1. I dont calculate any Jacobians or derivatives. I just pass the coefficient matrix and the RHS matrix to the SVD driver.
2. The actual values in this case range from 0.01 to 150.

I havent really tested the code (which is the MKL) on smaller problems as such, but for several problems that I am working with, the commercial tool provides better coefficients.
I am assuming it has something to do with the 80bit calculation.

Thanks again.

[ArturGuzik]

Quoting - atseifu

Thanks for the prompt response.
1. I dont calculate any Jacobians or derivatives. I just pass the coefficient matrix and the RHS matrix to the SVD driver.
2. The actual values in this case range from 0.01 to 150.

OK. I have to admit I don't understand it.

(1) If you have data fit, you need to calculate Jacobian and then solve LS problem. In your case is linear (means Jacobian doesn't depend on unknowns). The input to gelss (as A matrix) is the Jacobian, as far as I know.

(2) If you want to use SVD yourself to do exactly the same, the SVD of Jacobian gives you A = U*S*V^T and the LS solution of your problem is {x} = [U^T*(S^+)*V]*{y}, where {y} is vector containing your "observations".

I might be very, very wrong, but have an impression you're mixing something and that's why your MKL version gives you wrong (50% difference is just this) results.

A.

[atseifu]

Quoting - ArturGuzik

OK. I have to admit I don't understand it.

(1) If you have data fit, you need to calculate Jacobian and then solve LS problem. In your case is linear (means Jacobian doesn't depend on unknowns). The input to gelss (as A matrix) is the Jacobian, as far as I know.

(2) If you want to use SVD yourself to do exactly the same, the SVD of Jacobian gives you A = U*S*V^T and the LS solution of your problem is {x} = [U^T*(S^+)*V]*{y}, where {y} is vector containing your "observations".

I might be very, very wrong, but have an impression you're mixing something and that's why your MKL version gives you wrong (50% difference is just this) results.

A.

I apologize for the confusion, as I am not fully aware of the terminology.
I tried something else yesterday: I used Excel to fit the data with the same equation using the Regression function. I get the same coefficients as the SVD routine from the MKL. So I think, my code for calling the MKL SVD/LLS is correct.
Now (I think) the issue is only to somehow call the MKL routines in 80bit precision mode.
Thank you.

[ArturGuzik · Accepted solution]

Quoting - atseifu

I tried something else yesterday: I used Excel to fit the data with the same equation using the Regression function. I get the same coefficients as the SVD routine from the MKL. So I think, my code for calling the MKL SVD/LLS is correct.
Now (I think) the issue is only to somehow call the MKL routines in 80bit precision mode.

OK then. If for fitting to the straight line Excel and MKL (what about the commercial code here?) gave the same values, while when using the full function (6 params) gave you different results it must be something else (precision...) involved. It is still hard to believe that double precision and extended precision results in 50% error (in range 0.01 to 150) but.... unless you show some code and data we need to leave it as it is.

As I wrote, I believe, you can't use 80-bit math with MKL. The closest you can get, is to download SVD routines source (from Netlib) and convert them to 80-bit and then use if for OLS (as I described in previous post).

A.