Ridge regression - loss of accuracy between Composer 2017 up4 and 2018 up3

[Guillaume_A_]

Hi,

I notice a loss of accuracy and some unexpected crashes of the ridge regression learning when updating Composer from 2017 up4 to 2018 up3

Here is my code:

int ridgeLearn(double* ouputRegressionCoeffs, double* const trainingFeatureSamples, int nTrainingFeatures, double* const trainingTargetValues, int nTrainingSamples, double regularizationPar)
{
  const NumericTablePtr featureSamplesTable(new HomogenNumericTable<double>(trainingFeatureSamples, nTrainingFeatures, nTrainingSamples));
  const NumericTablePtr targetValuesTable(new HomogenNumericTable<double>(trainingTargetValues, 1, nTrainingSamples));

  algorithms::ridge_regression::training::Batch<> algorithm;

  // Pass a training data set to the algorithm
  algorithm.input.set(algorithms::ridge_regression::training::data, featureSamplesTable);
  algorithm.input.set(algorithms::ridge_regression::training::dependentVariables, targetValuesTable);

  // Ridge model config  
  const auto regularizationParTbl = new double[1]; regularizationParTbl[0] = regularizationPar;
  const NumericTablePtr ridgeParameter(new HomogenNumericTable<double>(regularizationParTbl, 1, 1));
  algorithm.parameter.ridgeParameters = ridgeParameter;

  // Composer 2018
  auto status = algorithm.compute(); bool isOk = status.ok();
  /*
  // Composer 2017
  algorithm.compute(); bool isOk = algorithm.getErrors()->isEmpty();
  */
  
  if(isOk)
  {
    const SharedPtr<algorithms::ridge_regression::training::Result> result = algorithm.getResult();
    NumericTablePtr resultCoeffs = result->get(algorithms::ridge_regression::training::model)->getBeta();

    BlockDescriptor<double> block;
    resultCoeffs->getBlockOfRows(0, resultCoeffs->getNumberOfRows(), readOnly, block);
    memcpy(ouputRegressionCoeffs, block.getBlockPtr(), block.getNumberOfColumns() * sizeof(double));
  }
  delete[] regularizationParTbl;

  return isOk ? 1:0;
}

void RidgeRegressionTest0()
{
  int nRows = 10;
  int nCols = 2;
  double * features = new double[nRows * nCols];
  double * labels = new double[nRows];
  for (int i = 0; i < nRows; ++i)
  {
    features[i * nCols]     = i + 1.0;  // x1
    features[i * nCols + 1] = i + 1.0;  // x2
    labels               = 4.0*(i+1);    // 0.0 + 4.0*x
  }
  double * beta = new double[nCols + 1];
  int statusResult = ridgeLearn(beta, features, nCols, labels, nRows, 1e-7);

  printf("Compute status: %d\n", statusResult);
  printf("result:\t%.16lf\t%.16lf\t%.16lf\n", beta[0], beta[1], beta[2]);
  printf("expect:\t%.16lf\t%.16lf\t%.16lf\n", 0.0, 2.0, 2.0);

  delete[] features;
  delete[] labels;
  delete[] beta;
}

void RidgeRegressionTest1()
{
  int nRows = 10;
  int nCols = 2;
  double * features = new double[nRows * nCols];
  double * labels = new double[nRows];
  for (int i = 0; i < nRows; ++i)
  {
    features[i * nCols]     = i + 1.0;                              // x1
    features[i * nCols + 1] = (i + 1.0) *(i + 1.0);                 // x2^2
    labels               = 3.0 * (i + 1) + (i + 1.0)*(i + 1.0);  // 0.0 + 3.0*x + x^2
  }
  double * beta = new double[nCols + 1];
  int statusResult = ridgeLearn(beta, features, nCols, labels, nRows, 1e-7);

  printf("Compute status: %d\n", statusResult);
  printf("result:\t%.16lf\t%.16lf\t%.16lf\n", beta[0], beta[1], beta[2]);
  printf("expect:\t%.16lf\t%.16lf\t%.16lf\n", 0.0, 3.0, 1.0);

  delete[] features;
  delete[] labels;
  delete[] beta;
}

RidgeRegressionTest0() should return : beta[0] = 0.0, beta[1] = 2.0, beta[2] = 2.0      (y = 0 + 4x)

• Composer 2017 outputs are good:  beta[0] = 0.0000000133333383    beta[0] = 2.0000009558174350    beta[0] = 1.999999041758322
• Composer 2018 returns an error: "Failed to solve the system of normal equations"

RidgeRegressionTest1() should return : beta[0] = 0.0, beta[1] = 3.0, beta[2] = 1.0       (y = 0 + 3x + x^2)

• Composer 2017 outputs are good:             beta[0] = 0.0000001533332496    beta[0] = 2.9999999296970059   beta[0] = 1.0000000060606029
• Composer 2018 outputs are not so good:  beta[0] = -0.0001530328445369   beta[0] = 3.0000724792480469   beta[0] = 0.9999936223030090

Are my inputs wrong for the new version ?

Can someone help me ?

My configuration:

• Composer 2017.4.210 and 2018.3.210
• BasePlatformToolset = V120
• Windows 10 Entreprise (version 1703, OS build 15063.1155)
• i7-6820HQ

Thanks,

Guillaume A.

[Gennady_F_Intel]

thanks Guillaume, we will check the problem on our side and  back asap.

[Gennady_F_Intel · Accepted solution]

this happens because of since DAAL 2018 the default precision has been changed since double to float. 

[Guillaume_A_]

That was obvious, my bad ...

Thanks you !