Issue with L2 regularization and intercept flag settings for MSE objective function

[Gusev__Dmitry]

Hello,

Can anyone please verify if the Mean Squared Error objective function with SAGA algorithm supports L2 regularization and the intercept flag settings?

I am getting the same results regardless of the interceptFlag and penaltyL2 settings.

My code is below:

        services::SharedPtr<mse::Batch<> > constrMSEObjFunc(new mse::Batch<>(nObs));
        constrMSEObjFunc->input.set(mse::data, data);
        constrMSEObjFunc->input.set(mse::dependentVariables, dependentVariables);

        constrMSEObjFunc->parameter().interceptFlag= false;
        constrMSEObjFunc->parameter().penaltyL2 = NumericTablePtr(new HomogenNumericTable<>(0, nVar, NumericTable::doAllocate, penaltyL2));

        
        saga::Batch<> sagaAlgorithm(constrMSEObjFunc);
        sagaAlgorithm.input.set(iterative_solver::inputArgument,NumericTablePtr(new HomogenNumericTable<DAAL_ALGORITHM_FP_TYPE>(initPoints, 1, nVar + 1)));
        
        sagaAlgorithm.parameter().nIterations = nIterations;
        sagaAlgorithm.parameter().batchSize = 1;
        sagaAlgorithm.parameter().accuracyThreshold = accuracyThreshold;

        s = sagaAlgorithm.compute();

Any thoughts or recommendations would be highly appreciated.

Regards,

Dmitry.

[Adweidh_Intel]

Hi Dmitry,

We have forwarded this case to SME, they will get back to you soon.

[Gusev__Dmitry]

Hello,

I would like to correct a typo in my previous post - correct statement for penalty setting is below

constrMSEObjFunc->parameter().penaltyL2 = NumericTablePtr(new HomogenNumericTable<>(1,1, NumericTable::doAllocate, penaltyL2));

It is still not clear how to set up MSE objective function with L2 penalty for the SAGA solver   - the solver returns the same results  regardless of the interceptFlag and penaltyL2 settings.

Can someone clarify how to initialize  MSE function with L2 penalty to use it with the SAGA solver?

Thanks,

Dmitry

[Gusev__Dmitry]

Hi Adweidh and everyone.

I am still waiting for a reply. I need Intel SMEs to address the issues I described,  it looks to me as it could be a serious problem with the MSE function implementation.

Can you please assist with this problem investigation?

Let me know if you need more information.

Looking forward to hear back from you or other Intel SMEs.

Thanks for your help,

Dmitry.

[Shailen_Sobhee]

Hi Dmitry,

We are looking into the details and syncing with an internal DAAL expert and will get back to you as soon as possible. 

Kind regards,

Shailen.

[Kirill_S_Intel]

Hi Dmitry,

Sorry for long response again.

Initially DAAL MSE function was implemented without any regularization parameters [https://software.intel.com/content/www/us/en/develop/documentation/daal-programming-guide/top/algorithms/analysis/optimization-solvers/objective-function/mean-squared-error-algorithm.html]. And the focus of the latest updates of DAAL MSE function was introducing possibility to calculate gradient, hessian diagonal, proximal projection, respectively to specific coordinate.

Currently there is no possibility to initialize MSE function with L2 penalty to use it with the SAGA solver (only Coordinate Descent handles it). But it will be fixed in further releases, thanks a lot for reporting this problem.

Best regards,

Kirill