Intel® oneAPI Data Analytics Library
Learn from community members on how to build compute-intensive applications that run efficiently on Intel® architecture.

Issue with L2 regularization and intercept flag settings for MSE objective function

Gusev__Dmitry
Beginner
1,569 Views

Hello,

Can anyone please verify if the Mean Squared Error objective function with SAGA algorithm supports L2 regularization and the intercept flag settings?

I am getting the same results regardless of the interceptFlag and penaltyL2 settings.


My code is below:

        services::SharedPtr<mse::Batch<> > constrMSEObjFunc(new mse::Batch<>(nObs));
        constrMSEObjFunc->input.set(mse::data, data);
        constrMSEObjFunc->input.set(mse::dependentVariables, dependentVariables);

 

        constrMSEObjFunc->parameter().interceptFlag= false;
        constrMSEObjFunc->parameter().penaltyL2 = NumericTablePtr(new HomogenNumericTable<>(0, nVar, NumericTable::doAllocate, penaltyL2));

        
        saga::Batch<> sagaAlgorithm(constrMSEObjFunc);
        sagaAlgorithm.input.set(iterative_solver::inputArgument,NumericTablePtr(new HomogenNumericTable<DAAL_ALGORITHM_FP_TYPE>(initPoints, 1, nVar + 1)));
        
        sagaAlgorithm.parameter().nIterations = nIterations;
        sagaAlgorithm.parameter().batchSize = 1;
        sagaAlgorithm.parameter().accuracyThreshold = accuracyThreshold;

        

        s = sagaAlgorithm.compute();

Any thoughts or recommendations would be highly appreciated.

Regards,

Dmitry.

0 Kudos
5 Replies
Adweidh_Intel
Moderator
1,569 Views

Hi Dmitry,

We have forwarded this case to SME, they will get back to you soon.

0 Kudos
Gusev__Dmitry
Beginner
1,569 Views

Hello,

I would like to correct a typo in my previous post - correct statement for penalty setting is below

constrMSEObjFunc->parameter().penaltyL2 = NumericTablePtr(new HomogenNumericTable<>(1,1, NumericTable::doAllocate, penaltyL2));

It is still not clear how to set up MSE objective function with L2 penalty for the SAGA solver   - the solver returns the same results  regardless of the interceptFlag and penaltyL2 settings.

Can someone clarify how to initialize  MSE function with L2 penalty to use it with the SAGA solver?

Thanks,

Dmitry

 

0 Kudos
Gusev__Dmitry
Beginner
1,569 Views

Hi Adweidh and everyone.

I am still waiting for a reply. I need Intel SMEs to address the issues I described,  it looks to me as it could be a serious problem with the MSE function implementation.

Can you please assist with this problem investigation?

Let me know if you need more information.

Looking forward to hear back from you or other Intel SMEs.

Thanks for your help,

Dmitry.

0 Kudos
Shailen_Sobhee
Employee
1,569 Views

Hi Dmitry,

We are looking into the details and syncing with an internal DAAL expert and will get back to you as soon as possible. 

Kind regards,

Shailen.

0 Kudos
Kirill_S_Intel
Employee
1,569 Views

Hi Dmitry,

Sorry for long response again.

Initially DAAL MSE function was implemented without any regularization parameters [https://software.intel.com/content/www/us/en/develop/documentation/daal-programming-guide/top/algorithms/analysis/optimization-solvers/objective-function/mean-squared-error-algorithm.html]. And the focus of the latest updates of DAAL MSE function was introducing possibility to calculate gradient, hessian diagonal, proximal projection, respectively to specific coordinate.

Currently there is no possibility to initialize MSE function with L2 penalty to use it with the SAGA solver (only Coordinate Descent handles it). But it will be fixed in further releases, thanks a lot for reporting this problem.

 

Best regards,

Kirill

 

0 Kudos
Reply