Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Beginner
193 Views

Issue with L2 regularization and intercept flag settings for MSE objective function

Hello,

Can anyone please verify if the Mean Squared Error objective function with SAGA algorithm supports L2 regularization and the intercept flag settings?

I am getting the same results regardless of the interceptFlag and penaltyL2 settings.


My code is below:

        services::SharedPtr<mse::Batch<> > constrMSEObjFunc(new mse::Batch<>(nObs));
        constrMSEObjFunc->input.set(mse::data, data);
        constrMSEObjFunc->input.set(mse::dependentVariables, dependentVariables);

 

        constrMSEObjFunc->parameter().interceptFlag= false;
        constrMSEObjFunc->parameter().penaltyL2 = NumericTablePtr(new HomogenNumericTable<>(0, nVar, NumericTable::doAllocate, penaltyL2));

        
        saga::Batch<> sagaAlgorithm(constrMSEObjFunc);
        sagaAlgorithm.input.set(iterative_solver::inputArgument,NumericTablePtr(new HomogenNumericTable<DAAL_ALGORITHM_FP_TYPE>(initPoints, 1, nVar + 1)));
        
        sagaAlgorithm.parameter().nIterations = nIterations;
        sagaAlgorithm.parameter().batchSize = 1;
        sagaAlgorithm.parameter().accuracyThreshold = accuracyThreshold;

        

        s = sagaAlgorithm.compute();

Any thoughts or recommendations would be highly appreciated.

Regards,

Dmitry.

Tags (1)
0 Kudos
5 Replies
Highlighted
Moderator
193 Views

Hi Dmitry,

We have forwarded this case to SME, they will get back to you soon.

0 Kudos
Highlighted
Beginner
193 Views

Hello,

I would like to correct a typo in my previous post - correct statement for penalty setting is below

constrMSEObjFunc->parameter().penaltyL2 = NumericTablePtr(new HomogenNumericTable<>(1,1, NumericTable::doAllocate, penaltyL2));

It is still not clear how to set up MSE objective function with L2 penalty for the SAGA solver   - the solver returns the same results  regardless of the interceptFlag and penaltyL2 settings.

Can someone clarify how to initialize  MSE function with L2 penalty to use it with the SAGA solver?

Thanks,

Dmitry

 

0 Kudos
Highlighted
Beginner
193 Views

Hi Adweidh and everyone.

I am still waiting for a reply. I need Intel SMEs to address the issues I described,  it looks to me as it could be a serious problem with the MSE function implementation.

Can you please assist with this problem investigation?

Let me know if you need more information.

Looking forward to hear back from you or other Intel SMEs.

Thanks for your help,

Dmitry.

0 Kudos
Highlighted
Employee
193 Views

Hi Dmitry,

We are looking into the details and syncing with an internal DAAL expert and will get back to you as soon as possible. 

Kind regards,

Shailen.

0 Kudos
Highlighted
Employee
193 Views

Hi Dmitry,

Sorry for long response again.

Initially DAAL MSE function was implemented without any regularization parameters [https://software.intel.com/content/www/us/en/develop/documentation/daal-programming-guide/top/algorithms/analysis/optimization-solvers/objective-function/mean-squared-error-algorithm.html]. And the focus of the latest updates of DAAL MSE function was introducing possibility to calculate gradient, hessian diagonal, proximal projection, respectively to specific coordinate.

Currently there is no possibility to initialize MSE function with L2 penalty to use it with the SAGA solver (only Coordinate Descent handles it). But it will be fixed in further releases, thanks a lot for reporting this problem.

 

Best regards,

Kirill

 

0 Kudos