Can anyone please verify if the Mean Squared Error objective function with SAGA algorithm supports L2 regularization and the intercept flag settings?
I am getting the same results regardless of the interceptFlag and penaltyL2 settings.
My code is below:
services::SharedPtr<mse::Batch<> > constrMSEObjFunc(new mse::Batch<>(nObs));
constrMSEObjFunc->parameter().penaltyL2 = NumericTablePtr(new HomogenNumericTable<>(0, nVar, NumericTable::doAllocate, penaltyL2));
sagaAlgorithm.input.set(iterative_solver::inputArgument,NumericTablePtr(new HomogenNumericTable<DAAL_ALGORITHM_FP_TYPE>(initPoints, 1, nVar + 1)));
sagaAlgorithm.parameter().nIterations = nIterations;
sagaAlgorithm.parameter().batchSize = 1;
sagaAlgorithm.parameter().accuracyThreshold = accuracyThreshold;
s = sagaAlgorithm.compute();
Any thoughts or recommendations would be highly appreciated.
I would like to correct a typo in my previous post - correct statement for penalty setting is below
constrMSEObjFunc->parameter().penaltyL2 = NumericTablePtr(new HomogenNumericTable<>(1,1, NumericTable::doAllocate, penaltyL2));
It is still not clear how to set up MSE objective function with L2 penalty for the SAGA solver - the solver returns the same results regardless of the interceptFlag and penaltyL2 settings.
Can someone clarify how to initialize MSE function with L2 penalty to use it with the SAGA solver?
Hi Adweidh and everyone.
I am still waiting for a reply. I need Intel SMEs to address the issues I described, it looks to me as it could be a serious problem with the MSE function implementation.
Can you please assist with this problem investigation?
Let me know if you need more information.
Looking forward to hear back from you or other Intel SMEs.
Thanks for your help,
Sorry for long response again.
Initially DAAL MSE function was implemented without any regularization parameters [https://software.intel.com/content/www/us/en/develop/documentation/daal-programming-guide/top/algorithms/analysis/optimization-solvers/objective-function/mean-squared-error-algorithm.html]. And the focus of the latest updates of DAAL MSE function was introducing possibility to calculate gradient, hessian diagonal, proximal projection, respectively to specific coordinate.
Currently there is no possibility to initialize MSE function with L2 penalty to use it with the SAGA solver (only Coordinate Descent handles it). But it will be fixed in further releases, thanks a lot for reporting this problem.