Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Custom Training] Add custom training for Cross Validation #334

Open
VictorMichonIntelcom opened this issue Apr 10, 2024 · 0 comments
Open

Comments

@VictorMichonIntelcom
Copy link

VictorMichonIntelcom commented Apr 10, 2024

Description

The current version of MLForecast does not support applying custom weights during cross-validation training. The custom training feature described in the documentation is only available for the final model training, not for the cross-validation process.

This is a limitation of the current implementation, as the cross-validation in MLForecast is designed to use the default training procedure without any customization. The cross-validation is used for model selection and hyperparameter tuning, where the ability to apply custom weights may be beneficial for certain use cases.

Use case

If I want to apply some weight on samples during training, in order to improve my forecasts on a given set of dates/periods, I need to use the custom training feature you explained in this guide https://nixtlaverse.nixtla.io/mlforecast/docs/how-to-guides/custom_training.html.
However, in the case I want to perform model selection over cross validation scores of multiple models, I need to be able to reproduce the same customed training during cross validation. This is where I am currently blocked, as the current version of MLForecast does not support applying custom weights during the cross-validation process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant