Speed up backtest in Ensembles

[alex-hse-repository]

🚀 Feature Request

Make the backtest time complexity O(n+m) instead of O(n*m), where n, m - number of folds for the backtest on fit and the actual backtest.

Motivation

Make our ensembles more efficient

Proposal

In BasePipeline:

1. Add attribute:
   • fold_id: Optional[int] = None
2. In method _run_fold:
   • Add parameter fold_id: int to the signature
   • Set attribute fold_id after this line
3. In method backtest:
   • Pass fold_id to the _run_fold here

In StackingEnsemble and VotingEnsemble:

1. Change the signature of _backtest_pipeline and make it static:
   • _backtest_pipeline(pipeline: BasePipeline, ts: TSDataset, n_folds: Union[int, List[FoldMask]])
2. Move it to EnsembleMixin
3. Improve the call of this method everywhere(pass self.n_folds)
4. Add attribute:
   • forecasts: Optional[List[List["TSDataset"]]] = None

In VotingEnsemble:

1. Override method backtest:

• If self.weights="auto" and n_folds is integer:
  • Set self.forecasts using the _backtest_pipeline with n_folds = n_folds + self.n_folds
• Call the backtest of the superclass with the same parameters
• Set self.forecasts = None

2. Change method fit:

• Fit the pipelines here only if self.forecasts = None

3. Change method _process_weights:

• If self.forecasts != None it should get the nessesary forecasts from self.forecasts using the self.fold_id here

5. Change method _forecast:

• If self.forecasts != None it should get the necessary forecasts from self.forecasts using the self.fold_id here

In StackingEnsemble:

1. Override method backtest:

• If n_folds is integer:
  • Set self.forecasts using the _backtest_pipeline with n_folds = n_folds + self.n_folds
• Call the backtest of the superclass with the same parameters
• Set self.forecasts = None

2. Change method fit:

• If self.forecasts != None it should get the necessary forecasts from self.forecasts using the self.fold_id here
• Fit the pipelines here only if self.forecasts = None

3. Change method _forecast:

• If self.forecasts != None it should get the nessesary forecasts from self.forecasts using the self.fold_id here

Notes:

1. In all the places the comparison with None should be done using is or is not
2. In all the places where self.forecasts = None leave the old logic

Test cases

No response

Alternatives

No response

Additional context

No response

Checklist

• I discussed this issue with ETNA Team

[alex-hse-repository]

Will be closed with adding caching in #655