Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix low speed of lagged implementation #1144

Merged
merged 3 commits into from
Aug 15, 2023
Merged

Fix low speed of lagged implementation #1144

merged 3 commits into from
Aug 15, 2023

Conversation

kasyanovse
Copy link
Collaborator

Replace chain concatenation with pandas by numpy bulk realization.

@kasyanovse kasyanovse linked an issue Aug 14, 2023 that may be closed by this pull request
@aim-pep8-bot
Copy link

aim-pep8-bot commented Aug 14, 2023

Hello @kasyanovse! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2023-08-15 07:20:28 UTC

@codecov
Copy link

codecov bot commented Aug 14, 2023

Codecov Report

Merging #1144 (6ae8f6c) into master (89ff552) will decrease coverage by 0.21%.
Report is 2 commits behind head on master.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1144      +/-   ##
==========================================
- Coverage   78.67%   78.47%   -0.21%     
==========================================
  Files         131      130       -1     
  Lines        9362     9323      -39     
==========================================
- Hits         7366     7316      -50     
- Misses       1996     2007      +11     
Files Changed Coverage Δ
...lementations/data_operations/ts_transformations.py 77.65% <100.00%> (-0.77%) ⬇️

... and 16 files with indirect coverage changes

@kasyanovse
Copy link
Collaborator Author

kasyanovse commented Aug 15, 2023

Speed test. Also some models are speeded up too because they use ts_to_table function that is implement in module with lagged.

Code (for IPython)
import numpy as np

from examples.simple.time_series_forecasting.ts_pipelines import ts_polyfit_pipeline, ts_complex_ridge_pipeline
from fedot.core.data.data import InputData
from fedot.core.repository.dataset_types import DataTypesEnum
from fedot.core.repository.tasks import Task, TaskTypesEnum, TsForecastingParams

series = np.random.rand(100000)
data = InputData(idx=np.arange(series.shape[0]),
               features=series, target=series,
               task=Task(TaskTypesEnum.ts_forecasting,
                         TsForecastingParams(forecast_length=100)),
               data_type=DataTypesEnum.ts)

pipeline = ts_polyfit_pipeline(2)
%timeit pipeline.fit(data)

pipeline = ts_complex_ridge_pipeline()
%timeit pipeline.fit(data)

Results.

  1. Old lagged
    1. 2.28 s ± 232 ms for polyfit
    2. 2.39 s ± 62.3 ms for lagged + ridge
  2. New lagged
    1. 48.3 ms ± 1.15 ms for polyfit
    2. 605 ms ± 11.3 ms for lagged + ridge

Pipeline with ridge spends half of all time for ridge fitting, therefore speed up is not so high.

@kasyanovse kasyanovse merged commit 0eb258b into master Aug 15, 2023
5 checks passed
@kasyanovse kasyanovse deleted the 1140-lagged-fix branch August 15, 2023 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Lagged node spends a lot of time
3 participants