Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(v2): adaptive sharding #3578

Merged
merged 48 commits into from
Oct 17, 2024
Merged

feat(v2): adaptive sharding #3578

merged 48 commits into from
Oct 17, 2024
+7,825 −634

Commits on Sep 20, 2024

  1. feat: uniform data distribution

    @kolesnikovae
    kolesnikovae committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    3176f76 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into feat/dynamic-sharding 

    # Conflicts:
#	pkg/experiment/distributor/distributor.go
#	pkg/experiment/distributor/placement/placement.go
    @kolesnikovae
    kolesnikovae committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    3958cc3 View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. remove shard shuffling

    @kolesnikovae
    kolesnikovae committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    d8ed814 View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2024

  1. improve distribution

    @kolesnikovae
    kolesnikovae committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    7e71f96 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. add more tests

    @kolesnikovae
    kolesnikovae committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    9f9aef0 View commit details
    Browse the repository at this point in the history

  2. add more tests

    @kolesnikovae
    kolesnikovae committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    c8ff4ff View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. refinements

    @kolesnikovae
    kolesnikovae committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    42d7c5c View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2024

  1. add stats tracking

    @kolesnikovae
    kolesnikovae committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    d0f603a View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2024

  1. ewma improvements

    @kolesnikovae
    kolesnikovae committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    3614a0a View commit details
    Browse the repository at this point in the history

  2. add more unit tests

    @kolesnikovae
    kolesnikovae committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    d96c3be View commit details
    Browse the repository at this point in the history

  3. refactor limits

    @kolesnikovae
    kolesnikovae committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    510b913 View commit details
    Browse the repository at this point in the history

  4. update proto

    @kolesnikovae
    kolesnikovae committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    6b0d7ae View commit details
    Browse the repository at this point in the history

  5. refinements

    @kolesnikovae
    kolesnikovae committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    dc3195e View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2024

  1. Merge branch 'main' into feat/dynamic-sharding

    @kolesnikovae
    kolesnikovae committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    b83dcf2 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2024

  1. add shard allocator

    @kolesnikovae
    kolesnikovae committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    6c15a71 View commit details
    Browse the repository at this point in the history

  2. test cases

    @kolesnikovae
    kolesnikovae committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    c52ddc4 View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2024

  1. add archetypes

    @kolesnikovae
    kolesnikovae committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    2843c1a View commit details
    Browse the repository at this point in the history

  2. refactor placement limits

    @kolesnikovae
    kolesnikovae committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    4b45db6 View commit details
    Browse the repository at this point in the history

  3. add ruler

    @kolesnikovae
    kolesnikovae committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    3337158 View commit details
    Browse the repository at this point in the history

  4. add an option to load rules to ruler

    @kolesnikovae
    kolesnikovae committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    df284e7 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. add ruler unit tests

    @kolesnikovae
    kolesnikovae committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    aa549f0 View commit details
    Browse the repository at this point in the history

  2. decouple distribution stats from metadata records

    @kolesnikovae
    kolesnikovae committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    78fdf2a View commit details
    Browse the repository at this point in the history

  3. expire obsolete placement rules

    @kolesnikovae
    kolesnikovae committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    dcf17f1 View commit details
    Browse the repository at this point in the history

  4. simplify data model

    @kolesnikovae
    kolesnikovae committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    5132af7 View commit details
    Browse the repository at this point in the history

  5. add unit tests for adaptive placement

    @kolesnikovae
    kolesnikovae committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    052ccd3 View commit details
    Browse the repository at this point in the history

  6. add agent

    @kolesnikovae
    kolesnikovae committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    3e07cfa View commit details
    Browse the repository at this point in the history

  7. add manager

    @kolesnikovae
    kolesnikovae committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    b1e4a21 View commit details
    Browse the repository at this point in the history

  8. add confidence period check

    @kolesnikovae
    kolesnikovae committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    9aa43c5 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2024

  1. fix load balancing test

    @kolesnikovae
    kolesnikovae committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    dea41b2 View commit details
    Browse the repository at this point in the history

  2. add manager metrics

    @kolesnikovae
    kolesnikovae committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    1fc1ebb View commit details
    Browse the repository at this point in the history

  3. config options

    @kolesnikovae
    kolesnikovae committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    007e0ed View commit details
    Browse the repository at this point in the history

  4. bind configs and limits

    @kolesnikovae
    kolesnikovae committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    ee5f41d View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into feat/dynamic-sharding 

    # Conflicts:
#	.mockery.yaml
    @kolesnikovae
    kolesnikovae committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    86d01ea View commit details
    Browse the repository at this point in the history

  6. integrate manager and agent

    @kolesnikovae
    kolesnikovae committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    d6b693f View commit details
    Browse the repository at this point in the history

  7. refinements

    @kolesnikovae
    kolesnikovae committed Oct 8, 2024
    Configuration menu
    Copy the full SHA
    5e3b1cd View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2024

  1. optimize data model

    @kolesnikovae
    kolesnikovae committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    64c0403 View commit details
    Browse the repository at this point in the history

  2. remove unused

    @kolesnikovae
    kolesnikovae committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    42bf422 View commit details
    Browse the repository at this point in the history

  3. test load balancing on single shard relocation

    @kolesnikovae
    kolesnikovae committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    fa91ca6 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2024

  1. cleanup mocks

    @kolesnikovae
    kolesnikovae committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    bfb7d55 View commit details
    Browse the repository at this point in the history

  2. implement bucket store

    @kolesnikovae
    kolesnikovae committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    502b0cb View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2024

  1. consistent naming

    @kolesnikovae
    kolesnikovae committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    631fb9d View commit details
    Browse the repository at this point in the history

  2. consistent naming

    @kolesnikovae
    kolesnikovae committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    46e80e1 View commit details
    Browse the repository at this point in the history

  3. ignore placement in all-in-one scenario

    @kolesnikovae
    kolesnikovae committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    de7354b View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2024

  1. fixes

    @kolesnikovae
    kolesnikovae committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    6643da0 View commit details
    Browse the repository at this point in the history

  2. drain in-flight requests in segment writer

    @kolesnikovae
    kolesnikovae committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    45590ae View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2024

  1. add docs

    @kolesnikovae
    kolesnikovae committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    b240aa6 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. add/fix comments

    @kolesnikovae
    kolesnikovae committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    e868fbf View commit details
    Browse the repository at this point in the history

  2. improve key hashing

    @kolesnikovae
    kolesnikovae committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    67217e5 View commit details
    Browse the repository at this point in the history