Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lightning: support prepared statement and client stmt cache in logical import mode #55482

Merged
merged 29 commits into from
Sep 21, 2024
Merged

lightning: support prepared statement and client stmt cache in logical import mode #55482

merged 29 commits into from
Sep 21, 2024

Commits on Aug 17, 2024

  1. connection pooling 

    set db properties in DBFromConfig

increase minDeliverBytes to 10m

add log to show session variable

support prepared statement in tidb lightning mode

fix bugs

try to reduce the inuse memory

try to avoid oom

Update import.go

change lru cache to kvcache

add configuration LogicalImportPrepStmt

add logical logicalImportPrepStmt config

add initial test

test single statement

fix the bug in test
    @dbsid
    dbsid committed Aug 17, 2024
    Configuration menu
    Copy the full SHA
    d457da8 View commit details
    Browse the repository at this point in the history

  2. cleanup

    @dbsid
    dbsid committed Aug 17, 2024
    Configuration menu
    Copy the full SHA
    8412357 View commit details
    Browse the repository at this point in the history

  3. fix bug

    @dbsid
    dbsid committed Aug 17, 2024
    Configuration menu
    Copy the full SHA
    b0c79df View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2024

  1. fix concurrent map writes

    @dbsid
    dbsid committed Aug 18, 2024
    Configuration menu
    Copy the full SHA
    7237d94 View commit details
    Browse the repository at this point in the history

  2. fix bug: add missing config

    @dbsid
    dbsid committed Aug 18, 2024
    Configuration menu
    Copy the full SHA
    df855d6 View commit details
    Browse the repository at this point in the history

  3. avoid connection creation and close

    @dbsid
    dbsid committed Aug 18, 2024
    Configuration menu
    Copy the full SHA
    07592bb View commit details
    Browse the repository at this point in the history

  4. remove the max open connection hard limit

    @dbsid
    dbsid committed Aug 18, 2024
    Configuration menu
    Copy the full SHA
    4be22d7 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2024

  1. try fix the nil pointer error

    @dbsid
    dbsid committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    5243b09 View commit details
    Browse the repository at this point in the history

  2. try fix the build error

    @dbsid
    dbsid committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    ee6704e View commit details
    Browse the repository at this point in the history

  3. fix build error 

    by: make bazel_prepare
    @dbsid
    dbsid committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    cb332bf View commit details
    Browse the repository at this point in the history

  4. fix unused key

    @dbsid
    dbsid committed Aug 19, 2024
    Configuration menu
    Copy the full SHA
    1fd3ce7 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2024

  1. Merge branch 'master' into prep-stmt-lightning-master

    @dbsid
    dbsid committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    17db4e6 View commit details
    Browse the repository at this point in the history

  2. Update pkg/lightning/backend/tidb/tidb.go 

    Co-authored-by: D3Hunter <[email protected]>
    @dbsid @D3Hunter
    dbsid and D3Hunter committed Aug 21, 2024
    Configuration menu
    Copy the full SHA
    8f2437e View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. address some of the comments

    @dbsid
    dbsid committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    afe73a5 View commit details
    Browse the repository at this point in the history

  2. fix overriding existing stmt

    @dbsid
    dbsid committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    3d6e4e4 View commit details
    Browse the repository at this point in the history

  3. make bazel_prepare

    @dbsid
    dbsid committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    4dc3fdc View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. avoid to create the LRU cache when LogicalImportPrepStmt is false

    @dbsid
    dbsid committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    a80c974 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. Merge remote-tracking branch 'upstream/master' into prep-stmt-lightni… 

    …ng-master
    @dbsid
    dbsid committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    066f7f8 View commit details
    Browse the repository at this point in the history

  2. remove the nil check

    @dbsid
    dbsid committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    5b808ae View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2024

  1. address comments

    @dbsid
    dbsid committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    132c4a3 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. address comments

    @dbsid
    dbsid committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    2c0bc66 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/master' into prep-stmt-lightni… 

    …ng-master
    @dbsid
    dbsid committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    539a0d1 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2024

  1. address comments

    @dbsid
    dbsid committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    3e08969 View commit details
    Browse the repository at this point in the history

  2. address comments

    @dbsid
    dbsid committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    0bae547 View commit details
    Browse the repository at this point in the history

  3. make bazel_prepare

    @dbsid
    dbsid committed Sep 14, 2024
    Configuration menu
    Copy the full SHA
    0c25621 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. add integration test

    @dbsid
    dbsid committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    1ce0977 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. Merge remote-tracking branch 'upstream/master' into prep-stmt-lightni… 

    …ng-master
    @dbsid
    dbsid committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    662f938 View commit details
    Browse the repository at this point in the history

  2. remove the printf for debug

    @dbsid
    dbsid committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    6727f20 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2024

  1. Merge remote-tracking branch 'upstream/master' into prep-stmt-lightni… 

    …ng-master
    @dbsid
    dbsid committed Sep 21, 2024
    Configuration menu
    Copy the full SHA
    8869768 View commit details
    Browse the repository at this point in the history