Perf/range

[KKould]

What problem does this PR solve?

Non-persistent Level 0 to reduce the hard disk read and write latency and compaction impact caused by Level 0

• Use BTreeTable replace SkipTable

In order to improve iteration performance and avoid the performance loss of multiple levels of iteration caused by iteration on the hard disk side, the number of Level layers is reduced (7 -> 4), and the threshold for triggering Sized Compaction for each level is also reduced(10 -> 4).

details: https://www.notion.so/KipDB-3f456f92e559458ca54904bd04376ca6

What is changed and how it works?

Code changes

• Has Rust code change
• Has CI related scripts change

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Note for reviewer

[github-actions]

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information

[github-actions]

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information

[lewiszlw]

Do you have benchmark about skiplist and btree? And can we keep both (behind a feature flag) and let user to choose?

[KKould]

Do you have benchmark about skiplist and btree? And can we keep both (behind a feature flag) and let user to choose?

The reason for using BTreeTable to replace SkipTable is actually not that SkipList's performance is weak, but because Table needs to implement the iter API while ensuring Sync + Send, and SkipList has no official response to this.

issue: JP-Ellis/rust-skiplist#24

Therefore I always comment out the code of SkipTableIter

But since the official did not respond for a long time, I switched to BTreeTable and removed the SkipTable

The Benchmark chart of Level 0 with memorization turned on and without memorization turned on is here :
benchmark_wtih_kipdb_and_sled.zip

[github-actions]

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information