Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(rdb_load): fix partial reads dropping elements #3853

Merged
merged 1 commit into from
Oct 3, 2024

Conversation

andydunstall
Copy link
Contributor

Found a bug in loading huge objects where we append to the wrong encoding.

Say you have a hmap with 4100 integers, we'll load 4092 then 8 elements. The first CreateHMap will see 4092 > 64 so create a StringMap, but the second CreateHMap will see 8 < 64 so creates an lpNew instead of appending to the existing StringMap (meaning the existing elements are lost). Same applies to set (when loading an 'intset') and zset.

Therefore adding an item.streamed to be really explicit about whether the item is being streamed so the larger encoding must always be used. We could probably infer from append and reserve but seemed better to be explicit.

@romange romange merged commit 44fbe09 into dragonflydb:main Oct 3, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants