Skip to content

Question about zstd stateless decoding #995

Answered by klauspost
luxflow asked this question in Q&A
Discussion options

You must be logged in to vote

The main difference is the memory requirements, but also some speed considerations.

The []byte -> []byte requires you to have both the input and the output in memory at the same time. If your payload is a 1GB JSON file, this is probably not a great idea.

Secondly []byte -> []byte means that every step must complete before the next can start. This means your flow will be Request -> io.ReadAll -> DecodeAll -> Process serial flow. With a stream, as soon as the first block is returned by the remote and has been compressed it is ready for your processor.

However streams require more setup, so typically for smaller payloads it is not worth setting it up, for effectively a single block (up to 12…

Replies: 1 comment 1 reply

Comment options

You must be logged in to vote
1 reply
@luxflow
Comment options

Answer selected by luxflow
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants
Converted from issue

This discussion was converted from issue #994 on August 30, 2024 15:51.