feat(rdb_load): add support for loading huge streams

[andydunstall]

Follows #3850 to add support for loading huge streams (#3760).

This loads the stream entries in partial reads, though loads the stream metadata and consumer groups in a single read (assuming consumer groups will be relatively small so don't need partial reads).

As with lists, loads streams in 512 segments as each stream node can contain 4kb of elements.

Also removes the outer Ltrace::arr as we now only use a single array. This means YieldIfNeeded is also redundant so removed.

Comparing a 5GB stream:

• main: 4.8s / ~13GB RSS
• load-huge-streams: 2.6s / ~7GB RSS

[romange]

I did not understand this comment. Can you explain please?

[andydunstall]

i've updated the comment

what I mean is ReadStreams is split into two sections:

• Reading the stream entries (ltrace->arr)
• Reading the stream metadata and consumer groups (ltrace->stream_trace)

loading the stream metadata and consumer groups in partial reads would be quite complex, and i'm guessing isn't expected to be large enough to require partial reads? so wasn't sure if it's worth trying to load consumer groups in partial reads?

the simplest option seems to be just load the stream entries (ltrace->arr) in partial reads, then on the final read also read the stream metadata and consumer groups (ltrace->stream_trace)

[andydunstall]

Added this as it's useful for testing, though it is a bit different from the other populate commands since XADD adds a single stream entry with multiple elements in that entry (but the key still has only a single entry which is why the test calls populate 2000 times)

Can remove if preferred and just move this logic into the test (though this sped up the test and useful for manual testing)