Sending Pre-Compressed Buffers?

[ronag]

Would it be possible to send pre-compressed data? i.e. if one caches messages one could cache the compressed version in order to avoid compressing during every send?

e.g.

Instead of:

cache.set(key, buf)
// ...
for (const socket of sockets) (  
  socket.send(cache.get(key)) // compressed by uws
}

cache.set(key, zlib.deflateSync(buf))
// ...
for (const socket of sockets) (
  socket.send(cache.get(key))
}

This would enable 2 things:

• Re-use compression
• Move compression to other thread

[ronag]

If this is possible to implement I would consider funding the work.

[uNetworkingAB]

Yes it is possible. It was the behaviour in v0.14. But doing more investigation I found that, this was very poor for both compressive performance, and for CPU performance.

My first idea was that sharing a compressor would be best, but the cost of compressing is directly (linearly) correlated with the difference from what you have in your sliding window and what you are compressing.

This means that dedicated (per connection) compression is both faster and obviously compresses better.

If the difference is minimal, compression costs almost nothing. If the difference is total, it's slow as a snail.

Most who use compression prefers the dedicated compression. So the shared pub sub compression was removed (because of complexity and added work of keeping multiple variants of the outgoing data).

Adding something like a shared compressor (only) for pub sub is most likely doable

[klauss194]

+1