WIP: Add "update a data structure" feature

[roblatham00]

Seems to me like there are a lot of overlaps between the 'cachercise' (https://github.com/roblatham00/cachercise) service I put together and quintain.

I don't know if this approach -- overloading the 'flags' in quintain_work -- is the best idea

[carns]

Sorry for the slow response @roblatham00 . I like having this feature here.

Stepping back a little and looking at the API, I wonder if instead of overloading flags we shouldn't have an additional ops field, with power-of-two predefined values to indicate what operations each RPC should perform. We have at least 4 we could start with, being bulk get, bulk put, hoard get, hoard put. Later we might want to indicate abt-io operations too. If they are power of 2 values we could specify operations that do a bulk get, abt-io write, and a hoard put, for example.

That's a little scope expansion for your PR because right now the bulk transfer is only one direction and is implied by the size field being non-zero, so it's already kind of sneaking operations specifications in places they don't really belong. Do you want to tackle that, or would you rather I refactored a little to add that and then you rebase your PR atop it?

[roblatham00]

I am glad to add an ops field, maybe in a separate pull request.

While we are designing API refinements: I also found myself wanting a data validation path. We can do puts and gets but the "quintain_work" only indicates "yes I did that for you".

[carns]

That makes sense. So maybe an ops field for what to do (do an rdma pull), and use the flags fields for modifiers on it (like validate contents, or use buffer pool)?

[roblatham00]

Can we time the validation separately? Maybe I need to think about validation differently, but in this "concurrent cache update" test, I want the validation to ensure I'm not corupting data but I also want to know that the locking and coordination isn't adding overhead.

[carns]

Can we time the validation separately? Maybe I need to think about validation differently, but in this "concurrent cache update" test, I want the validation to ensure I'm not corupting data but I also want to know that the locking and coordination isn't adding overhead.

I would think so in cases where validation happens on the client. On the server its probably harder to separate, because even if you discount the validation time, that's time the server used doing something besides processing other requests.