Validator memorizes values about previous calls to oracle

[djrtwo]

Issue

Right now, the validator reinitializes a new Oracle when checking estimate safety regardless of any previous checks to estimate safety and any intermittent results/calculations. This is a huge source of inefficiencies.

Proposed implementation

• Validator keeps Oracle data around (as long as needed).
• Validator updates Oracle data upon receiving new message
  • Oracles should all implement a method update_estimate_cache (not sure about name) that is called upon receiving a new message. Cache would be dependent on Oracle type.
  • Implementation decision:
    • Validator can either keep a dict of Oracles (one per candidate estimate) -- updating each upon new messages, and removing particular Oracles as candiate_estimates become non-relevant
    • OR Validator only has one Oracle and the Oracle is responsible for organizing info per candidate estimate.

[djrtwo]

@naterush More details need to be fleshed out. Just wanted to get this up before you dive into memoization tonight.

[naterush]

Thanks for adding this. I've started work here and should be able to get up a WIP branch tomorrow.

Not sure what the best trade-off is here. Seems to me the best option might actually be having the validators keep a list of edges between other validators per estimate (or every 10 blocks). If we make them directed, I think we can translate freely between any type or lower-bound-side-effects-free-oracle we want (aversary, turans, clique, etc). Will give a bit of a more formal argument for this tomorrow, too.

I'll keep banging away at it tomorrow - should have a good couple hours to get a preliminary WIP up so we can discuss :)

[naterush]

Here's how I'm approaching it as of now:

1. When a validator checks estimate safety on some estimate for the first time, we create the directed edges in the view and store them. It is also possible we could store the messages themselves that make these edges valid - and this might be the more future proof solution - but I don't think it's necessary as of now.
2. If there are fewer edges than could possibly exist for a node to meet their safety threshold, then we don't check safety at all.
3. Whenever new messages are received, we update the edges for all estimates for which we are currently storing edges for.
4. As soon as there are enough edges that we can possibly detect safety, we check safety (they have to "poke" the estimate safety check again - but maybe this isn't the best). If they detect estimate safety on that estimate, we delete the edges that we have for that estimate + any estimates that are implicitly finalized and/or "anti-finalized" by that estimate.

Things might be a bit more complicated than is worthwhile - but I think if we decide to simplify most of the logic for updating edges (or storing the viewables) can be re-used.

[djrtwo]

This looks good. A couple of questions:

• Should the view be responsible for creating/maintaining edges or should the Oracle? I like the idea of the oracle maintaining the cache relevant to it and the view being dumb, but I could be persuaded otherwise.
• 1: Should we only store edges related to an estimate once we check estimate safety on it or should we start storing edges once we see a new estimate?
• As for 2, I think the Oracle should be responsible for this quick check. So the validator would ask the oracle if safe on that estimate, and the oracle can use the Turan shortcut. Rather than the validator/view needing to know about the Turan shortcut.

Sweet! super cool stuff

[djrtwo]

Honestly, looks like a great start. We can debate design on the other side

…

On Tuesday, October 17, 2017, Nate Rush ***@***.***> wrote: Here's how I'm approaching it as of now: 1. When a validator checks estimate safety on some estimate for the first time, we create the directed edges in the view and store them. It is also possible we could store the messages themselves that make these edges valid - and this might be the more future proof solution - but I don't think it's necessary as of now. 2. If there are fewer edges than could possibly exist for a node to meet their safety threshold, then we don't check safety at all. 3. Whenever new messages are received, we update the edges for all estimates for which we are currently storing edges for. 4. As soon as there are enough edges that we can possibly detect safety, we check safety (they have to "poke" the estimate safety check again - but maybe this isn't the best). If they detect estimate safety on that estimate, we delete the edges that we have for that estimate + any estimates that are implicitly finalized and/or "anti-finalized" by that estimate. Things might be a bit more complicated than is worthwhile - but I think if we decide to simplify most of the logic for updating edges (or storing the viewables) can be re-used. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#59 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABXf-2QoAguhtJUd1iEwX0W0Zx8dwavaks5stSJQgaJpZM4P7U-k> .

[naterush]

https://github.com/karlfloersch/cbc-casper/tree/feat/oracle_cache work begun here.