Configuring multiple subnetworks

[morph-dev]

Configuring trin to run multiple subnetworks is currently not ideal for a couple of reasons:

Subnetwork selection

We have a flag for enabling subnetworks (portal-subnetworks), but not all combination should be valid.

Once #1377 is merged, state network won't be able to run without history network. In a few months, history network won't be able to run without beacon network.

Potential solutions:

1. Crash execution at the start if flags are not configured correctly (approach I took in feat(state): validate state content by anchoring it to the state root #1377 )
2. Introduce different flag (e.g. mode) that enables all required subnetworks
   • For example, in the short/medium term, valid values could be history (beacon+history) and state (beacon+history+state)
     • Not sure if it makes sense to run beacon only, but we could add it if it does
   • This can get more complicated when we add more subnetworks (e.g. transactions), but we an easily support multiple modes (union of all subnetworks)

Side note: Validating state network requires having history network, but we don't need to store any history data ourself. However, I don't think we want to go this path (see section below).

Storage

We have only one flag (mb) for configuring the storage capacity. Currently, it applies on all subnetworks separately. Meaning, if user enables all 3 networks, and specifies 1GB, we will actually set capacity of 1GB per subnetwork (meaning 3GB in total).

In the context of beacon network, my understanding is that storage capacity doesn't make sense. Firstly, it uses little storage to begin with (compare to history and state), and secondly it will never prune or do anything to stay under capacity, no matter how small it is. @ogenev is this correct?

Potential solutions:

1. Have a way to configure capacity per network
   • While this gives more granular control, it can also be complicated for the end user
2. Keep only one flag, but it should mean total capacity. It will be up to us to decide how much capacity each network has. Some ideas:
   1. Use the same radius for all subnetworks
      • Nice and clean, but might require bigger refactoring of the current code (which we might need to do to support "big radius node" anyway)
   2. Have a fixed weight of how much capacity should be allocated for each subnetwork
      • Only consider weights of the enabled subnetworks
   3. Have a combinations of fixed weight that are dependent on the mode from section above (if we agree on that approach)
      • For example, let's consider two scenarios: A: both history and state mode and B: only state mode. In both scenarios, we would run all 3 subnetworks, but more storage would be allocated for state subnetwork in scenario B.

Side note: We currently estimate how much storage we are using (by adding lengths of content-id, content-key and content-value) which isn't totally accurate (we underestimate). This estimate is used when we decide whether to prune or not, resulting in us using more than specified with the capacity. The error is not too big (percentage wise), but I believe we can do better.

Bootnodes

Current flags only allow selecting one set of bootnodes, that is used for all subnetworks. This might be ok, as long as that set contains at least some bootnodes for every selected subnetwork.

[morph-dev]

My opinion is that in the long term, we should have:

• mode flag for selecting the mode(s) we are operating in (and we will enable all required subnetworks) (option 2. above)
• one flag for storage capacity and the same radius for all subnetworks (option 2.i. above)

Both of these require some amount of refactoring (to be investigated how much), and I would rather have something that works better than the current approach and do more improvements in the future. So in the short term, we could:

• Keep the portal-subnetworks as is, but check that flags are configured correctly (option 1. above)
  • Easy to add as we create dependency (see change in #1377 )
• Replace mb flag with a respective flag for each subnetwork: history-mb and state-mb (and beacon-mb if it makes sense) (option 1. above)
  • This is aligned with the current architecture, so it shouldn't be big change