Parallel Startup with Lazy Dependencies to speed up Agent Startup and Bootstrapping

[CMCDragonkai]

Specification

It appears agent startup may be slowing down our tests in the CI/CD. There are opportunities here to exploit IO parallelism during the startup procedures. What we need to is to define a Lazy type constructor indicating a lazy dependency.

Then afterwards, for dependencies that are only needed after the domain is started, it can done like this:

import { performance } from 'perf_hooks';

type Lazy<T> = T | PromiseLike<T>;

class X {
  public static async createX() {
    const x = new this();
    await x.start();
    return x;
  }

  constructor() {
  }

  public async start() {
    console.log('X starting', performance.now());
    // Do regular startup
    await new Promise<void>((resolve) => {
      setTimeout(resolve, 1000);
    });
    console.log('X started', performance.now());
    return;
  }

  public async doSomething() {
    return 1;
  }
}

class Y {
  public static async createY({
    x
  }: {
    x: Lazy<X>
  }) {
    const y = new this();
    await y.start({ x });
    return y;
  }

  protected x: X;

  constructor() {
  }

  public async start({ x }: { x: Lazy<X> }) {
    console.log('Y starting', performance.now());
    // Do regular startup
    await new Promise<void>((resolve) => {
      setTimeout(resolve, 1000);
    });
    this.x = await x;
    console.log('Y started', performance.now());
    return;
  }

  public async doSomething () {
    return (await this.x.doSomething()) + 4;
  }
}

async function main() {
  const xLazy = X.createX();
  const yLazy = Y.createY({ x: xLazy });
  const [x, y] = await Promise.all([
    xLazy,
    yLazy
  ]);
  console.log(await y.doSomething());
}

void main();

The result looks like this:

X starting 202.94423389434814
Y starting 204.55536937713623
X started 1205.8660507202148
Y started 1206.1837797164917
5

You can see here that Y depends on X, but Y and X can start at the same time because Y only needs X when it is finished starting. Assuming both Y and X are IO-bound operations, this can speed up the bootup procedure quite a bit as we now have a lazy promise graph of dependencies.

Not all dependencies can do this, only that which is not required to be ready by the time the downstream dependency is starting.

This also adds additional complexity one has to understand:

• There are required dependencies (these are not optional, and their lifetimes are not encapsulated)
• There are optional dependencies (these are optional, and their lifetimes are encapsulated)
• There are fully encapsulated dependencies
• There are dependencies that run on creation, prior to construction, these are dependencies that must exist or be constructed beforehand
• There are dependencies that are used in start after construction, where start is for asynchronous startup and construction for synchronous startup
• Now there are "lazy" dependencies that can be awaited at any point in the construction, notice that lazy dependencies that are used in start are not taken in the constructor, but lazy dependencies that are needed during creation would be set into the constructor
• CDSS, SS, CD are all relevant now

Additional context

• Updates to Docker execution #439 (comment) - nat tests are starting multiple PK agents at the same time, this appears to take 15 seconds even on local testing

Tasks

1. ...
2. ...
3. ...

[CMCDragonkai]

IoC containers could reduce our test boilerplates when it comes to spinning up all the dependencies. I'm finding alot of boiler plate now. #446 (comment)

However it's a good idea to prevent tight coupling. Right now our classes declare requirements through the constructor/asynchronous creator. The dependencies are however highly contextual.

1. Static required dependencies (expects that this is already created and should be injected in)
2. Static optional external dependencies (won't create them, won't use them)
3. Static internal optional dependencies (will create them if you don't pass it)
4. Start parameters - required and optional

The IoC may be to fill in some of them, but some of them, they will still require the user top pass in.

What would the API look like? On the outside, when we want a object like CertManager, we would ask:

container[CertManager] = async (c, config = {}) => {
  return CertManager.createCertManager({
    db: c[DB],
    keyRing: c[KeyRing],
    taskManager: c[TaskManager],
    ...config
  });
};

const certMgr = await container[CertManager]()

If we enable the ability to take Lazy<Dependency> then, it's possible that we don't have to await within the functions to instantiate it.

However what about the other parameters? It would make sense that for any given function, you have to call it with parameters that are "left over".

await container[CertManager]({ ... })
await container[CertManager]()

Then further parameters could pass into the system.

What about things like Logger that may need to be configured separately? And how would we "override" the dependencies? This seems like a nix sort of override problem. It's like you can take container[CertManager] and apply a fixed point override that changes the underlying dependencies.

I wonder if a IoC container has already been developed that does this.

// each time you override, you get a copy of the `c`, and the returned value is a copy
// this allows you override the container and fetch dependencies using the new override
// remember this does end up with a new instance...
container.override(c => {
  // mutate c, but, the return is going to be a new container
  c[DB] = async (c) => c[DB]('some param');
  return c;
})[CertManager];

Remember that container[DB]() will require a DB path. That means all the "required" dependencies have to "bubble up" if I want the CertManager, I will also need to pass the dbPath.

Unless it's defaulted to somewhere. That comes down to how the container is setup. Any parameter not passed in where required should result in an error. We could try to make TS propagate these required parameters up so it can be determined statically.

[CMCDragonkai]

Working on PKE has shown that IOC container had to be synchronous, it was not able to create objects asynchronously. This is because using hooks to do service location cannot be asynchronous.

On the otherhand, if the IOC container was made dynamic, and dynamic registration was possible, then we could create objects asynchronously.

So the issue is that we have classes that can be asynchronously created using the js-async-init pattern. But this would not work in React, as such classes could not be injected using useHook pattern....

Actually that's not entirely true. We could do something where asynchronous classes can be instantiated, and if they are synchronously returned, then the React component can use it straight away. On the other hand, if they come as through a Promise, then one could argue that the component would be blocked on rendering until the object is created.

Is this possible? We have observables that if they don't have a default value would end up blocking on render and thus triggering a suspense component. Alternatively one could argue that the entire component would just not render until the promise is resolved.

If it is possible, one could then create a universal IOC container that supports both synchronous classes and asynchronous classes. (Support both synchronous creation and destruction, and asynchronous creation and destruction).

Whether it is dynamic or not is another story. Dynamic registration would not be type-safe. One could argue that dynamic registration should not be required. Anything that is likely to have dynamic registration would have to exist at a level above the object-layer.

Remember that in our instantiation procedure we have:

Instance Layer
     ^
     |
Module Layer

This is due to lacking first-class modules in JS and also lacking top-level await in JS. Now that JS has top-level await, it still lacks first-class modules, but does support module monkey-patching, but that does not count as first-class modules.

Anyway, introducing dynamic registration is one more layer:

State Layer
     ^
     |
Instance Layer
     ^
     |
Module Layer

Therefore the IOC container operates at the instance layer. Therefore any dynamic registration should be done as part of the instance operational state.

[CMCDragonkai]

I think the conclusion here is that we don't need dynamic registration.

However if we investigate further how blocking react component rendering works, it'd be possible to then create a js-ioc package that wraps this up all nicely.

It can then work in both React contexts AND PK contexts where there's no React at all.

@amydevs Thoughts?

[CMCDragonkai]

This has relationship to #444. There #444 is investigating how the Observable concept fits into the overall architecture of PK.