question about experimental vendor functionality

[dlethin]

I was curious to learn more about the idea behind terramate's evolving vendor experiment.

Is the idea to eventually encourage the practice of terramate pulling down modules through some kind of manifest and then terramate code generation and have the stack point the module source to the appropriate path in the vendor directory?

I think this would improve the performance of our current terragrunt implementation. While you can cache terraform providers, you can't cache modules, so if you are running terragrunt across multiple projects, the same module(s) have to be downloaded remotely each time which is time consuming.

Thanks.

[mariux]

Hi @dlethin,

That's to the point.

The upcoming terramate vendor will support recursively downloading modules to a local directory. Those can be referenced locally and do not need to be downloaded in terraform init phase over and over again.

The motivation is - as you mentioned - performance improvements on scaling repositories.

But another criteria here is trust: While supply-chain attacks are getting more popular and also availability of specific repositories and versions is not guaranteed forever as they are outside of the control of it's users, vendoring helps to guarantee access to the dependencies a project has at all times. Ofc the terraform module universe is mostly maintained by well known sources and the likeliness of supply-chain issues is still very very low.. but also deprecation might make modules unavailable at some point.

It also comes with improved stability as the likeliness of temporary failures decreases due to the fact to have all code and dependencies in a single point of truth.

On the manifest, we already support a manifest configuration inside of a module that Terramate will honor when vendoring a repository. This way the actual amount of duplicated (vendored) code can be minimized to the actual files needed to run each modules and its submodules (e.g. skipping tests and examples etc.).

So what is the current state and roadmap for our experiment here:

• Currently we are in an alpha with a very unstable interface and unstable behavior.
• Within the next two weeks we want to stabilize the interface and allow for recursive vendoring (source will be automatically overridden).
• Once the stand-alone vendor command is done, we want to integrate it natively into the code generation - so no manual vendoring would be needed anymore.

Terramate vendor will be usable without using any other Terramate feature like Code Generation or Orchestration.

Once done we will provide an introduction blog post ;)

[dlethin]

That's a really informative response.

I'm particularly interested in this feature:

Within the next two weeks we want to stabilize the interface and allow for recursive vendoring (source will be automatically overridden).

I was testing out the recursive feature when I saw it was added but noticed source path to inner modules was not automatically updated, but now understand with the featuring being experimental not all the functionality is in place yet.

I'll keep following along and watching for updates.

Thanks for creating & sharing this tool.