How to implement re-export behavior in ScopeComputation?

[georg-schwarz]

Hey,

I'd like to add a feature to our language that allows re-exporting imported elements. Therefore, I extended the DefaultScopeComputation implementation and added my custom logic.

However, my implementation is running into the following error:

Attempted reference resolution before document reached ComputedScopes state.

Example

The code snippets below are from our project Jayvee. I omitted everything that is not interesting for the example.

grammar.langium

ExportDefinition:
  'publish' element=[ExportableElement] ';';

ImportDefinition:
  'use' '*' 'from' path=STRING ';';

publishing.jv (originally publishing file)

// originally publish the element "PublishedBlockType"
publish builtin blocktype PublishedBlockType {
  // [...]
}

publishing-transitive.jv (re-publishing file)

use * from './publishing.jv';

// re-publish the imported element "PublishedBlockType"
publish PublishedBlockType;

using.jv (using file)

use * from './publishing-transitive.jv';

// reference the imported element "PublishedBlockType"
block UsingBlock oftype PublishedBlockType {}

Implementation

export class JayveeScopeComputation extends DefaultScopeComputation {
  constructor(services: LangiumCoreServices) {
    super(services);
  }

  protected override exportNode(
    node: AstNode,
    exports: AstNodeDescription[],
    document: LangiumDocument,
  ): void {
    // [...]

    const isDelayedExportDefinition = isExportDefinition(node);
    if (isDelayedExportDefinition) {
      const exportedNode = node.element.ref;
      if (exportedNode === undefined) {
        return;
      }

      return super.exportNode(exportedNode, exports, document);
    }
  }
}

Questions

1. I assume the issue is trying to follow the reference via node.element.ref since the linking has not been done yet.
   How can I circumvent this?
2. Is it even possible to export the node from a document where it was not originally defined?

Thanks already in advance!

[msujew]

Hey @georg-schwarz,

great question! In general, I believe you're trying to achieve too much with the scope computation mechanism. While in theory this could work with a smarter compute scope strategy (i.e. a sorting of the documents), this would also break down in case of a cyclic dependency between files.

My solution would attempt to do this using the scoping mechanism directly instead of using the scope computation. During the scoping, we're allowed to resolve cross references. In your case it should be enough to adjust the global scope:

// compute the scope that can be accessed from the file that contains the reference
getGlobalScope(referenceType: string, context: ReferenceInfo): Scope {
  const entryElement = getContainerOfType(context.container, isModel); // assuming the most top level element in your grammar is `Model`
  const imports = entryElement.imports; // collect all `use` statements
  const resolvedFiles = resolveImports(imports); // this should be a list of all imported documents
  const elements: AstNodeDescription[] = [];
  for (const file of resolvedFiles) {
    const allPublishedElements = getPublishedElements(file); // gets all elements annotated with `publish`
    elements.push(...allPublishedElements.map(e => this.descriptions.createDescription(e, e.name));
  }
  return new MapScope(elements);
}

getPublishedElements(document: LangiumDocument): Element[] {
  const model = document.parseResult.value;
  const publishedElements = model.elements.filter(e => e.published);
  const exportedElements: Element[] = [];
  for (const published of publishedElements) {
    if (isPublishStatement(published) {
      // follow the chain of published elements here
      // i.e. use published.element.ref recursively until you find the defining element (or run into a cycle)
      exportedElements.push(followPublishStatementChain(published)); 
    } else {
      exportedElements.push(published);
    }
  }
  return exportedElements;
}

Note that the code above is a very rough outline. Unless you perform caching, you will quickly run into performance issues with this. Also, this doesn't deal with renaming of objects yet, you might need an intermediate layer for that. I.e. don't expose the Element directly from Element but use a { element: Element, name: string } object instead. Additionally, some work might be required to get this to perform well in cyclic dependency cases. But this is the general gist of it, and I'm sure you will be able to figure out the rest :)

[georg-schwarz]

Thanks for the very fast response! I'll give your suggestion a try and report back.

Regarding the performance issues that you recommend using caching, do you have any advice on how to approach that efficiently? Or do you have some resources/examples available that we could read up on?

[msujew]

We have a few types of caches available, see here. You can find them in use within Langium's scoping mechanism as well here and here.

For your case, it would be useful to cache whatever each document is publishing in a DocumentCache. That way, it is automatically flushed when the document is changed/needs to be re-scoped.

[msujew]

@georg-schwarz FYI we have a new guide in our pipeline for this :) See eclipse-langium/langium-website#239

[georg-schwarz]

Awesome, thank you! :)
We just released a new version of Jayvee (our DSL) version with our implementation of the file import.

Maybe it helps anyone coming across this discussion as well:

• Grammar
• JayveeScopeProvider
• ImportResolver service

Feedback is very welcome :)
If you like it, feel free to link to our implementation where you see fit 😄