Create a JSON serializer can fully serialize AST nodes content including cross references?

[jindong-zhannng]

Hi team,

I'm facing a requirement that needs to transfer parsed AST to another process via a textual way. To achieve it, I think I need a JSON serializer that:

1. can print a (nearly) complete textual representation of an AST, including cross-referenced nodes.
2. has zero dependency, at least for the deserializer.

There is one DefaultJsonSerializer in langium. But it depends on language services and documents in memory to handle cross-references.

So do we need such a new json serializer to be added to langium if my need is not too rare? And any ideas about implementation?

My rough idea: serialize node to...

{
  "node":  { ... },  // json representation as is
  "references": {
    [reference-id]: astnode
  }
}

[msujew]

Hey @jindong-zhannng,

you can take some inspiration from the old json serializer. It is less powerful (cannot resolve cross references to other files), but has far fewer dependencies. Stuff like the AstNodeLocator and NameProvider can be inlined. That might be a good fit for you.

[jindong-zhannng]

Well, cross files reference is a pain point and necessary for me. Is there any idea about how to do it?

[msujew]

I mean, you just need to reproduce the same deserialization algorithm that is currently employed in the JSON serializer. It's not like we do any complex stuff in there. I believe @sailingKieler once wrote a deserializer for those files for his own uses.

We could possibly provide a standalone deserializer as a package, but I currently don't have time to work on that.

[jindong-zhannng]

Ok, got it.

We could possibly provide a standalone deserializer as a package, but I currently don't have time to work on that.

I'm happy to contribute to this since I have to write them anyway. But before I start, I'd like to know langium team's opinions firstly about things like specifications of json, APIs, how to export it to user... (a RFC maybe?) Once we got alignment I can start coding soon.

[msujew]

I'm not sure how usable a specification is here; Adopters of Langium can override their serializer however they like and the spec becomes less of a spec and more of a general recommendation. I don't think that the general structure of the default serializer will change anymore, so writing it against the current behavior should be good enough. The effort to spec it out doesn't seem worth it in comparison to the value, given that most users are probably fine with just including the DefaultJsonSerializer in their projects.