RFC: Principles of modularity

[saem]

The compiler needs to be reworked so it's easier to work with, extend, maintain, learn, etc. As part of the nkError refactor and my broader experience, the following are a first attempt at principles of modularity.

The aim is to be more useful than the oft repeated:

• separation of concerns
• single responsibility
• encapsulation
• narrow interface

I don't find the above all that useful and it's not really actionable advice as it amounts to "guess your responsibility correctly".

The lesson in nkError is that there are, at time of writing, at least two modules, errorhandling (types, create error node, iteration) and errorreporting (turning errors nodes into strings etc). The separation is required because handling happens potentially very "early" and pervasively in the code. It needs to have the absolute minimal amount of dependencies. Whereas reporting happens late and harnesses many different complier facilities to generate meaningful error messages.

An insight from this experience is that a responsibility is about purpose and there seem to be at least two major kinds I've discovered with respect to the compiler. The types of modules (kinds of resposibilitiy) are:

1. collecting actions/procedures together used at a particular circumstance/time rather than about a domain subject
2. schema, data format, data interchange, associated predicates and functions

A module may have both the first and the second kind within if no other module requires the schema/data format/etc alone, as an early warning make any exposed data types opaque to force a revisitation of this arrangement. A module that focuses solely on the second kind of responsibility is for situations where "peer" modules, two or more modules that need to work together (cohesion) but not depend upon each other (coupling), use the schema/formats as an intermediate dependency or as a way of communicating together.

Presently, we have modules around semantic analysis of calls, expressions, statements, and more. This seems to be a mistake in many cases, look no further than the includes that comprise sem. Instead, when doing semantic analysis, the actions and relevant timing is more likely validation, query, logic, and production. There might be further layering around other timing within the domain itself such as lowerings, which is the concept of higher level features (generics) being "rendered" down to a weaker version of the language that doesn't have that feature.

Example of the second is fairly easy, AST, though it has issues of course. These principles should also help adjudicate code reviews or hard decisions where we have to reject or pushback on things as a matter of taste.

[ShalokShalom]

Well, this may be off the charts, but the most impressive modularity to me offers OCaml with its higher order modules.

Those are functors, so probably not for the scope of this project and too difficult/unfitting to implement?

I am guessing looking into their implementation and particularly how it works, might still help and can you throw a look at the last section, called "Living Without First-Class Modules"

See more about functors here: https://en.wikipedia.org/wiki/Functor

[saem]

I'm know a tiny bit about OCaml and first class modules and at the very least it's inspired some things I'd like to implement in the project -- especially parameterized modules, but that's a different discussion all together.