Revamped: How Backend/Targets are Handled

docs/Loader/Backends/About.md

@@ -4,13 +4,17 @@
 
     For some programming languages, it might be necessary to manually bootstrap a runtime.
 
-!!! note "Assume each backend links to the common C runtime for the platform."
+## Platforms
 
-    In other words:
+!!! info "The following platform names are standardised."
 
-    - `win-x86` really means `win-x86-msvc`.
-    - `linux-x86` really means `linux-x86-glibc`.
-    - `dotnet-latest` really means `dotnet-latest-coreclr`.
+| Backend  | Description      | MVP | MVP Note                                                |
+| -------- | ---------------- | --- | ------------------------------------------------------- |
+| `win`    | Windows (`.dll`) | ✅   | Support MSVC or Static Linked                           |
+| `linux`  | Linux (`.so`)    | ✅   | Support glibc or Static Linked                          |
+| `macos`  | macOS (`.dylib`) | ❌   | Lacking necessary hardware.                             |
+| `switch` | Horizon (`.nro`) | ❌   | Lacking in expertise. But there's interest.             |
+| `dotnet` | .NET (`.dll`)    | ✅   | CoreCLR. Including support for platforms targeting R2R. |
 
 !!! tip "It is preferred to use universally recognisable names over internal ones."
 
@@ -19,45 +23,86 @@
     - `switch` over `horizon`
     - `dotnet` over `coreclr`
 
-Planned backends include:
+## Architectures
+
+!!! info "The following architectures are standardised."
+
+| Arch      | Note                             |
+| --------- | -------------------------------- |
+| `x86-any` | Assuming `i686` as the baseline. |
+| `x64-any` | Also known as `x86-64-v1`        |
+| `arm64`   |                                  |
+
+### Microarchitecture Levels
+
+!!! tip "[Microarchitecture levels][microarchitecture-levels] for purposes of micro-optimisation are also supported."
+
+    This is present for high performance dependencies, where every nanosecond counts.
+
+    Generally, it is not expected that mod authors will manually leverage this functionality however,
+    that said; it is hoped we can make it easy to use during the [publish process][mod-publishing] if possible.
+
+| Type   | Name   | Description                                   |
+| ------ | ------ | --------------------------------------------- |
+| string | x64-v2 | Path to DLL targeting x86-64-v2               |
+| string | x64-v3 | Path to DLL targeting x86-64-v3               |
+| string | x64-v4 | Path to DLL targeting x86-64-v4               |
+| string | x86-v2 | Path to DLL targeting x86-64-v2 (32-bit mode) |
+| string | x86-v3 | Path to DLL targeting x86-64-v3 (32-bit mode) |
+| string | x86-v4 | Path to DLL targeting x86-64-v4 (32-bit mode) |
+
+Compilers based on LLVM (Clang, Rust etc.) can directly target these.
+
+For example, if the backend specified is `x64-v3`, it is assumed the CPU supports AVX2.
+
+!!! tip "For more information, including how the `x86` targets are derived, see [Research: Microarchitecture Levels][research-march-levels]."
 
-| Backend         | Description                   |
-| --------------- | ----------------------------- |
-| `win-x86`       | Native x86 Support on Windows |
-| `win-x64`       | Native x64 Support on Windows |
-| `linux-x64`     | Native x64 Support on Linux   |
-| `linux-x86`     | Native x86 Support on Linux   |
-| `dotnet-latest` | Latest .NET Runtime           |
+## TOML Representation
 
-Backends that will be delivered if there's community interest in terms of donations/code contributions:
+!!! info "Backends have arbitrary information, thus in all serialized data in configs are represented as dictionaries"
 
-| Backend       | Description                     |
-| ------------- | ------------------------------- |
-| `win-arm64`   | Native ARM64 Support on Windows |
-| `linux-arm64` | Native ARM64 Support on Linux   |
+Take for example [Package Metadata], where the [`Targets`][mod-configurations-targets] field
+is a nested dictionary.
 
-Some potential backends could only be delivered with community involvement, due to lack of knowledge:
+```toml
+[Targets.win]
+x64-any = "mod.dll"
+```
 
-| Backend        | Description                              |
-| -------------- | ---------------------------------------- |
-| `switch-arm64` | Native ARM64 Support on Switch (Horizon) |
+The dictionary name is [`Targets`][mod-configurations-targets], with the first level
+[Platform](#platforms) key being `win`, and the second level [Architecture](#architectures) key
+being `x64-any`.
 
-These backends are directly specified inside the [mod configurations][mod-configurations-targets].
+## Runtime Library Assumptions
 
-When the loader is about to load a mod, it looks up a dictionary specified in the mod config,
+!!! note "Assume each backend links to the common C runtime for the platform."
+
+    In other words:
+
+    - `win` really means `win` + `msvc`.
+    - `linux` really means `linux` + `glibc`.
+    - `dotnet` really means `dotnet-coreclr`.
+
+## Loader Behaviour
+
+When the loader is about to load a mod, it looks up a dictionary specified in the [Package Metadata],
 and starts the mod using the appropriate backend.
 
+If the backend is not one supported by the loader, the loader will try to find an already loaded
+mod which may have [registered the backend](#custom-backends).
+
 It's possible to ship a mod for multiple platforms by including multiple backends in a mod config
 and shipping separate binaries for multiple platforms.
 
-## Custom Backends
+### Custom Backends
 
 !!! note "In some cases, some custom bootstrapping might be required."
 
 For example:
 
 - For a .NET game, it might be desirable to execute mods inside the game's own runtime.
-- For supporting mods for older mod loaders, a backend can act as a wrapper translating exports and implementing old loader APIs.
+- For a `Unity` game handler, you may want to swap between `mono` and `dotnet-latest` based on runtime version & `Il2Cpp` status.
+- Backwards compatibility for legacy mods written for a previous mod loader.
 
 | Example Backend                | Description                                                                |
 | ------------------------------ | -------------------------------------------------------------------------- |
@@ -68,6 +113,20 @@ For example:
 To allow for maximum modularity; custom backends are implemented via [regular mods][regular-mods];
 allowing them to be updated independently from the loader.
 
+## Information for Project Template Authors
+
+!!! info "For project templates it is recommended to target LLVM-based toolchains (Clang, Rust, etc.)"
+
+For the following reasons:
+
+- They can directly target the microarchitecture levels mentioned above.
+- More portable, for example, you can `cross compile` (e.g. build for Windows from Linux).
+- They can generate better code.
+
 <!-- Links -->
 [mod-configurations-targets]: ../../Server/Packaging/Package-Metadata.md#targets
-[regular-mods]: ../Core-Architecture.md#regular-mods-layer-3
+[regular-mods]: ../Core-Architecture.md#regular-mods-layer-3
+[Package Metadata]: ../../Server/Packaging/Package-Metadata.md
+[microarchitecture-levels]: https://en.wikipedia.org/wiki/X86-64#Microarchitecture_levels
+[mod-publishing]: ../../Server/Packaging/Publishing-Packages.md
+[research-march-levels]: ../../Research/Microarchitecture-Levels.md

docs/Mods/Libraries/Signature-Scanner/Requirements.md

@@ -111,4 +111,4 @@ I (Sewer) would prefer Rust.
 [lazysimd]: https://github.com/Raytwo/lazysimd
 [reloaded-memory-sigscan]: https://github.com/Reloaded-Project/Reloaded.Memory.SigScan
 [r2-sigscan-mod]: https://github.com/Reloaded-Project/Reloaded.Memory.SigScan/blob/master/External/Reloaded.Memory.SigScan.ReloadedII/StartupScanner.cs
-[microarch-levels]: ../../../Loader/Backends/Native.md#instruction-sets
+[microarch-levels]: ../../../Loader/Backends/About.md#microarchitecture-levels

docs/Research/Microarchitecture-Levels.md

@@ -0,0 +1,51 @@
+!!! info "This page provides more info on [Microarchitecture Levels][microarchitecture-levels] in Reloaded3."
+
+## Determining Supported Instruction Set on Local Machine
+
+Start with [core-detect][core-detect] crate, and make a separate library with helper methods to
+determine which category the current CPU falls under.
+
+You can use `rustc --print=cfg -C target-cpu=x86-64-v3` to print the specific `target_feature`(s)
+available to a CPU.
+
+## Emulating Instruction Sets for x86 (32-bit) Targets
+
+!!! warning "x86-64-v* don't have equivalents for 32-bit targets in LLVM."
+
+    But we can derive them!
+
+To derive them, use the Rust compiler like so `rustc --print=cfg -C target-cpu=x86-64-v4`.
+This will print a list of details for the target, alongside, most importantly, ***target features***.
+
+!!! example "Example for x86-64-v2"
+
+    ```
+    target_feature="avx"
+    target_feature="avx2"
+    target_feature="avx512bw"
+    target_feature="avx512cd"
+    target_feature="avx512dq"
+    target_feature="avx512f"
+    target_feature="avx512vl"
+    ```
+
+From there, we have to filter out features which are not available in 32-bit mode.
+
+- `avx512bw`, `avx512cd`, `avx512dq`, `avx512f`, `avx512vl`: These AVX-512 extensions are 64-bit only.
+- `cmpxchg16b`: This extension is only available in 64-bit mode. May be possible to use in 32-bit programs but atomic read/write of 128-bit values is not guaranteed in 32-bit mode.
+
+Therefore we can derive the following:
+
+| Type   | Name   | Features                                                                       |
+| ------ | ------ | ------------------------------------------------------------------------------ |
+| string | x86-v2 | `fxsr` `lahfsahf` `popcnt` `sse` `sse2` `sse3` `sse4.1` `sse4.2` `ssse3`       |
+| string | x86-v3 | all previous + `avx` `avx2` `bmi1` `bmi2` `f16c` `fma` `lzcnt` `movbe` `xsave` |
+
+The microarch level `i686-v4` is redundant, because all new features in `x86-64-v4` are not
+supported in 32-bit mode.
+
+These features come from LLVM, so can also be used with Clang, etc.
+
+
+[core-detect]: https://docs.rs/core_detect/latest/core_detect/
+[microarchitecture-levels]: https://en.wikipedia.org/wiki/X86-64#Microarchitecture_levels