Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EPIC] Full TS-style resolution hooks #1514

Open
1 task
cspotcode opened this issue Oct 15, 2021 · 4 comments
Open
1 task

[EPIC] Full TS-style resolution hooks #1514

cspotcode opened this issue Oct 15, 2021 · 4 comments
Labels
research Needs design work, investigation, or prototyping. Implementation uncertain.

Comments

@cspotcode
Copy link
Collaborator

cspotcode commented Oct 15, 2021

Note: if you're here following a link from our docs about experimentalResolver, you know that our docs for this feature are currently quite limited. Feel free to experiment and ask questions in Discussions or on Discord. This epic tracks the development work to finish all the resolver features and eventually promote them out of experimental status.

Current status

Describes state of our main branch, may not be published to npm yet

  • All file extensions are supported, both CommonJS and ECMAScript modules
    • Use module: NodeNext, esm: true, and experimentalResolver: true. Everything works: cjs, cts, mjs, mts, with or without file extensions in import specifiers, in CommonJS or ESM files

Motivation

In TS, you import the emitted output path, and the compiler resolves this to the input source. They might have different extensions or be in different directories.

In ts-node, we should do the same.

Creating an epic to link together all the related tickets and tasks to make this happen.


Related tickets in no particular order:

How this works in TS

It's been a while since I read the TS code.

I believe that TS creates a mapping from all source files to their emit paths. It can do this because it starts with a glob of all source files, then computes the output path for each in a loop.

Aside: Does this work with files not included in a files array? Does it trigger inclusion of those files in compilation? If yes, that would violate my understanding above.

How it'll work in ts-node

ts-node must do the opposite of TS: we start from a runtime require() or import request, then work backwards to a source TS file. We don't know if such a source file exists; we might be importing a non-compiled .js file.

Mappings to keep in mind:

  • package.json exports
  • package.json main
  • node_modules search
  • Searching all matching file extensions
  • tsconfig.json outDir and rootDir correspondence
  • tsconfig.json paths
  • tsconfig.json baseUrl
  • tsconfig.json rootDirs

Supported extensions are affected by:

  • resolveJsonModule

  • allowJs

  • jsx

  • TS >=4.5 supports mts, cts, mjs, cjs

  • moduleResolution or module?

  • break down this list by (a) those needed when resolving relative specifiers (b) those needed when resolving non-relative specifiers, such as foo

Questions

"Resolve to source path" vs "pretend emit exists on disk"

When resolving from ./dist/foo.js to ./src/foo.ts we have 2x options:

  • A) Tell node that the path is ./dist/foo.js and give it the compiled output of ./src/foo.ts
    • Resolve hook resolves to ./dist/foo.js
    • Load hook understands ./dist -> ./src mapping
    • Sourcemap must map from ./dist to ./src
  • B) Tell node that the path is ./src/foo.ts and give it the compiled output of ./src/foo.ts
    • Resolve hook resolves to ./src/foo.ts
    • Load hook does not need to understand any mappings
    • Sourcemap must map to same filename

ts-node today does (B) but does not do most of the advanced resolution we'd be implementing in this epic.

(B) is closer to deno and probably what people want. __filename and import.meta.url will refer to the actual TS file being executed. This is technically slightly different than pre-compilation execution. However, my gut tells me users will be happier with an intuitive __filename and won't mind the slight difference with pre-compilation.

Note: resolve hook is equivalent to _resolveFilename, load hook is equivalent to compile() hook

ignore with resolver

If src/index.ts is ignored and dist/index.js resolves to src/index.ts, should the resolver do that mapping?

@cspotcode cspotcode changed the title [EPIC] Full TS-style resolution hooking [EPIC] Full TS-style resolution hookds Oct 15, 2021
@cspotcode cspotcode changed the title [EPIC] Full TS-style resolution hookds [EPIC] Full TS-style resolution hooks Oct 15, 2021
@cspotcode cspotcode added the research Needs design work, investigation, or prototyping. Implementation uncertain. label Oct 15, 2021
@cspotcode cspotcode pinned this issue Oct 15, 2021
crapStone pushed a commit to Calciumdibromid/CaBr2 that referenced this issue Feb 16, 2022
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [ts-node](https://typestrong.org/ts-node) ([source](https://github.com/TypeStrong/ts-node)) | devDependencies | minor | [`10.4.0` -> `10.5.0`](https://renovatebot.com/diffs/npm/ts-node/10.4.0/10.5.0) |

---

### Release Notes

<details>
<summary>TypeStrong/ts-node</summary>

### [`v10.5.0`](https://github.com/TypeStrong/ts-node/releases/v10.5.0)

[Compare Source](TypeStrong/ts-node@v10.4.0...v10.5.0)

<!--
  I don't make a discussion thread for every release.  Github has a button to make a discussion thread for a release.
  Then I update the discussion thread to remove the release notes and instead link to the release.
-->

Questions about this release? Ask in the official discussion thread: [#&#8203;1634](TypeStrong/ts-node#1634)

**Added**

-   Eliminate "Emit Skipped" errors ([#&#8203;693](TypeStrong/ts-node#693), [#&#8203;1345](TypeStrong/ts-node#1345), [#&#8203;1629](TypeStrong/ts-node#1629))
    -   Avoids all "Emit Skipped" errors by performing a fallback `transpileOnly`-style transformation.
    -   Does not affect typechecking.  Type errors are still detected and thrown.
    -   Fallback has the same limitations as `isolatedModules`. This will only affect rare cases such as using `const enums` with `preserveConstEnums` disabled.
    -   Fixes [#&#8203;693](TypeStrong/ts-node#693)
-   Graduate swc transpiler out of experimental; add `swc: true` convenience option ([docs](https://typestrong.org/ts-node/docs/transpilers)) ([#&#8203;1487](TypeStrong/ts-node#1487), [#&#8203;1536](TypeStrong/ts-node#1536), [#&#8203;1613](TypeStrong/ts-node#1613), [#&#8203;1627](TypeStrong/ts-node#1627))
    -   `"swc": true` or `--swc` will use swc for faster execution
    -   This feature is no longer marked "experimental."  Thank you to everyone who filed bugs!
-   swc transpiler attempts to load `@swc/core` or `@swc/wasm` dependencies from your project before falling-back to global installations ([#&#8203;1613](TypeStrong/ts-node#1613), [#&#8203;1627](TypeStrong/ts-node#1627))
    -   global fallback only occurs when using a global installation of ts-node
-   Add support for TypeScript's `traceResolution` output ([docs](https://www.typescriptlang.org/tsconfig/#traceResolution)) ([#&#8203;1128](TypeStrong/ts-node#1128), [#&#8203;1491](TypeStrong/ts-node#1491)) [@&#8203;TheUnlocked](https://github.com/TheUnlocked)
-   Support import assertions in ESM loader ([docs](https://nodejs.org/dist/latest-v17.x/docs/api/esm.html#import-assertions)) ([#&#8203;1557](TypeStrong/ts-node#1557), [#&#8203;1558](TypeStrong/ts-node#1558), [#&#8203;1559](TypeStrong/ts-node#1559), [#&#8203;1573](TypeStrong/ts-node#1573)) [@&#8203;Pokute](https://github.com/Pokute), [@&#8203;geigerzaehler](https://github.com/geigerzaehler)
    -   Allows importing JSON files from ESM with the requisite flag ([docs](https://nodejs.org/dist/latest-v17.x/docs/api/esm.html#json-modules))
-   `ts-node -vvv` also logs absolute paths to `ts-node` and `typescript`, to make it more obvious when you're accidentally using globally-installed versions ([#&#8203;1323](TypeStrong/ts-node#1323), [#&#8203;1620](TypeStrong/ts-node#1620))
-   Add swc target "es2022" ([#&#8203;1535](TypeStrong/ts-node#1535), [#&#8203;1540](TypeStrong/ts-node#1540))
    -   When you have target es2022 in tsconfig, will use swc's es2022 target

**Changed**

-   Initialize TypeScript compiler before starting REPL prompt ([#&#8203;1498](TypeStrong/ts-node#1498)) [@&#8203;TheUnlocked](https://github.com/TheUnlocked)
    -   Improves responsiveness for first line of REPL input
-   Use `v8-compile-cache-lib` to load typescript
    -   improves startup time ([#&#8203;1339](TypeStrong/ts-node#1339), [#&#8203;1603](TypeStrong/ts-node#1603))
-   Support both `--camelCase` and `--hyphen-case` for all CLI flags; update documentation to use `--camelCase` ([#&#8203;1598](TypeStrong/ts-node#1598), [#&#8203;1599](TypeStrong/ts-node#1599))
    -   Not a breaking change; CLI continues to accept both forms
-   Make `TSError` `diagnosticText` property non-enumerable to prevent it from being logged below the stack ([#&#8203;1632](TypeStrong/ts-node#1632))

**Fixed**

-   Fix [#&#8203;1538](TypeStrong/ts-node#1538): REPL inputs fail to transpile via swc ([#&#8203;1538](TypeStrong/ts-node#1538), [#&#8203;1541](TypeStrong/ts-node#1541), [#&#8203;1602](TypeStrong/ts-node#1602))
-   Fix [#&#8203;1478](TypeStrong/ts-node#1478): REPL erroneously logged `undefined` for all inputs after the first when using swc transpiler ([#&#8203;1478](TypeStrong/ts-node#1478), [#&#8203;1580](TypeStrong/ts-node#1580), [#&#8203;1602](TypeStrong/ts-node#1602))
-   Fix [#&#8203;1389](TypeStrong/ts-node#1389): In `--showConfig` output, emit accurate `moduleTypes` paths resolved relative to the `tsconfig.json` which declared them ([#&#8203;1389](TypeStrong/ts-node#1389), [#&#8203;1619](TypeStrong/ts-node#1619))
-   Fix: Remove indentation from `ts-node --help` output ([#&#8203;1597](TypeStrong/ts-node#1597), [#&#8203;1600](TypeStrong/ts-node#1600))
-   Fix [#&#8203;1425](TypeStrong/ts-node#1425): Merged definitions correctly into `tsconfig.schemastore-schema.json` ([#&#8203;1425](TypeStrong/ts-node#1425), [#&#8203;1618](TypeStrong/ts-node#1618))
-   Fix: Allow disabling `"use strict"` emit in SWC transpiler ([#&#8203;1531](TypeStrong/ts-node#1531), [#&#8203;1537](TypeStrong/ts-node#1537))
-   Fix: Add missing `ERR_UNKNOWN_FILE_EXTENSION` constructor; was throwing `ERR_UNKNOWN_FILE_EXTENSION is not a constructor` ([#&#8203;1562](TypeStrong/ts-node#1562)) [@&#8203;bluelovers](https://github.com/bluelovers)
-   Fix [#&#8203;1565](TypeStrong/ts-node#1565): entrypoint resolution failed on node v12.0.x and v12.1.x ([#&#8203;1565](TypeStrong/ts-node#1565), [#&#8203;1566](TypeStrong/ts-node#1566)) [@&#8203;davidmurdoch](https://github.com/davidmurdoch)

#### Docs

-   Explain `env -S` flag for shebangs ([docs](https://typestrong.org/ts-node/docs/usage#shebang)) ([#&#8203;1448](TypeStrong/ts-node#1448), [#&#8203;1545](TypeStrong/ts-node#1545)) [@&#8203;sheeit](https://github.com/sheeit), [@&#8203;chee](https://github.com/chee)
-   Suggest `skipIgnore` when you want to compile files in node_modules ([docs](https://typestrong.org/ts-node/docs/how-it-works)) ([#&#8203;1553](TypeStrong/ts-node#1553)) [@&#8203;webstrand](https://github.com/webstrand)
-   Fix typo in `moduleTypes` on options page ([docs](https://typestrong.org/ts-node/docs/options)) ([#&#8203;1630](TypeStrong/ts-node#1630), [#&#8203;1633](TypeStrong/ts-node#1633))

#### Misc

-   Adds experimental `experimentalResolverFeatures` option, but it does not do anything yet ([#&#8203;1514](TypeStrong/ts-node#1514), [#&#8203;1614](TypeStrong/ts-node#1614))

https://github.com/TypeStrong/ts-node/milestone/4

</details>

---

### Configuration

📅 **Schedule**: At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox.

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

Co-authored-by: cabr2-bot <[email protected]>
Reviewed-on: https://codeberg.org/Calciumdibromid/CaBr2/pulls/1156
Reviewed-by: crapStone <[email protected]>
Co-authored-by: Calciumdibromid Bot <[email protected]>
Co-committed-by: Calciumdibromid Bot <[email protected]>
@cainrus
Copy link

cainrus commented Feb 18, 2022

If src/index.ts is ignored and dist/index.js resolves to src/index.ts, should the resolver do that mapping?

Sounds like an edge case. Lib can throw an error with any unclear behavior for the first feature-version.

@hitsthings
Copy link

Stumbled across this while looking for support for the --conditions flag in tsnode.

Just wanted to mention that the list of "Mappings to keep in mind" might also need to consider the "imports" field in package.json. I had trouble using it with paths outside the package. E.g. "#other": "../sibling-package/index.js". Either TypeScript or ts-node didn't like it (unfortunately I changed approaches before figuring out exactly what went wrong there).

@cainrus
Copy link

cainrus commented Feb 1, 2023

Just wanted to mention that the list of "Mappings to keep in mind" might also need to consider the "imports" field in package.json. I had trouble using it with paths outside the package. E.g. "#other": "../sibling-package/index.js". Either TypeScript or ts-node didn't like it (unfortunately I changed approaches before figuring out exactly what went wrong there).

Do other tools also suffer from the same issue of varied dependency mapping across different configuration files (such as package.json mappings, tsconfig, jsconfig, swc config, etc.)? Has anyone developed any external loaders to solve this problem?

@JavaScriptBach
Copy link

Is this still being worked on? I work on a large Typescript mono-repo and I can't have libraries shared by both ts-node and Node until this is implemented, because we use bare imports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
research Needs design work, investigation, or prototyping. Implementation uncertain.
Projects
None yet
Development

No branches or pull requests

4 participants