Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

module: expose getPackageJSON utility #55229

Open
wants to merge 26 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
48ab696
module: expose `getPackageJSON` utility
JakobJingleheimer Sep 30, 2024
dcddd1e
fixup: convert `path_value` to string before fn casting to path
JakobJingleheimer Oct 1, 2024
98c72f4
fixup: remove obsolete reference to `findNearestPackageJSON`
JakobJingleheimer Oct 1, 2024
d4ede54
fixup: add some tests cases
JakobJingleheimer Oct 1, 2024
92f2c62
feat: adjust `getNearestParentPackageJSON().path` to pjsonPath
JakobJingleheimer Oct 1, 2024
b83707b
feat: conditionally include non-required fields only when they exist
JakobJingleheimer Oct 1, 2024
e0beefa
fixup: remove unnecessary `JSON.parse()` (it's already parsed)
JakobJingleheimer Oct 1, 2024
6e715ec
test: add more cases
JakobJingleheimer Oct 1, 2024
458dcca
fix: provide missing `pjsonPath` for `GetNearestRawParentPackageJSON`
JakobJingleheimer Oct 1, 2024
903d201
fix: correct internal bindings type decs
JakobJingleheimer Oct 1, 2024
b567e9a
fixup: remove unnecessary string manipulation
JakobJingleheimer Oct 1, 2024
ed4db64
fixup: de-lint
JakobJingleheimer Oct 1, 2024
9a4b1d1
this is awful. why does anyone want this lint rule
JakobJingleheimer Oct 1, 2024
c23c8f1
fixup: correct yaml version needle
JakobJingleheimer Oct 2, 2024
c865d55
fixup: americanize name
JakobJingleheimer Oct 2, 2024
a9bf393
fixup: mangle md
JakobJingleheimer Oct 3, 2024
190e315
fixup: mangle c++ code
JakobJingleheimer Oct 3, 2024
a9e3ded
fixup: specific → nondescript to satisfy linting
JakobJingleheimer Oct 4, 2024
926b2b0
fixup: correct return dec in md
JakobJingleheimer Oct 5, 2024
485b1e3
fixup: remove temp fields & correct type decs
JakobJingleheimer Oct 5, 2024
f824ce9
fixup: wordsmith doc & expand examples
JakobJingleheimer Oct 5, 2024
7deeb7f
fixup: leverage validate* utils
JakobJingleheimer Oct 5, 2024
cf49f71
fixup: `ToString` → `ToStringView `
JakobJingleheimer Oct 5, 2024
358486c
fixup: support file URL strings, update arg name, add test case
JakobJingleheimer Oct 5, 2024
d9bdf23
fixup: damn this limited docs types validator
JakobJingleheimer Oct 5, 2024
bf5265d
fixup: require url → internal/url
JakobJingleheimer Oct 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions doc/api/module.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,39 @@ added: v22.8.0
* Returns: {string|undefined} Path to the [module compile cache][] directory if it is enabled,
or `undefined` otherwise.

### `module.getPackageJSON(startPath[, everything])`

<!-- YAML
added: VERSION
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
-->

> Stability: 1.1 - Active Development

* `startPath` {URL['pathname']} Where to start looking
* `everything` {boolean} Whether to return the full contents of the found package.json
Copy link
Contributor

@aduh95 aduh95 Oct 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure it is a good idea to introduce yet-another way of loading a JSON file, wouldn't it be simpler to return the path and let the user deal with reading/parsing the file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered that (there's another PR I opened and closed that exposed findPackageJSON, which did exactly that). We're already reading the pjson and caching it, so IMO that would force a de-op as well as force the user to manually do what we're already doing. So, why?

Copy link
Member

@joyeecheung joyeecheung Oct 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think returning the parsed data means every time we consider parsing another field in the package.json, it needs to be surfaced into this API, even though that field may not even be useful for tools, but we'll end up paying the serialization/deserialization cost even though internally only selected places need selected fields. IMO even just returning the string would be better than this. Users need this for the lookup algorithm, they tend to have other fields to parse anyway.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May I recommend a different naming and API surface?

module.findPackageJson(path, { includeContents: true })

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, @joyeecheung I'm not sure I follow in your message. I think you've misunderstood the behaviour here: internally, we do not use the full version—we do use getNearestParentPackageJSON but do not set everything to true. When everything is not true, only the select fields we use internally are parsed. That has not changed in this PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May I recommend a different naming and API surface?

module.findPackageJson(path, { includeContents: true })

@anonrig if you recall, when we were working on this a few days ago, that did not make sense due to how the C++ worked, which is what caused me to abandon the findPackageJSON + getPackageJSON route in the first place.

Regarding "find" + includeContents, this seems counter-intuitive to me: I expect something called "find" to only locate, not retrieve. I could perhaps see the inverse

module.getPackageJSON(path, { includeContents: false })

I do not like either option though because the shape of the return is changed significantly in a foot-gun way:

getPackageJSON(path, false) // '…/package.json'

getPackageJSON(path, true)  // { data: {…}, path: '…/package.json' }

(I simplified the 2nd arg for brevity, not necessarily to say we shouldn't use an object)

I suppose it could always return an object for consistency, but that seems like a clumsy compromise

getPackageJSON(path, false) // { path: '…/package.json' }

getPackageJSON(path, true)  // { data: {…}, path: '…/package.json' }

If we were to do this, I think exposing 2 different utils (findPackageJSON + getPackageJSON) would be better.

* Returns: {undefined | {
data: {
name?: string,
type?: 'commonjs' | 'module' | 'none',
exports?: string | string[] | Record<string, unknown>,
imports?: string | string[] | Record<string, unknown>,
[key: string]?: unknown,
},
path: URL['pathname'],
}}

In addition to being available to users, this utility is used internally when
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
resolving various aspects about a module.

```mjs
import { getPackageJSON } from 'node:module';

const pjson = getPackageJSON(import.meta.resolve('some-package'), true)?.data;

pjson?.name; // 'some-package-real-name'
pjson?.types; // './index.d.ts'
```

### `module.isBuiltin(moduleName)`

<!-- YAML
Expand Down
18 changes: 9 additions & 9 deletions lib/internal/modules/cjs/loader.js
Original file line number Diff line number Diff line change
Expand Up @@ -604,11 +604,11 @@ function trySelf(parentPath, request) {
try {
const { packageExportsResolve } = require('internal/modules/esm/resolve');
return finalizeEsmResolution(packageExportsResolve(
pathToFileURL(pkg.path + '/package.json'), expansion, pkg.data,
pathToFileURL(pkg.path), expansion, pkg.data,
pathToFileURL(parentPath), getCjsConditions()), parentPath, pkg.path);
} catch (e) {
if (e.code === 'ERR_MODULE_NOT_FOUND') {
throw createEsmNotFoundErr(request, pkg.path + '/package.json');
throw createEsmNotFoundErr(request, pkg.path);
}
throw e;
}
Expand Down Expand Up @@ -1219,9 +1219,10 @@ Module._resolveFilename = function(request, parent, isMain, options) {
try {
const { packageImportsResolve } = require('internal/modules/esm/resolve');
return finalizeEsmResolution(
packageImportsResolve(request, pathToFileURL(parentPath),
getCjsConditions()), parentPath,
pkg.path);
packageImportsResolve(request, pathToFileURL(parentPath), getCjsConditions()),
parentPath,
pkg.path,
);
} catch (e) {
if (e.code === 'ERR_MODULE_NOT_FOUND') {
throw createEsmNotFoundErr(request);
Expand Down Expand Up @@ -1281,8 +1282,7 @@ function finalizeEsmResolution(resolved, parentPath, pkgPath) {
if (actual) {
return actual;
}
const err = createEsmNotFoundErr(filename,
path.resolve(pkgPath, 'package.json'));
const err = createEsmNotFoundErr(filename, pkgPath);
throw err;
}

Expand Down Expand Up @@ -1614,7 +1614,7 @@ function loadTS(module, filename) {

const parent = module[kModuleParent];
const parentPath = parent?.filename;
const packageJsonPath = path.resolve(pkg.path, 'package.json');
const packageJsonPath = pkg.path;
const usesEsm = containsModuleSyntax(content, filename);
const err = new ERR_REQUIRE_ESM(filename, usesEsm, parentPath,
packageJsonPath);
Expand Down Expand Up @@ -1673,7 +1673,7 @@ Module._extensions['.js'] = function(module, filename) {
// This is an error path because `require` of a `.js` file in a `"type": "module"` scope is not allowed.
const parent = module[kModuleParent];
const parentPath = parent?.filename;
const packageJsonPath = path.resolve(pkg.path, 'package.json');
const packageJsonPath = pkg.path;
const usesEsm = containsModuleSyntax(content, filename);
const err = new ERR_REQUIRE_ESM(filename, usesEsm, parentPath,
packageJsonPath);
Expand Down
2 changes: 1 addition & 1 deletion lib/internal/modules/esm/resolve.js
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ const legacyMainResolveExtensionsIndexes = {
* 4. TRY(pkg_url/index.js, pkg_url/index.json, pkg_url/index.node)
* 5. NOT_FOUND
* @param {URL} packageJSONUrl
* @param {import('typings/internalBinding/modules').PackageConfig} packageConfig
* @param {import('typings/internalBinding/modules').RecognisedPackageConfig} packageConfig
* @param {string | URL | undefined} base
* @returns {URL}
*/
Expand Down
98 changes: 69 additions & 29 deletions lib/internal/modules/package_json_reader.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,26 @@ const {
ArrayIsArray,
JSONParse,
ObjectDefineProperty,
StringPrototypeLastIndexOf,
StringPrototypeSlice,
} = primordials;
const modulesBinding = internalBinding('modules');
const { resolve, sep } = require('path');
const { resolve } = require('path');
const { kEmptyObject } = require('internal/util');
const {
codes: {
ERR_INVALID_ARG_TYPE,
},
} = require('internal/errors');

/**
* @typedef {import('typings/internalBinding/modules').FullPackageConfig} FullPackageConfig
* @typedef {import('typings/internalBinding/modules').RecognisedPackageConfig} RecognisedPackageConfig
* @typedef {import('typings/internalBinding/modules').SerializedPackageConfig} SerializedPackageConfig
*/

/**
* @param {string} path
* @param {import('typings/internalBinding/modules').SerializedPackageConfig} contents
* @returns {import('typings/internalBinding/modules').PackageConfig}
* @param {SerializedPackageConfig} contents
* @returns {RecognisedPackageConfig}
*/
function deserializePackageJSON(path, contents) {
if (contents === undefined) {
Expand Down Expand Up @@ -51,19 +60,23 @@ function deserializePackageJSON(path, contents) {
exists: true,
pjsonPath,
name,
main,
type,
// This getters are used to lazily parse the imports and exports fields.
get imports() {
const value = requiresJSONParse(plainImports) ? JSONParse(plainImports) : plainImports;
ObjectDefineProperty(this, 'imports', { __proto__: null, value });
return this.imports;
},
get exports() {
const value = requiresJSONParse(plainExports) ? JSONParse(plainExports) : plainExports;
ObjectDefineProperty(this, 'exports', { __proto__: null, value });
return this.exports;
},
...(main != null && { main }),
...(type != null && { type }),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you change these lines? Meaning anything after the highlighted lines included 63-64.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because as it was before, empty fields are forced to be present but with no value:

{
  "name": "foo",
  "exports": "./index.js"
}
{
  name: 'foo'
  main: undefined,
  exports: './index.js',
}

The only result is hassle and gotchas: ex 'main' in pkgtrue, which is a lie ("main" is not in the package.json).

And in tests, it forces you to add extra properties that don't matter:

assert.deepStrictEqual(pkg, {
  name: 'foo'
  exports: './index.js',
});

Pedantically fails because main: undefined is not explicitly specified.

...(plainImports != null && {
// This getters are used to lazily parse the imports and exports fields.
get imports() {
const value = requiresJSONParse(plainImports) ? JSONParse(plainImports) : plainImports;
ObjectDefineProperty(this, 'imports', { __proto__: null, value });
return this.imports;
},
}),
...(plainExports != null && {
get exports() {
const value = requiresJSONParse(plainExports) ? JSONParse(plainExports) : plainExports;
ObjectDefineProperty(this, 'exports', { __proto__: null, value });
return this.exports;
},
}),
};
}

Expand All @@ -75,7 +88,7 @@ function deserializePackageJSON(path, contents) {
* specifier?: URL | string,
* isESM?: boolean,
* }} options
* @returns {import('typings/internalBinding/modules').PackageConfig}
* @returns {RecognisedPackageConfig}
*/
function read(jsonPath, { base, specifier, isESM } = kEmptyObject) {
// This function will be called by both CJS and ESM, so we need to make sure
Expand All @@ -94,7 +107,7 @@ function read(jsonPath, { base, specifier, isESM } = kEmptyObject) {
* @deprecated Expected to be removed in favor of `read` in the future.
* Behaves the same was as `read`, but appends package.json to the path.
* @param {string} requestPath
* @return {PackageConfig}
* @return {RecognisedPackageConfig}
*/
function readPackage(requestPath) {
// TODO(@anonrig): Remove this function.
Expand All @@ -104,29 +117,56 @@ function readPackage(requestPath) {
/**
* Get the nearest parent package.json file from a given path.
* Return the package.json data and the path to the package.json file, or undefined.
* @param {string} checkPath The path to start searching from.
* @returns {undefined | {data: import('typings/internalBinding/modules').PackageConfig, path: string}}
* @param {URL['href'] | URL['pathname']} startPath The path to start searching from.
* @param {boolean} everything Whether to include the full contents of the package.json.
* @returns {undefined | {
* data: everything extends true ? FullPackageConfig : RecognisedPackageConfig,
* path: URL['pathname'],
* }}
*/
function getNearestParentPackageJSON(checkPath) {
const result = modulesBinding.getNearestParentPackageJSON(checkPath);
function getNearestParentPackageJSON(startPath, everything = false) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of everything, can we change this to includeContents?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And preferably make this an object rather than a plain argument. getPackageJSON(path, true) does not help the developer, unless they look into the documentation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

includeContents does not make sense: it will include some of the contents regardless.

does not help the developer, unless they look into the documentation.

IntelliSense :)

I think an object makes sense when there are (or potentially will be) multiple configuration options. Here it seems verbose.

if (typeof startPath !== 'string') {
throw new ERR_INVALID_ARG_TYPE('startPath', 'string', startPath);
}
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
if (
everything !== undefined &&
typeof everything !== 'boolean'
) {
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
throw new ERR_INVALID_ARG_TYPE('everything', 'boolean', everything);
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved
}

if (everything) {
const result = modulesBinding.getNearestRawParentPackageJSON(startPath);
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved

return {
data: {
__proto__: null,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having null-prototype object returned to the object feels weird

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. I did it for consistency between everything (not everything is used internally and needs the null prototype).

...JSONParse(result[0]),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes an unnecessary deep copy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I need to spread here in order to void the prototype. Or is voiding the prototype here not necessary because it came from c-land?

},
path: result[1],
};
}

const result = modulesBinding.getNearestParentPackageJSON(startPath);

if (result === undefined) {
return undefined;
}

const data = deserializePackageJSON(checkPath, result);
const data = deserializePackageJSON(startPath, result);

const { pjsonPath: path } = data;

// Path should be the root folder of the matched package.json
// For example for ~/path/package.json, it should be ~/path
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: Why does this statement doesn't hold anymore?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const path = StringPrototypeSlice(data.pjsonPath, 0, StringPrototypeLastIndexOf(data.pjsonPath, sep));
delete data.exists;
delete data.pjsonPath;
JakobJingleheimer marked this conversation as resolved.
Show resolved Hide resolved

return { data, path };
}

/**
* Returns the package configuration for the given resolved URL.
* @param {URL | string} resolved - The resolved URL.
* @returns {import('typings/internalBinding/modules').PackageConfig} - The package configuration.
* @returns {RecognisedPackageConfig} - The package configuration.
*/
function getPackageScopeConfig(resolved) {
const result = modulesBinding.getPackageScopeConfig(`${resolved}`);
Expand Down
4 changes: 4 additions & 0 deletions lib/module.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@ const {
flushCompileCache,
getCompileCacheDir,
} = require('internal/modules/helpers');
const {
getNearestParentPackageJSON,
} = require('internal/modules/package_json_reader');

Module.findSourceMap = findSourceMap;
Module.register = register;
Expand All @@ -19,4 +22,5 @@ Module.enableCompileCache = enableCompileCache;
Module.flushCompileCache = flushCompileCache;

Module.getCompileCacheDir = getCompileCacheDir;
Module.getPackageJSON = getNearestParentPackageJSON;
module.exports = Module;
31 changes: 31 additions & 0 deletions src/node_modules.cc
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,32 @@ const BindingData::PackageConfig* BindingData::TraverseParent(
return nullptr;
}

void BindingData::GetNearestRawParentPackageJSON(
const v8::FunctionCallbackInfo<v8::Value>& args
) {
CHECK_GE(args.Length(), 1);
CHECK(args[0]->IsString());

Realm* realm = Realm::GetCurrent(args);
BufferValue path_value(realm->isolate(), args[0]);

// Required for long paths in Windows
ToNamespacedPath(realm->env(), &path_value);

auto package_json = TraverseParent(
realm,
std::filesystem::path(path_value.ToString()));

if (package_json != nullptr) {
Local<Value> result[2] = {
ToV8Value(realm->context(), package_json->raw_json).ToLocalChecked(),
ToV8Value(realm->context(), package_json->file_path).ToLocalChecked(),
};

args.GetReturnValue().Set(Array::New(realm->isolate(), result, 2));
}
}

void BindingData::GetNearestParentPackageJSON(
const v8::FunctionCallbackInfo<v8::Value>& args) {
CHECK_GE(args.Length(), 1);
Expand Down Expand Up @@ -494,6 +520,10 @@ void BindingData::CreatePerIsolateProperties(IsolateData* isolate_data,
target,
"getNearestParentPackageJSON",
GetNearestParentPackageJSON);
SetMethod(isolate,
target,
"getNearestRawParentPackageJSON",
GetNearestRawParentPackageJSON);
SetMethod(isolate, target, "getPackageScopeConfig", GetPackageScopeConfig);
SetMethod(isolate, target, "enableCompileCache", EnableCompileCache);
SetMethod(isolate, target, "getCompileCacheDir", GetCompileCacheDir);
Expand Down Expand Up @@ -527,6 +557,7 @@ void BindingData::RegisterExternalReferences(
registry->Register(ReadPackageJSON);
registry->Register(GetNearestParentPackageJSONType);
registry->Register(GetNearestParentPackageJSON);
registry->Register(GetNearestRawParentPackageJSON);
registry->Register(GetPackageScopeConfig);
registry->Register(EnableCompileCache);
registry->Register(GetCompileCacheDir);
Expand Down
2 changes: 2 additions & 0 deletions src/node_modules.h
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,8 @@ class BindingData : public SnapshotableObject {
const v8::FunctionCallbackInfo<v8::Value>& args);
static void GetNearestParentPackageJSONType(
const v8::FunctionCallbackInfo<v8::Value>& args);
static void GetNearestRawParentPackageJSON(
const v8::FunctionCallbackInfo<v8::Value>& args);
static void GetPackageScopeConfig(
const v8::FunctionCallbackInfo<v8::Value>& args);
static void GetPackageJSONScripts(
Expand Down
Empty file.
9 changes: 9 additions & 0 deletions test/fixtures/packages/nested-types-field/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"name": "package-with-unrecognised-fields",
"type": "module",
"exports": {
"default": "./index.js",
"types": "./index.d.ts"
},
"unrecognised": true
}
Empty file.
5 changes: 5 additions & 0 deletions test/fixtures/packages/root-types-field/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"name": "package-with-unrecognised-fields",
"type": "module",
"types": "./index.d.ts"
}
Loading
Loading