node-api: add napi_create_buffer_from_arraybuffer method

[mertcanaltin]

introduces a new N-API method napi_create_buffer_from_arraybuffer that allows the creation of a Buffer from an ArrayBuffer. This new method provides functionality similar to napi_create_typedarray, enabling developers to specify a byte offset and length to create a Buffer from a portion of the ArrayBuffer. This feature addresses the limitation where it was previously not possible to create a Buffer directly from an ArrayBuffer using N-API.

Closes: #54440

[nodejs-github-bot]

Review requested:

• @nodejs/node-api

[mertcanaltin]

I made make in my local environment and ran the make only-tesy command and saw that it passed the build and tests, I will check the places that give errors again.

[ronag]

I'm not sure this is actually creating a node buffer.

[ronag]

LGTM but would like another review from someone more familiar with this part of the code.

[codecov]

Codecov Report

Attention: Patch coverage is 50.00000% with 8 lines in your changes missing coverage. Please review.

Project coverage is 88.41%. Comparing base (aac8ba7) to head (6c4e386).
Report is 97 commits behind head on main.

Files with missing lines	Patch %	Lines
src/node_api.cc	50.00%	3 Missing and 5 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #54505      +/-   ##
==========================================
+ Coverage   88.24%   88.41%   +0.16%     
==========================================
  Files         651      652       +1     
  Lines      183856   186628    +2772     
  Branches    35853    36065     +212     
==========================================
+ Hits       162251   165001    +2750     
+ Misses      14896    14885      -11     
- Partials     6709     6742      +33     

Files with missing lines	Coverage Δ
src/node_api.cc	76.26% <50.00%> (-0.53%)	⬇️

... and 135 files with indirect coverage changes

[legendecas]

@ronag would you mind sharing a use case for this method? Does https://nodejs.org/api/n-api.html#napi_create_typedarray work for the use case?

We talked about this in Node-API meeting, a new node-api can not be easily changed or deprecated, a concrete use case would really help us understand the necessity to add the new API and if the API interface sufficient.

[ronag]

@ronag would you mind sharing a use case for this method? Does https://nodejs.org/api/n-api.html#napi_create_typedarray work for the use case?

We talked about this in Node-API meeting, a new node-api can not be easily changed or deprecated, a concrete use case would really help us understand the necessity to add the new API and if the API interface sufficient.

It doesn't work if I need to return node buffers in the js api. I guess I could change the prototype in js but that seems a bit ugly.

[ronag]

@nodejs/node-api

[mertcanaltin]

I wonder if you have a suggestion for incompatible signatures here, I tried const but I got an error again

gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  CC(target) Release/obj.target/test_buffer/test_buffer.o
../test_buffer.c:47:42: warning: incompatible function pointer types passing 'void (napi_env, void *, void *)' (aka 'void (struct napi_env__ *, void *, void *)') to parameter of type 'node_api_basic_finalize' (aka 'void (*)(const struct napi_env__ *, void *, void *)') [-Wincompatible-function-pointer-types]
          env, sizeof(theText), theCopy, deleteTheText,
                                         ^~~~~~~~~~~~~
../../../js-native-api/common.h:90:27: note: expanded from macro 'NODE_API_CALL'
  NODE_API_CALL_BASE(env, the_call, NULL)
                          ^~~~~~~~
../../../js-native-api/common.h:74:10: note: expanded from macro 'NODE_API_CALL_BASE'
    if ((the_call) != napi_ok) {                                         \
         ^~~~~~~~
/Users/mert/Library/Caches/node-gyp/20.17.0/include/node/node_api.h:134:53: note: passing argument to parameter 'finalize_cb' here
                            node_api_basic_finalize finalize_cb,
                                                    ^
../test_buffer.c:106:49: warning: incompatible function pointer types passing 'void (napi_env, void *, void *)' (aka 'void (struct napi_env__ *, void *, void *)') to parameter of type 'node_api_basic_finalize' (aka 'void (*)(const struct napi_env__ *, void *, void *)') [-Wincompatible-function-pointer-types]
          env, sizeof(theText), (void*)theText, noopDeleter,
                                                ^~~~~~~~~~~
../../../js-native-api/common.h:90:27: note: expanded from macro 'NODE_API_CALL'
  NODE_API_CALL_BASE(env, the_call, NULL)
                          ^~~~~~~~
../../../js-native-api/common.h:74:10: note: expanded from macro 'NODE_API_CALL_BASE'
    if ((the_call) != napi_ok) {                                         \
         ^~~~~~~~
/Users/mert/Library/Caches/node-gyp/20.17.0/include/node/node_api.h:134:53: note: passing argument to parameter 'finalize_cb' here
                            node_api_basic_finalize finalize_cb,
                                                    ^
2 warnings generated.
  SOLINK_MODULE(target) Release/test_buffer.node
  CC(target) Release/obj.target/test_finalizer/test_finalizer.o
  SOLINK_MODULE(target) Release/test_finalizer.node
gyp info ok 

[legendecas]

The type of the first argument of a finalizer with NAPI_EXPERIMENTAL should be node_api_basic_env: https://nodejs.org/api/n-api.html#node_api_basic_finalize, and per suggestion #54505 (comment) for `node_api_post_finalizer.

[vmoroz]

Just to be on the same page: my suggestion regarding the use of node_api_post_finalizer was in the context of the failing tests caused by the new NAPI_EXPERIMENTAL in the gyp file. The malignDeleter function in the test_finalizer.c file calls functions like napi_get_global that cause the crash in the new immediate finalizers while running tests. To address this issue the code in the malignDeleter function must be changed to a call to a single function node_api_post_finalizer which would call a new function with the body of the former malignDeleter function.

I made an update @vmoroz

@mertcanaltin , @legendecas , looking back at this code I think I was completely wrong about the test_finalizer.c.
I did not realize that it is a different DLL.
I previously saw the issue in the unit test related to the finalizers, and then lately found the finalizer to be only in the test_finalizer.c, and wrongly assumed that the issue is in this file instead of it being in the original file where it was then removed.
Sorry about that.
It must be OK to keep the change as long as the finalizers signature match to the declaration in the js_native_api.h.
Otherwise, you could rollback changes in the test_finalizer.c and remove the NAPI_EXPERIMENTAL for the finalizer test.

[vmoroz]

The type of the first argument of a finalizer with NAPI_EXPERIMENTAL should be node_api_basic_env: https://nodejs.org/api/n-api.html#node_api_basic_finalize, and per suggestion #54505 (comment) for `node_api_post_finalizer.

It is interesting that MSVC compiler does not have any issues compiling the code.
The issue shows up only for the Linux and Mac compilers.
MSVC does not implement the true "C" compilation and it seems that the new basic env type trick does not work for it.

[mertcanaltin]

I made the signature definitions 🚀

[mertcanaltin]

I set napi signature but get call to undeclared function error, I wonder if anyone has a comment on this

gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  CC(target) Release/obj.target/test_buffer/test_buffer.o
../test_buffer.c:148:12: error: call to undeclared function 'node_api_create_buffer_from_arraybuffer'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
  status = node_api_create_buffer_from_arraybuffer(
           ^
../test_buffer.c:148:12: note: did you mean 'napi_create_external_arraybuffer'?
/Users/mert/Library/Caches/node-gyp/20.17.0/include/node/js_native_api.h:433:1: note: 'napi_create_external_arraybuffer' declared here
napi_create_external_arraybuffer(napi_env env,
^
1 error generated.
make: *** [Release/obj.target/test_buffer/test_buffer.o] Error 1

[mertcanaltin]

I set napi signature but get call to undeclared function error, I wonder if anyone has a comment on this

gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]

  CC(target) Release/obj.target/test_buffer/test_buffer.o

../test_buffer.c:148:12: error: call to undeclared function 'node_api_create_buffer_from_arraybuffer'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]

  status = node_api_create_buffer_from_arraybuffer(

           ^

../test_buffer.c:148:12: note: did you mean 'napi_create_external_arraybuffer'?

/Users/mert/Library/Caches/node-gyp/20.17.0/include/node/js_native_api.h:433:1: note: 'napi_create_external_arraybuffer' declared here

napi_create_external_arraybuffer(napi_env env,

^

1 error generated.

make: *** [Release/obj.target/test_buffer/test_buffer.o] Error 1

hmm I'm getting errors on my local, but here the tests seem to have passed 🤔

[vmoroz]

hmm I'm getting errors on my local, but here the tests seem to have passed 🤔

I suspect you are running gyp for the installed Node.js instance instead of the one built on your machine. This is why it cannot find the new function that is added in your PR.

I typically build the tests on Windows with vcbuild.bat and then iterate on them inside of Visual Studio. To run them using the built Node.js instance I usually do out\Debug\node.exe test\node-api\test_buffer\test.js. I.e. the key here is to use the Node.js instance from the out folder. I guess on Mac it can be done in a similar way.

[mertcanaltin]

I suspect you are running gyp for the installed Node.js instance instead of the one built on your machine. This is why it cannot find the new function that is added in your PR.

I typically build the tests on Windows with vcbuild.bat and then iterate on them inside of Visual Studio. To run them using the built Node.js instance I usually do out\Debug\node.exe test\node-api\test_buffer\test.js. I.e. the key here is to use the Node.js instance from the out folder. I guess on Mac it can be done in a similar way.

this worked very well thank you very much

[vmoroz]

LGTM