feat(bundler): implement enum inlining / more constant folding #12144

paperdave · 2024-06-25T07:47:35Z

Initially was a PR to fix an enum bug, but it turned out the bug was due to a lack of a feature: enum inlining. So I decided to do it, since it was a feature I personally have a big use for (For example, the bun.report project has a regex hack workaround to force enum inlining, because the codegen otherwise is an abomination).

edit: As time went, this PR turned into not just that, but a lot of changes to constant folding. i am much happier with how the bundler handles this now.

Fixes #11764
Fixes #2945
Fixes #2889
Fixes #12285
Fixes
Draft TODO:

e_inlined_enum
inline enum as well as in simplify functions
Enum use-before-define behavior
Implement required changes in TS namespaces
Fix sibling namespace scopes
Cross-file enums while bundling
Tests
Note: late-stage numeric folding (js printer) is not implemented, and will not be for now

extras:

constnat folding now happens with all numeric operators
constant folding now happens with n + "baa", where n is any constant number. this uses webkit's code to properly adhere to JS spec
constant folding now happens with +"1.2". this uses webkit's code to properly adhere to JS spec
constant folding requires --minify-syntax, where previously it seemed to always be applied. Though keep in mind the JS runtime transpiles all files with --minify-syntax
constant folding is now enabled in the context of require, import, and macro calls; even if minify-syntax is disabled. This was previously not an issue because it was always enabled.
string and template string folding is much more advanced now. this may need more tests

Simple Sample

enum A {
  Hello = 1,
  World,
}

console.log(A.Hello);
console.log(A.World);
console.log(A.Hello + A.World);

After:

// index.ts
console.log(1 /* Hello */);
console.log(2 /* World */);
console.log(3);

Before:

// index.ts
var A;
(function(A2) {
  A2[A2["Hello"] = 1] = "Hello";
  A2[A2["World"] = 2] = "World";
})(A || (A = {}));
console.log(A.Hello);
console.log(A.World);
console.log(A.Hello + A.World);

If you reference the enum directly, you can get the two-way map. This even works with const enum, as it is TSC's job to prohibit such usage.

enum A {
  Hello = 1,
  World,
}

console.log(A);

The codegen of these enums have been improved. Projects with heavy enum usage should see smaller bundle sizes. Now:

var A;
((A2) => {
  A2[A2.Hello = 1] = "Hello";
  A2[A2.World = 2] = "World";
})(A ||= {});

Before:

// index.ts
var A;
(function(A2) {
  A2[A2["Hello"] = 1] = "Hello";
  A2[A2["World"] = 2] = "World";
})(A || (A = {}));
console.log(A);

It would be interesting as a future project to see if the size of these enums could be shrunk further, but esbuild doesn't do that, and I think personally it's fine to leave enum in the dust as const enum and inlining all values becomes the new norm.

github-actions · 2024-06-25T08:39:12Z

❌ @Jarred-Sumner, your commit has failing tests :(

🐧💪 1 failing tests Linux AARCH64

test/js/deno/crypto/webcrypto.test.ts 1 failing

🐧🖥 1 failing tests Linux x64 baseline

test/js/bun/shell/leak.test.ts 1 failing

🪟💻 4 failing tests Windows x64 baseline

test/cli/install/registry/bun-install-registry.test.ts 1 failing
test/cli/install/registry/bun-install-windowsshim.test.ts code 1
test/js/bun/dns/resolve-dns.test.ts SIGKILL
test/js/node/child_process/child_process.test.ts 1 failing

🪟💻 3 failing tests Windows x64

test/cli/install/registry/bun-install-registry.test.ts SIGKILL
test/cli/install/registry/bun-install-windowsshim.test.ts code 1
test/js/node/child_process/child_process.test.ts 1 failing

View logs

paperdave · 2024-06-29T03:59:34Z

src/js_ast.zig

+        /// by calling out to the APIs in WebKit which are responsible for this operation.
+        ///
+        /// This can return `null` in wasm builds to avoid linking JSC
+        pub fn toString(this: Number, allocator: std.mem.Allocator) ?string {


note: this is renamed from toStringSafely to toString, not because it is now less safe, but because the original intent of "we are taking a safer approach" does not apply.

paperdave · 2024-06-29T04:00:59Z

build.zig

@@ -123,7 +123,7 @@ pub fn getOSGlibCVersion(os: OperatingSystem) ?Version {
 }

 pub fn build(b: *Build) !void {
-    std.debug.print("zig build v{s}\n", .{builtin.zig_version_string});
+    std.log.info("zig compiler v{s}", .{builtin.zig_version_string});


apologies for doing a ton of other random things in this pr. i fixed the banned.json linter which resulted in needing to apply a ton of small transformations. i did this because i was sick of std.debug.print showing at all in my cmd+shift+f menu. this also resulted in the deletion of some old files, which some did not even past astgen.

additionally, i have been using a client side spell-checker, so it just becomes annoying to see a ton of yellow underlines. maybe i will upstream a ci check to keep spelling somewhat intact (i have been very impressed with the typos cli and lsp, it is very low false positive).

paperdave · 2024-06-29T04:02:47Z

src/bun.zig

-        @compileError("This function can only be called in comptime.");
-    }
+    var x = 0; // if you hit an error on this line, you are not in a comptime context
+    _ = &x;
 }


comptime @inComptime is a compiler error in 0.14 and in 0.13 does not do what we want.

paperdave · 2024-06-29T04:03:53Z

src/feature_flags.zig

@@ -46,8 +46,6 @@ pub const allow_json_single_quotes = true;

 pub const react_specific_warnings = true;

-pub const log_allocations = false;


this behavior is now provided via BUN_DEBUG_mimalloc=1

src/bundler/bundle_v2.zig

Jarred-Sumner · 2024-06-29T05:13:47Z

src/js_ast.zig

+    /// generated proxy symbols that represent the property access "x3.y". This
+    /// map is unique per namespace block because "x3" is the argument symbol that
+    /// is specific to that particular namespace block.
+    property_accesses: std.StringArrayHashMapUnmanaged(Ref) = .{},


Suggested change

property_accesses: std.StringArrayHashMapUnmanaged(Ref) = .{},

property_accesses: bun.StringArrayHashMapUnmanaged(Ref) = .{},

src/js_parser.zig

paperdave · 2024-07-03T00:27:40Z

two bugs remain

utf16 identifiers
template rope thing

Jarred-Sumner · 2024-07-03T04:54:05Z

src/js_parser.zig

@@ -6819,7 +7200,7 @@ fn NewParser_(
                }
            }

-            if (comptime !Environment.isRelease) {
+            if (comptime Environment.allow_assert) {


Suggested change

if (comptime Environment.allow_assert) {

if (comptime Environment.isDebug) {

Jarred-Sumner · 2024-07-03T04:58:56Z

src/js_parser.zig

@@ -8286,11 +8650,7 @@ fn NewParser_(
                            else => {
                                if (comptime get_metadata) {
                                    const find_result = p.findSymbol(logger.Loc.Empty, p.lexer.identifier) catch unreachable;
-                                    if (p.known_enum_values.contains(find_result.ref)) {


What was this change?

known_enum_values was removed in favor of real enum inlining.

Jarred-Sumner · 2024-07-03T04:59:30Z

src/js_parser.zig

+                                        .loc = ns.name.loc,
+                                    },
+                                );
+                                // try p.ref_to_ts_namespace_member.put(


Was this deliberately commented out?

this second put is seen in esbuild's code but i cannot think of a situation where the value is not already in the map

paperdave added 3 commits June 24, 2024 15:01

start the enum pr

982fa2c

Merge remote-tracking branch 'origin/main' into dave/the-enum-pr

32561d8

wuh

a30741f

paperdave added 3 commits June 25, 2024 15:20

Merge remote-tracking branch 'origin/main' into dave/the-enum-pr

692a3e6

compiler crash snapshot

1f9c562

enum hell

25a0695

paperdave changed the title ~~feat(bundler): implement enum inlining / ts namespace fixes~~ feat(bundler): implement enum inlining / more constant folding Jun 27, 2024

paperdave added 2 commits June 27, 2024 18:13

wuh

856c895

a

3aec7f1

paperdave force-pushed the dave/the-enum-pr branch from 76e7420 to 3aec7f1 Compare June 28, 2024 01:56

This comment was marked as outdated.

Sign in to view

paperdave added 4 commits June 27, 2024 18:58

Merge remote-tracking branch 'origin/main' into dave/the-enum-pr

86f3145

Merge remote-tracking branch 'origin/main' into dave/the-enum-pr

659c429

yay

443664e

more

8eaca9c

paperdave commented Jun 29, 2024

View reviewed changes

paperdave marked this pull request as ready for review June 29, 2024 04:25

paperdave added 2 commits June 28, 2024 21:41

ok

ffc5a14

j

420bb9e

Jarred-Sumner reviewed Jun 29, 2024

View reviewed changes

src/bundler/bundle_v2.zig Outdated Show resolved Hide resolved

Jarred-Sumner reviewed Jun 29, 2024

View reviewed changes

paperdave marked this pull request as draft June 29, 2024 05:29

paperdave added 3 commits June 29, 2024 00:32

guh

dcacb31

btw re-disabling constant inlining since it never was enabled

14bab40

hell

8464487

This was referenced Jul 2, 2024

Reduce the size of JS code when using --minify-syntax and enum #12285

Closed

bun build could do better minification #9418

Closed

paperdave added 2 commits July 1, 2024 19:46

resolve build failure

8f8ce3f

Merge remote-tracking branch 'origin/main' into dave/the-enum-pr

c3f24ad

dylan-conway reviewed Jul 2, 2024

View reviewed changes

src/js_parser.zig Show resolved Hide resolved

paperdave added 2 commits July 1, 2024 21:53

stuff

4c6c253

remove submodule i added on windows on accident, thanks vsc

e18be43

dylan-conway reviewed Jul 2, 2024

View reviewed changes

src/js_parser.zig Outdated Show resolved Hide resolved

dylan-conway reviewed Jul 2, 2024

View reviewed changes

src/js_parser.zig Show resolved Hide resolved

fix namespaces

16526a7

paperdave and others added 5 commits July 2, 2024 18:15

fix bugs

7cc34cc

Merge remote-tracking branch 'origin/main' into dave/the-enum-pr

9ba8659

resolve banned code on main

f873bb4

silly

9e50b7a

Apply formatting changes

bf69323

Jarred-Sumner reviewed Jul 3, 2024

View reviewed changes

paperdave and others added 7 commits July 2, 2024 22:14

remove known_enum_values in deinit

bdfd100

Add micro-optimization

f0e7779

Test comment

a7ad760

Test comment

95c5464

Test github action

6b61829

Apply formatting changes

cc97f1a

test github action

68913df

github-actions bot added the updated-submodule label Jul 3, 2024

Apply formatting changes

b6d0cf8

Jarred-Sumner removed the updated-submodule label Jul 3, 2024

Jarred-Sumner added 2 commits July 3, 2024 03:16

Merge branch 'main' into dave/the-enum-pr

b479bd3

Merge branch 'main' into dave/the-enum-pr

79dd97f

Jarred-Sumner marked this pull request as ready for review July 3, 2024 11:23

Jarred-Sumner merged commit 688ddbd into main Jul 3, 2024
49 of 53 checks passed

Jarred-Sumner deleted the dave/the-enum-pr branch July 3, 2024 11:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(bundler): implement enum inlining / more constant folding #12144

feat(bundler): implement enum inlining / more constant folding #12144

paperdave commented Jun 25, 2024 •

edited

Loading

github-actions bot commented Jun 25, 2024 •

edited

Loading

This comment was marked as outdated.

paperdave Jun 29, 2024

paperdave Jun 29, 2024

paperdave Jun 29, 2024

paperdave Jun 29, 2024

Jarred-Sumner Jun 29, 2024

paperdave commented Jul 3, 2024

Jarred-Sumner Jul 3, 2024

Jarred-Sumner Jul 3, 2024

paperdave Jul 3, 2024

Jarred-Sumner Jul 3, 2024

paperdave Jul 3, 2024

		@@ -46,8 +46,6 @@ pub const allow_json_single_quotes = true;

		pub const react_specific_warnings = true;

		pub const log_allocations = false;

	property_accesses: std.StringArrayHashMapUnmanaged(Ref) = .{},
	property_accesses: bun.StringArrayHashMapUnmanaged(Ref) = .{},

	if (comptime Environment.allow_assert) {
	if (comptime Environment.isDebug) {

feat(bundler): implement enum inlining / more constant folding #12144

feat(bundler): implement enum inlining / more constant folding #12144

Conversation

paperdave commented Jun 25, 2024 • edited Loading

github-actions bot commented Jun 25, 2024 • edited Loading

🐧💪 1 failing tests Linux AARCH64

🐧🖥 1 failing tests Linux x64 baseline

🪟💻 4 failing tests Windows x64 baseline

🪟💻 3 failing tests Windows x64

This comment was marked as outdated.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paperdave commented Jul 3, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paperdave commented Jun 25, 2024 •

edited

Loading

github-actions bot commented Jun 25, 2024 •

edited

Loading