Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

breaking: use webpack builtin cache #1040

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Conversation

JLHwung
Copy link
Contributor

@JLHwung JLHwung commented Sep 4, 2024

Please Read the CONTRIBUTING Guidelines
In particular the portion on Commit Message Formatting

Please check if the PR fulfills these requirements

  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

  • Bugfix
  • Feature
  • Code style update (formatting, local variables)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • CI related changes
  • Other... Please describe: Use webpack builtin cache

What is the current behavior? (You can also link to an open issue here)
The babel-loader uses per-input filesystem cache.

What is the new behavior?
The babel-loader option cacheCompression will be removed, users can configure the webpack option `cache.
babel-loader will use the builtin webpack cache.

The builtin webpack cache offers many options and users will be able to compose their own cache plugin by tapping the compiler cache hooks.

Does this PR introduce a breaking change?

  • Yes
  • No

If this PR contains a breaking change, please describe the following...

  • Impact: Big
  • Migration path for existing applications:
    If you are using the babel-loader cacheDirectory or cacheCompression option, set cacheDirectory to true and enable the webpack persistent cache via
// webpack.config.js
{
  cache: {
    type: "filesystem",
    compression: "gzip"
  }
}

The default webpack fs cache location is ./node_modules/.cache/webpack. You can change the cache directory via the webpack option cache.cacheDirectory. See the official docs for more webpack cache options.

Other information:

This is an early WIP, obviously most cache tests are currently failing due to config errors and incorrect filesystem structures. The updated cache test serves as a proof-of-concept that this approach works with the memory cache (The default of webpack 5 in development mode). Note that the memory cache will be purged when the compiler shutdown, so for those who enabled the babel-loader cache, they should enable the webpack filesystem cache.

When the filesystem cache is enabled, users can simply disable the Babel cache in most circumstances, such as 1) babel-loader is the only webpack loader for the js/ts inputs or 2) every js/ts loader is cachebale (the webpack defaults for all loaders unless the loader explicitly called this.cacheable(false)). The transformed output will be cached by webpack into the filesystem even without babel-loader cache. So in the proof-of-concept test we have to plug in an uncacheable loader for babel-loader cache to be hit again.

Alternative Solutions:

A cacheProvider: "webpack" | "babel-loader" option to switch between webpack cache and our own cache. But the new option will further complicates the cache config, and the babel-loader cache does not offer very much functionalities than the webpack builtin cache.

I would like to get feedback from team members, @liuxingbaoyu and @nicolo-ribaudo Do you think this approach is a good direction or we should keep our own cache? Do you think we should ship this in v10 or defer the changes to v11?

@@ -20,13 +20,12 @@ if (/^6\./.test(babel.version)) {
}

const { version } = require("../package.json");
const cache = require("./cache");
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ./cache is now unused, we can remove this file and the only dependency find-up after tests are fixed.

await addTimestamps(result.externalDependencies, getFileTimestamp);
logger.debug(`caching result for '${filename}'`);
await itemCache.storePromise(result);
logger.debug(`cached result for '${filename}'`);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cache log is updated from "writing result to cache file" to current wordings to reflect the fact that the loader is unaware of the cache implementation: it may be written to the filesystem, it may be written to a database or it is retained in the memory.

@liuxingbaoyu
Copy link
Member

Do you think this approach is a good direction or we should keep our own cache? Do you think we should ship this in v10 or defer the changes to v11?

This looks good to me. Both v10 and v11 are ok for me and the migration steps seem to be simple.

@JLHwung JLHwung marked this pull request as ready for review September 23, 2024 19:38
@JLHwung
Copy link
Contributor Author

JLHwung commented Sep 23, 2024

This PR is ready for review. The options docs are also updated.

@liuxingbaoyu
Copy link
Member

liuxingbaoyu commented Sep 23, 2024

Can we still utilize webpack cache system when webpack options disable cache?🤔

@JLHwung
Copy link
Contributor Author

JLHwung commented Sep 23, 2024

Can we still utilize webpack cache system when webpack options disable cache?🤔

In most case, no. When webpack cache is disabled, the builtin cache is doing nothing. The cache facade provided by webpack is essentially an event emitter. In this case, the cache action event is triggered but there is no cache event handler to actually do the cache job.

If this PR gets merged, I would also add to the release note that to enable the babel-loader cache, you will have to turn on the webpack cache. But for most cases, the webpack cache is good enough and you don't have to enable the babel-loader cache at all, which is also our default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants