Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: built in env files support #3759

Open
wants to merge 76 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
abe80f0
feat: handle env files
Apr 27, 2023
f5bcec5
chore: undo package json changes
Apr 27, 2023
011627e
refactor: undo changes to webpack cli
Apr 28, 2023
bb735ec
feat: add dotenv webpack plugin package
Apr 28, 2023
5a1b4db
refactor: remove cache
Apr 30, 2023
95ca34e
fix: default mode should be production
Apr 30, 2023
08d16ff
refactor: provide env files as default value
Apr 30, 2023
e3a23ce
Merge branch 'master' into feature/env-files
Apr 30, 2023
e2cf250
feat: add webpack cli runner code
May 2, 2023
ca5a782
Merge branch 'master' into feature/env-files
May 2, 2023
fd6d2ba
fix: add env plugin in build config
May 3, 2023
d544597
fix: add webpack as dev dep for dotenv webpack plugin
May 3, 2023
b83ccaf
fix: incorrect path being used in dotenv
May 3, 2023
a4388d7
test: add a simple test
May 3, 2023
ab00786
fix: check if webpack is installed before adding dotenv plugin
May 4, 2023
4e039ba
test: add test for value override
May 4, 2023
3790945
test: add test for the case when variable does not start with webpack_
May 4, 2023
c3d97d7
refactor: simplify test code
May 4, 2023
1aef466
test: add missing assertion
May 4, 2023
bd6c4a6
chore: debug failing smoketests
May 4, 2023
74ad4c8
feat: add cli option
May 6, 2023
afcbf8c
Merge branch 'webpack:master' into feature/env-files
burhanuday May 6, 2023
f203e8b
Merge branch 'master' into feature/env-files
May 6, 2023
9286cb5
refactor: redo applying the plugin
May 6, 2023
8689a2e
test: update snapshots
May 6, 2023
ca6fc8d
test: update tests
May 6, 2023
75ebe62
Merge branch 'feature/env-files' of github.com:burhanuday/webpack-cli…
May 6, 2023
4388c69
test: update snapshots
May 6, 2023
7ab13cf
fix: undo package json changes
May 6, 2023
bf8cb58
feat: add .example env file
May 8, 2023
ff7246f
chore: move dotenv webpack plugin to peerdeps
May 8, 2023
9d7e1db
feat: rename cli flag
May 8, 2023
8a45c8b
test: update snapshots
May 8, 2023
7401474
Merge branch 'master' into feature/env-files
May 8, 2023
f4b8835
test: add more test cases
May 8, 2023
9ee072d
test: add test case for mode local files
May 8, 2023
f7f79b8
fix: update types
May 8, 2023
88d3660
test: add test case for multiple configs
May 8, 2023
0158dca
fix: do validation of passed config
May 9, 2023
f8a6142
test: add more test cases
May 9, 2023
a165f38
fix: use correct logger name
May 10, 2023
06399c5
feat: add types and jsdoc
May 10, 2023
a0534f5
Merge branch 'master' into feature/env-files
May 10, 2023
7f8c4c7
Merge branch 'master' into feature/env-files
May 13, 2023
e00469c
chore: bump
May 15, 2023
d89d73c
feat: add schema utils package
May 16, 2023
0c477b0
feat: use schema utils package to validate input
May 16, 2023
8911ba7
test: update tests with new validation errors
May 16, 2023
00fb0b8
refactor: avoid using object assign
May 16, 2023
cdeb09f
perf: use webpack fs for better perf
May 16, 2023
07dbcd5
feat: add env files to build dependencies
May 17, 2023
99a7163
refactor: extract file exists into a function
May 17, 2023
03d285c
fix: use push instead of unshift for perf
May 17, 2023
ac666c4
Merge branch 'master' into feature/env-files
May 17, 2023
7de5892
feat: make env var prefix configurable
May 17, 2023
07ac04e
test: add tests for env var prefix
May 17, 2023
a3e81fa
test: update tests
May 17, 2023
8e8b78c
Merge branch 'master' into feature/env-files
May 20, 2023
c191cb2
feat: update priority order to make it same as next
May 20, 2023
67d8aed
test: add test case to check priority order of variables
May 20, 2023
a29fc1b
feat: reduce fs call
May 20, 2023
ca7a264
test: add test case to validate files
May 20, 2023
8a98a05
refactor: filter variables together
May 20, 2023
03211fe
feat: check if plugin is installed before adding
May 20, 2023
1cb9309
chore: update snapshots
May 20, 2023
26d136f
docs: add documentation for plugin
May 20, 2023
cdd9bde
docs: update cli docs
May 20, 2023
8acb20c
docs: update schema
May 23, 2023
1388d5e
feat: throw error when an empty value is passed
May 23, 2023
81edc4c
feat: update types
May 23, 2023
55b64a1
test: add tests for allowEmptyValues
May 23, 2023
c4ec39b
docs: add docs for allowemptyvalues option
May 23, 2023
f0e32aa
feat: fail the build on errors
May 27, 2023
6681233
feat: use webpack error to throw
May 27, 2023
8d92b67
Merge branch 'master' into feature/env-files
May 27, 2023
ccf341e
chore: remove author field from package file
Jun 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -64,5 +64,7 @@ test/**/**/**/bin/
test/**/**/binary/**
test/**/dist
test/**/**/dist
test/**/**/dist1
test/**/**/dist2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test/**/**/dist*

test/**/**/**/dist
test/**/stats.json
1 change: 1 addition & 0 deletions OPTIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -706,6 +706,7 @@ Options:
--performance-max-entrypoint-size <value> Total size of an entry point (in bytes).
--profile Capture timing information for each module.
--no-profile Negative 'profile' option.
--read-dot-env Read environment variables from dotenv files
--records-input-path <value> Store compiler state to a json file.
--no-records-input-path Negative 'records-input-path' option.
--records-output-path <value> Load compiler state from a json file.
Expand Down
1 change: 1 addition & 0 deletions packages/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ This folder is the collection of those packages.
3. [info](https://github.com/webpack/webpack-cli/tree/master/packages/info)
4. [serve](https://github.com/webpack/webpack-cli/tree/master/packages/serve)
5. [webpack-cli](https://github.com/webpack/webpack-cli/tree/master/packages/webpack-cli)
6. [dotenv-webpack-plugin](https://github.com/webpack/webpack-cli/tree/master/packages/dotenv-webpack-plugin)

## Generic Installation

Expand Down
91 changes: 91 additions & 0 deletions packages/dotenv-webpack-plugin/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Dotenv Webpack Plugin

`dotenv-webpack-plugin` is a webpack plugin that enables consumers to load environment variables from dotenv files. This plugin simplifies the process of managing environment-specific configurations in your webpack projects.

## Features

- Loads environment variables from dotenv files
- Provides a convenient way to manage environment-specific configurations
- Fully configurable via an options API

## Installation

Install `dotenv-webpack-plugin` using npm:

```bash
npm install dotenv-webpack-plugin --save-dev
```

or using yarn:

```bash
yarn add dotenv-webpack-plugin --dev
```

or using pnpm:

```bash
pnpm add dotenv-webpack-plugin --save-dev
```

## Usage

To use `dotenv-webpack-plugin`, follow these steps:

1. Create a `.env` file in the root directory of your project. Add each environment variable on a new lines in the form of `PUBLIC_NAME=VALUE`. By default only variables that are prefixed with `PUBLIC_` will be exposed to webpack. The prefix can be changed by passing the `envVarPrefix` option to the plugin.

1. Import `dotenv-webpack-plugin` in your webpack configuration file:

```javascript
const DotenvWebpackPlugin = require("dotenv-webpack-plugin");
```

1. Add an instance of DotenvWebpackPlugin to your webpack plugins:

```javascript
module.exports = {
// Your webpack configuration options...
plugins: [new DotenvWebpackPlugin()],
};
```

## Configuration Options

DotenvWebpackPlugin accepts the following configuration options:

1. `envFiles`: An array of dotenv files to load. By default, DotenvWebpackPlugin will look for the following files in the root directory of your project:

- `.env.[mode].local`
- `.env.local`
- `.env.[mode]`
- `.env`
- `.env.example`

The `[mode]` placeholder will be replaced with the current webpack mode. For example, if the current webpack mode is `development`, DotenvWebpackPlugin will look for the following files:

- `.env.development.local`
- `.env.local`
- `.env.development`
- `.env`
- `.env.example`

If the same variable is defined in multiple files, the value from the file with the highest precedence will be used. The precedence order is same as the order of files listed above.

While passing an array of dotenv files, the path towards the right of the array will have the highest precedence. For example, if you pass `["./.env", "./.env.local"]`, the value from `.env.local` will be used if the same variable is defined in both files.

1. `envVarPrefix`: The prefix to use when loading environment variables. By default, DotenvWebpackPlugin will look for variables prefixed with `PUBLIC_`.

1. `prefixes`: An array of prefixes to prepend to the names of environment variables. By default, DotenvWebpackPlugin will prepend `process.env.` and `import.meta.env.` to the names of environment variables.

1. `allowEmptyValues`: A boolean value indicating whether to allow empty values. By default this value is set to `false`, DotenvWebpackPlugin will throw an error if an environment variable is defined without a value.

You can pass these options when creating an instance of DotenvWebpackPlugin:

```javascript
new DotenvWebpackPlugin({
envFiles: ["./.env", "./.env.local"],
prefixes: ["process.env.", "import.meta.env."],
envVarPrefix: "PUBLIC_",
allowEmptyValues: false,
});
```
21 changes: 21 additions & 0 deletions packages/dotenv-webpack-plugin/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"name": "dotenv-webpack-plugin",
"version": "1.0.0",
"description": "A webpack plugin to support env files",
"main": "src/index.js",
"types": "src/types.d.ts",
"repository": "https://github.com/webpack/webpack-cli",
"license": "MIT",
"private": true,
"dependencies": {
"dotenv": "^16.0.3",
"dotenv-expand": "^10.0.0",
"schema-utils": "^4.0.1"
},
"peerDependencies": {
"webpack": "5.x.x"
},
"devDependencies": {
"webpack": "^5.81.0"
}
}
212 changes: 212 additions & 0 deletions packages/dotenv-webpack-plugin/src/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
const dotenv = require("dotenv");
const dotenvExpand = require("dotenv-expand");
const { DefinePlugin, WebpackError } = require("webpack");
const { validate } = require("schema-utils");
const schema = require("./options.json");

/** @typedef {import("./types").Config} Config */
/** @typedef {import("schema-utils/declarations/validate").Schema} Schema */
/** @typedef {import("webpack").Compiler} Compiler */
/** @typedef {import("webpack").Compilation} Compilation */

class DotenvWebpackPlugin {
/**
* Dotenv Webpack Plugin
* @param {Config} options - Configuration options
*/
constructor(options = {}) {
validate(/** @type {Schema} */ (schema), options || {}, {
name: "DotenvWebpackPlugin",
baseDataPath: "options",
});

const currentDirectory = process.cwd();

this.defaultFileList = [
`${currentDirectory}/.env.example`, // loaded in all cases
`${currentDirectory}/.env`, // loaded in all cases
`${currentDirectory}/.env.[mode]`, // only loaded in specified mode
`${currentDirectory}/.env.local`, // loaded in all cases, ignored by git
`${currentDirectory}/.env.[mode].local`, // only loaded in specified mode, ignored by git
];

const {
// priority is in ascending order
// files at the end of the array have higher priority
envFiles = this.defaultFileList,
prefixes = ["process.env.", "import.meta.env."],
envVarPrefix = "PUBLIC_",
alexander-akait marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's allow it to be string | string[], some env variables can be from other environments, for example SENTRY_ (error reporting) requre prefix SENTRY_, and it can be used in client, so you can have ["PUBLIC_", "SENTRY_"]

allowEmptyValues = false,
} = options;

this.options = {
envFiles,
prefixes,
envVarPrefix,
allowEmptyValues,
};

this.errors = [];
}

/**
* Default file list and the options file list are updated with the
* value of the mode if [mode] placeholder is used
* @param {String} mode - Webpack mode
*/
updateFileListWithMode(mode) {
this.options.envFiles = this.options.envFiles.map((environmentFile) =>
environmentFile.replace(/\[mode\]/g, mode),
);
this.defaultFileList = this.defaultFileList.map((environmentFile) =>
environmentFile.replace(/\[mode\]/g, mode),
);
}

/**
* Read file from path and parse it
* @param {Compiler} compiler - Webpack compiler
* @param {string} environmentFile - Path to environment file
*/
readFile(compiler, environmentFile) {
return new Promise((resolve, reject) => {
const fs = compiler.inputFileSystem;

fs.readFile(environmentFile, (err, environmentFileContents) => {
if (err) {
if (!this.defaultFileList.includes(environmentFile)) {
this.collectError(`Could not read ${environmentFile}`);
return reject(err);
} else {
return resolve();
}
}

const parsedEnvVariables = dotenv.parse(environmentFileContents);

resolve(parsedEnvVariables);
});
alexander-akait marked this conversation as resolved.
Show resolved Hide resolved
});
}

filterVariables(envVariables) {
const filteredEnvVariables = {};

for (const [key, value] of Object.entries(envVariables)) {
// only add variables starting with the provided prefix
if (key.startsWith(this.options.envVarPrefix)) {
filteredEnvVariables[key] = value;
}
}

return filteredEnvVariables;
}

assignPrefixes(envVariables) {
const prefixedEnvVariables = {};

for (const [key, value] of Object.entries(envVariables)) {
for (let index = 0; index < this.options.prefixes.length; index++) {
const prefix = this.options.prefixes[index];
prefixedEnvVariables[`${prefix}${key}`] = value;
}
}

return prefixedEnvVariables;
}

/**
* Throw collected errors to fail the build
* @param {Compilation} compilation - Webpack compilation
*/
throwErrors(compilation) {
const errors = this.errors.map((error) => {
const webpackError = new WebpackError(error);
webpackError.name = "DotenvWebpackPluginError";
return webpackError;
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from what i understand, that plugin implements a "blueprint" feature. We have not implemented it. Although i dont find it entirely necessary, I am open to suggestions on why this is required

Copy link
Member

@alexander-akait alexander-akait May 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it allows to catch problems on startup

compilation.errors.push(...errors);
}

collectError(errorMessage) {
this.errors.push(errorMessage);
}

/**
* Get list of empty values
* @param {Object} envVariables - Environment variables
* @returns {Array} - List of empty values
*/
getEmptyValues(envVariables) {
const emptyValues = [];

for (const [key, value] of Object.entries(envVariables)) {
if (value === "") {
emptyValues.push(key);
}
}

return emptyValues;
}

/**
* Webpack apply hook
* @param {Compiler} compiler - Webpack compiler
* @returns {void}
*/
apply(compiler) {
const mode = compiler.options.mode || "production";
this.updateFileListWithMode(mode);

compiler.hooks.beforeRun.tapPromise("DotenvWebpackPlugin", (compiler) => {
compiler.hooks.compilation.tap("DotenvWebpackPlugin", (compilation) => {
compilation.buildDependencies.addAll(this.options.envFiles);
if (this.errors.length > 0) {
this.throwErrors(compilation);
}
});

return Promise.all(
this.options.envFiles.map((environmentFile) => this.readFile(compiler, environmentFile)),
)
.then((valuesList) => {
const envVariables = {};

valuesList.forEach((values) => {
if (values) {
Object.entries(values).forEach(([key, value]) => {
envVariables[key] = value;
});
}
});

const filteredEnvVariables = this.filterVariables(envVariables);

const emptyValues = this.getEmptyValues(filteredEnvVariables);

if (!this.options.allowEmptyValues && emptyValues.length > 0) {
this.collectError(
`Environment variables cannot have an empty value. The following variables are empty: ${emptyValues.join(
", ",
)}`,
);
return;
evenstensberg marked this conversation as resolved.
Show resolved Hide resolved
}

const prefixedEnvVariables = this.assignPrefixes(filteredEnvVariables);

// expand environment vars
const expandedEnvVariables = dotenvExpand.expand({
parsed: prefixedEnvVariables,
// don't write to process.env
ignoreProcessEnv: true,
}).parsed;

new DefinePlugin(expandedEnvVariables).apply(compiler);
})
.catch(() => {});
});
}
}

module.exports = DotenvWebpackPlugin;
30 changes: 30 additions & 0 deletions packages/dotenv-webpack-plugin/src/options.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"definitions": {},
"title": "DotenvWebpackPlugin",
"type": "object",
"additionalProperties": false,
"properties": {
"envFiles": {
"description": "The paths to the .env files to load",
"type": "array",
"items": {
"type": "string"
}
},
"prefixes": {
"description": "The prefixes to prepend to the environment variables",
"type": "array",
"items": {
"type": "string"
}
},
"envVarPrefix": {
"description": "The prefix to filter environment variables by",
"type": "string"
},
"allowEmptyValues": {
"description": "Whether to allow empty values for environment variables",
"type": "boolean"
}
}
}
6 changes: 6 additions & 0 deletions packages/dotenv-webpack-plugin/src/types.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
export type Config = {
envFiles?: string[];
prefixes?: string[];
envVarPrefix?: string;
allowEmptyValues?: boolean;
};
1 change: 1 addition & 0 deletions packages/webpack-cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ Options:
-m, --merge Merge two or more configurations using 'webpack-merge'.
--disable-interpret Disable interpret for loading the config file.
--env <value...> Environment passed to the configuration when it is a function.
--read-dot-env Read environment variables from dotenv files
--node-env <value> Sets process.env.NODE_ENV to the specified value.
--define-process-env-node-env <value> Sets process.env.NODE_ENV to the specified value. (Currently an alias for `--node-env`)
--analyze It invokes webpack-bundle-analyzer plugin to get bundle information.
Expand Down
Loading