Merge branch 'main' into origin/issues/3107-method-filters

CONTRIBUTING.md

@@ -28,10 +28,10 @@ Issue: #999
 ## Pull Requests
 
 Our [Definition of Done](https://github.com/junit-team/junit5/wiki/Definition-of-Done)
-offers some guidelines on what we expect from a pull request.
+(DoD) offers some guidelines on what we expect from a pull request.
 Feel free to open a pull request that does not fulfill all criteria, e.g. to discuss
 a certain change before polishing it, but please be aware that we will only merge it
-in case the DoD is met.
+once the DoD is met.
 
 Please add the following lines to your pull request description:
 
@@ -95,16 +95,15 @@ code -- class names, method names, variable names, etc.
 ### Javadoc
 
 - Javadoc comments should be wrapped after 80 characters whenever possible.
-- This first paragraph must be a single, concise sentence that ends with a period (".").
-- Place `<p>` on the same line as the first line in a new paragraph and precede `<p>` with a blank line.
+- This first paragraph must be a single, concise sentence that ends with a period (`.`).
+- Place `<p>` on the same line as the first line of a new paragraph and precede `<p>` with a blank line.
 - Insert a blank line before at-clauses/tags.
 - Favor `{@code foo}` over `<code>foo</code>`.
 - Favor literals (e.g., `{@literal @}`) over HTML entities.
-- New classes and methods should have `@since ...` annotation.
-- Use `@since 5.0` instead of `@since 5.0.0`.
-- Do not use `@author` tags. Instead, contributors are listed on [GitHub](https://github.com/junit-team/junit5/graphs/contributors).
-- Do not use verbs in third person form (e.g. use "Discover tests..." instead of "Discovers tests...")
-  in the first sentence describing a method.
+- New classes and methods should declare a `@since ...` tag.
+- Use `@since 5.10` instead of `@since 5.10.0`.
+- Do not use `@author` tags. Instead, contributors are listed on the [GitHub](https://github.com/junit-team/junit5/graphs/contributors) page.
+- Do not use verbs in third-person form in the first sentence of the Javadoc for a method -- for example, use "Discover tests..." instead of "Discovers tests...".
 
 #### Examples
 
@@ -121,11 +120,11 @@ See [`ExtensionContext`](junit-jupiter-api/src/main/java/org/junit/jupiter/api/e
 
 #### Assertions
 
-- Use `org.junit.jupiter.api.Assertions` wherever possible.
+- Use `org.junit.jupiter.api.Assertions` for simple assertions.
 - Use AssertJ when richer assertions are needed.
 - Do not use `org.junit.Assert` or `junit.framework.Assert`.
 
-#### Mocking
+#### Mocking and Stubbing
 
 - Use either [Mockito](https://github.com/mockito/mockito) or hand-written test doubles.
 
@@ -143,10 +142,11 @@ See [`ExtensionContext`](junit-jupiter-api/src/main/java/org/junit/jupiter/api/e
 
 ### Deprecation
 
-Publicly available interfaces, classes and methods have a defined lifecycle
+The JUnit 5 project uses the `@API` annotation from [API Guardian](https://github.com/apiguardian-team/apiguardian).
+Publicly available interfaces, classes, and methods have a defined lifecycle
 which is described in detail in the [User Guide](https://junit.org/junit5/docs/current/user-guide/#api-evolution).
-This process is using the `@API` annotation from [API Guardian](https://github.com/apiguardian-team/apiguardian).
-It also describes the deprecation process followed for API items.
+
+That following describes the deprecation process followed for API items.
 
 To deprecate an item:
 - Update the `@API.status` to `DEPRECATED`.

documentation/src/docs/asciidoc/release-notes/release-notes-5.10.0-RC1.adoc

@@ -28,8 +28,11 @@ JUnit repository on GitHub.
 ==== New Features and Improvements
 
 * New `@SelectMethod` selector support in the `@Suite` test engine.
-* Classes may now be selected by fully-qualified name via new the `names` attribute in
+* Classes may now be selected by fully-qualified name via the `names` attribute in
   `@SelectClasses`.
+* New `selectMethod()` and `selectNestedMethod()` variants in `DiscoverySelectors` that
+  accept a `Class<?>...` argument of parameter types as a type-safe alternative to
+  providing the names of parameter types as a comma-delimited string.
 
 
 [[release-notes-5.10.0-RC1-junit-jupiter]]
@@ -52,6 +55,11 @@ JUnit repository on GitHub.
 * Lifecycle and thread-safety semantics are now documented for the `TempDirFactory` SPI.
 * New `reason` attribute in `@Execution` which can be used to document the reason for
   using the selected execution mode.
+* The <<../user-guide/index.adoc#extensions-RandomNumberExtension, User Guide>> now
+  includes an example implementation of the `RandomNumberExtension` in order to improve
+  the documentation for extension registration via `@ExtendWith` on fields.
+* The scope of applicability for `TestWatcher` implementations is now more extensively
+  documented in the User Guide and Javadoc.
 
 
 [[release-notes-5.10.0-RC1-junit-vintage]]

documentation/src/docs/asciidoc/user-guide/extensions.adoc

@@ -115,6 +115,25 @@ include::{testDir}/example/extensions/Random.java[tags=user_guide]
 include::{testDir}/example/extensions/RandomNumberDemo.java[tags=user_guide]
 ----
 
+[[extensions-RandomNumberExtension]]
+The following code listing provides an example of how one might choose to implement such a
+`RandomNumberExtension`. This implementation works for the use cases in
+`RandomNumberDemo`; however, it may not prove robust enough to cover all use cases – for
+example, the random number generation support is limited to integers, it uses
+`java.util.Random` instead of `java.security.SecureRandom`, etc. In any case, it is
+important to note which extension APIs are implemented and for what reasons.
+
+Specifically, `RandomNumberExtension` implements the following extension APIs:
+
+- `BeforeAllCallback`: to support static field injection
+- `TestInstancePostProcessor`: to support non-static field injection
+- `ParameterResolver`: to support constructor and method injection
+
+[source,java,indent=0]
+----
+include::{testDir}/example/extensions/RandomNumberExtension.java[tags=user_guide]
+----
+
 [TIP]
 .Extension Registration Order for `@ExtendWith` on Fields
 ====
@@ -386,11 +405,11 @@ test instance, invoking custom de-initialization methods on the test instance, e
 runtime.
 
 If a _test class_ constructor, _test method_, or _lifecycle method_ (see
-<<writing-tests-classes-and-methods>>) declares a parameter, the parameter must be
-_resolved_ at runtime by a `ParameterResolver`. A `ParameterResolver` can either be
-built-in (see `{TestInfoParameterResolver}`) or <<extensions-registration,registered by
-the user>>. Generally speaking, parameters may be resolved by _name_, _type_,
-_annotation_, or any combination thereof.
+<<writing-tests-definitions>>) declares a parameter, the parameter must be _resolved_ at
+runtime by a `ParameterResolver`. A `ParameterResolver` can either be built-in (see
+`{TestInfoParameterResolver}`) or <<extensions-registration,registered by the user>>.
+Generally speaking, parameters may be resolved by _name_, _type_, _annotation_, or any
+combination thereof.
 
 If you wish to implement a custom `{ParameterResolver}` that resolves parameters based
 solely on the type of the parameter, you may find it convenient to extend the
@@ -436,20 +455,44 @@ information for the following events.
 * `testFailed`: invoked after a _test method_ has failed
 
 NOTE: In contrast to the definition of "test method" presented in
-<<writing-tests-classes-and-methods>>, in this context _test method_ refers to any
-`@Test` method or `@TestTemplate` method (for example, a `@RepeatedTest` or
-`@ParameterizedTest`).
+<<writing-tests-definitions>>, in this context _test method_ refers to any `@Test` method
+or `@TestTemplate` method (for example, a `@RepeatedTest` or `@ParameterizedTest`).
+
+Extensions implementing this interface can be registered at the class level, instance
+level, or method level. When registered at the class level, a `TestWatcher` will be
+invoked for any contained _test method_ including those in `@Nested` classes. When
+registered at the method level, a `TestWatcher` will only be invoked for the _test method_
+for which it was registered.
+
+[WARNING]
+====
+If a `TestWatcher` is registered via a non-static (instance) field – for example, using
+`@RegisterExtension` – and the test class is configured with
+`@TestInstance(Lifecycle.PER_METHOD)` semantics (which is the default lifecycle mode), the
+`TestWatcher` will **not** be invoked with events for `@TestTemplate` methods (for
+example, `@RepeatedTest` or `@ParameterizedTest`).
+
+To ensure that a `TestWatcher` is invoked for all _test methods_ in a given class, it is
+therefore recommended that the `TestWatcher` be registered at the class level with
+`@ExtendWith` or via a `static` field with `@RegisterExtension` or `@ExtendWith`.
+====
+
+If there is a failure at the class level — for example, an exception thrown by a
+`@BeforeAll` method — no test results will be reported. Similarly, if the test class is
+disabled via an `ExecutionCondition` — for example, `@Disabled` — no test results will be
+reported.
 
-Extensions implementing this interface can be registered at the method level or at the
-class level. In the latter case they will be invoked for any contained _test method_
-including those in `@Nested` classes.
+In contrast to other Extension APIs, a `TestWatcher` is not permitted to adversely
+influence the execution of tests. Consequently, any exception thrown by a method in the
+`TestWatcher` API will be logged at `WARNING` level and will not be allowed to propagate
+or fail test execution.
 
 [WARNING]
 ====
 Any instances of `ExtensionContext.Store.CloseableResource` stored in the `Store` of the
-provided `{ExtensionContext}` will be closed _before_ methods in this API are invoked (see
-<<extensions-keeping-state>>). You can use the parent context's `Store` to work with such
-resources.
+provided `{ExtensionContext}` will be closed _before_ methods in the `TestWatcher` API are
+invoked (see <<extensions-keeping-state>>). You can use the parent context's `Store` to
+work with such resources.
 ====
 
 [[extensions-lifecycle-callbacks]]
@@ -797,7 +840,7 @@ callbacks implemented by `Extension2`. `Extension1` is therefore said to _wrap_
 `Extension2`.
 
 JUnit Jupiter also guarantees _wrapping_ behavior within class and interface hierarchies
-for user-supplied _lifecycle methods_ (see <<writing-tests-classes-and-methods>>).
+for user-supplied _lifecycle methods_ (see <<writing-tests-definitions>>).
 
 * `@BeforeAll` methods are inherited from superclasses as long as they are not _hidden_,
   _overridden_, or _superseded_ (i.e., replaced based on signature only, irrespective of
@@ -945,7 +988,6 @@ image::extensions_BrokenLifecycleMethodConfigDemo.png[caption='',title='BrokenLi
 [TIP]
 ====
 Due to the aforementioned behavior, the JUnit Team recommends that developers declare at
-most one of each type of _lifecycle method_ (see <<writing-tests-classes-and-methods>>)
-per test class or test interface unless there are no dependencies between such lifecycle
-methods.
+most one of each type of _lifecycle method_ (see <<writing-tests-definitions>>) per test
+class or test interface unless there are no dependencies between such lifecycle methods.
 ====

documentation/src/docs/asciidoc/user-guide/writing-tests.adoc

@@ -592,8 +592,7 @@ deterministic but intentionally nonobvious. This ensures that subsequent runs of
 suite execute test classes and test methods in the same order, thereby allowing for
 repeatable builds.
 
-NOTE: See <<writing-tests-classes-and-methods>> for a definition of _test method_ and
-_test class_.
+NOTE: See <<writing-tests-definitions>> for a definition of _test method_ and _test class_.
 
 [[writing-tests-test-execution-order-methods]]
 ==== Method Order
@@ -665,8 +664,8 @@ dependencies between test classes, or you may wish to order test classes to opti
 time as outlined in the following scenarios.
 
 * Run previously failing tests and faster tests first: "fail fast" mode
-* With parallel execution enabled, run longer tests first: "shortest test plan execution
-  duration" mode
+* With parallel execution enabled, schedule longer tests first: "shortest test plan
+  execution duration" mode
 * Various other use cases
 
 To configure test class execution order _globally_ for the entire test suite, use the
@@ -728,8 +727,8 @@ include::{testDir}/example/OrderedNestedTestClassesDemo.java[tags=user_guide]
 In order to allow individual test methods to be executed in isolation and to avoid
 unexpected side effects due to mutable test instance state, JUnit creates a new instance
 of each test class before executing each _test method_ (see
-<<writing-tests-classes-and-methods>>). This "per-method" test instance lifecycle is the
-default behavior in JUnit Jupiter and is analogous to all previous versions of JUnit.
+<<writing-tests-definitions>>). This "per-method" test instance lifecycle is the default
+behavior in JUnit Jupiter and is analogous to all previous versions of JUnit.
 
 NOTE: Please note that the test class will still be instantiated if a given _test method_
 is _disabled_ via a <<writing-tests-conditional-execution,condition>> (e.g., `@Disabled`,
@@ -840,8 +839,8 @@ constructors and methods.
 
 `{ParameterResolver}` defines the API for test extensions that wish to _dynamically_
 resolve parameters at runtime. If a _test class_ constructor, a _test method_, or a
-_lifecycle method_ (see <<writing-tests-classes-and-methods>>) accepts a parameter, the
-parameter must be resolved at runtime by a registered `ParameterResolver`.
+_lifecycle method_ (see <<writing-tests-definitions>>) accepts a parameter, the parameter
+must be resolved at runtime by a registered `ParameterResolver`.
 
 There are currently three built-in resolvers that are registered automatically.
 
@@ -2303,6 +2302,12 @@ might conflict with the configured execution order. Thus, in both cases, test me
 such test classes are only executed concurrently if the `@Execution(CONCURRENT)`
 annotation is present on the test class or method.
 
+When parallel execution is enabled and a default `{ClassOrderer}` (see
+<<writing-tests-test-execution-order-classes>> for details) is registered, top-level test
+classes will initially be sorted accordingly and scheduled in that order. However, they
+are not guaranteed to be started in exactly that order since the threads they are executed
+on are not controlled directly by JUnit.
+
 All nodes of the test tree that are configured with the `CONCURRENT` execution mode will
 be executed fully in parallel according to the provided
 <<writing-tests-parallel-execution-config, configuration>> while observing the

documentation/src/test/java/example/extensions/RandomNumberDemo.java

@@ -11,31 +11,34 @@
 package example.extensions;
 
 import org.junit.jupiter.api.BeforeEach;
-import org.junit.jupiter.api.Disabled;
 import org.junit.jupiter.api.Test;
 
-@Disabled("RandomNumberExtension has not been implemented")
-//tag::user_guide[]
+// tag::user_guide[]
 class RandomNumberDemo {
 
-	// use random number field in test methods and @BeforeEach
-	// or @AfterEach lifecycle methods
+	// Use static randomNumber0 field anywhere in the test class,
+	// including @BeforeAll or @AfterEach lifecycle methods.
+	@Random
+	private static Integer randomNumber0;
+
+	// Use randomNumber1 field in test methods and @BeforeEach
+	// or @AfterEach lifecycle methods.
 	@Random
 	private int randomNumber1;
 
 	RandomNumberDemo(@Random int randomNumber2) {
-		// use random number in constructor
+		// Use randomNumber2 in constructor
 	}
 
 	@BeforeEach
 	void beforeEach(@Random int randomNumber3) {
-		// use random number in @BeforeEach method
+		// Use randomNumber3 in @BeforeEach method.
 	}
 
 	@Test
 	void test(@Random int randomNumber4) {
-		// use random number in test method
+		// Use randomNumber4 in test method.
 	}
 
 }
-//end::user_guide[]
+// end::user_guide[]

documentation/src/test/java/example/extensions/RandomNumberExtension.java

@@ -10,30 +10,85 @@
 
 package example.extensions;
 
+// tag::user_guide[]
+
+import static org.junit.platform.commons.support.AnnotationSupport.findAnnotatedFields;
+
+import java.lang.reflect.Field;
+import java.util.function.Predicate;
+
 import org.junit.jupiter.api.extension.BeforeAllCallback;
-import org.junit.jupiter.api.extension.BeforeEachCallback;
 import org.junit.jupiter.api.extension.ExtensionContext;
 import org.junit.jupiter.api.extension.ParameterContext;
 import org.junit.jupiter.api.extension.ParameterResolver;
+import org.junit.jupiter.api.extension.TestInstancePostProcessor;
+import org.junit.platform.commons.support.ModifierSupport;
 
-class RandomNumberExtension implements BeforeAllCallback, BeforeEachCallback, ParameterResolver {
+// end::user_guide[]
+// @formatter:off
+// tag::user_guide[]
+class RandomNumberExtension
+		implements BeforeAllCallback, TestInstancePostProcessor, ParameterResolver {
 
+	private final java.util.Random random = new java.util.Random(System.nanoTime());
+
+	/**
+	 * Inject a random integer into static fields that are annotated with
+	 * {@code @Random} and can be assigned an integer value.
+	 */
 	@Override
-	public boolean supportsParameter(ParameterContext parameterContext, ExtensionContext extensionContext) {
-		return false;
+	public void beforeAll(ExtensionContext context) {
+		Class<?> testClass = context.getRequiredTestClass();
+		injectFields(testClass, null, ModifierSupport::isStatic);
 	}
 
+	/**
+	 * Inject a random integer into non-static fields that are annotated with
+	 * {@code @Random} and can be assigned an integer value.
+	 */
 	@Override
-	public Object resolveParameter(ParameterContext parameterContext, ExtensionContext extensionContext) {
-		return null;
+	public void postProcessTestInstance(Object testInstance, ExtensionContext context) {
+		Class<?> testClass = context.getRequiredTestClass();
+		injectFields(testClass, testInstance, ModifierSupport::isNotStatic);
 	}
 
+	/**
+	 * Determine if the parameter is annotated with {@code @Random} and can be
+	 * assigned an integer value.
+	 */
 	@Override
-	public void beforeAll(ExtensionContext context) {
+	public boolean supportsParameter(ParameterContext pc, ExtensionContext ec) {
+		return pc.isAnnotated(Random.class) && isInteger(pc.getParameter().getType());
 	}
 
+	/**
+	 * Resolve a random integer.
+	 */
 	@Override
-	public void beforeEach(ExtensionContext context) {
+	public Integer resolveParameter(ParameterContext pc, ExtensionContext ec) {
+		return this.random.nextInt();
+	}
+
+	private void injectFields(Class<?> testClass, Object testInstance,
+			Predicate<Field> predicate) {
+
+		predicate = predicate.and(field -> isInteger(field.getType()));
+		findAnnotatedFields(testClass, Random.class, predicate)
+			.forEach(field -> {
+				try {
+					field.setAccessible(true);
+					field.set(testInstance, this.random.nextInt());
+				}
+				catch (Exception ex) {
+					throw new RuntimeException(ex);
+				}
+			});
+	}
+
+	private static boolean isInteger(Class<?> type) {
+		return int.class.isAssignableFrom(type);
 	}
 
 }
+// end::user_guide[]
+// @formatter:on