Add thread-per-test-class execution model

The thread-per-test-class execution model runs tests via one thread per test class. This model is primarily useful in situations when test executions attach `ThreadLocal`s to the test worker thread, which "leak" into other test class executions. This problem can lead to out-of-memory errors, if the `ThreadLocal`s reference (large) object trees, either as its value or via its initial-value `Supplier`, which cannot be cleaned up / garbage collected, because the `ThreadLocal` is referenced by the test worker thread.

The problem becomes even worse, if the test class creates its own class loader and attaches `ThreadLocal`s that reference classes loaded by such class loaders. In such cases the whole class loader including all its loaded classes and (static) state is not eligible for garbage collection, leaking more heap (and non-heap) memory.

Using one thread per test class works around the above problem(s), because once the (ephemeral) thread per test-class finishes, the whole thread and all its `ThreadLocal`s become eligible for garbage collection.

Particularly Quarkus unit tests (`@QuarkusTest` annotated test classes) benefit from this execution model.

This change cannot eliminate other sources of similar leaks, like threads spawned from tests or not removed MBeans. Those kinds of leaks are better handled by the test code or the tested code providing "proper" cleanup mechanisms.

This new execution is implemented via the introduced `ThreadPerClassHierarchicalTestExecutorService`, a 3rd model in addition to `SameThreadHierarchicalTestExecutorService` and `ForkJoinPoolHierarchicalTestExecutorService`. It is enabled if `junit.jupiter.execution.threadperclass.enabled` is set to `true` and `junit.jupiter.execution.parallel.enabled` is `false`.

Issue: junit-team#3939

documentation/src/docs/asciidoc/release-notes/release-notes-5.12.0-M1.adoc

@@ -26,7 +26,7 @@ JUnit repository on GitHub.
 [[release-notes-5.12.0-M1-junit-platform-new-features-and-improvements]]
 ==== New Features and Improvements
 
-* ❓
+* Introduce thread-per-test-class execution model.
 
 
 [[release-notes-5.12.0-M1-junit-jupiter]]

documentation/src/docs/asciidoc/user-guide/writing-tests.adoc

@@ -2963,6 +2963,29 @@ include::{testDir}/example/SharedResourcesDemo.java[tags=user_guide]
 ----
 
 
+[[writing-tests-isolated-execution]]
+=== Thread-per-class Isolated Execution
+
+By default, JUnit Jupiter tests are run sequentially from a single thread. The
+thread-per-class isolated execution, set `junit.jupiter.execution.threadperclass.enabled`
+to `true`. Each test class will be executed in its own thread.
+
+The thread-per-class execution model is useful, if test classes need to ensure that
+per-thread resources, for example instances of `ThreadLocal`, do not leak to other test
+classes. This becomes relevant if 3rd party libraries provide no way to clean up the
+`ThreadLocal` instances they created. If such a `ThreadLocal` references a class that has
+been loaded via a different class loader, this can lead to class-leaks and eventually
+out-of-memory errors. Running test classes using the thread-per-class execution model allows
+the JVM to eventually garbage collect those `ThreadLocal` instances and prevent such
+out-of-memory errors.
+
+`junit.jupiter.execution.threadperclass.enabled` is only evaluated, if
+`junit.jupiter.execution.parallel.enabled` is `false`.
+
+Since every test class requires a new thread to be created and requires some synchronization,
+execution with the thread-per-class model has a little overhead.
+
+
 [[writing-tests-built-in-extensions]]
 === Built-in Extensions
 

junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/JupiterTestEngine.java

@@ -30,6 +30,7 @@
 import org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService;
 import org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine;
 import org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutorService;
+import org.junit.platform.engine.support.hierarchical.ThreadPerClassHierarchicalTestExecutorService;
 import org.junit.platform.engine.support.hierarchical.ThrowableCollector;
 
 /**
@@ -74,9 +75,15 @@ public TestDescriptor discover(EngineDiscoveryRequest discoveryRequest, UniqueId
 	protected HierarchicalTestExecutorService createExecutorService(ExecutionRequest request) {
 		JupiterConfiguration configuration = getJupiterConfiguration(request);
 		if (configuration.isParallelExecutionEnabled()) {
+			if (configuration.isThreadPerClassExecutionEnabled()) {
+				throw new IllegalArgumentException("Parallel execution and thread-per-class is not supported");
+			}
 			return new ForkJoinPoolHierarchicalTestExecutorService(new PrefixedConfigurationParameters(
 				request.getConfigurationParameters(), Constants.PARALLEL_CONFIG_PREFIX));
 		}
+		if (configuration.isThreadPerClassExecutionEnabled()) {
+			return new ThreadPerClassHierarchicalTestExecutorService(request.getConfigurationParameters());
+		}
 		return super.createExecutorService(request);
 	}
 

junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/config/CachingJupiterConfiguration.java

@@ -61,6 +61,12 @@ public boolean isParallelExecutionEnabled() {
 			key -> delegate.isParallelExecutionEnabled());
 	}
 
+	@Override
+	public boolean isThreadPerClassExecutionEnabled() {
+		return (boolean) cache.computeIfAbsent(THREAD_PER_CLASS_EXECUTION_ENABLED_PROPERTY_NAME,
+			key -> delegate.isThreadPerClassExecutionEnabled());
+	}
+
 	@Override
 	public boolean isExtensionAutoDetectionEnabled() {
 		return (boolean) cache.computeIfAbsent(EXTENSIONS_AUTODETECTION_ENABLED_PROPERTY_NAME,

junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/config/DefaultJupiterConfiguration.java

@@ -84,6 +84,11 @@ public boolean isParallelExecutionEnabled() {
 		return configurationParameters.getBoolean(PARALLEL_EXECUTION_ENABLED_PROPERTY_NAME).orElse(false);
 	}
 
+	@Override
+	public boolean isThreadPerClassExecutionEnabled() {
+		return configurationParameters.getBoolean(THREAD_PER_CLASS_EXECUTION_ENABLED_PROPERTY_NAME).orElse(false);
+	}
+
 	@Override
 	public boolean isExtensionAutoDetectionEnabled() {
 		return configurationParameters.getBoolean(EXTENSIONS_AUTODETECTION_ENABLED_PROPERTY_NAME).orElse(false);

junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/config/JupiterConfiguration.java

@@ -36,6 +36,7 @@ public interface JupiterConfiguration {
 
 	String DEACTIVATE_CONDITIONS_PATTERN_PROPERTY_NAME = "junit.jupiter.conditions.deactivate";
 	String PARALLEL_EXECUTION_ENABLED_PROPERTY_NAME = "junit.jupiter.execution.parallel.enabled";
+	String THREAD_PER_CLASS_EXECUTION_ENABLED_PROPERTY_NAME = "junit.jupiter.execution.threadperclass.enabled";
 	String DEFAULT_EXECUTION_MODE_PROPERTY_NAME = Execution.DEFAULT_EXECUTION_MODE_PROPERTY_NAME;
 	String DEFAULT_CLASSES_EXECUTION_MODE_PROPERTY_NAME = Execution.DEFAULT_CLASSES_EXECUTION_MODE_PROPERTY_NAME;
 	String EXTENSIONS_AUTODETECTION_ENABLED_PROPERTY_NAME = "junit.jupiter.extensions.autodetection.enabled";
@@ -50,6 +51,8 @@ public interface JupiterConfiguration {
 
 	boolean isParallelExecutionEnabled();
 
+	boolean isThreadPerClassExecutionEnabled();
+
 	boolean isExtensionAutoDetectionEnabled();
 
 	ExecutionMode getDefaultExecutionMode();

junit-platform-engine/src/main/java/org/junit/platform/engine/support/hierarchical/NodeTestTask.java

@@ -69,6 +69,10 @@ class NodeTestTask<C extends EngineExecutionContext> implements TestTask {
 		this.finalizer = finalizer;
 	}
 
+	TestDescriptor getTestDescriptor() {
+		return testDescriptor;
+	}
+
 	@Override
 	public ResourceLock getResourceLock() {
 		return taskContext.getExecutionAdvisor().getResourceLock(testDescriptor);

junit-platform-engine/src/main/java/org/junit/platform/engine/support/hierarchical/ThreadPerClassHierarchicalTestExecutorService.java

@@ -0,0 +1,133 @@
+/*
+ * Copyright 2015-2024 the original author or authors.
+ *
+ * All rights reserved. This program and the accompanying materials are
+ * made available under the terms of the Eclipse Public License v2.0 which
+ * accompanies this distribution and is available at
+ *
+ * https://www.eclipse.org/legal/epl-v20.html
+ */
+
+package org.junit.platform.engine.support.hierarchical;
+
+import static java.lang.String.format;
+import static java.time.Duration.ofMinutes;
+import static java.util.concurrent.CompletableFuture.completedFuture;
+import static java.util.concurrent.TimeUnit.MILLISECONDS;
+import static org.apiguardian.api.API.Status.STABLE;
+
+import java.time.Duration;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Future;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.apiguardian.api.API;
+import org.junit.platform.commons.JUnitException;
+import org.junit.platform.engine.ConfigurationParameters;
+import org.junit.platform.engine.TestDescriptor;
+import org.junit.platform.engine.UniqueId;
+
+/**
+ * A {@linkplain HierarchicalTestExecutorService executor service} that creates a new thread for
+ * each test class, all {@linkplain TestTask test tasks}.
+ *
+ * <p>This execution model is useful to prevent some kinds of class / class-loader leaks. For
+ * example, if a test creates {@link ClassLoader}s and the tests or any of the code and libraries
+ * create {@link ThreadLocal}s, those thread locals would accumulate in the single {@link
+ * SameThreadHierarchicalTestExecutorService} causing a class-(loader)-leak.
+ *
+ * @since 5.12
+ */
+@API(status = STABLE, since = "5.12")
+public class ThreadPerClassHierarchicalTestExecutorService implements HierarchicalTestExecutorService {
+
+	private final AtomicInteger threadCount = new AtomicInteger();
+	private final Duration interruptWaitDuration;
+
+	static final Duration DEFAULT_INTERRUPT_WAIT_DURATION = ofMinutes(5);
+	static final String THREAD_PER_CLASS_INTERRUPTED_WAIT_TIME_SECONDS = "junit.jupiter.execution.threadperclass.interrupted.waittime.seconds";
+
+	public ThreadPerClassHierarchicalTestExecutorService(ConfigurationParameters config) {
+		interruptWaitDuration = config.get(THREAD_PER_CLASS_INTERRUPTED_WAIT_TIME_SECONDS).map(Integer::parseInt).map(
+			Duration::ofSeconds).orElse(DEFAULT_INTERRUPT_WAIT_DURATION);
+	}
+
+	@Override
+	public Future<Void> submit(TestTask testTask) {
+		executeTask(testTask);
+		return completedFuture(null);
+	}
+
+	@Override
+	public void invokeAll(List<? extends TestTask> tasks) {
+		tasks.forEach(this::executeTask);
+	}
+
+	protected void executeTask(TestTask testTask) {
+		NodeTestTask<?> nodeTestTask = (NodeTestTask<?>) testTask;
+		TestDescriptor testDescriptor = nodeTestTask.getTestDescriptor();
+
+		UniqueId.Segment lastSegment = testDescriptor.getUniqueId().getLastSegment();
+
+		if ("class".equals(lastSegment.getType())) {
+			executeOnDifferentThread(testTask, lastSegment);
+		}
+		else {
+			testTask.execute();
+		}
+	}
+
+	private void executeOnDifferentThread(TestTask testTask, UniqueId.Segment lastSegment) {
+		CompletableFuture<Object> future = new CompletableFuture<>();
+		Thread threadPerClass = new Thread(() -> {
+			try {
+				testTask.execute();
+				future.complete(null);
+			}
+			catch (Exception e) {
+				future.completeExceptionally(e);
+			}
+		}, threadName(lastSegment));
+		threadPerClass.setDaemon(true);
+		threadPerClass.start();
+
+		try {
+			try {
+				future.get();
+			}
+			catch (InterruptedException e) {
+				// propagate a thread-interrupt to the executing class
+				threadPerClass.interrupt();
+				try {
+					future.get(interruptWaitDuration.toMillis(), MILLISECONDS);
+				}
+				catch (InterruptedException ie) {
+					threadPerClass.interrupt();
+				}
+				catch (TimeoutException to) {
+					throw new JUnitException(format("Test class %s was interrupted but did not terminate within %s",
+						lastSegment.getValue(), interruptWaitDuration), to);
+				}
+			}
+		}
+		catch (ExecutionException e) {
+			Throwable cause = e.getCause();
+			if (cause instanceof RuntimeException) {
+				throw (RuntimeException) cause;
+			}
+			throw new JUnitException("TestTask execution failure", cause);
+		}
+	}
+
+	private String threadName(UniqueId.Segment lastSegment) {
+		return format("TEST THREAD #%d FOR %s", threadCount.incrementAndGet(), lastSegment.getValue());
+	}
+
+	@Override
+	public void close() {
+		// nothing to do
+	}
+}