Skip to content

Fix #3303: distinguish JUnit 6 ParameterizedClass invocations #3432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AzazelSensei wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Fix #3303: distinguish JUnit 6 ParameterizedClass invocations #3432

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a7dd1ff
Fix #3303: keep JUnit 6 ParameterizedClass invocations distinct
AzazelSensei Aug 13, 2026
0931118
Fix #3303: keep nested ParameterizedClass indices
AzazelSensei Aug 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 50 additions & 2 deletions ...it-platform/src/main/java/org/apache/maven/surefire/junitplatform/RunListenerAdapter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@
@Override
public void setRunMode(RunMode runMode) {
this.runMode = runMode;
}

Check failure on line 95 in surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/RunListenerAdapter.java

View workflow job for this annotation

GitHub Actions / Verify / ubuntu-latest jdk-21-zulu 3.10.0-rc-1 

error: cannot find symbol

Check failure on line 95 in surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/RunListenerAdapter.java

View workflow job for this annotation

GitHub Actions / Verify / ubuntu-latest jdk-21-zulu 3.10.0-rc-1 

error: cannot find symbol

@Override
public void testPlanExecutionStarted(TestPlan testPlan) {
Expand Down Expand Up @@ -208,6 +208,42 @@
.orElseGet(Stream::empty);
}

/**
* Build a {@code [outer][inner]} suffix from every JUnit Jupiter
* {@code [class-template-invocation:#N]} unique-id segment. Nested
* {@code @ParameterizedClass} declarations emit more than one; taking only
* the last would collapse outer #1/inner #1 with outer #2/inner #1.
*
* @param uniqueId the platform unique id string
* @return the suffix, or an empty string when the id has no class-template invocation
*/
private static String extractClassTemplateInvocationSuffix(String uniqueId) {
if (uniqueId == null) {
return "";
}
String marker = "[class-template-invocation:";

@olamy olamy Aug 17, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there anything available to extract segments from a uniqueId representation rather than parsing manually?
Maybe?

UniqueId.parse(uniqueId).getSegments()

StringBuilder suffix = new StringBuilder();
int from = 0;
while (true) {
int start = uniqueId.indexOf(marker, from);
if (start < 0) {
break;
}
start += marker.length();
int end = uniqueId.indexOf(']', start);
if (end <= start) {
break;
}
String value = uniqueId.substring(start, end);
if (value.startsWith("#")) {
value = value.substring(1);
}
suffix.append('[').append(value).append(']');
from = end + 1;
}
return suffix.toString();
}

private String safeGetMessage(Throwable throwable) {
try {
SafeThrowable t = throwable == null ? null : new SafeThrowable(throwable);
Expand Down Expand Up @@ -462,14 +498,26 @@
.map(TestIdentifier::getLegacyReportingName)
.anyMatch(legacyReportingName -> legacyReportingName.matches("^\\[.+]$"));
boolean isTestTemplate = testIdentifier.getLegacyReportingName().matches("^.*\\[\\d+]$");

boolean parameterized = isParameterized || hasParameterizedParent || isTestTemplate;
// JUnit 6 @ParameterizedClass parents have a ClassSource, so they are missed by
// hasParameterizedParent, and the method legacy name no longer includes [N] (#3303).
String classTemplateInvocationSuffix = extractClassTemplateInvocationSuffix(testIdentifier.getUniqueId());

boolean parameterized = isParameterized
|| hasParameterizedParent
|| isTestTemplate
|| !classTemplateInvocationSuffix.isEmpty();
String methodName = methodSource.getMethodName();
String description = testIdentifier.getLegacyReportingName();
boolean equalDescriptions = methodDisplay.equals(description);
boolean hasLegacyDescription = description.startsWith(methodName + '(');
boolean hasDisplayName = !equalDescriptions || !hasLegacyDescription;
String methodDesc = parameterized ? description : methodName;
// JUnit 6.1+ already puts the class index before a method invocation index

@olamy olamy Aug 17, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the class-invocation index is appended after the method index (foo()[m][c]), or collapses on the diagonal (c==m → foo()[c]). This contradicts the [class][method].
2 class invocations × method invocations #1/#2 → foo(String)[1], foo(String)[2][1], foo(String)[1][2], foo(String)[2]

// (method()[1][2]); do not append again just because the name does not end
// with the last class index.
if (!classTemplateInvocationSuffix.isEmpty() && !methodDesc.contains(classTemplateInvocationSuffix)) {
methodDesc = methodDesc + classTemplateInvocationSuffix;
}
String methodDisp = hasDisplayName ? methodDisplay : methodDesc;

// The behavior of methods getLegacyReportingName() and getDisplayName().
Expand Down
169 changes: 169 additions & 0 deletions ...latform/src/test/java/org/apache/maven/surefire/junitplatform/RunListenerAdapterTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@
import static org.mockito.Mockito.inOrder;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoMoreInteractions;
import static org.mockito.Mockito.when;
Expand Down Expand Up @@ -142,6 +143,127 @@ public void notifiedWithCompatibleNameForMethodWithArguments() throws Exception
assertNull(entry.getStackTraceWriter());
}

@Test
public void distinguishedJUnit6ParameterizedClassInvocationsByIndex() throws Exception {
// JUnit 6 @ParameterizedClass uses [class-template-invocation:#N] parents with a ClassSource.
// The method's own legacy name has no [N] suffix (unlike JUnit 5.14 / @ParameterizedTest),
// so invocations must still be reported under distinct names or rerunFailingTestsCount
// aggregates a pass+fail pair as a flake (#3303).
EngineDescriptor engine = new EngineDescriptor(UniqueId.forEngine("junit-jupiter"), "JUnit Jupiter");
TestDescriptor classTemplate = newParameterizedClassTemplateDescriptor(engine.getUniqueId());
engine.addChild(classTemplate);

TestDescriptor invocation1 = newParameterizedClassInvocationDescriptor(classTemplate.getUniqueId(), 1);
TestDescriptor method1 = newUnparameterizedMethodDescriptor(invocation1.getUniqueId());
classTemplate.addChild(invocation1);
invocation1.addChild(method1);

TestDescriptor invocation2 = newParameterizedClassInvocationDescriptor(classTemplate.getUniqueId(), 2);
TestDescriptor method2 = newUnparameterizedMethodDescriptor(invocation2.getUniqueId());
classTemplate.addChild(invocation2);
invocation2.addChild(method2);

TestPlan plan = TestPlan.from(false, singletonList(engine), CONFIG_PARAMS, OUTPUT_DIRECTORY);
adapter.testPlanExecutionStarted(plan);

adapter.executionStarted(TestIdentifier.from(engine));
adapter.executionStarted(TestIdentifier.from(classTemplate));
adapter.executionStarted(TestIdentifier.from(invocation1));
adapter.executionStarted(TestIdentifier.from(method1));
adapter.executionFinished(TestIdentifier.from(method1), successful());
adapter.executionFinished(TestIdentifier.from(invocation1), successful());
adapter.executionStarted(TestIdentifier.from(invocation2));
adapter.executionStarted(TestIdentifier.from(method2));
adapter.executionFinished(TestIdentifier.from(method2), failed(new AssertionError("fail")));

ArgumentCaptor<ReportEntry> started = ArgumentCaptor.forClass(ReportEntry.class);
verify(listener, times(2)).testStarting(started.capture());
assertEquals(
MY_TEST_METHOD_NAME + "()[1]", started.getAllValues().get(0).getName());
assertEquals(
MY_TEST_METHOD_NAME + "()[2]", started.getAllValues().get(1).getName());

ArgumentCaptor<ReportEntry> failed = ArgumentCaptor.forClass(ReportEntry.class);
verify(listener).testFailed(failed.capture());
assertEquals(MY_TEST_METHOD_NAME + "()[2]", failed.getValue().getName());
assertThat(failed.getValue().getName())
.isNotEqualTo(started.getAllValues().get(0).getName());
}

@Test
public void distinguishedNestedJUnit6ParameterizedClassInvocations() throws Exception {
EngineDescriptor engine = new EngineDescriptor(UniqueId.forEngine("junit-jupiter"), "JUnit Jupiter");
TestDescriptor outerTemplate = newParameterizedClassTemplateDescriptor(engine.getUniqueId());
engine.addChild(outerTemplate);

TestDescriptor outer1 = newParameterizedClassInvocationDescriptor(outerTemplate.getUniqueId(), 1);
TestDescriptor innerTemplate1 = newParameterizedClassTemplateDescriptor(outer1.getUniqueId());
TestDescriptor inner1 = newParameterizedClassInvocationDescriptor(innerTemplate1.getUniqueId(), 1);
TestDescriptor method11 = newUnparameterizedMethodDescriptor(inner1.getUniqueId());
outerTemplate.addChild(outer1);
outer1.addChild(innerTemplate1);
innerTemplate1.addChild(inner1);
inner1.addChild(method11);

TestDescriptor outer2 = newParameterizedClassInvocationDescriptor(outerTemplate.getUniqueId(), 2);
TestDescriptor innerTemplate2 = newParameterizedClassTemplateDescriptor(outer2.getUniqueId());
TestDescriptor inner2 = newParameterizedClassInvocationDescriptor(innerTemplate2.getUniqueId(), 1);
TestDescriptor method21 = newUnparameterizedMethodDescriptor(inner2.getUniqueId());
outerTemplate.addChild(outer2);
outer2.addChild(innerTemplate2);
innerTemplate2.addChild(inner2);
inner2.addChild(method21);

TestPlan plan = TestPlan.from(false, singletonList(engine), CONFIG_PARAMS, OUTPUT_DIRECTORY);
adapter.testPlanExecutionStarted(plan);

adapter.executionStarted(TestIdentifier.from(engine));
adapter.executionStarted(TestIdentifier.from(outerTemplate));
adapter.executionStarted(TestIdentifier.from(outer1));
adapter.executionStarted(TestIdentifier.from(innerTemplate1));
adapter.executionStarted(TestIdentifier.from(inner1));
adapter.executionStarted(TestIdentifier.from(method11));
adapter.executionFinished(TestIdentifier.from(method11), successful());
adapter.executionStarted(TestIdentifier.from(outer2));
adapter.executionStarted(TestIdentifier.from(innerTemplate2));
adapter.executionStarted(TestIdentifier.from(inner2));
adapter.executionStarted(TestIdentifier.from(method21));
adapter.executionFinished(TestIdentifier.from(method21), failed(new AssertionError("fail")));

ArgumentCaptor<ReportEntry> started = ArgumentCaptor.forClass(ReportEntry.class);
verify(listener, times(2)).testStarting(started.capture());
assertEquals(
MY_TEST_METHOD_NAME + "()[1][1]", started.getAllValues().get(0).getName());
assertEquals(
MY_TEST_METHOD_NAME + "()[2][1]", started.getAllValues().get(1).getName());
}

@Test
public void doesNotReappendClassIndexWhenLegacyNameAlreadyHasIt() throws Exception {
// JUnit 6.1+ parameterized methods already report method()[class][method].
EngineDescriptor engine = new EngineDescriptor(UniqueId.forEngine("junit-jupiter"), "JUnit Jupiter");
TestDescriptor classTemplate = newParameterizedClassTemplateDescriptor(engine.getUniqueId());
engine.addChild(classTemplate);

TestDescriptor invocation = newParameterizedClassInvocationDescriptor(classTemplate.getUniqueId(), 1);
TestDescriptor method = newLegacyIndexedMethodDescriptor(invocation.getUniqueId(), "()[1][2]");
classTemplate.addChild(invocation);
invocation.addChild(method);

TestPlan plan = TestPlan.from(false, singletonList(engine), CONFIG_PARAMS, OUTPUT_DIRECTORY);
adapter.testPlanExecutionStarted(plan);

adapter.executionStarted(TestIdentifier.from(engine));
adapter.executionStarted(TestIdentifier.from(classTemplate));
adapter.executionStarted(TestIdentifier.from(invocation));
adapter.executionStarted(TestIdentifier.from(method));
adapter.executionFinished(TestIdentifier.from(method), successful());

ArgumentCaptor<ReportEntry> started = ArgumentCaptor.forClass(ReportEntry.class);
verify(listener).testStarting(started.capture());
assertEquals(MY_TEST_METHOD_NAME + "()[1][2]", started.getValue().getName());
}

@Test
public void notifiedEagerlyForTestSetWhenClassExecutionStarted() throws Exception {
EngineDescriptor engine = newEngineDescriptor();
Expand Down Expand Up @@ -800,6 +922,53 @@ public Type getType() {
assertEquals("Run a dummy cucumber test", entry.getName());
}

private static TestDescriptor newParameterizedClassTemplateDescriptor(UniqueId engineId) {
return new ClassTestDescriptor(
engineId.append("class-template", MyTestClass.class.getName()),
MyTestClass.class,
new DefaultJupiterConfiguration(CONFIG_PARAMS, OUTPUT_DIRECTORY));
}

private static TestDescriptor newParameterizedClassInvocationDescriptor(UniqueId classTemplateId, int index) {
return new AbstractTestDescriptor(
classTemplateId.append("class-template-invocation", "#" + index),
"Parameterization with index: [" + index + "]",
ClassSource.from(MyTestClass.class)) {
@Override
public Type getType() {
return CONTAINER;
}

@Override
public String getLegacyReportingName() {
return MyTestClass.class.getSimpleName() + "[" + index + "]";
}
};
}

private static TestDescriptor newUnparameterizedMethodDescriptor(UniqueId parentId) throws Exception {
return newLegacyIndexedMethodDescriptor(parentId, "()");
}

private static TestDescriptor newLegacyIndexedMethodDescriptor(UniqueId parentId, String legacySuffix)
throws Exception {
Method method = MyTestClass.class.getDeclaredMethod(MY_TEST_METHOD_NAME);
return new AbstractTestDescriptor(
parentId.append("method", method.getName() + "()"),
method.getName() + "()",
MethodSource.from(MyTestClass.class, method)) {
@Override
public Type getType() {
return TEST;
}

@Override
public String getLegacyReportingName() {
return method.getName() + legacySuffix;
}
};
}

private static TestIdentifier newMethodIdentifier() throws Exception {
return TestIdentifier.from(newMethodDescriptor());
}
Expand Down
Loading