Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for @AfterAll in XML report #300

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

fzakaria
Copy link
Contributor

@fzakaria fzakaria commented Oct 10, 2024

Writing a simple test like

package com.library;

import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Test;

public class AlwaysFailTest {
    @Test
    public void doNothingTest() {
        System.out.println("Hi there!");
    }

    @AfterAll
    public static void alwaysFail() {
        throw new RuntimeException("Always failing.");
    }
}

Produces an XML that seems to look like it succeeds.

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="com.library.AlwaysFailTest" timestamp="2024-10-10T21:19:11.522176Z" hostname="KHK9NLVQGN" tests="2" failures="0" errors="0" disabled="0" skipped="0" package="">
    <properties/>
    <testcase name="doNothingTest" classname="com.library.AlwaysFailTest" time="0.03">
      <system-out><![CDATA[Hi there!
]]></system-out>
    </testcase>
  </testsuite>
</testsuites>

Bazel itself correctly identifies it fails. The Junit4 reporter also included with Bazel natively correclty reports the failure in the XML output.

Augment the Junit listener to add support for static methods and add them to the JUnit output.

Doing so produces the following XML.

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="com.library.AlwaysFailTest" timestamp="2024-10-10T21:49:02.096648Z" hostname="KHK9NLVQGN" tests="3" failures="1" errors="0" disabled="0" skipped="0" package="">
    <properties/>
    <testcase name="com.library.AlwaysFailTest" classname="com.library.AlwaysFailTest" time="0.05">
      <failure message="Always failing." type="java.lang.RuntimeException"><![CDATA[java.lang.RuntimeException: Always failing.
	at com.library.AlwaysFailTest.alwaysFail(AlwaysFailTest.java:20)
...
	at com.github.bazel_contrib.contrib_rules_jvm.junit5.JUnit5Runner.main(JUnit5Runner.java:39)
]]></failure>
    </testcase>
    <testcase name="doNothingTest" classname="com.library.AlwaysFailTest" time="0.03">
      <system-out><![CDATA[Hi there!
]]></system-out>
    </testcase>
  </testsuite>
</testsuites>

Co-authored-by: Vince Rose [email protected]


if (segments.size() == 3 || segments.size() == 5) {

if (segments.size() == 2) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here was the printout of the segment for the static methods:

[Segment [type = 'engine', value = 'junit-jupiter'], Segment [type = 'class', value = 'com.library.AlwaysFailTest']]

@@ -161,7 +164,7 @@ private List<TestData> findTestCases_locked() {
// are identified by the fact that they have no child test cases in the
// test plan, or they are marked as tests.
TestIdentifier id = result.getId();
return id.isTest() || testPlan.getChildren(id).isEmpty();
return id.getSource() != null || id.isTest() || testPlan.getChildren(id).isEmpty();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% on what the right filter here.
I am choosing source but not clear why getChildren is a filter
For the @AfterAll methods it looks like they have children.

Here is an example of children from the PR description @AfterAll method:

[TestIdentifier [uniqueId = [engine:junit-jupiter]/[class:com.library.AlwaysFailTest]/[method:doNothingTest()], parentId = [engine:junit-jupiter]/[class:com.library.AlwaysFailTest], displayName = 'doNothingTest()', legacyReportingName = 'doNothingTest()', source = MethodSource [className = 'com.library.AlwaysFailTest', methodName = 'doNothingTest', methodParameterTypes = ''], tags = [], type = TEST], TestIdentifier [uniqueId = [engine:junit-jupiter]/[class:com.library.AlwaysFailTest]/[method:executeFirst()], parentId = [engine:junit-jupiter]/[class:com.library.AlwaysFailTest], displayName = 'executeFirst()', legacyReportingName = 'executeFirst()', source = MethodSource [className = 'com.library.AlwaysFailTest', methodName = 'executeFirst', methodParameterTypes = ''], tags = [], type = TEST]]

@@ -146,4 +146,17 @@ public boolean isDynamic() {
public Instant getStarted() {
return this.started;
}

@Override
public String toString() {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was just helpful.

Writing a simple test like
```
package com.library;

import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Test;

public class AlwaysFailTest {
    @test
    public void doNothingTest() {
        System.out.println("Hi there!");
    }

    @afterall
    public static void alwaysFail() {
        throw new RuntimeException("Always failing.");
    }
}
```

Produces an XML that seems to look like it succeeds.
```xml
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="com.library.AlwaysFailTest" timestamp="2024-10-10T21:19:11.522176Z" hostname="KHK9NLVQGN" tests="2" failures="0" errors="0" disabled="0" skipped="0" package="">
    <properties/>
    <testcase name="doNothingTest" classname="com.library.AlwaysFailTest" time="0.03">
      <system-out><![CDATA[Hi there!
]]></system-out>
    </testcase>
  </testsuite>
</testsuites>
```

Bazel itself correctly identifies it fails. The Junit4 reporter also included with Bazel natively correclty reports the failure in the XML output.

Augment the Junit listener to add support for static methods and add them to the JUnit output.

Doing so produces the following XML.
```xml
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="com.library.AlwaysFailTest" timestamp="2024-10-10T21:49:02.096648Z" hostname="KHK9NLVQGN" tests="3" failures="1" errors="0" disabled="0" skipped="0" package="">
    <properties/>
    <testcase name="com.library.AlwaysFailTest" classname="com.library.AlwaysFailTest" time="0.05">
      <failure message="Always failing." type="java.lang.RuntimeException"><![CDATA[java.lang.RuntimeException: Always failing.
	at com.library.AlwaysFailTest.alwaysFail(AlwaysFailTest.java:20)
...
	at com.github.bazel_contrib.contrib_rules_jvm.junit5.JUnit5Runner.main(JUnit5Runner.java:39)
]]></failure>
    </testcase>
    <testcase name="doNothingTest" classname="com.library.AlwaysFailTest" time="0.03">
      <system-out><![CDATA[Hi there!
]]></system-out>
    </testcase>
  </testsuite>
</testsuites>
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant