Add test results visualization #360

mw-kapilg · 2024-09-18T08:53:51Z

This feature creates an in-platform view for MATLAB test results (whenever a default test runner or pre-made TestTask is used) if the user has a MATLAB Test license.

Key highlights:

Each row of the table represents the test file.
All tests for a single file are shown under that file.
The files with same name and different parent directories are shown separately and you can hover over the file name to see its path.
The table in table format allows easy management of test cases for a test file
You see the logs in-place for a failed/incomplete. If there are more than failures within a test case, say multiple verification failures, each verification failure will have its own dropdown and log box.
Each of the passed, failed, etc. buttons act as filters to respective type of tests
The "Open in MATLAB Online" button helps to redirect users to MATLAB Online. If their Jenkins project is connected to a git repository, they are prompted to open the in MATLAB Online for easy debugging.
The help button redirects users to our plugin's configuration guide.

Here's a few snapshots:

src/main/java/com/mathworks/ci/utilities/MatlabCommandRunner.java

mw-kapilg · 2024-10-23T14:42:24Z

@mw-hrastega, for text review you can go through all the .jelly files.

src/main/resources/com/mathworks/ci/TestResultsViewAction/index.jelly

mw-hrastega · 2024-10-23T15:15:43Z

src/main/resources/com/mathworks/ci/TestResultsViewAction/index.jelly

+            <button id="Incomplete" onclick="showTests(id)" class="jenkins-button jenkins-!-margin-4 jenkins-!-padding-0" style="display: inline; width:8em;" tooltip="Incomplete tests">
+                <p>${it.incompleteCount}<br>
+                    <l:icon class="symbol-alert-circle-outline plugin-ionicons-api icon-lg jenkins-!-warning-color"/>
+                </br><br></br>Incomplete</p>


The test browser uses different icons for incomplete and "not run" tests.

src/main/resources/com/mathworks/ci/TestResultsViewAction/index.jelly

mw-hrastega · 2024-10-23T15:20:47Z

src/main/resources/com/mathworks/ci/TestResultsViewAction/index.jelly

+                                Test Name
+                            </th>
+                            <th class="pane-header" style="width: auto; min-width: 10em; text-align: right;">
+                                Duration (seconds)


(s) instead of (seconds)?

Is it better to include + and - signs or drop-down arrows instead?

Updated (seconds) to (s)

Is it better to include + and - signs or drop-down arrows instead?

I didn't get this, could you explain?

Nothing major. Currently, it seems that you use the + and - buttons to expand and collapse sections in the report. I was wondering whether a drop-down arrow could do the same. In the test browser, we resort to the arrow for expansion: https://www.mathworks.com/help/matlab/matlab_prog/run-tests-using-test-browser.html#:~:text=You%20can%20also%20add%20tests%20to%20the%20test%20browser%20by%20clicking%20the%20drop%2Ddown%20arrow

src/main/resources/com/mathworks/ci/TestResultsViewAction/index.jelly

nbhoski · 2024-10-24T10:52:02Z

pom.xml

-				<artifactId>bom-2.164.x</artifactId>
-				<version>4</version>
+				<artifactId>bom-2.346.x</artifactId>
+				<version>1763.v092b_8980a_f5e</version>


since We are updating this, could we consider updating it to the latest ? https://github.com/jenkinsci/bom/blob/master/README.md

src/main/java/com/mathworks/ci/TestCase.java

src/main/java/com/mathworks/ci/TestDiagnostics.java

src/main/java/com/mathworks/ci/TestFile.java

src/main/java/com/mathworks/ci/TestDiagnostics.java

src/main/java/com/mathworks/ci/TestCase.java

src/main/java/com/mathworks/ci/TestFile.java

src/main/java/com/mathworks/ci/TestResultsViewAction.java

sameagen-MW · 2024-10-25T12:10:43Z

src/main/java/com/mathworks/ci/TestResultsViewAction.java

+
+    public List<List<TestFile>> getTestResults() throws ParseException, InterruptedException, IOException {
+        List<List<TestFile>> testResults = new ArrayList<>();
+        FilePath fl = new FilePath(new File(build.getRootDir().getAbsolutePath() + File.separator + MatlabBuilderConstants.TEST_RESULTS_VIEW_ARTIFACT + this.actionID + ".json"));


Does this work for a Windows server and Linux agent and vice-versa? I've sometimes run into trouble with using File.separator in those cases.

I'm always a little nervous about working with file paths in the Jenkins plugin because it's really easy to get a path to a file on the agent that doesn't exist on the server and vice-versa. Something I mess up all the time haha.

This case looks good to me though as all of the action code should be running on the agent!

Don't get me started on file paths issue with Jenkins, I've devoted so much time to fixing those 😭
Haven't tested this combination yet but sure will test it before releasing.

src/main/java/com/mathworks/ci/TestResultsViewAction.java

src/main/java/com/mathworks/ci/utilities/MatlabCommandRunner.java

src/test/java/integ/com/mathworks/ci/TestResultsViewActionTest.java

mw-kapilg added 30 commits August 1, 2024 10:57

commit to draft test procedure

b7b737a

update finally block

a65e93f

fix merge conflicts

49e7bee

Update actions

a7bd44b

Change build colors

eb83cb6

fix merge conflicts

dc059ae

update colour scheme

9d79737

add tests

12ebdc3

remove extra comments

f917c1f

try fix for ci test failure

284d2df

try fix for ci test failure

0f4b801

add debug statements

fe6fab8

debug failure

cfb1795

debug

2071d66

update input stream

e50a7c7

update paths

620bceb

update test workspace

1c068e4

more debugging

7725c3a

fix testcase workspace

16f0988

final fix (hopefully)

213cfad

revert filepath

46666c9

remove absolute path

cb2f6e8

update workspace path

fb7793b

add logger

678a12d

update test

7b746fd

update test data

2d81d41

Specify java version in yml

06b2d12

update slave label

c15e013

add null checks

40a7af4

add null check

3517803

mw-kapilg requested review from nbhoski, mw-hrastega and sameagen-MW October 23, 2024 08:05

mw-kapilg assigned mcafaro, Vahila and davidbuzinski Oct 23, 2024

mw-kapilg marked this pull request as ready for review October 23, 2024 08:07