Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add plugin and template_id to resource and attachment results in submission.json (bugfix) #1596

Merged
merged 1 commit into from
Nov 13, 2024

Conversation

pieqq
Copy link
Collaborator

@pieqq pieqq commented Nov 12, 2024

When using check-jsonschema using a recent version of a submission.json file against the JSON schema in the repo, the following errors show up:

check-jsonschema --schemafile submission-schema/schema.json /home/pieq/.local/share/checkbox-ng/submission_2024-11-12T13.48.37.650193/submission.json 
Schema validation errors were encountered.
(...)
  /home/pieq/.local/share/checkbox-ng/submission_2024-11-12T13.48.37.650193/submission.json::$.resource-results[0]: 'plugin' is a required property
  /home/pieq/.local/share/checkbox-ng/submission_2024-11-12T13.48.37.650193/submission.json::$.resource-results[0]: 'template_id' is a required property

This is because resource jobs stored in the resource-results do not contain these fields, as the plugin is assumed to be resource. Same for attachment-results.

However, these are all jobs and therefore the different results arrays should match in keys and values returned (it's a bit of a mystery why they were separated to begin with...).

Adding these properties to resource-results and attachment-results.

Description

Resolved issues

Documentation

JSON schema updated.

Tests

New generated submission with resource results:

    "resource-results": [
        {
            "id": "bluez-internal-bnep-tests",
            "full_id": "com.canonical.certification::bluez-internal-bnep-tests",
            "name": "Gather list of tests provided by bluez bnep test binary",
            "certification_status": "non-blocker",
            "category": "Uncategorised",
            "category_id": "com.canonical.plainbox::uncategorised",
            "status": "pass",
            "outcome": "pass",
            "comments": null,
            "io_log": "bash: line 1: bnep-tester: command not found\n",
            "type": "test",
            "project": "certification",
            "duration": 0.1284027099609375,
            "plugin": "resource",
            "template_id": null
        },
...

and attachment results:

    "attachment-results": [
        {
...
        },
        {
            "id": "lspci_standard_config_json",
            "full_id": "com.canonical.certification::lspci_standard_config_json",
            "name": "Attach PCI configuration space hex dump",
            "certification_status": "non-blocker",
            "category": "Informational tests",
            "category_id": "com.canonical.plainbox::info",
            "status": "pass",
            "outcome": "pass",
            "comments": null,
            "io_log": "\"0b08\"\n",
            "duration": 1.9314079284667969,
            "plugin": "attachment",
            "template_id": null
        },

Copy link
Collaborator

@Hook25 Hook25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that those fields aren't required is because they are missing from *-results fields, which in my opinion is silly (I don't even understand why we split them, we have a plugin field, idk). Either way, this is not the solution in my opinion, I would add them to the other results.

These values should be there, since all of these are jobs.
@pieqq pieqq force-pushed the schema-json-optional-fields branch from aa3a89b to 4ca16eb Compare November 13, 2024 14:02
@pieqq pieqq changed the title Remove plugin and template_id from required fields in JSON schema (bugfix) Add plugin and template_id to resource and attachment results in submission.json (bugfix) Nov 13, 2024
Copy link

codecov bot commented Nov 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 48.03%. Comparing base (542c748) to head (4ca16eb).
Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1596   +/-   ##
=======================================
  Coverage   48.03%   48.03%           
=======================================
  Files         371      371           
  Lines       39852    39852           
  Branches     6734     6734           
=======================================
  Hits        19142    19142           
  Misses      19993    19993           
  Partials      717      717           
Flag Coverage Δ
checkbox-ng 68.65% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@pieqq
Copy link
Collaborator Author

pieqq commented Nov 13, 2024

@Hook25 thanks for your feedback. I've modified the PR to do as suggested. Maybe one day we'll be able to merge those arrays back together! xD

Copy link
Collaborator

@Hook25 Hook25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, ty

@Hook25 Hook25 merged commit b965681 into main Nov 13, 2024
54 checks passed
@Hook25 Hook25 deleted the schema-json-optional-fields branch November 13, 2024 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants