Collect system information (New)

[Hook25]

Description

We have decided in CR041 that we are going to rely, among others, on inxi to fetch the hardware of a system. The best place to inject this new step is the bootstrap sequence before any resource job is collected and executed.

This should result in the collection being included as a map in the session storage and dumped at the end of the execution under a new key of the submission.json file in this form:

If a tool runs successfully

"system_information" : {
  "tool_name" : { 
    “tool_version” :  tool_version,
     "success" : True,
     “outputs” : { 
       "payload" : tool_json_output,
       "stderr" : tool_stderr
    }
  } 
}

If a tool fails or produced an invalid json output:

"system_information" : {
  "tool_name" : { 
    “tool_version” :  tool_version, 
    "success" : False,
    “outputs” : { 
      “stdout” : tool_stdout,
      “stderr” : tool_stderr
    }
  } 
}

Resolved issues

Resolves: https://warthogs.atlassian.net/browse/CHECKBOX-900

Documentation

WIP

Tests

The change was tested via metabox both in snaps and source runs

[codecov]

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (2e0f376) 36.58% compared to head (d37eb71) 36.78%.
Report is 9 commits behind head on main.

Files	Patch %	Lines
...box-ng/plainbox/impl/session/system_information.py	93.25%	6 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #760      +/-   ##
==========================================
+ Coverage   36.58%   36.78%   +0.20%     
==========================================
  Files         310      311       +1     
  Lines       34639    34759     +120     
  Branches     5965     5976      +11     
==========================================
+ Hits        12671    12786     +115     
- Misses      21399    21405       +6     
+ Partials      569      568       -1     

Flag	Coverage Δ
checkbox-ng	62.92% <95.04%> (+0.27%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[kissiel]

Many things need changing, most of them posted below.
Generally to much unnecessary tight coupling, for something that's practically static.
Now for the whole keeping the state of the session in the state object -> why?
The sysinfo is gathered at the start of the session, and kept there and submitted with the submission. Awesome. But it may be done at the end of the session as well, and wouldn't require any change to the suspend/resume in Checkbox. "what if sysinfo changes during the session", well, with the code submitted here we also wouldn't notice.

[yphus]

The sysinfo is gathered at the start of the session

Getting hw info at the start of the session is IMHO a good idea, to reuse the same data for resources expression (whatever the form of such expression) or to combine one collector with another, or even with the hw manifest. Needless to call again inxi if the data collected at session start can be turned into something jobs requirements can consume.

[kissiel]

I think I found something in need of deletion

[kissiel]

Have you tested how it affects other subcommands that create temporary sessions?
I remember other subcommands creating a "throw-away" sessions, and if this collection is run when, for instance, listing units using checkbox-cli list it would become extremely slow.

[kissiel]

The code looks great. The problem I've got is I think its complexity is unnecessary in places.
It may be best if we sit on it together, so one of two outcomes happen:

• I see what I'm missing, and why the things included here are necessary
• the outputs gets simpler

[kissiel]

Monumental work!
Thanks for all the tweaks and improvements!

[kissiel]

LGTM