feat: Invoking a sub-command from an operation workflow #2792

didier-wenzek · 2024-03-19T13:40:20Z

Proposed changes

Reference documentation for workflow sub-command execution
Trigger a sub-command and await its completion
Pass arguments to a sub-command
Inject sub-command outcome in the invoking command state
Give the ability to use a script to compute the sub-command init state.
Logging a sub-command in the same log file as the invoking command

Types of changes

Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
Documentation Update (if none of the other choices apply)
Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

Invoking a sub-command from an operation workflow #2790

Checklist

I have read the CONTRIBUTING doc
I have signed the CLA (in all commits with git commit -s)
I ran cargo fmt as mentioned in CODING_GUIDELINES
I used cargo clippy as mentioned in CODING_GUIDELINES
I have added tests that prove my fix is effective or that my feature works
I have added necessary documentation (if appropriate)

Further comments

Follow-up tasks

Handle sub-command execution timeouts
Improve workflow log output to ease debugging
Add syntactic sugar so users don't have to add explicit await_operation_completion steps.
Deprecate built-in restart action
Consider alternative designs to track the invoking command of a sub-command. See feat: Invoking a sub-command from an operation workflow #2792 (comment)
Improve test coverage feat: Invoking a sub-command from an operation workflow #2792 (comment)
Finalize naming of workflow definition items, see feat: Invoking a sub-command from an operation workflow #2792 (comment)
Fix the behavior of sub-sub-commands. See feat: Invoking a sub-command from an operation workflow #2792 (comment)
Consider to publish a failed state on state parsing error (see feat: Invoking a sub-command from an operation workflow #2792 (comment))

docs/src/references/agent/operation-workflow.md

github-actions · 2024-03-20T13:59:18Z

Robot Results

✅ Passed	❌ Failed	⏭️ Skipped	Total	Pass %	⏱️ Duration
432	0	3	432	100	0s

codecov · 2024-03-20T15:56:46Z

Codecov Report

Attention: Patch coverage is 40.16393% with 365 lines in your changes are missing coverage. Please review.

Project coverage is 76.3%. Comparing base (fc1da52) to head (25ca6e6).
Report is 5 commits behind head on main.

Additional details and impacted files

Files	Coverage Δ
crates/common/mqtt_channel/src/topics.rs	`86.2% <100.0%> (+0.3%)`	⬆️
crates/core/tedge_api/src/commands.rs	`86.8% <100.0%> (-0.2%)`	⬇️
crates/core/tedge_api/src/lib.rs	`100.0% <100.0%> (ø)`
...s/core/tedge_agent/src/operation_file_cache/mod.rs	`0.0% <0.0%> (ø)`
crates/core/tedge_api/src/workflow/error.rs	`0.0% <0.0%> (ø)`
crates/core/tedge_api/src/workflow/on_disk.rs	`40.9% <0.0%> (+3.4%)`	⬆️
crates/core/tedge_api/src/mqtt_topics.rs	`87.5% <87.5%> (-0.1%)`	⬇️
crates/core/tedge_api/src/workflow/mod.rs	`60.1% <0.0%> (-6.8%)`	⬇️
crates/core/tedge_api/src/workflow/supervisor.rs	`56.3% <34.6%> (-7.2%)`	⬇️
crates/core/tedge_api/src/workflow/toml_config.rs	`74.4% <0.0%> (-3.8%)`	⬇️
... and 3 more

... and 4 files with indirect coverage changes

docs/src/references/agent/operation-workflow.md

tests/RobotFramework/tests/tedge_agent/workflows/restart_sub_command.toml

docs/src/references/agent/operation-workflow.md

crates/core/tedge_agent/src/tedge_operation_converter/actor.rs

albinsuresh · 2024-03-27T14:23:23Z

crates/core/tedge_api/src/workflow/state.rs

+/// A set of values to be injected/extracted into/from a [GenericCommandState]
+#[derive(Clone, Debug, Deserialize, Eq, PartialEq)]
+#[serde(try_from = "Option<Value>")]
+pub enum StateExcerpt {


Some unit tests for the conversions to/from this struct would be good.

You seem to have missed this. Can be added in a follow-up PR as well, to avoid blocking this one.

crates/core/tedge_api/src/workflow/state.rs

Signed-off-by: Didier Wenzek <[email protected]>

- A super-command can trigger a sub-command defined by another workflow - Data can be injected from the current super-command state payload into the sub-command init payload. - Data can be injected back from the successful sub-command payload into the super-command current state payload. Signed-off-by: Didier Wenzek <[email protected]>

AwaitHandlers will also be used to await the completion of a subcommand. Signed-off-by: Didier Wenzek <[email protected]>

Signed-off-by: Didier Wenzek <[email protected]>

Extracting a value from a command state will be required for the next which is to extract data from a command state and to inject this excerpt into a sub-command state. Signed-off-by: Didier Wenzek <[email protected]>

Signed-off-by: Didier Wenzek <[email protected]>

…kflow The word *command* is easily confused with *script*. Signed-off-by: Didier Wenzek <[email protected]>

Signed-off-by: Didier Wenzek <[email protected]>

… payload Signed-off-by: Didier Wenzek <[email protected]>

Signed-off-by: Didier Wenzek <[email protected]>

Signed-off-by: Reuben Miller <[email protected]>

didier-wenzek had a problem deploying to Test Pull Request March 19, 2024 13:40 — with GitHub Actions Failure

didier-wenzek commented Mar 19, 2024

View reviewed changes

docs/src/references/agent/operation-workflow.md Show resolved Hide resolved

docs/src/references/agent/operation-workflow.md Outdated Show resolved Hide resolved

docs/src/references/agent/operation-workflow.md Show resolved Hide resolved

didier-wenzek requested a review from albinsuresh March 19, 2024 13:53

didier-wenzek had a problem deploying to Test Pull Request March 20, 2024 13:28 — with GitHub Actions Failure

didier-wenzek temporarily deployed to Test Pull Request March 20, 2024 13:37 — with GitHub Actions Inactive

didier-wenzek had a problem deploying to Test Auto March 20, 2024 13:47 — with GitHub Actions Failure

didier-wenzek had a problem deploying to Test Pull Request March 20, 2024 15:45 — with GitHub Actions Failure

reubenmiller reviewed Mar 20, 2024

View reviewed changes

docs/src/references/agent/operation-workflow.md Outdated Show resolved Hide resolved

didier-wenzek had a problem deploying to Test Pull Request March 21, 2024 18:24 — with GitHub Actions Failure

didier-wenzek temporarily deployed to Test Pull Request March 25, 2024 11:22 — with GitHub Actions Inactive

didier-wenzek had a problem deploying to Test Auto March 25, 2024 11:28 — with GitHub Actions Failure

didier-wenzek temporarily deployed to Test Pull Request March 26, 2024 17:41 — with GitHub Actions Inactive

didier-wenzek had a problem deploying to Test Auto March 26, 2024 17:48 — with GitHub Actions Failure

didier-wenzek temporarily deployed to Test Pull Request March 26, 2024 18:23 — with GitHub Actions Inactive

didier-wenzek had a problem deploying to Test Auto March 26, 2024 18:29 — with GitHub Actions Failure

didier-wenzek temporarily deployed to Test Pull Request March 27, 2024 08:38 — with GitHub Actions Inactive

didier-wenzek had a problem deploying to Test Auto March 27, 2024 08:43 — with GitHub Actions Failure

didier-wenzek temporarily deployed to Test Pull Request March 27, 2024 09:25 — with GitHub Actions Inactive

didier-wenzek marked this pull request as ready for review March 27, 2024 09:27

didier-wenzek requested review from Bravo555, jarhodes314, rina23q and a team as code owners March 27, 2024 09:27

didier-wenzek commented Mar 27, 2024

View reviewed changes

tests/RobotFramework/tests/tedge_agent/workflows/restart_sub_command.toml Outdated Show resolved Hide resolved

didier-wenzek temporarily deployed to Test Auto March 27, 2024 09:31 — with GitHub Actions Inactive

didier-wenzek commented Mar 27, 2024

View reviewed changes

docs/src/references/agent/operation-workflow.md Outdated Show resolved Hide resolved

didier-wenzek commented Mar 27, 2024

View reviewed changes

docs/src/references/agent/operation-workflow.md Outdated Show resolved Hide resolved

albinsuresh reviewed Mar 27, 2024

View reviewed changes

didier-wenzek and others added 22 commits April 25, 2024 09:34

Reference documentation for workflow sub-command execution

ce97c18

Signed-off-by: Didier Wenzek <[email protected]>

Impl TryFrom<TomlExitHandlers> for AwaitHandlers

782ae0c

AwaitHandlers will also be used to await the completion of a subcommand. Signed-off-by: Didier Wenzek <[email protected]>

Trigger a sub-command from a main command

bfcd349

Signed-off-by: Didier Wenzek <[email protected]>

Improve GenericCommandState::extract_value signature

ec896b5

Extracting a value from a command state will be required for the next which is to extract data from a command state and to inject this excerpt into a sub-command state. Signed-off-by: Didier Wenzek <[email protected]>

Pass data back and forth from invoking commands to sub-commands

904a22d

Signed-off-by: Didier Wenzek <[email protected]>

Test workflow restarting the device with a sub-command

39522ae

Signed-off-by: Didier Wenzek <[email protected]>

Remove unnecessary to_owned invocation

2b1f55f

Signed-off-by: Didier Wenzek <[email protected]>

Log sub-command steps in the invoking command log file

383675a

Signed-off-by: Didier Wenzek <[email protected]>

Add ability to use scripts to compute sub-command init states

3883cca

Signed-off-by: Didier Wenzek <[email protected]>

Use operation instead of command to trigger another operation wor…

56fef00

…kflow The word *command* is easily confused with *script*. Signed-off-by: Didier Wenzek <[email protected]>

Log sub-operation input script

c7674f9

Signed-off-by: Didier Wenzek <[email protected]>

Use early return instead of nested matches

33cb085

Signed-off-by: Didier Wenzek <[email protected]>

Simplify usage of MqttChannel::entity_channel_of

497c0ae

Signed-off-by: Didier Wenzek <[email protected]>

Represent command clearing message by GenericCommandState with a null…

3eb4c8b

… payload Signed-off-by: Didier Wenzek <[email protected]>

Store invoking command topic in sub command state

9466f74

Signed-off-by: Didier Wenzek <[email protected]>

Move topic helper functions under MqttSchema

03a9abc

Signed-off-by: Didier Wenzek <[email protected]>

docs: add construction notice to sub-command execution

aa18f4e

Signed-off-by: Reuben Miller <[email protected]>

rewording the awaiting sub-command instructions

e5bb662

Signed-off-by: Reuben Miller <[email protected]>

using formal language

021853d

Signed-off-by: Reuben Miller <[email protected]>

Rewording sub-operation execution doc

ff106c5

test: detect reboot via temp file creation

25ca6e6

Signed-off-by: Reuben Miller <[email protected]>

didier-wenzek force-pushed the feat/trigger-a-workflow-from-a-workflow branch from d6aef9a to 25ca6e6 Compare April 25, 2024 08:02

didier-wenzek temporarily deployed to Test Pull Request April 25, 2024 08:02 — with GitHub Actions Inactive

didier-wenzek temporarily deployed to Test Auto April 25, 2024 08:08 — with GitHub Actions Inactive

didier-wenzek added this pull request to the merge queue Apr 25, 2024

Merged via the queue into thin-edge:main with commit d5478ce Apr 25, 2024
32 checks passed

didier-wenzek deleted the feat/trigger-a-workflow-from-a-workflow branch April 25, 2024 08:54

didier-wenzek mentioned this pull request May 3, 2024

A sub-operation cannot trigger a sub-sub operation #2858

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Invoking a sub-command from an operation workflow #2792

feat: Invoking a sub-command from an operation workflow #2792

didier-wenzek commented Mar 19, 2024 •

edited

Loading

github-actions bot commented Mar 20, 2024 •

edited

Loading

codecov bot commented Mar 20, 2024 •

edited

Loading

albinsuresh Mar 27, 2024

albinsuresh Apr 24, 2024

feat: Invoking a sub-command from an operation workflow #2792

feat: Invoking a sub-command from an operation workflow #2792

Conversation

didier-wenzek commented Mar 19, 2024 • edited Loading

Proposed changes

Types of changes

Paste Link to the issue

Checklist

Further comments

Follow-up tasks

github-actions bot commented Mar 20, 2024 • edited Loading

Robot Results

codecov bot commented Mar 20, 2024 • edited Loading

Codecov Report

albinsuresh Mar 27, 2024

Choose a reason for hiding this comment

albinsuresh Apr 24, 2024

Choose a reason for hiding this comment

didier-wenzek commented Mar 19, 2024 •

edited

Loading

github-actions bot commented Mar 20, 2024 •

edited

Loading

codecov bot commented Mar 20, 2024 •

edited

Loading