c8y-mapper sends software list to advanced software management endpoint #2771

rina23q · 2024-03-08T19:33:33Z

Proposed changes

Since c8y 10.14, advanced software management feature is supported together with the microservice "Advanced-software-mgmt". This PR enables users to use the feature. For more details about the feature, please read the c8y user doc.

Introduced two tedge configs:

c8y.software_management.api
To switch legacy software management (= sending list via HTTP) and advanced one (= sending list via SmartREST).
By default, it is set to legacy. To enhance the detection, the ticket Cumulocity software api configuration options #2778 is created.
c8y.software_management.with_types
If mapper publishes c8y_SupportedSoftwareTypes to the inventory or not. If the type is set in its managed object, c8y stop using c8y_SoftwareList fragment to look up the installed software list. By default, it is set to false so that we can still use c8y_SoftwareList fragment for testing and so on.

Types of changes

Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
Documentation Update (if none of the other choices apply)
Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

#2654

Checklist

I have read the CONTRIBUTING doc
I have signed the CLA (in all commits with git commit -s)
I ran cargo fmt as mentioned in CODING_GUIDELINES
I used cargo clippy as mentioned in CODING_GUIDELINES
I have added tests that prove my fix is effective or that my feature works
I have added necessary documentation (if appropriate)

Further comments

codecov · 2024-03-08T19:43:11Z

Codecov Report

Attention: Patch coverage is 87.41259% with 36 lines in your changes are missing coverage. Please review.

Project coverage is 77.3%. Comparing base (19773d6) to head (be43806).
Report is 10 commits behind head on main.

❗ Current head be43806 differs from pull request most recent head 667652a. Consider uploading reports for the commit 667652a to get more accurate results

Additional details and impacted files

Files	Coverage Δ
...core/c8y_api/src/smartrest/smartrest_serializer.rs	`89.6% <99.2%> (+5.9%)`	⬆️
.../tedge_config/src/tedge_config_cli/tedge_config.rs	`80.1% <80.0%> (-0.1%)`	⬇️
crates/extensions/c8y_mapper_ext/src/inventory.rs	`85.5% <81.8%> (-0.1%)`	⬇️
crates/extensions/c8y_mapper_ext/src/config.rs	`46.9% <50.0%> (-0.2%)`	⬇️
crates/extensions/c8y_mapper_ext/src/tests.rs	`92.7% <94.3%> (+<0.1%)`	⬆️
...tedge_config_cli/models/c8y_software_management.rs	`56.2% <56.2%> (ø)`
crates/extensions/c8y_mapper_ext/src/converter.rs	`83.4% <60.0%> (+0.7%)`	⬆️

... and 2 files with indirect coverage changes

github-actions · 2024-03-11T19:57:23Z

Robot Results

✅ Passed	❌ Failed	⏭️ Skipped	Total	Pass %	⏱️ Duration
426	0	3	426	100	0s

tests/RobotFramework/tests/cumulocity/software_management/software.robot

crates/common/tedge_config/src/tedge_config_cli/models/c8y_software_management.rs

didier-wenzek · 2024-03-19T15:27:10Z

crates/extensions/c8y_mapper_ext/src/converter.rs

+        // Send c8y_SupportedSoftwareTypes, which is introduced in c8y >= 10.14
+        let data = SoftwareCommandMetadata::from_json(message.payload_str()?)?;
+        let payload = json!({"c8y_SupportedSoftwareTypes": data.types}).to_string();
+        let topic = self.get_inventory_update_topic(target)?;


This PR is not the right place to improve on this, but we will have to find a better pattern before being overwhelmed by this kind of artificial and error-prone complexity.

Artificial, because we only need to translate a tedge MQTT message into a c8y one. This is not obvious from these lines.

Error-prone, because the message must be partially parsed before calling this method (to extract the target and message topic from the message.topic).

I agree this is not so easy, as we need to access the inventory to translate the tedge entity id into a c8y external id.

Yeah we can work to improve, however, I'm not sure how much I should do in this PR...

By the way, since this action updates the device's c8y inventory, should the mapper publish to our twin topic, instead of directly publishing to the JSON over MQTT topic?

I'm not sure how much I should do in this PR...

Definitely not in this PR as this would impact all the messages received by the c8y converter.

should the mapper publish to our twin topic, instead of directly publishing to the JSON over MQTT topic?

No, the twin topic is where child devices publish their metadata. The c8y mapper should react on these messages, not push data there.

reubenmiller · 2024-03-20T16:04:34Z

@rina23q you'll have to rebase your PR to get the workflows to work again (since #2781 was merged)

didier-wenzek

Approved

Since Cumulocity 10.14, it is recommended to send software list to the advanced software management endpoint. By default, the mapper publishes its software list via SmartREST. If `c8y.software_management.api` is set to 'true', the mapper also sends c8y_SupportedSoftwareTypes to inventory via JSON over MQTT. To keep backwards compatibility, if `c8y.software_management.api` is set to "legacy", the mapper posts to c8y_SoftwareList fragment to the inventory API via HTTP. Signed-off-by: Rina Fujino <[email protected]>

Signed-off-by: Rina Fujino <[email protected]>

rina23q had a problem deploying to Test Pull Request March 8, 2024 19:33 — with GitHub Actions Failure

rina23q temporarily deployed to Test Pull Request March 11, 2024 19:26 — with GitHub Actions Inactive

rina23q had a problem deploying to Test Auto March 11, 2024 19:44 — with GitHub Actions Failure

rina23q had a problem deploying to Test Pull Request March 11, 2024 20:05 — with GitHub Actions Failure

rina23q force-pushed the improve/2654/c8y-mapper-supports-advanced-software-type branch from a764643 to 6ede9d3 Compare March 11, 2024 20:33

rina23q had a problem deploying to Test Pull Request March 11, 2024 20:33 — with GitHub Actions Failure

rina23q marked this pull request as ready for review March 11, 2024 20:34

rina23q requested review from didier-wenzek, albinsuresh and jarhodes314 as code owners March 11, 2024 20:34

rina23q force-pushed the improve/2654/c8y-mapper-supports-advanced-software-type branch from 6ede9d3 to 7e81021 Compare March 11, 2024 20:54

rina23q had a problem deploying to Test Pull Request March 11, 2024 20:54 — with GitHub Actions Failure

rina23q force-pushed the improve/2654/c8y-mapper-supports-advanced-software-type branch from 7e81021 to ee8f32f Compare March 12, 2024 23:08

rina23q temporarily deployed to Test Pull Request March 12, 2024 23:08 — with GitHub Actions Inactive

rina23q had a problem deploying to Test Auto March 12, 2024 23:17 — with GitHub Actions Failure

rina23q temporarily deployed to Test Pull Request March 13, 2024 13:42 — with GitHub Actions Inactive

rina23q had a problem deploying to Test Auto March 13, 2024 13:42 — with GitHub Actions Failure

rina23q requested a review from a team as a code owner March 13, 2024 14:29

rina23q had a problem deploying to Test Pull Request March 13, 2024 14:29 — with GitHub Actions Failure

rina23q force-pushed the improve/2654/c8y-mapper-supports-advanced-software-type branch from a70e8dc to 30741ee Compare March 13, 2024 14:35

rina23q had a problem deploying to Test Pull Request March 13, 2024 14:35 — with GitHub Actions Failure

rina23q force-pushed the improve/2654/c8y-mapper-supports-advanced-software-type branch from 30741ee to d2655fa Compare March 13, 2024 14:53

rina23q temporarily deployed to Test Pull Request March 13, 2024 14:53 — with GitHub Actions Inactive

rina23q had a problem deploying to Test Auto March 13, 2024 14:58 — with GitHub Actions Failure

reubenmiller mentioned this pull request Mar 13, 2024

Cumulocity software api configuration options #2778

Closed

rina23q force-pushed the improve/2654/c8y-mapper-supports-advanced-software-type branch from d2655fa to c96de86 Compare March 13, 2024 16:37

rina23q temporarily deployed to Test Pull Request March 13, 2024 16:37 — with GitHub Actions Inactive

rina23q temporarily deployed to Test Auto March 13, 2024 16:44 — with GitHub Actions Inactive

rina23q force-pushed the improve/2654/c8y-mapper-supports-advanced-software-type branch from c96de86 to d2655fa Compare March 13, 2024 16:52

rina23q had a problem deploying to Test Pull Request March 13, 2024 16:52 — with GitHub Actions Failure

rina23q had a problem deploying to Test Pull Request March 13, 2024 17:05 — with GitHub Actions Failure

rina23q force-pushed the improve/2654/c8y-mapper-supports-advanced-software-type branch from fa391cb to cb75ce0 Compare March 13, 2024 17:15

rina23q temporarily deployed to Test Pull Request March 13, 2024 17:15 — with GitHub Actions Inactive

rina23q temporarily deployed to Test Auto March 13, 2024 17:22 — with GitHub Actions Inactive

gligorisaev reviewed Mar 15, 2024

View reviewed changes

tests/RobotFramework/tests/cumulocity/software_management/software.robot Show resolved Hide resolved

gligorisaev reviewed Mar 15, 2024

View reviewed changes

tests/RobotFramework/tests/cumulocity/software_management/software.robot Show resolved Hide resolved

didier-wenzek reviewed Mar 19, 2024

View reviewed changes

rina23q had a problem deploying to Test Pull Request March 20, 2024 15:52 — with GitHub Actions Failure

rina23q force-pushed the improve/2654/c8y-mapper-supports-advanced-software-type branch from c594fc3 to be43806 Compare March 20, 2024 16:09

rina23q temporarily deployed to Test Pull Request March 20, 2024 16:09 — with GitHub Actions Inactive

rina23q temporarily deployed to Test Auto March 20, 2024 16:16 — with GitHub Actions Inactive

rina23q had a problem deploying to Test Pull Request March 20, 2024 18:46 — with GitHub Actions Failure

rina23q temporarily deployed to Test Pull Request March 21, 2024 09:55 — with GitHub Actions Inactive

rina23q temporarily deployed to Test Auto March 21, 2024 10:21 — with GitHub Actions Inactive

didier-wenzek approved these changes Mar 21, 2024

View reviewed changes

rina23q added 2 commits March 21, 2024 15:00

Add robot tests to support advanced software management

667652a

Signed-off-by: Rina Fujino <[email protected]>

rina23q force-pushed the improve/2654/c8y-mapper-supports-advanced-software-type branch from 5546a35 to 667652a Compare March 21, 2024 14:00

rina23q temporarily deployed to Test Pull Request March 21, 2024 14:00 — with GitHub Actions Inactive

rina23q temporarily deployed to Test Auto March 21, 2024 14:07 — with GitHub Actions Inactive

rina23q added this pull request to the merge queue Mar 21, 2024

Merged via the queue into thin-edge:main with commit 4109ef6 Mar 21, 2024
30 checks passed

rina23q mentioned this pull request Mar 25, 2024

Align software management feature to the Cumulocity advanced software management API #2654

Closed

rina23q deleted the improve/2654/c8y-mapper-supports-advanced-software-type branch July 18, 2024 14:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

c8y-mapper sends software list to advanced software management endpoint #2771

c8y-mapper sends software list to advanced software management endpoint #2771

rina23q commented Mar 8, 2024 •

edited

Loading

codecov bot commented Mar 8, 2024 •

edited

Loading

github-actions bot commented Mar 11, 2024 •

edited

Loading

didier-wenzek Mar 19, 2024

rina23q Mar 20, 2024

didier-wenzek Mar 21, 2024

didier-wenzek Mar 21, 2024

reubenmiller commented Mar 20, 2024 •

edited

Loading

didier-wenzek left a comment

c8y-mapper sends software list to advanced software management endpoint #2771

c8y-mapper sends software list to advanced software management endpoint #2771

Conversation

rina23q commented Mar 8, 2024 • edited Loading

Proposed changes

Types of changes

Paste Link to the issue

Checklist

Further comments

codecov bot commented Mar 8, 2024 • edited Loading

Codecov Report

github-actions bot commented Mar 11, 2024 • edited Loading

Robot Results

didier-wenzek Mar 19, 2024

Choose a reason for hiding this comment

rina23q Mar 20, 2024

Choose a reason for hiding this comment

didier-wenzek Mar 21, 2024

Choose a reason for hiding this comment

didier-wenzek Mar 21, 2024

Choose a reason for hiding this comment

reubenmiller commented Mar 20, 2024 • edited Loading

didier-wenzek left a comment

Choose a reason for hiding this comment

rina23q commented Mar 8, 2024 •

edited

Loading

codecov bot commented Mar 8, 2024 •

edited

Loading

github-actions bot commented Mar 11, 2024 •

edited

Loading

reubenmiller commented Mar 20, 2024 •

edited

Loading