tedge-log-plugin specification

[rina23q]

Proposed changes

This is a PR for tedge-log-plugin specification.

Open topics

• States. Need to sync with the state machine spec later.

Types of changes

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
• Documentation Update (if none of the other choices apply)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

#2048

Checklist

• I have read the CONTRIBUTING doc
• I have signed the CLA (in all commits with git commit -s)
• I ran cargo fmt as mentioned in CODING_GUIDELINES
• I used cargo clippy as mentioned in CODING_GUIDELINES
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

Further comments

[didier-wenzek]

I really like this proposal, notably the fact the very same plugin can run either on the main device or a child device.

However, the text is not always easy to read.

[github-actions]

Robot Results

✅ Passed	❌ Failed	⏭️ Skipped	Total	Pass %
239	0	5	239	100

Passed Tests

Name	⏱️ Duration	Suite
Define Child device 1 ID	0.016 s	C8Y Child Alarms Rpi
Normal case when the child device does not exist on c8y cloud	2.45 s	C8Y Child Alarms Rpi
Normal case when the child device already exists	0.874 s	C8Y Child Alarms Rpi
Reconciliation when the new alarm message arrives, restart the mapper	1.091 s	C8Y Child Alarms Rpi
Reconciliation when the alarm that is cleared	65.447 s	C8Y Child Alarms Rpi
Prerequisite Parent	13.788 s	Child Conf Mgmt Plugin
Prerequisite Child	0.213 s	Child Conf Mgmt Plugin
Child device bootstrapping	12.045 s	Child Conf Mgmt Plugin
Snapshot from device	62.432 s	Child Conf Mgmt Plugin
Child device config update	62.193 s	Child Conf Mgmt Plugin
Configuration types should be detected on file change (without restarting service)	47.068 s	Inotify Crate
Check lock file existence in default folder	1.595 s	Lock File
Check PID number in lock file	1.572 s	Lock File
Check PID number in lock file after restarting the services	2.158 s	Lock File
Check starting same service twice	1.051 s	Lock File
Switch off lock file creation	2.011 s	Lock File
Set configuration when file exists	14.644 s	Configuration Operation
Set configuration when file does not exist	5.06 s	Configuration Operation
Set configuration with broken url	4.402 s	Configuration Operation
Get configuration	4.859 s	Configuration Operation
Get non existent configuration file	4.514 s	Configuration Operation
Get non existent configuration type	4.433 s	Configuration Operation
Update configuration plugin config via cloud	5.48 s	Configuration Operation
Modify configuration plugin config via local filesystem modify inplace	4.596 s	Configuration Operation
Modify configuration plugin config via local filesystem overwrite	4.873 s	Configuration Operation
Update configuration plugin config via local filesystem copy	2.789 s	Configuration Operation
Update configuration plugin config via local filesystem move (different directory)	2.455 s	Configuration Operation
Update configuration plugin config via local filesystem move (same directory)	2.36 s	Configuration Operation
Update the custom operation dynamically	52.008 s	Dynamically Reload Operation
Custom operation successful	120.828 s	Custom Operation
Custom operation fails	66.723 s	Custom Operation
Successful firmware operation	120.134 s	Firmware Operation
Install with empty firmware name	110.345 s	Firmware Operation
Prerequisite Parent	15.083 s	Firmware Operation Child Device
Prerequisite Child	7.905 s	Firmware Operation Child Device
Child device firmware update	6.166 s	Firmware Operation Child Device
Child device firmware update with cache	4.026 s	Firmware Operation Child Device
Firmware plugin supports restart via service manager #1932	4.497 s	Firmware Operation Child Device Retry
Update Inventory data via inventory.json	1.431 s	Inventory Update
Inventory includes the agent fragment with version information	0.996 s	Inventory Update
Retrieve a JWT tokens	29.557 s	Jwt Request
Mapper recovers and processes output of ongoing software update request	15.201 s	Recover And Publish Software Update Message
Check running collectd	1.319 s	Monitor Device Collectd
Is collectd publishing MQTT messages?	2.922 s	Monitor Device Collectd
Check thin-edge monitoring	3.817 s	Monitor Device Collectd
Check grouping of measurements	8.361 s	Monitor Device Collectd
Main device registration	2.073 s	Device Registration
Child device registration	2.251 s	Device Registration
Supports restarting the device	103.57 s	Restart Device
Update tedge version from previous using Cumulocity	62.937 s	Tedge Self Update
Test if all c8y services are up	101.313 s	Service Monitoring
Test if all c8y services are down	108.512 s	Service Monitoring
Test if all c8y services are using configured service type	164.702 s	Service Monitoring
Test if all c8y services using default service type when service type configured as empty	91.983 s	Service Monitoring
Check health status of tedge-mapper-c8y service on broker stop start	20.455 s	Service Monitoring
Check health status of tedge-mapper-c8y service on broker restart	23.481 s	Service Monitoring
Check health status of child device service	19.721 s	Service Monitoring
Successful shell command with output	4.132 s	Shell Operation
Check Successful shell command with literal double quotes output	3.456 s	Shell Operation
Execute multiline shell command	3.516 s	Shell Operation
Failed shell command	3.285 s	Shell Operation
Software list should be populated during startup	37.185 s	Software
Install software via Cumulocity	31.873 s	Software
tedge-agent should terminate on SIGINT while downloading file	32.209 s	Software
Software list should only show currently installed software and not candidates	45.204 s	Software
Create and publish the tedge agent supported operations on mapper restart	38.638 s	Mapper-Publishing-Agent-Supported-Ops
Agent gets the software list request once it comes up	22.328 s	Mapper-Publishing-Agent-Supported-Ops
Child devices support sending simple measurements	1.5819999999999999 s	Child Device Telemetry
Child devices support sending custom measurements	1.11 s	Child Device Telemetry
Child devices support sending custom events	1.163 s	Child Device Telemetry
Child devices support sending custom events overriding the type	1.186 s	Child Device Telemetry
Child devices support sending custom alarms #1699	3.349 s	Child Device Telemetry
Child devices support sending inventory data via c8y topic	1 s	Child Device Telemetry
Child device supports sending custom child device measurements directly to c8y	1.464 s	Child Device Telemetry
Check retained alarms	208.575 s	Raise Alarms
Thin-edge devices support sending simple measurements	1.6840000000000002 s	Thin-Edge Device Telemetry
Thin-edge devices support sending simple measurements with custom type	1.337 s	Thin-Edge Device Telemetry
Thin-edge devices support sending custom measurements	1.186 s	Thin-Edge Device Telemetry
Thin-edge devices support sending custom events	1.142 s	Thin-Edge Device Telemetry
Thin-edge devices support sending custom events overriding the type	1.08 s	Thin-Edge Device Telemetry
Thin-edge devices support sending custom alarms #1699	1.003 s	Thin-Edge Device Telemetry
Thin-edge device supports sending custom Thin-edge device measurements directly to c8y	1.525 s	Thin-Edge Device Telemetry
Thin-edge device support sending inventory data via c8y topic	1.434 s	Thin-Edge Device Telemetry
thin-edge components support a custom config-dir location via flags	26.432 s	Config Dir
Validate updated data path used by tedge-agent	0.481 s	Data Path Config
Validate updated data path used by c8y-firmware-plugin	10.564 s	Data Path Config
Validate updated data path used by tedge-agent	0.463 s	Log Path Config
Check existence of init directories	1.342 s	Tedge Init
Tedge init and check creation of folders	1.6219999999999999 s	Tedge Init
Check ownership of the folders	1.173 s	Tedge Init
Change user/group and check the change	1.246 s	Tedge Init
Tedge init and check if default values are restored	1.9409999999999998 s	Tedge Init
Install thin-edge via apt	46.11 s	Install Apt
Install latest via script (from current branch)	27.667 s	Install Tedge
Install specific version via script (from current branch)	23.93 s	Install Tedge
Install latest tedge via script (from main branch)	22.188 s	Install Tedge
Install then uninstall latest tedge via script (from main branch)	41.28 s	Install Tedge
Support starting and stopping services	39.557 s	Service-Control
Supports a reconnect	53.2 s	Test-Commands
Supports disconnect then connect	33.951 s	Test-Commands
Update unknown setting	32.999 s	Test-Commands
Update known setting	32.911 s	Test-Commands
It checks MQTT messages using a pattern	63.377 s	Test-Mqtt
Stop c8y-configuration-plugin	0.198 s	Health C8Y-Configuration-Plugin
Update the service file	0.159 s	Health C8Y-Configuration-Plugin
Reload systemd files	0.709 s	Health C8Y-Configuration-Plugin
Start c8y-configuration-plugin	0.187 s	Health C8Y-Configuration-Plugin
Start watchdog service	10.146 s	Health C8Y-Configuration-Plugin
Check PID of c8y-configuration-plugin	0.079 s	Health C8Y-Configuration-Plugin
Kill the PID	0.147 s	Health C8Y-Configuration-Plugin
Recheck PID of c8y-configuration-plugin	6.284 s	Health C8Y-Configuration-Plugin
Compare PID change	0.001 s	Health C8Y-Configuration-Plugin
Stop watchdog service	0.12 s	Health C8Y-Configuration-Plugin
Remove entry from service file	0.074 s	Health C8Y-Configuration-Plugin
Stop c8y-log-plugin	0.115 s	Health C8Y-Log-Plugin
Update the service file	0.086 s	Health C8Y-Log-Plugin
Reload systemd files	0.237 s	Health C8Y-Log-Plugin
Start c8y-log-plugin	0.076 s	Health C8Y-Log-Plugin
Start watchdog service	10.178 s	Health C8Y-Log-Plugin
Check PID of c8y-log-plugin	0.08 s	Health C8Y-Log-Plugin
Kill the PID	0.149 s	Health C8Y-Log-Plugin
Recheck PID of c8y-log-plugin	6.34 s	Health C8Y-Log-Plugin
Compare PID change	0.001 s	Health C8Y-Log-Plugin
Stop watchdog service	0.138 s	Health C8Y-Log-Plugin
Remove entry from service file	0.081 s	Health C8Y-Log-Plugin
Stop tedge-mapper	60.085 s	Health Tedge Mapper C8Y
Update the service file	0.107 s	Health Tedge Mapper C8Y
Reload systemd files	0.322 s	Health Tedge Mapper C8Y
Start tedge-mapper	0.112 s	Health Tedge Mapper C8Y
Start watchdog service	10.164 s	Health Tedge Mapper C8Y
Check PID of tedge-mapper	0.137 s	Health Tedge Mapper C8Y
Kill the PID	0.315 s	Health Tedge Mapper C8Y
Recheck PID of tedge-mapper	6.642 s	Health Tedge Mapper C8Y
Compare PID change	0.001 s	Health Tedge Mapper C8Y
Stop watchdog service	0.472 s	Health Tedge Mapper C8Y
Remove entry from service file	0.267 s	Health Tedge Mapper C8Y
Stop tedge-agent	0.139 s	Health Tedge-Agent
Update the service file	0.124 s	Health Tedge-Agent
Reload systemd files	0.638 s	Health Tedge-Agent
Start tedge-agent	0.172 s	Health Tedge-Agent
Start watchdog service	10.258 s	Health Tedge-Agent
Check PID of tedge-mapper	0.082 s	Health Tedge-Agent
Kill the PID	0.156 s	Health Tedge-Agent
Recheck PID of tedge-agent	6.288 s	Health Tedge-Agent
Compare PID change	0.001 s	Health Tedge-Agent
Stop watchdog service	0.088 s	Health Tedge-Agent
Remove entry from service file	0.076 s	Health Tedge-Agent
Stop tedge-mapper-az	0.084 s	Health Tedge-Mapper-Az
Update the service file	0.15 s	Health Tedge-Mapper-Az
Reload systemd files	0.41 s	Health Tedge-Mapper-Az
Start tedge-mapper-az	0.158 s	Health Tedge-Mapper-Az
Start watchdog service	10.172 s	Health Tedge-Mapper-Az
Check PID of tedge-mapper-az	0.081 s	Health Tedge-Mapper-Az
Kill the PID	0.143 s	Health Tedge-Mapper-Az
Recheck PID of tedge-agent	6.5440000000000005 s	Health Tedge-Mapper-Az
Compare PID change	0.001 s	Health Tedge-Mapper-Az
Stop watchdog service	0.217 s	Health Tedge-Mapper-Az
Remove entry from service file	0.205 s	Health Tedge-Mapper-Az
Stop tedge-mapper-collectd	0.405 s	Health Tedge-Mapper-Collectd
Update the service file	0.266 s	Health Tedge-Mapper-Collectd
Reload systemd files	0.621 s	Health Tedge-Mapper-Collectd
Start tedge-mapper-collectd	0.141 s	Health Tedge-Mapper-Collectd
Start watchdog service	10.247 s	Health Tedge-Mapper-Collectd
Check PID of tedge-mapper-collectd	0.118 s	Health Tedge-Mapper-Collectd
Kill the PID	0.265 s	Health Tedge-Mapper-Collectd
Recheck PID of tedge-mapper-collectd	6.454 s	Health Tedge-Mapper-Collectd
Compare PID change	0.001 s	Health Tedge-Mapper-Collectd
Stop watchdog service	0.17 s	Health Tedge-Mapper-Collectd
Remove entry from service file	0.108 s	Health Tedge-Mapper-Collectd
tedge-collectd-mapper health status	5.4030000000000005 s	Health Tedge-Mapper-Collectd
c8y-log-plugin health status	5.807 s	MQTT health endpoints
c8y-configuration-plugin health status	5.7 s	MQTT health endpoints
Publish on a local insecure broker	0.157 s	Basic Pub Sub
Publish on a local secure broker	1.855 s	Basic Pub Sub
Publish on a local secure broker with client authentication	2.336 s	Basic Pub Sub
Publish events to subscribed topic	0.279 s	Custom Sub Topics Tedge-Mapper-Aws
Publish measurements to unsubscribed topic	5.362 s	Custom Sub Topics Tedge-Mapper-Aws
Publish measurements to subscribed topic	0.26 s	Custom Sub Topics Tedge-Mapper-Az
Publish measurements to unsubscribed topic	5.284 s	Custom Sub Topics Tedge-Mapper-Az
Publish events to subscribed topic	0.254 s	Custom Sub Topics Tedge-Mapper-C8Y
Publish measurements to unsubscribed topic	5.31 s	Custom Sub Topics Tedge-Mapper-C8Y
Check remote mqtt broker #1773	3.612 s	Remote Mqtt Broker
Apply name filter	0.256 s	Filter Packages List Output
Apply maintainer filter	0.244 s	Filter Packages List Output
Apply both filters	0.197 s	Filter Packages List Output
No filters	0.218 s	Filter Packages List Output
Both filters but name filter as empty string	0.24 s	Filter Packages List Output
Both filters but maintainer filter as empty string	0.193 s	Filter Packages List Output
Both filters as empty string	0.318 s	Filter Packages List Output
Wrong package name	0.143 s	Improve Tedge Apt Plugin Error Messages
Wrong version	0.172 s	Improve Tedge Apt Plugin Error Messages
Wrong type	0.658 s	Improve Tedge Apt Plugin Error Messages
tedge_connect_test_positive	0.181 s	Tedge Connect Test
tedge_connect_test_negative	0.536 s	Tedge Connect Test
tedge_connect_test_sm_services	7.312 s	Tedge Connect Test
tedge_disconnect_test_sm_services	60.397 s	Tedge Connect Test
Install thin-edge.io	20.212 s	Call Tedge
call tedge -V	0.241 s	Call Tedge
call tedge -h	0.09 s	Call Tedge
call tedge -h -V	0.126 s	Call Tedge
call tedge help	0.088 s	Call Tedge
tedge config list	0.074 s	Call Tedge Config List
tedge config list --all	0.073 s	Call Tedge Config List
set/unset device.type	0.312 s	Call Tedge Config List
set/unset device.key_path	0.299 s	Call Tedge Config List
set/unset device.cert_path	0.265 s	Call Tedge Config List
set/unset c8y.root_cert_path	0.307 s	Call Tedge Config List
set/unset c8y.smartrest.templates	0.28 s	Call Tedge Config List
set/unset c8y.topics	0.275 s	Call Tedge Config List
set/unset az.root_cert_path	0.293 s	Call Tedge Config List
set/unset az.topics	0.304 s	Call Tedge Config List
set/unset aws.topics	0.289 s	Call Tedge Config List
set/unset aws.url	0.304 s	Call Tedge Config List
set/unset aws.root_cert_path	0.401 s	Call Tedge Config List
set/unset aws.mapper.timestamp	0.329 s	Call Tedge Config List
set/unset az.mapper.timestamp	0.216 s	Call Tedge Config List
set/unset mqtt.bind.address	0.276 s	Call Tedge Config List
set/unset mqtt.bind.port	0.298 s	Call Tedge Config List
set/unset http.bind.port	0.343 s	Call Tedge Config List
set/unset tmp.path	0.216 s	Call Tedge Config List
set/unset logs.path	0.281 s	Call Tedge Config List
set/unset run.path	0.282 s	Call Tedge Config List
set/unset firmware.child.update.timeout	0.369 s	Call Tedge Config List
set/unset c8y.url	0.376 s	Call Tedge Config List
set/unset az.url	0.424 s	Call Tedge Config List
set/unset mqtt.external.bind.port	0.382 s	Call Tedge Config List
mqtt.external.bind.address	0.235 s	Call Tedge Config List
mqtt.external.bind.interface	0.232 s	Call Tedge Config List
set/unset mqtt.external.ca_path	0.21 s	Call Tedge Config List
set/unset mqtt.external.cert_file	0.264 s	Call Tedge Config List
set/unset mqtt.external.key_file	0.303 s	Call Tedge Config List
set/unset software.plugin.default	0.221 s	Call Tedge Config List
Get Put Delete	1.159 s	Http File Transfer Api
Set keys should return value on stdout	0.127 s	Tedge Config Get
Unset keys should not return anything on stdout and warnings on stderr	0.178 s	Tedge Config Get
Invalid keys should not return anything on stdout and warnings on stderr	0.207 s	Tedge Config Get
Set configuration via environment variables	0.617 s	Tedge Config Get
Set configuration via environment variables for topics	0.243 s	Tedge Config Get
Set unknown configuration via environment variables	0.086 s	Tedge Config Get