From 47f7738122eca4e0ec2d158c39de6ea4dc227822 Mon Sep 17 00:00:00 2001 From: Rina Fujino <18257209+rina23q@users.noreply.github.com> Date: Tue, 18 Jul 2023 18:36:39 +0000 Subject: [PATCH] Update tedge-log-management plugin spec Signed-off-by: Rina Fujino <18257209+rina23q@users.noreply.github.com> --- docs/src/references/tedge-log-management.md | 79 +++++++++------------ 1 file changed, 32 insertions(+), 47 deletions(-) diff --git a/docs/src/references/tedge-log-management.md b/docs/src/references/tedge-log-management.md index e9f8f56a651..1ec25d55c2f 100644 --- a/docs/src/references/tedge-log-management.md +++ b/docs/src/references/tedge-log-management.md @@ -7,27 +7,27 @@ draft: true # Log file management plugin -Thin-edge provides a command plugin to fetch log files from the device. +Thin-edge provides an operation plugin to fetch log files from the device. -* Log file management feature is provided with a `tedge-log-plugin` which runs as a daemon on thin-edge. +* The log file management feature is provided with a `tedge-log-plugin` which runs as a daemon on thin-edge. * The device owner can define the list of log files in the plugin's configuration file named `tedge-log-plugin.toml`. -* Each entry in the the `tedge-log-plugin.toml` file contains a log `type` and a `path` pattern, +* Each entry in the `tedge-log-plugin.toml` file contains a log `type` and a `path` pattern, where the `type` is used to represent the logical group of log files matching the `path` pattern. -* On receipt of a log file upload command for a given `type`, +* Upon receiving a log file upload command for a given `type`, the log files for that type are retrieved using the `path` pattern defined in this `tedge-log-plugin.toml`, matched against the requested time range, search text and maximum line count. -* The plugin uploads the requested log file to the tedge file transfer repository, - which url is given by the received log upload command. -* The list of managed log files in `tedge-log-plugin.toml` can be updated both locally as well as from cloud, for instance, by using the configuration management feature. -* However, the plugin provides no direct connection to clouds, which is responsibility of another component, i.e. cloud mapper. -* The plugin has dependency on the `tedge.toml` device configuration file to get the MQTT hostname, port and the device identifier. +* The plugin uploads the requested log file to the tedge file transfer repository. + Its url is given by the received log upload command. +* The list of managed log files in `tedge-log-plugin.toml` can be updated both locally as well as from clouds, for instance, by using the configuration management feature. +* However, the plugin provides no direct connection to clouds, which is the responsibility of another component, i.e. the cloud mapper. +* The plugin has a dependency on the `tedge.toml` configuration file to get the MQTT hostname, port, and device identifier. ## Installation As part of this plugin installation: -* On systemd enabled devices, the service definition file for this `tedge-log-plugin` daemon is also installed as part of this plugin installation. +* On systemd-enabled devices, the service definition file for this `tedge-log-plugin` daemon is also installed. -Once installed, the `tedge-log-plugin` is run as a daemon on the device listening to log update commands on `//cmd/log_upload/+` MQTT topic. +Once installed, the `tedge-log-plugin` runs as a daemon on the device, listening to log update commands on the [`//cmd/log_upload/+` MQTT topic](mqtt-topic-design.md). ## Configuration @@ -46,10 +46,13 @@ files = [ ``` The `tedge-log-plugin` parses this configuration file on startup for all the `type` values specified, -and sends the supported log types message to MQTT local broker on `//cmd/log_upload` topic with a retained flag as follows: +and sends the supported log types message to the MQTT local broker on the `//cmd/log_upload` topic with a retained flag. +The `/` is defined in `tedge.toml` file as `device.topic`. + +Given that `device.topic` is set to `te/device/main//`, the message to declare the supported log types is as follows. ```sh te2mqtt -tedge mqtt pub -r '//cmd/log_upload' '{ +tedge mqtt pub -r 'te/device/main///cmd/log_upload' '{ "types" : [ "mosquitto", "software-management", "c8y_CustomOperation" ] }' ``` @@ -64,12 +67,11 @@ then a JSON message with an empty array for the `types` field is sent, indicatin ## Handling log upload commands -The plugin establishes the MQTT connection to a broker for `mqtt.bind.address` and `mqtt.bind.port` from the `tedge.toml` device configuration. -Then, it subscribes to the `//cmd/log_upload/+` topic, listening for -log file upload commands. -The `/` is defined in `tedge.toml` file as the `xxx` key. +The plugin establishes an MQTT connection to the broker using the `mqtt.bind.address` and `mqtt.bind.port` values from the `tedge.toml` configuration. +Then, it subscribes to the `//cmd/log_upload/+` topic, listening for log file upload commands. +The `/` is defined in `tedge.toml` file as the `device.topic` key. -For instance, if the `/` is defined as `te/device/main//`, the plugin subscribes to the following topic. +For instance, if `device.topic` is set to `te/device/main//`, the plugin subscribes to the following topic. ```sh te2mqtt tedge mqtt sub 'te/device/main///cmd/log_upload/+' @@ -94,23 +96,23 @@ retrieves the log files using the `target` glob pattern provided in the plugin c including only the ones modified within the date range(`2013-06-22T17:03:14.000+02:00` to `2013-06-23T18:03:14.000+02:00`), with the content filtered by the search text(`ERROR`) and the maximum line count(`1000`). -This filtered content is then uploaded to the given url as `uploadUrl` via HTTP PUT request. +This filtered content is then uploaded to the given URL as `uploadUrl` via an HTTP PUT request. During the process, the plugin updates the command status via MQTT by publishing a retained message to the topic `//cmd/log_upload/`, -which is the same topic as the one where the command is received. +which is the same topic where the command is received. -On a reception of a new log file upload command, the plugin updates the status to `executing `. -On a reception of the HTTP success status code after uploading a file to the file transfer repository, the plugin updates the status to `successful`. -If any unexpected error occurs, the plugin updates the status to `failed` with `reason`. +On the reception of a new log file upload command, the plugin updates the status to `executing `. +On the reception of the HTTP success status code after uploading a file to the file transfer repository, the plugin updates the status to `successful`. +If any unexpected error occurs, the plugin updates the status to `failed` with a `reason`. -So, the operation status update message for the above example looks like below. +Thus, the operation status update message for the above example looks like below. ```sh te2mqtt tedge mqtt pub -r 'te/device/main///cmd/log_upload/1234' '{ "status": "failed", "reason": "The target log file for 'mosquitto' does not exist.", - "uploadUrl": "http://127.0.0.1:8000/tedge/file-transfer/main/log_upload/mosquitto", + "uploadUrl": "http://127.0.0.1:8000/tedge/file-transfer/main/log_upload/mosquitto-1234", "logType": "mosquitto", "dateFrom": "2013-06-22T17:03:14.000+02:00", "dateTo": "2013-06-22T18:03:14.000+02:00", @@ -152,7 +154,7 @@ USAGE: OPTIONS: --config-dir - [default: /etc/tedge/plugins] + [default: /etc/tedge] --debug Turn-on the debug log level. @@ -165,31 +167,14 @@ OPTIONS: -V, --version Print version information -The thin-edge `CONFIG_DIR` is used to store: - * tedge-log-plugin.toml - the configuration file that specifies which logs to be retrieved +The thin-edge `CONFIG_DIR` is used: + * to find where to connect the MQTT bus: `tedge config get mqtt.bind.address` and `tedge config get mqtt.bind.port` + * to find where to publish/subscribe: `tedge config get device.topic` + * to store tedge-log-plugin.toml: the configuration file that specifies which logs to be retrieved ``` ## Logging The `tedge-log-plugin` reports progress and errors to the OS journal which can be retrieved using `journalctl`. -## Check points for developer (this section will be deleted later) - -* Stop creating any supported operation file from the plugin. Mapper will address it on the reception of the log type declaration message. -* Renaming the toml file from `c8y-log-plugin.toml` to `tedge-log-plugin.toml`. -* The location of the toml file. Change from `/etc/tedge/c8y/c8y-log-plugin.toml` to `/etc/tedge/plugins/tedge-log-plugin.toml`. -* Create a `tedge-log-plugin.toml` file with example contents if it does not exist on startup. -* All MQTT messages must be published with a retain flag. -* All JSON deserializer must accept unknown fields. - -### Drop 1 only - -* The `/` is hard-coded to `te/device/main//`, thus, supporting only main device. So, - * The topic for log type declaration: hard-code to `te/device/main///cmd/log_upload`. - * The topic for receiving/updating operation: hard-code to `te/device/main///cmd/log_upload/+`. (`+` is for a command id) -* Use `mqtt.bind.address` and `mqtt.bind.port` from tedge config API for MQTT connection. - -### Drop 2 - -* Get `` from `xxx` of tedge config API. (The key `xxx` must be specified later.)