Skip to content

Commit

Permalink
fixup! Update after review
Browse files Browse the repository at this point in the history 
Signed-off-by: Rina Fujino <[email protected]>
  • Loading branch information
@rina23q
rina23q committed Jul 17, 2023
1 parent f9710fc commit 3954bf2
Showing 1 changed file with 95 additions and 132 deletions.
227 changes: 95 additions & 132 deletions docs/src/references/tedge-log-management.md
View file
Original file line number Diff line number Diff line change
@@ -1,36 +1,39 @@
---
title: New Log File Management
title: New Log File Management Plugin
tags: [Reference, Log Files]
sidebar_position: 7
draft: true
---

# Log file management from cloud
# Log file management plugin

Thin-edge provides an operation plugin to fetch log files from the device on to cloud.
Thin-edge provides a command plugin to fetch log files from the device.

* Log file management from cloud is provided with a `tedge-log-plugin` which runs as a daemon on thin-edge together with cloud mapper.
* The device owner can define the list of log files that can be retrieved from cloud,
in the plugin's configuration file named `tedge-log-plugin.toml`.
* Log file management feature is provided with a `tedge-log-plugin` which runs as a daemon on thin-edge.
* The device owner can define the list of log files in the plugin's configuration file named `tedge-log-plugin.toml`.
* Each entry in the the `tedge-log-plugin.toml` file contains a log `type` and a `path` pattern,
where the `type` is used to represent the logical group of log files matching the `path` pattern.
* On receipt of a log file request for a given `type`,
* On receipt of a log file upload command for a given `type`,
the log files for that type are retrieved using the `path` pattern defined in this `tedge-log-plugin.toml`,
matched against the requested time range, search text and maximum line count.
* The list of managed log files in `tedge-log-plugin.toml` can be updated both locally as well as from cloud, for instance,
using the configuration management feature of Cumulocity, combined with the `tedge-configuration-plugin` of thin-edge.
* The plugin uploads the requested log file to the tedge file transfer repository,
which url is given by the received log upload command.
* The list of managed log files in `tedge-log-plugin.toml` can be updated both locally as well as from cloud, for instance, by using the configuration management feature.
* However, the plugin provides no direct connection to clouds, which is responsibility of another component, i.e. cloud mapper.
* The plugin has dependency on the `tedge.toml` device configuration file to get the MQTT hostname, port and the device identifier.

## Installation

As part of this plugin installation:
* On systemd enabled devices, the service definition file for this `tedge-log-plugin` daemon is also installed as part of this plugin installation.

Once installed, the `tedge-log-plugin` is run as a daemon on the device listening to log requests from cloud mapper on `te/device/main///cmd/log_upload/+` MQTT topic.
Once installed, the `tedge-log-plugin` is run as a daemon on the device listening to log update commands on `<root>/<identifier>/cmd/log_upload/+` MQTT topic.

## Configuration

The `tedge-log-plugin` configuration is stored by default under `/etc/tedge/plugins/tedge-log-plugin.toml`.
The `tedge-log-plugin` configuration is stored by default under `/etc/tedge/plugins/tedge-log-plugin.toml`. If it does not exist on startup, the plugin creates the file with example contents.

This [TOML](https://toml.io/en/) file defines the list of log files that can be retrieved from the cloud tenant.
This [TOML](https://toml.io/en/) file defines the list of log files that should be managed by the plugin.
The paths to these files can be represented using [glob](https://en.wikipedia.org/wiki/Glob_(programming)) patterns.
The `type` given to these paths are used as the log type associated to a log path.

Expand All @@ -43,12 +46,12 @@ files = [
```

The `tedge-log-plugin` parses this configuration file on startup for all the `type` values specified,
and sends the supported log types message to MQTT local broker on `te/device/main///cmd/log_upload` topic as follows:
and sends the supported log types message to MQTT local broker on `<root>/<identifier>/cmd/log_upload` topic with a retained flag as follows:

```json
{
```sh te2mqtt
tedge mqtt pub -r '<root>/<identifier>/cmd/log_upload' '{
"types" : [ "mosquitto", "software-management", "c8y_CustomOperation" ]
}
}'
```

The plugin continuously watches this configuration file for any changes and resends the JSON message with the `type`s in this file,
Expand All @@ -59,174 +62,134 @@ If the file `/etc/tedge/plugins/tedge-log-plugin.toml` is ill-formed or cannot b
then a JSON message with an empty array for the `types` field is sent, indicating no log files are tracked.
:::

## Startup
## Handling log upload commands

The plugin declares **supported operation** and **supported log types** on startup.
It's up to cloud mappers, if it converts the declaration messages to the corresponding cloud format or discard them.
The plugin establishes the MQTT connection to a broker for `mqtt.bind.address` and `mqtt.bind.port` from the `tedge.toml` device configuration.
Then, it subscribes to the `<root>/<identifier>/cmd/log_upload/+` topic, listening for
log file upload commands.
The `<root>/<identifier>` is defined in `tedge.toml` file as the `xxx` key.

### Supported operation
For instance, if the `<root>/<identifier>` is defined as `te/device/main//`, the plugin subscribes to the following topic.

On startup, the plugin declares to the Cumulocity mapper that it supports `c8y_LogfileRequest` operation
by creating an empty file at `/etc/tedge/operations/c8y/c8y_LogfileRequest`.
For other cloud platforms, it does nothing as of now.

### Supported log types
On startup, the plugin also reports all the log file types that it manages, defined in the `tedge-log-plugin.toml`, to the MQTT topic `te/device/main///cmd/log_upload` with JSON payload,
as defined in the Configuration section.

## Handling log requests from cloud mapper

After all initialization, this plugin subscribes to `te/device/main///cmd/log_upload/+` topic, listening for
log file upload request from any cloud mapper, like this one.

Topic (provided "1234" is operation ID):
```text
te/device/main///cmd/log_upload/1234
```sh te2mqtt
tedge mqtt sub 'te/device/main///cmd/log_upload/+'
```

Payload:
```json
{
A new log file upload command with the ID "1234" is published by another component as below.

```sh te2mqtt
tedge mqtt pub -r 'te/device/main///cmd/log_upload/1234' '{
"status": "init",
"uploadUrl": "http://127.0.0.1:8000/tedge/file-transfer/main/log_upload/mosquitto",
"uploadUrl": "http://127.0.0.1:8000/tedge/file-transfer/main/log_upload/mosquitto-1234",
"logType": "mosquitto",
"dateFrom": "2013-06-22T17:03:14.000+02:00",
"dateTo": "2013-06-22T18:03:14.000+02:00",
"dateTo": "2013-06-23T18:03:14.000+02:00",
"searchText": "ERROR",
"maximumLines": 1000
}
}'
```

The plugin then checks the `tedge-log-plugin.toml` file for the log type in the incoming message (`mosquitto`),
retrieves the log files using the `target` glob pattern provided in the plugin config file,
including only the ones modified within the date range(`2013-06-22T17:03:14.000+02:00` to `2013-06-22T18:03:14.000+02:00`),
including only the ones modified within the date range(`2013-06-22T17:03:14.000+02:00` to `2013-06-23T18:03:14.000+02:00`),
with the content filtered by the search text(`ERROR`) and the maximum line count(`1000`).

This filtered content is then uploaded to the given url as `uploadUrl` via HTTP PUT request.

During this process, the plugin need to update the operation status via MQTT for `executing `, `uploading` and `failed`
by publishing a message to the topic `te/device/main///cmd/log_upload/<op_id>`.
During the process, the plugin updates the command status via MQTT
by publishing a retained message to the topic `<root>/<identifier>/cmd/log_upload/<id>`,
which is the same topic as the one where the command is received.

So, for example, the operation status update message should look like below. The topic is `te/device/main///cmd/log_upload/1234`.
On a reception of a new log file upload command, the plugin updates the status to `executing `.
On a reception of the HTTP success status code after uploading a file to the file transfer repository, the plugin updates the status to `successful`.
If any unexpected error occurs, the plugin updates the status to `failed` with `reason`.

```json
// Why do we need to repeat other fileds than status and reason?
{
So, the operation status update message for the above example looks like below.

```sh te2mqtt
tedge mqtt pub -r 'te/device/main///cmd/log_upload/1234' '{
"status": "failed",
"reason": "The target log file for 'mosquitto' does not exist."
"reason": "The target log file for 'mosquitto' does not exist.",
"uploadUrl": "http://127.0.0.1:8000/tedge/file-transfer/main/log_upload/mosquitto",
"logType": "mosquitto",
"dateFrom": "2013-06-22T17:03:14.000+02:00",
"dateTo": "2013-06-22T18:03:14.000+02:00",
"searchText": "ERROR",
"maximumLines": 1000,
"reason": "The type 'typeA' is not configured."
}
"maximumLines": 1000
}'
```

### Successful flow

The status transition: `init` -> `executing` -> `uploading` -> `successful`
### Flow

```mermaid
sequenceDiagram
participant Cloud
participant Mapper
participant Mapper/others
participant Plugin
Cloud->>Mapper: Cloud-specific Log file upload request (Status: init)
Mapper->>Plugin: Cloud-agnostic Log file upload request (Status: init)
Plugin->>Mapper: Status: executing
alt if necessary
Mapper->>Cloud: Status: executing
end
Plugin->>Plugin: Extract log
Plugin->>Mapper: File transfer [HTTP]
Plugin->>Mapper: Status: uploading
alt if necessary
Mapper->>Cloud: Status: uploading
end
Mapper->>Cloud: Upload log file [HTTP]
Cloud-->> Mapper: Return the http endpoint of the uploaded file [HTTP]
alt if necessary
Mapper->>Cloud: Status: successful
Mapper->>Cloud: Cloud-specific something if any
Mapper/others->>Plugin: Cloud-agnostic Log file upload command (Status: init)
Plugin->>Mapper/others: Status: executing
alt No error
Plugin->>Plugin: Extract log
Plugin->>Mapper/others: File transfer [HTTP]
Mapper/others-->>Plugin: Success status code [HTTP]
Plugin->>Mapper/others: Status: successful
else Any error occurs
Plugin->>Mapper/others: Status: failed
end
```

### Failed flows
## Usage

a. Failure on the plugin side
```sh
tedge-log-plugin --help
```

The status transition: `init` -> `executing` -> `failed`
```run command="tedge-log-plugin --help" lang="text" title="Output"
Thin-edge device log file retriever
```mermaid
sequenceDiagram
participant Cloud
participant Mapper
participant Plugin
USAGE:
tedge-log-plugin [OPTIONS]
Cloud->>Mapper: Cloud-specific Log file upload request (Status: init)
Mapper->>Plugin: Cloud-agnostic Log file upload request (Status: init)
Plugin->>Mapper: Status: executing
alt if necessary
Mapper->>Cloud: Status: executing
end
Plugin->>Plugin: Extract log (FAILED!)
Plugin->>Mapper: Status: failed with reason
alt if necessary
Mapper->>Cloud: Status: failed with reason
end
```

b. Failure on uploading log (mapper side)
OPTIONS:
--config-dir <CONFIG_DIR>
[default: /etc/tedge/plugins]
The status transition: `init` -> `executing` -> `uploading` -> `failed`
--debug
Turn-on the debug log level.
```mermaid
sequenceDiagram
participant Cloud
participant Mapper
participant Plugin
If off only reports ERROR, WARN, and INFO If on also reports DEBUG and TRACE
Cloud->>Mapper: Cloud-specific Log file upload request (Status: init)
Mapper->>Plugin: Cloud-agnostic Log file upload request (Status: init)
Plugin->>Mapper: Status: executing
alt if necessary
Mapper->>Cloud: Status: executing
end
Plugin->>Plugin: Extract log
Plugin->>Mapper: File transfer [HTTP]
Plugin->>Mapper: Status: uploading
alt if necessary
Mapper->>Cloud: Status: uploading
end
Mapper->>Cloud: Upload log file [HTTP] (FAILED!)
Cloud-->> Mapper: Return the http endpoint of the uploaded file [HTTP] (FAILED CODE!)
alt if necessary
Mapper->>Cloud: Cloud-specific Status failed
end
```
-h, --help
Print help information
## Usage
-V, --version
Print version information
TBD
The thin-edge `CONFIG_DIR` is used to store:
* tedge-log-plugin.toml - the configuration file that specifies which logs to be retrieved
```

## Logging

The `tedge-log-plugin` reports progress and errors to the OS journal which can be retrieved using `journalctl`.

## Check points for developer (this section will be deleted later)

## Not clear / undecided

* Stop creating any supported operation file from the plugin. Mapper will address it on the reception of the log type declaration message.
* Renaming the toml file from `c8y-log-plugin.toml` to `tedge-log-plugin.toml`.
* Supported operation. Create automatically `c8y_LogfileRequest` as it is now? Or ask user to create it?
* The location of the toml file. Change from `/etc/tedge/c8y/c8y-log-plugin.toml` to `/etc/tedge/plugins/tedge-log-plugin.toml`?
* The location of the toml file. Change from `/etc/tedge/c8y/c8y-log-plugin.toml` to `/etc/tedge/plugins/tedge-log-plugin.toml`.
* Create a `tedge-log-plugin.toml` file with example contents if it does not exist on startup.
* All MQTT messages must be published with a retain flag.
* All JSON deserializer must accept unknown fields.

### Drop 1 only

* The `<root>/<identifier>` is hard-coded to `te/device/main//`, thus, supporting only main device. So,
* The topic for log type declaration: hard-code to `te/device/main///cmd/log_upload`.
* The topic for receiving/updating operation: hard-code to `te/device/main///cmd/log_upload/+`. (`+` is for a command id)
* Use `mqtt.bind.address` and `mqtt.bind.port` from tedge config API for MQTT connection.

## Check points (MQTT topic names and payloads)
### Drop 2

* Supported log type declaration format. CSV vs JSON? I prefer JSON as it's easy to extend fields in the future.
* Type declaration topic: `te/device/main///cmd/log_upload`?
* For receiving operation from mapper: `te/device/main///cmd/log_upload`?
* Do we need to include other fields in payload for status update message?
* Get `<root/identifier>` from `xxx` of tedge config API. (The key `xxx` must be specified later.)

0 comments on commit 3954bf2

Please sign in to comment.