-
Notifications
You must be signed in to change notification settings - Fork 87
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Init hash plugin * Add docs&tests * Optimize regexps * Fix doc * Fix after merge * Fix doc * Add LM normalizer * Remove re normalizer * Fix url regexp * Rename hash_field to result_fields * gen-doc * Add max_size * Change default max_size * Fix doc
- Loading branch information
Showing
16 changed files
with
927 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| # Hash plugin | ||
| @introduction | ||
|
|
||
| ## Examples | ||
| @examples | ||
|
|
||
| ## Config params | ||
| @config-params|description |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| # Hash plugin | ||
| It calculates the hash for one of the specified event fields and adds a new field with result in the event root. | ||
| > Fields can be of any type except for an object and an array. | ||
| ## Examples | ||
| Hashing without normalization (first found field is `error.code`): | ||
| ```yaml | ||
| pipelines: | ||
| example_pipeline: | ||
| ... | ||
| actions: | ||
| - type: hash | ||
| fields: | ||
| - field: error.code | ||
| - field: level | ||
| result_field: hash | ||
| ... | ||
| ``` | ||
| The original event: | ||
| ```json | ||
| { | ||
| "level": "error", | ||
| "error": { | ||
| "code": "unauthenticated", | ||
| "message": "bad token format" | ||
| } | ||
| } | ||
| ``` | ||
| The resulting event: | ||
| ```json | ||
| { | ||
| "level": "error", | ||
| "error": { | ||
| "code": "unauthenticated", | ||
| "message": "bad token format" | ||
| }, | ||
| "hash": 6584967863753642363, | ||
| } | ||
| ``` | ||
| --- | ||
| Hashing with normalization (first found field is `message`): | ||
| ```yaml | ||
| pipelines: | ||
| example_pipeline: | ||
| ... | ||
| actions: | ||
| - type: hash | ||
| fields: | ||
| - field: error.code | ||
| - field: message | ||
| format: normalize | ||
| result_field: hash | ||
| ... | ||
| ``` | ||
| The original event: | ||
| ```json | ||
| { | ||
| "level": "error", | ||
| "message": "2023-10-30T13:35:33.638720813Z error occurred, client: 10.125.172.251, upstream: \"http://10.117.246.15:84/download\", host: \"mpm-youtube-downloader-38.name.com:84\"" | ||
| } | ||
| ``` | ||
|
|
||
| Normalized 'message': | ||
| `<datetime> error occurred, client: <ip>, upstream: "<url>", host: "<host>:<int>"` | ||
|
|
||
| The resulting event: | ||
| ```json | ||
| { | ||
| "level": "error", | ||
| "message": "2023-10-30T13:35:33.638720813Z error occurred, client: 10.125.172.251, upstream: \"http://10.117.246.15:84/download\", host: \"mpm-youtube-downloader-38.name.com:84\"", | ||
| "hash": 13863947727397728753, | ||
| } | ||
| ``` | ||
|
|
||
| ## Config params | ||
| **`fields`** *`[]Field`* *`required`* | ||
|
|
||
| Prioritized list of fields. The first field found will be used to calculate the hash. | ||
|
|
||
| `Field` params: | ||
| * **`field`** *`cfg.FieldSelector`* *`required`* | ||
|
|
||
| The event field for calculating the hash. | ||
|
|
||
| * **`format`** *`string`* *`default=no`* *`options=no|normalize`* | ||
|
|
||
| The field format for various hashing algorithms. | ||
|
|
||
| * **`max_size`** *`int`* *`default=0`* | ||
|
|
||
| The maximum field size used in hash calculation of any format. | ||
| If set to `0`, the entire field will be used in hash calculation. | ||
|
|
||
| > If the field size is greater than `max_size`, then | ||
| the first `max_size` bytes will be used in hash calculation. | ||
| > | ||
| > It can be useful in case of performance degradation when calculating the hash of long fields. | ||
|
|
||
| <br> | ||
|
|
||
| **`result_field`** *`cfg.FieldSelector`* *`required`* | ||
|
|
||
| The event field to which put the hash. | ||
|
|
||
| <br> | ||
|
|
||
|
|
||
| <br>*Generated using [__insane-doc__](https://github.com/vitkovskii/insane-doc)* |
Oops, something went wrong.