From 328ca46e0087b005366a1fb2a4194b2a8233c5a8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Samuel=20D=C3=A9fago?= Date: Wed, 25 Sep 2024 09:51:14 +0200 Subject: [PATCH] Update monitoring specifications (#99) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Gaëtan Muller --- .../specifications/monitoring/MONITORING.md | 43 +++++++++++++------ .../monitoring/schemas/error-schema.json | 2 +- 2 files changed, 30 insertions(+), 15 deletions(-) diff --git a/public/specifications/monitoring/MONITORING.md b/public/specifications/monitoring/MONITORING.md index b85e727..e5913e9 100644 --- a/public/specifications/monitoring/MONITORING.md +++ b/public/specifications/monitoring/MONITORING.md @@ -18,6 +18,19 @@ Pillarbox integrates with a monitoring platform that provides real-time and hist This article describes the flexible event model that allows our players to send data to our monitoring platform. +## JSON Key / Value Naming Conventions + +The following naming conventions are applied for key / values appearing in JSON payloads: + +- Keys use snake case. +- Values are provided in a form best suited for display, usually capitalized, with possible exceptions when brand + names are involved (e.g. macOS must not be capitalized as MacOS). +- The only values provided in uppercase are event types (e.g. `START`). + +> [!WARNING] +> Values provided by the system (e.g. device names) can be assumed as having a form already best suited for display. +> Their case must not be changed. + ## General Event Format Events provide data related to specific points of interests in the lifetime of a playback session. Three kinds of event @@ -38,7 +51,7 @@ following keys: | `timestamp` | The timestamp at the time the event is sent | [Unix timestamp](https://unixtime.org) in milliseconds | `1717665997932` | | `version` | The version of the JSON format | Number | 1 | -> [!ATTENTION] +> [!WARNING] > All keys listed above are mandatory. All events associated with the same session **MUST** be assigned the same `session_id`. The `data` format associated @@ -65,13 +78,13 @@ The associated event data dictionary supports the following keys: | `qoe_timings` | QoE timings | JSON dictionary | `{ ... }` | | `qos_timings` | QoS timings | JSON dictionary | `{ ... }` | -> [!ATTENTION] +> [!WARNING] > Requirements for each key are not provided explicitly but implementations **SHOULD** fill as much information as > possible. If some information cannot be reliably determined it **SHOULD** be omitted. ### JSON Schema -[start-schema.json](/specifications/monitoring/schemas/start-schema.json ':ignore') +[start-schema.json](specifications/monitoring/schemas/start-schema.json ':ignore') ### Browser @@ -138,7 +151,7 @@ The `qoe_timings` JSON data dictionary supports the following keys: | `metadata` | Time the user waited for metadata to be retrieved | Time in milliseconds | `412` | | `total` | Total time the user waited for playback to start | Time in milliseconds | `1763` | -> [!ATTENTION] +> [!WARNING] > QoE timings measure the perceived user experience. If content preloading in a playlist makes it possible to start > instantaneously (or almost), these values **SHOULD** be zero or close to zero. @@ -153,7 +166,7 @@ The `qos_timings` JSON data dictionary supports the following keys: | `metadata` | Time for metadata to be retrieved by the player | Time in milliseconds | `412` | | `token` | Time to fetch an authorization token | Time in milliseconds | `356` | -> [!ATTENTION] +> [!WARNING] > QoS timings measure actual system performance. They **SHOULD** reflect the time technically required to fetch content, > whether content preloading could take place or not. @@ -217,7 +230,7 @@ An event with the name `ERROR` **MUST** be sent when an error, either fatal or n - A non-fatal error (warning) informs about potential issues that occur behind the scenes and might affect the playback experience negatively. -> [!ATTENTION] +> [!WARNING] > A fatal `ERROR` at startup **MUST** always be preceded by a `START` event. If playback is restarted after a fatal `ERROR` a new session **MUST** be created, beginning with a new `START` event. @@ -231,10 +244,10 @@ The associated event data dictionary supports the following keys: | `name` | The name of the error | String | `ERR-404` | | `position` | The current player position, relative to the beginning of the playlist. Negative values are admitted | Time in milliseconds | `16548` | | `position_timestamp` | The current player timestamp, as retrieved from the playlist. Omitted if not available | [Unix timestamp](https://unixtime.org) in milliseconds | `1717665997932` | -| `severity` | The error severity | `WARNING`, `FATAL` | `WARNING` | +| `severity` | The error severity | `Warning`, `Fatal` | `Warning` | | `url` | The URL that was affected by the error | String | `https://...` | -> [!ATTENTION] +> [!WARNING] > Requirements for each key are not provided explicitly but implementations **SHOULD** fill as much information as > possible. If some information cannot be reliably determined it **SHOULD** be omitted. @@ -248,7 +261,7 @@ Some remarks: ### JSON Schema -[error-schema.json](/specifications/monitoring/schemas/error-schema.json ':ignore') +[error-schema.json](specifications/monitoring/schemas/error-schema.json ':ignore') ### Example @@ -258,7 +271,7 @@ Some remarks: "message": "Segment exceeds specified bandwidth for variant", "name": "CoreMediaErrorDomain(-12318)", "position": 1024, - "severity": "WARNING", + "severity": "Warning", "url": "https://rts-vod-amd.akamaized.net/ww/14895342/85891228-1e53-371b-997a-094380f533e2/index-f4-v1.m3u8" }, "event_name": "ERROR", @@ -286,6 +299,7 @@ The associated event data dictionary supports the following keys: | `bitrate` | Bitrate of the content being played | Number in bits per second | `1000000` | | `buffered_duration` | Duration of the content currently available in buffer | Time in milliseconds | `12000` | | `duration` | The content duration, as retrieved from the playlist | Time in milliseconds | `16548` | +| `frame_drops` | The total number of frame drops experienced during the session | Number | `12` | | `playback_duration` | The duration of the playback session | Time in milliseconds | `40000` | | `position` | The current player position, relative to the beginning of the playlist. Negative values are admitted | Time in milliseconds | `16548` | | `position_timestamp` | The current player timestamp, as retrieved from the playlist. Omitted if not available | [Unix timestamp](https://unixtime.org) in milliseconds | `1717665997932` | @@ -294,7 +308,7 @@ The associated event data dictionary supports the following keys: | `url` | The URL that is being played | String | `https://...` | | `vpn` | A value indicating whether a VPN is enabled on the device | Boolean | `true` | -> [!ATTENTION] +> [!WARNING] > Requirements for each key are not provided explicitly but implementations **SHOULD** fill as much information as > possible. If some information cannot be reliably determined it **SHOULD** be omitted. @@ -308,7 +322,7 @@ Some remarks: ### JSON Schema -[status-schema.json](/specifications/monitoring/schemas/status-schema.json ':ignore') +[status-schema.json](specifications/monitoring/schemas/status-schema.json ':ignore') ### Stall @@ -321,7 +335,7 @@ The `stall` JSON data dictionary supports the following keys: The stall duration **MUST** be measured in wall-clock time, independently of playback speed adjustments. -> [!ATTENTION] +> [!WARNING] > If a player is able to provide stall information, both `count` and `duration` **MUST** be supplied, even if zero. ### Example @@ -334,13 +348,14 @@ The stall duration **MUST** be measured in wall-clock time, independently of pla "bitrate": 6129146, "buffered_duration": 36000, "duration": 2386040, + "frame_drops": 2, "playback_duration": 10663, "position": 10618, "stall": { "count": 0, "duration": 0 }, - "stream_type": "on-demand", + "stream_type": "On-demand", "url": "https://rts-vod-amd.akamaized.net/ww/14895342/85891228-1e53-371b-997a-094380f533e2/index-f5-v1.m3u8", "vpn": false }, diff --git a/public/specifications/monitoring/schemas/error-schema.json b/public/specifications/monitoring/schemas/error-schema.json index 52d3d31..a3628b2 100644 --- a/public/specifications/monitoring/schemas/error-schema.json +++ b/public/specifications/monitoring/schemas/error-schema.json @@ -37,7 +37,7 @@ }, "severity": { "type": "string", - "enum": ["WARNING", "FATAL"], + "enum": ["Warning", "Fatal"], "description": "The severity of the error." }, "url": {