feat: Documentation for client reports (getsentry#429)

src/components/sidebar.tsx

@@ -248,6 +248,7 @@ export default () => {
             </SidebarLink>
           </SidebarLink>
           <SidebarLink to="/sdk/sessions/">Sessions</SidebarLink>
+          <SidebarLink to="/sdk/client-reports/">Client Reports</SidebarLink>
           <SidebarLink to="/sdk/research/performance">
             Research: Performance Monitoring API
           </SidebarLink>

src/docs/sdk/client-reports.mdx

@@ -0,0 +1,101 @@
+---
+title: Client Reports
+sidebar_order: 20
+---
+
+Client reports are a protocol feature that let clients send status reports
+about themselves to Sentry.  They are currently mainly used to emit outcomes
+for events that were never sent.
+
+## Basic Operation
+
+Client reports are sent as envelope items to Sentry, typically as separate
+envelopes or with one of the already scheduled envelopes.  They should not
+be sent too frequently but not too infrequently either.  Their main purpose
+is to bring visibility into what is happening on the SDK side which affects
+the user experience.
+
+For instance SDKs might drop events in a few places in the SDK and this loss
+of events can be invisible to a customer.  Client reports let an SDK emit
+such event outcomes to provide data about how often this is happening.  For
+instance SDKs might drop events if the transports hit their maximum internal
+queue size, because rate limits instruct the SDK to drop events as they are
+over quota etc.
+
+## Envelope Item Payload
+
+A client report is an item in an envelope called `client_report`. It consists
+of a JSON payload that looks roughly like this:
+
+```json
+{
+  "timestamp": "2020-02-07T14:16:00Z",
+  "discarded_events": [
+    {
+      "reason": "queue_overflow",
+      "category": "error",
+      "quantity": 23
+    },
+    {
+      "reason": "queue_overflow",
+      "category": "transaction",
+      "quantity": 1321
+    }
+  ]
+}
+```
+
+Note that this must be enclosed in an envelope. So the full event looks
+something like this:
+
+```json
+{}
+{"type":"client_report"}
+{"timestamp":"..."}
+```
+
+The following fields exist:
+
+`timestamp`
+
+: _String | Number, optional_. The timestamp of when the client report was created.
+Must be an ISO DateTime string or a UNIX timestamp. If not sent, the server
+will assume the current UTC timestamp. In the data model, this is called
+`received`.
+
+`discarded_events`
+
+: _List of outcome objects_ {`discard_reason`, `category`, `quantity`}
+
+- `discard_reason`: A string reason that defines why events were lost.
+- `category`: The data category for which the discard reason applies.
+- `quantity`: The number of events which were lost
+
+The following discard reasons are currently defined:
+
+- `queue_overflow`: a SDK internal queue (eg: transport queue) overflowed
+- `cache_overflow`: an SDK internal cache (eg: offline event cache) overflowed
+- `ratelimit_backoff`: the SDK dropped events because an earlier rate limit
+  instructed the SDK to back off.
+- `network_error`: events were dropped because of network errors and were not retried.
+- `sample_rate`: an event was dropped because of the configured sample rate.
+
+Additionally the following discard reasons are reserved but there is no expectation
+that SDKs send these under normal operation:
+
+- `before_send`: an event was dropped in `before_send`
+- `event_processor`: an event was dropped by an event processor
+
+## SDK Side Recommendations
+
+SDKs are encouraged to reduce the total amount of needless communciation.  As
+such a recommended approach is to keep track of the counts for the discard
+reasons in the transport directly and to periodically flush them out as separate
+envelope item or attach it to an already scheduled envelope.  As some SDKs still
+send legacy events instead of envelopes for backwards compatibility with older
+Sentry servers it would be recommended in such cases to send it as separate envelope
+instead or attach it to pending session envelopes.
+
+There is no expectation that such bookkeping can work transparently for custom
+transports which is why it's generally acceptable if client reports are optional
+for custom transports.

src/docs/sdk/index.mdx

@@ -15,3 +15,5 @@ typically look and behave.
 - <Link to="/sdk/data-handling/">Data Handling</Link>
 - <Link to="/sdk/store/">Store Endpoint</Link>
 - <Link to="/sdk/envelopes/">Envelopes</Link>
+- <Link to="/sdk/sessions/">Sessions</Link>
+- <Link to="/sdk/client-reports/">Client Reports</Link>

src/docs/sdk/rate-limiting.mdx

@@ -78,9 +78,16 @@ As stated earlier, SDKs can ignore the `scope` dimension. These definitions are
   - `security`: Events with event_type `csp`, `hpkp`, `expectct`, `expectstaple`
   - `attachment`: Attachment bytes stored (unused for rate limiting)
   - `session`: Session update events
+  - `internal`: a sentry/system internal event[^internal]
 
 
 - **Scope**: The unit / model in Sentry that quotas are enforced for.
   - `organization`
   - `project`
   - `key`
+
+[^internal]: Special note on the internal data category: this data category exists but is promised not to be emitted as we do not enforcd
+rate limits on internal items.  This might appear odd but the purpose of this is to allow envelope implementations in SDKs to
+assign a data category to internal items such as `client_report`.  SDKs can handle `internal` like any other data category but
+they can rely on the fact that rate limits are not communicated for these explicitly.  That said, the special _all_ category
+does apply to `internal` event types as well.