Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename service invocation identifier -> invocation identifier #135

Merged
merged 7 commits into from
Aug 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions docs/references/restate-sql-introspection.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,10 @@ The schema of these tables is described in the following.
| `service_key_int32` | `Int32` | Only contains meaningful values when a service uses an `int32` Restate key. | `1337` |
| `service_key_uuid` | `Utf8` | Only contains meaningful values for unkeyed Restate services. | `0189bab3b3397d828e979e2548980fa6` |
| `status` | `Utf8` | Enum value: `invoked` or `suspended`. | |
| `invocation_id` | `Utf8` | Restate-generated invocation ID. | `0189baba86ea76b2a0efc37db88ea9a5` |
| `sid` | `Utf8` | Service invocation ID, an opaque token that can be used to cancel an invocation of a service with the [admin API](/references/admin-api). | |
| `id` | `Utf8` | [Invocation ID](/services/invocation#invocation-identifier), an opaque token that can be used to cancel an invocation of a service with the [admin API](/references/admin-api). | |
| `invoked_by` | `Utf8` | Enum describing the invoker of this service: `ingress` = invoked externally / `service` = invoked by a service. | |
| `invoked_by_service` | `Utf8` | The name of the invoking service. Or `null` if invoked via the ingress. | `com.example.Greeter` |
| `invoked_by_sid` | `Utf8` | The service invocation ID of the service that triggered this invocation. Or `null` if invoked via the ingress. | |
| `invoked_by_id` | `Utf8` | The Invocation ID of the service that triggered this invocation. Or `null` if invoked externally. | |
| `trace_id` | `Utf8` | The ID of the trace that is assigned to this invocation. Only relevant when tracing is enabled. | |
| `journal_size` | `UInt32` | The number of journal entries durably logged for this invocation. | |
| `created_at` | `Date64` | Timestamp indicating the start of this invocation. | |
Expand Down
2 changes: 1 addition & 1 deletion docs/restate/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Most log events generated by Restate will also have the following attached attri

* `rpc.service`: The related service
* `rpc.method`: The related method
* `restate.invocation.sid`: The [service invocation identifier](/services/invocation#service-invocation-identifier)
* `restate.invocation.id`: The [invocation identifier](/services/invocation#invocation-identifier)

## Recommendations

Expand Down
8 changes: 4 additions & 4 deletions docs/restate/managed_service.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,19 +57,19 @@ Grafana supports correlating logs and traces. In your Loki data source configura
set up a [Derived field](https://grafana.com/docs/grafana/latest/datasources/loki/configure-loki-data-source/#derived-fields)
with the following parameters:
```yaml
name: restate.invocation.sid
name: restate.invocation.id
# pull invocation ids out of logs
regex: '"restate.invocation.sid":"(.+?)"' # single quotes not needed
regex: '"restate.invocation.id":"(.+?)"' # single quotes not needed
# find traces with that invocation id
query: '{ .restate.invocation.sid="${__value.raw}" }' # single quotes not needed
query: '{ .restate.invocation.id="${__value.raw}" }' # single quotes not needed
internal_link:
enabled: true
data_source: <your Tempo data source>
```

In your Tempo configuration, after you have a Loki data source configured, set up
[Trace to logs](https://grafana.com/docs/grafana/latest/datasources/tempo/#trace-to-logs) with data source set to your Loki data source,
and query set to `{kubernetes_container_name="restate"} |= "${__span.tags["restate.invocation.sid"]}"`.
and query set to `{kubernetes_container_name="restate"} |= "${__span.tags["restate.invocation.id"]}"`.

You can view logs in Grafana on the explore page by selecting the Loki data source. To get started you can query:
```logql
Expand Down
10 changes: 5 additions & 5 deletions docs/restate/tracing.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,21 +80,21 @@ You can import the trace files using the Jaeger UI:
## Understanding traces
The traces contain detailed information about the context calls that were done during the invocation (e.g. sleep, one-way calls, interaction with state):

![Understanding traces](/img/understanding_traces.png)
![Understanding traces](/img/jaeger_tour_background_call_handler.png)
slinkydeveloper marked this conversation as resolved.
Show resolved Hide resolved

The initial `ingress_service_invocation` spans show when the gRPC/Connect HTTP request was received by Restate. The `invoke` span beneath it shows when Restate invoked the service endpoint to process the request.
The initial `ingress_invoke` spans show when the gRPC/Connect HTTP request was received by Restate. The `invoke` span beneath it shows when Restate invoked the service endpoint to process the request.

The tags of the spans contain the metadata of the context calls (e.g. call arguments, invocation id).

When a service invokes another service, the child invocation is linked automatically to the parent invocation, as you can see in the image.
Note that the spans of one-way calls are shown as separate traces. The parent invocation only shows that the one-way call was scheduled, not its entire tracing span.
To see this information, search for the trace of the one-way call by filtering on the invocation id tag:
```
restate.invocation.sid="example.MyExampleService-AzEyMw==-0189b536906b746c8da6f83f0257acda"
restate.invocation.id="T4pIkIJIGAsBiiGDV2dxK7PkkKnWyWHE"
```

## Searching traces

Traces export attributes and tags that correlate the trace with the service and/or invocation. For example, in the Jaeger UI, you can filter on the invocation id (`restate.invocation.sid`) or any other tag:
Traces export attributes and tags that correlate the trace with the service and/or invocation. For example, in the Jaeger UI, you can filter on the invocation id (`restate.invocation.id`) or any other tag:

![Jaeger invocation id search](/img/jaeger_docs_invocationid_search.png)
![Jaeger invocation id search](/img/jaeger_invocationid_search_handler.png)
20 changes: 10 additions & 10 deletions docs/services/introspection.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ draft: false

Restate exposes information on invocations and application state via its Introspection SQL API. You can use this to gain insight into the status of invocations and the service state that is stored.

This can be useful for troubleshooting. For example, a handler might be blocked for a specific key, and you want to kill the invocation that is blocking it but you don't know the service invocation ID. Or you want to check what is currently stored in the state of a service.
This can be useful for troubleshooting. For example, a handler might be blocked for a specific key, and you want to kill the invocation that is blocking it but you don't know the invocation ID. Or you want to check what is currently stored in the state of a service.

To let you do this, Restate exposes two SQL tables that you can query via the [*psql* client](https://www.postgresql.org/docs/current/app-psql.html):
- `sys_status` table to inspect invocation status.
Expand Down Expand Up @@ -51,33 +51,33 @@ This following list contains some key queries on the `sys_status` table to give
select * from sys_status;
```

- Retrieve the service invocation ID (`sid`) that is currently blocking a service-key combination via:
- Retrieve the invocation ID (`id`) that is currently blocking a service-key combination via:
```sql
select method, status, sid from sys_status where service = 'test.MyServiceName' and service_key_utf8 = 'myKey';
select method, status, id from sys_status where service = 'test.MyServiceName' and service_key_utf8 = 'myKey';
```
You can then use the service invocation ID to kill the invocation via the [admin API](/references/admin-api#tag/invocation/operation/cancel_invocation).
You can then use the invocation ID to [kill the invocation](./invocation.md#cancel-an-invocation).

- Check the status of an invocation via:
```sql
select service, method, status from sys_status where sid = 'my_sid';
select service, method, status from sys_status where id = 'T4pIkIJIGAsBiiGDV2dxK7PkkKnWyWHE';
```
The status field shows that the invocation is either in status `invoked`, meaning actively processing, or in status is `suspended`, meaning waiting on some external input (e.g. request-response call, awakeable, sleep, ...).

- Check when an invocation was modified the last time via:
```sql
select modified_at from sys_status where sid = 'my_sid';
select modified_at from sys_status where id = 'T4pIkIJIGAsBiiGDV2dxK7PkkKnWyWHE';
```
This includes any modification to the row in the table (e.g. when the service last switched its status from `invoked` to `suspended`, or when the last journal entry was added).

- To find out if an invocation was triggered via the ingress or by another service:
```sql
select invoked_by, invoked_by_service, invoked_by_sid from sys_status where sid = 'my_sid';
select invoked_by, invoked_by_service, invoked_by_id from sys_status where id = 'T4pIkIJIGAsBiiGDV2dxK7PkkKnWyWHE';
```
The `invoked_by` field contains either `ingress` or `service`. If the invocation was triggered by another service, then the fields `invoked_by_service` and `invoked_by_sid` will supply more information about the invoking service.
The `invoked_by` field contains either `ingress` or `service`. If the invocation was triggered by another service, then the fields `invoked_by_service` and `invoked_by_id` will supply more information about the invoking service.

- Retrieve the trace ID based on a service invocation ID:
- Retrieve the trace ID based on a invocation ID:
```sql
select trace_id from sys_status where sid = 'my_sid';
select trace_id from sys_status where id = 'T4pIkIJIGAsBiiGDV2dxK7PkkKnWyWHE';
```
Afterwards, you can use this trace ID to [search for spans in Jaeger](/restate/tracing#searching-traces).

Expand Down
16 changes: 8 additions & 8 deletions docs/services/invocation.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ For example, using [Connect](#connect-grpc-on-http) and `curl`:
curl -X POST http://<your-restate-runtime-host-port>/dev.restate.Ingress/Invoke -H 'content-type: application/json' -d '{"service": "org.example.Greeter", "method": "Greet", "payload": {"name": "Pete"}}'
```

The response contains the [service invocation identifier](/services/invocation#service-invocation-identifier). You can use this identifier to manage the invocation as described in [the respective documention](/services/invocation).
The response contains the [Invocation identifier](#invocation-identifier). You can use this identifier to cancel or kill the invocation as described in [the below paragraph](#cancel-an-invocation).

For a complete documentation of the `dev.restate.Ingress` built-in service, check out the [Restate protobuf definitions](https://github.com/restatedev/proto/blob/main/dev/restate/services.proto).

Expand Down Expand Up @@ -103,11 +103,11 @@ For more details on the API, refer to the [admin API docs](/references/admin-api

Restate offers several tools to manage the ongoing invocations.

## Service invocation identifier
## Invocation identifier

Every invocation to a service gets a unique identifier assigned by Restate, called _Service invocation identifier_. You can use this identifier to filter your structured logs, find traces, and execute some management operations such as cancelling an invocation.
Every invocation to a service gets a unique identifier assigned by Restate, called _Invocation identifier_. You can use this identifier to filter your structured logs, find traces, and execute some management operations such as cancelling an invocation.

You can find this identifier in the runtime logs and OpenTelementry traces by looking for the `restate.invocation.sid`, for example:
You can find this identifier in the runtime logs and OpenTelementry traces by looking for the `restate.invocation.id`, for example:

```log {7}
2023-05-19T15:02:28.656467Z INFO restate_invoker::invocation_task
Expand All @@ -116,12 +116,12 @@ You can find this identifier in the runtime logs and OpenTelementry traces by lo
in restate_invoker::invocation_task::invoker_invocation_task
rpc.system: "restate"
rpc.service: coordinator.Coordinator
restate.invocation.sid: coordinator.Coordinator-AYguNHjOdG+gM+TY9k1qeA==-01882e3478ce79579999ecabfd7f4129
restate.invocation.id: T4pIkIJIGAsBiiGDV2dxK7PkkKnWyWHE
```

:::note

The service invocation identifier is opaque and its current format should not be relied on, as it might change in future Restate versions.
The Invocation identifier is opaque and its current format should not be relied on, as it might change in future Restate versions.

:::

Expand Down Expand Up @@ -152,13 +152,13 @@ Killing an invocation might leave the service instance in an inconsistent state,
To kill an invocation, send the following request to the Restate meta endpoint:

```shell
$ curl -X DELETE <META_ENDPOINT>/invocations -H 'content-type: application/json' -d '{"sid": "<SERVICE_INVOCATION_IDENTIFIER>"}'
$ curl -X DELETE <META_ENDPOINT>/invocations/<INVOCATION_IDENTIFIER>
```

For example:

```shell
$ curl -X DELETE http://localhost:8081/invocations -H 'content-type: application/json' -d '{"sid": "coordinator.Coordinator-AYguNHjOdG+gM+TY9k1qeA==-01882e3478ce79579999ecabfd7f4129"}'
$ curl -X DELETE http://localhost:8081/invocations/T4pIkIJIGAsBiiGDV2dxK7PkkKnWyWHE
```

For more details on the API, refer to the [admin API docs](/references/admin-api).
Loading