Frontend Refactor: Streaming tags (#3460)

* log request Signed-off-by: Joe Elliott <[email protected]> * move stuff a bit Signed-off-by: Joe Elliott <[email protected]> * oh my. e2e tests pass Signed-off-by: Joe Elliott <[email protected]> * add handlers Signed-off-by: Joe Elliott <[email protected]> * streaming tags Signed-off-by: Joe Elliott <[email protected]> * add cli support Signed-off-by: Joe Elliott <[email protected]> * improve logging Signed-off-by: Joe Elliott <[email protected]> * fix Signed-off-by: Joe Elliott <[email protected]> * docs Signed-off-by: Joe Elliott <[email protected]> * pipe overrides Signed-off-by: Joe Elliott <[email protected]> * cleanup Signed-off-by: Joe Elliott <[email protected]> * cleanup Signed-off-by: Joe Elliott <[email protected]> * support limits Signed-off-by: Joe Elliott <[email protected]> * docs Signed-off-by: Joe Elliott <[email protected]> * e2e tests and caching Signed-off-by: Joe Elliott <[email protected]> * key prefixes Signed-off-by: Joe Elliott <[email protected]> * cache keys Signed-off-by: Joe Elliott <[email protected]> * Fixed distinct collection in combiners Signed-off-by: Joe Elliott <[email protected]> * fixed combiner bugs and revived tests Signed-off-by: Joe Elliott <[email protected]> * restored all tests Signed-off-by: Joe Elliott <[email protected]> * lint Signed-off-by: Joe Elliott <[email protected]> * made search handler utilities generic Signed-off-by: Joe Elliott <[email protected]> * Added handler tests for tags Signed-off-by: Joe Elliott <[email protected]> * add diff support Signed-off-by: Joe Elliott <[email protected]> * lint Signed-off-by: Joe Elliott <[email protected]> * add distinct value collector tests Signed-off-by: Joe Elliott <[email protected]> * fix integration tests Signed-off-by: Joe Elliott <[email protected]> * diff tests Signed-off-by: Joe Elliott <[email protected]> * swapped query for the more robust ExtractMatchers(query) Signed-off-by: Joe Elliott <[email protected]> * tests Signed-off-by: Joe Elliott <[email protected]> * moved e2e tests to a more sensible place Signed-off-by: Joe Elliott <[email protected]> * fix non-deterministic test Signed-off-by: Joe Elliott <[email protected]> * changelog Signed-off-by: Joe Elliott <[email protected]> * fix tests for 429 handling Signed-off-by: Joe Elliott <[email protected]> * Update docs/sources/tempo/operations/tempo_cli.md Co-authored-by: Mario <[email protected]> * review Signed-off-by: Joe Elliott <[email protected]> * Update docs/sources/tempo/api_docs/_index.md Co-authored-by: Kim Nylander <[email protected]> * Correctly cancel GRPC context beneath the HTTP server (#3443) * cancel context Signed-off-by: Joe Elliott <[email protected]> * update dskit Signed-off-by: Joe Elliott <[email protected]> * focused timeouts Signed-off-by: Joe Elliott <[email protected]> * docs Signed-off-by: Joe Elliott <[email protected]> * lint N docs Signed-off-by: Joe Elliott <[email protected]> * more lint Signed-off-by: Joe Elliott <[email protected]> * make update-mod Signed-off-by: Joe Elliott <[email protected]> --------- Signed-off-by: Joe Elliott <[email protected]> * Bump anchore/sbom-action from 0.15.8 to 0.15.9 (#3476) Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.15.8 to 0.15.9. - [Release notes](https://github.com/anchore/sbom-action/releases) - [Commits](anchore/sbom-action@v0.15.8...v0.15.9) --- updated-dependencies: - dependency-name: anchore/sbom-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Doc update (#3482) * doc: remove reference to previously purged script * doc: correct label for docs updates * [TraceQL Metrics] Use new per-tenant max_metrics_duration and fix duration check (#3484) * Use new per-tenant max_metrics_duration, and fix duration timestamp handling * Update docs and defaults * Handle prefixes when listing blocks from S3 and GCS (#3466) * Handle prefixes when listing blocks from S3 fixes #3465 * Handle prefixes when listing blocks from GCS * Add test for prefixes when listing blocks from Azure * Update unit tests to check for actual block IDs instead of just length of the slices Cleanup unit tests * Further refine S3/GCS backend for ListBlocks Brings logic more in line with Azure object parsing. Also has the benefit of handling prefixes without a trailing slash. * Update poller integration test to exercise prefixes * Update e2e test to exercise prefixes * Fix format check error * Fix failing e2e tests * Remove unnecessary prefix permutations from e2e test * Remove unnecessary test config file copy * Ignore lint --------- Co-authored-by: Zach Leslie <[email protected]> * Update doc-validator.yml (#3483) Updates the doc-validator to the latest version. Note that this changes the reference format to use the full URL (https://....) instead of /docs/blah * [DOC] Document Tempo Operator Monolithic mode (#3474) * [DOC] Document Tempo Operator Monolithic mode Signed-off-by: Andreas Gerstmayr <[email protected]> * clarify supported storages Signed-off-by: Andreas Gerstmayr <[email protected]> * fix case of title Signed-off-by: Andreas Gerstmayr <[email protected]> * Apply suggestions from code review * Apply suggestions from code review --------- Signed-off-by: Andreas Gerstmayr <[email protected]> Co-authored-by: Kim Nylander <[email protected]> * [DOC] document Grafana data source setup using Grafana and Tempo operators (#3473) * [docs] document Grafana data source setup using Grafana and Tempo operators * move the Grafana data source setup page to the operator folder (this page is only relevant for the operator) * document Grafana data source setup using Grafana and Tempo operators Signed-off-by: Andreas Gerstmayr <[email protected]> * Apply suggestions from code review --------- Signed-off-by: Andreas Gerstmayr <[email protected]> Co-authored-by: Kim Nylander <[email protected]> * [DOC] fix typo in setup/operator/monolithic.md (#3496) Signed-off-by: Andreas Gerstmayr <[email protected]> * Bump github.com/prometheus/client_golang from 1.18.0 to 1.19.0 (#3455) * Bump github.com/prometheus/client_golang from 1.18.0 to 1.19.0 Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.18.0 to 1.19.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](prometheus/client_golang@v1.18.0...v1.19.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Update serverless gomod --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: grafanabot <[email protected]> * Add support for dashes, quotes and spaces in attribute names (#3458) * Add support for dashes, quotes and spaces in attribute names * chlog * [TraceQL Metrics] Step align query_range time range (#3490) * Step align query_range time range * Time range error: improve message and fix format for prom format. * oops remove printlns * lint * changelog * 2.4.1 changelog (#3503) Signed-off-by: Joe Elliott <[email protected]> * [DOC] Add 2.4.1 release notes (#3504) * fix tests due to interface changing Signed-off-by: Joe Elliott <[email protected]> * Pass context Signed-off-by: Joe Elliott <[email protected]> --------- Signed-off-by: Joe Elliott <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Andreas Gerstmayr <[email protected]> Co-authored-by: Mario <[email protected]> Co-authored-by: Kim Nylander <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Matt Simonsen <[email protected]> Co-authored-by: Martin Disibio <[email protected]> Co-authored-by: Ben Foster <[email protected]> Co-authored-by: Zach Leslie <[email protected]> Co-authored-by: Andreas Gerstmayr <[email protected]> Co-authored-by: grafanabot <[email protected]>
grafana · Mar 20, 2024 · c61bd1b · c61bd1b
1 parent 85cb5de
commit c61bd1b
Show file tree

Hide file tree

Showing 39 changed files with 2,907 additions and 2,158 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,6 @@
 ## main / unreleased
 
+* [FEATURE] Added gRPC streaming endpoints for all tag queries. [#3460](https://github.com/grafana/tempo/pull/3460) (@joe-elliott)
 * [CHANGE] Align metrics query time ranges to the step parameter [#3490](https://github.com/grafana/tempo/pull/3490) (@mdisibio)
 * [ENHANCEMENT] Add string interning to TraceQL queries [#3411](https://github.com/grafana/tempo/pull/3411) (@mapno)
 * [ENHANCEMENT] Add new (unsafe) query hints for metrics queries [#3396](https://github.com/grafana/tempo/pull/3396) (@mdisibio)

diff --git a/cmd/tempo-cli/cmd-query-search-tag-values.go b/cmd/tempo-cli/cmd-query-search-tag-values.go
@@ -1,23 +1,117 @@
 package main
 
 import (
+	"context"
+	"errors"
+	"io"
+	"path"
+	"time"
+
+	"github.com/grafana/dskit/user"
 	"github.com/grafana/tempo/pkg/httpclient"
+	"github.com/grafana/tempo/pkg/tempopb"
+	"google.golang.org/grpc"
+	"google.golang.org/grpc/credentials/insecure"
 )
 
 type querySearchTagValuesCmd struct {
-	APIEndpoint string `arg:"" help:"tempo api endpoint"`
-	Tag         string `arg:"" help:"tag name"`
+	HostPort string `arg:"" help:"tempo host and port. scheme and path will be provided based on query type. e.g. localhost:3200"`
+	Tag      string `arg:"" help:"tag name"`
+	Start    string `arg:"" optional:"" help:"start time in ISO8601 format"`
+	End      string `arg:"" optional:"" help:"end time in ISO8601 format"`
 
-	OrgID string `help:"optional orgID"`
+	Query      string `help:"TraceQL query to filter attribute results by (supported by GRPC only)"`
+	OrgID      string `help:"optional orgID"`
+	UseGRPC    bool   `help:"stream search results over GRPC"`
+	PathPrefix string `help:"string to prefix all http paths with"`
 }
 
 func (cmd *querySearchTagValuesCmd) Run(_ *globalOptions) error {
-	client := httpclient.New(cmd.APIEndpoint, cmd.OrgID)
+	var start, end int64
+
+	if cmd.Start != "" {
+		startDate, err := time.Parse(time.RFC3339, cmd.Start)
+		if err != nil {
+			return err
+		}
+		start = startDate.Unix()
+	}
+
+	if cmd.End != "" {
+		endDate, err := time.Parse(time.RFC3339, cmd.End)
+		if err != nil {
+			return err
+		}
+		end = endDate.Unix()
+	}
+
+	if cmd.UseGRPC {
+		return cmd.searchGRPC(start, end)
+	}
+	return cmd.searchHTTP(start, end)
+}
+
+// nolint: goconst // goconst wants us to make http:// a const
+func (cmd *querySearchTagValuesCmd) searchHTTP(start, end int64) error {
+	if cmd.PathPrefix != "" {
+		cmd.HostPort = path.Join(cmd.HostPort, cmd.PathPrefix)
+	}
+	client := httpclient.New("http://"+cmd.HostPort, cmd.OrgID)
+
+	var tags *tempopb.SearchTagValuesV2Response
+	var err error
+	if start != 0 || end != 0 {
+		tags, err = client.SearchTagValuesV2WithRange(cmd.Tag, start, end)
+	} else {
+		tags, err = client.SearchTagValuesV2(cmd.Tag, "")
+	}
+
+	if err != nil {
+		return err
+	}
+
+	return printAsJSON(tags)
+}
+
+func (cmd *querySearchTagValuesCmd) searchGRPC(start, end int64) error {
+	ctx := user.InjectOrgID(context.Background(), cmd.OrgID)
+	ctx, err := user.InjectIntoGRPCRequest(ctx)
+	if err != nil {
+		return err
+	}
 
-	tagValues, err := client.SearchTagValues(cmd.Tag)
+	clientConn, err := grpc.DialContext(ctx, cmd.HostPort, grpc.WithTransportCredentials(insecure.NewCredentials()))
 	if err != nil {
 		return err
 	}
 
-	return printAsJSON(tagValues)
+	client := tempopb.NewStreamingQuerierClient(clientConn)
+
+	tagsRequest := &tempopb.SearchTagValuesRequest{
+		TagName: cmd.Tag,
+		Start:   uint32(start),
+		End:     uint32(end),
+		Query:   cmd.Query,
+	}
+
+	resp, err := client.SearchTagValuesV2(ctx, tagsRequest)
+	if err != nil {
+		return err
+	}
+
+	for {
+		searchResp, err := resp.Recv()
+		if searchResp != nil {
+			err = printAsJSON(searchResp)
+			if err != nil {
+				return err
+			}
+		}
+		if errors.Is(err, io.EOF) {
+			return nil
+		}
+		if err != nil {
+			return err
+		}
+	}
 }
diff --git a/cmd/tempo-cli/cmd-query-search-tags.go b/cmd/tempo-cli/cmd-query-search-tags.go
@@ -1,22 +1,113 @@
 package main
 
 import (
+	"context"
+	"errors"
+	"io"
+	"path"
+	"time"
+
+	"github.com/grafana/dskit/user"
 	"github.com/grafana/tempo/pkg/httpclient"
+	"github.com/grafana/tempo/pkg/tempopb"
+	"google.golang.org/grpc"
+	"google.golang.org/grpc/credentials/insecure"
 )
 
 type querySearchTagsCmd struct {
-	APIEndpoint string `arg:"" help:"tempo api endpoint"`
+	HostPort string `arg:"" help:"tempo host and port. scheme and path will be provided based on query type. e.g. localhost:3200"`
+	Start    string `arg:"" optional:"" help:"start time in ISO8601 format"`
+	End      string `arg:"" optional:"" help:"end time in ISO8601 format"`
 
-	OrgID string `help:"optional orgID"`
+	OrgID      string `help:"optional orgID"`
+	UseGRPC    bool   `help:"stream search results over GRPC"`
+	PathPrefix string `help:"string to prefix all http paths with"`
 }
 
 func (cmd *querySearchTagsCmd) Run(_ *globalOptions) error {
-	client := httpclient.New(cmd.APIEndpoint, cmd.OrgID)
+	var start, end int64
+
+	if cmd.Start != "" {
+		startDate, err := time.Parse(time.RFC3339, cmd.Start)
+		if err != nil {
+			return err
+		}
+		start = startDate.Unix()
+	}
+
+	if cmd.End != "" {
+		endDate, err := time.Parse(time.RFC3339, cmd.End)
+		if err != nil {
+			return err
+		}
+		end = endDate.Unix()
+	}
+
+	if cmd.UseGRPC {
+		return cmd.searchGRPC(start, end)
+	}
+	return cmd.searchHTTP(start, end)
+}
+
+// nolint: goconst // goconst wants us to make http:// a const
+func (cmd *querySearchTagsCmd) searchHTTP(start, end int64) error {
+	if cmd.PathPrefix != "" {
+		cmd.HostPort = path.Join(cmd.HostPort, cmd.PathPrefix)
+	}
+	client := httpclient.New("http://"+cmd.HostPort, cmd.OrgID)
+
+	var tags *tempopb.SearchTagsV2Response
+	var err error
+	if start != 0 || end != 0 {
+		tags, err = client.SearchTagsV2WithRange(start, end)
+	} else {
+		tags, err = client.SearchTagsV2()
+	}
 
-	tags, err := client.SearchTags()
 	if err != nil {
 		return err
 	}
 
 	return printAsJSON(tags)
 }
+
+func (cmd *querySearchTagsCmd) searchGRPC(start, end int64) error {
+	ctx := user.InjectOrgID(context.Background(), cmd.OrgID)
+	ctx, err := user.InjectIntoGRPCRequest(ctx)
+	if err != nil {
+		return err
+	}
+
+	clientConn, err := grpc.DialContext(ctx, cmd.HostPort, grpc.WithTransportCredentials(insecure.NewCredentials()))
+	if err != nil {
+		return err
+	}
+
+	client := tempopb.NewStreamingQuerierClient(clientConn)
+
+	tagsRequest := &tempopb.SearchTagsRequest{
+		Start: uint32(start),
+		End:   uint32(end),
+	}
+
+	resp, err := client.SearchTagsV2(ctx, tagsRequest)
+	if err != nil {
+		return err
+	}
+
+	for {
+		searchResp, err := resp.Recv()
+		if searchResp != nil {
+			err = printAsJSON(searchResp)
+			if err != nil {
+				return err
+			}
+		}
+		if errors.Is(err, io.EOF) {
+			return nil
+		}
+		if err != nil {
+			return err
+		}
+	}
+}
diff --git a/cmd/tempo-cli/cmd-query-search.go b/cmd/tempo-cli/cmd-query-search.go
@@ -95,6 +95,7 @@ func (cmd *querySearchCmd) searchGRPC(req *tempopb.SearchRequest) error {
 	}
 }
 
+// nolint: goconst // goconst wants us to make http:// a const
 func (cmd *querySearchCmd) searchHTTP(req *tempopb.SearchRequest) error {
 	httpReq, err := http.NewRequest("GET", "http://"+path.Join(cmd.HostPort, cmd.PathPrefix, api.PathSearch), nil)
 	if err != nil {

diff --git a/docs/sources/tempo/api_docs/_index.md b/docs/sources/tempo/api_docs/_index.md
@@ -631,52 +631,14 @@ To enable the streaming service over the HTTP port for use with Grafana, set the
 stream_over_http_enabled: true
 ```
 
-The below `rpc` call returns only traces that are new or have updated each time `SearchResponse` is returned except for the last response.
-The final response sent is guaranteed to have the entire resultset.
+The query frontend supports the following interface. Refer to [`tempo.proto`](https://github.com/grafana/tempo/blob/main/pkg/tempopb/tempo.proto) for complete details of all objects.
 
 ```protobuf
 service StreamingQuerier {
   rpc Search(SearchRequest) returns (stream SearchResponse);
-}
-message SearchRequest {
-  map<string, string> Tags = 1
-  uint32 MinDurationMs = 2;
-  uint32 MaxDurationMs = 3;
-  uint32 Limit = 4;
-  uint32 start = 5;
-  uint32 end = 6;
-  string Query = 8;
-}
-message SearchResponse {
-  repeated TraceSearchMetadata traces = 1;
-  SearchMetrics metrics = 2;
-}
-message TraceSearchMetadata {
-  string traceID = 1;
-  string rootServiceName = 2;
-  string rootTraceName = 3;
-  uint64 startTimeUnixNano = 4;
-  uint32 durationMs = 5;
-  SpanSet spanSet = 6; // deprecated. use SpanSets field below
-  repeated SpanSet spanSets = 7;
-}
-message SpanSet {
-  repeated Span spans = 1;
-  uint32 matched = 2;
-}
-message Span {
-  string spanID = 1;
-  string name = 2;
-  uint64 startTimeUnixNano = 3;
-  uint64 durationNanos = 4;
-  repeated tempopb.common.v1.KeyValue attributes = 5;
-}
-message SearchMetrics {
-  uint32 inspectedTraces = 1;
-  uint64 inspectedBytes = 2;
-  uint32 totalBlocks = 3;
-  uint32 completedJobs = 4;
-  uint32 totalJobs = 5;
-  uint64 totalBlockBytes = 6;
+  rpc SearchTags(SearchTagsRequest) returns (stream SearchTagsResponse) {}
+  rpc SearchTagsV2(SearchTagsRequest) returns (stream SearchTagsV2Response) {}
+  rpc SearchTagValues(SearchTagValuesRequest) returns (stream SearchTagValuesResponse) {}
+  rpc SearchTagValuesV2(SearchTagValuesRequest) returns (stream SearchTagValuesV2Response) {}
 }
 ```
diff --git a/docs/sources/tempo/operations/tempo_cli.md b/docs/sources/tempo/operations/tempo_cli.md
@@ -80,7 +80,7 @@ tempo-cli query api trace-id http://tempo:3200 f1cfe82a8eef933b
 Call the Tempo API and search using TraceQL.
 
 ```bash
-tempo-cli query api search <host-port> <trace-ql> <start> <end>
+tempo-cli query api search <host-port> <trace-ql> [<start> <end>]
 ```
 Arguments:
 - `host-port` A host/port combination for Tempo. The scheme will be inferred based on the options provided.
@@ -99,6 +99,46 @@ Options:
 Streaming over HTTP requires the `stream_over_http_enabled` flag to be set. For more information, refer to [Tempo GRPC API documentation]({{< relref "../api_docs" >}}).
 {{% /admonition %}}
 
+### Search tags
+Call the Tempo API and search attribute names.
+
+```bash
+tempo-cli query api search-tags <host-port> [<start> <end>]
+```
+Arguments:
+- `host-port` A host/port combination for Tempo. The scheme will be inferred based on the options provided.
+- `start` Start of the time range to search: (YYYY-MM-DDThh:mm:ss)
+- `end` End of the time range to search: (YYYY-MM-DDThh:mm:ss)
+
+Options:
+- `--org-id <value>`      Organization ID (for use in multi-tenant setup).
+- `--use-grpc`            Use GRPC streaming
+- `--path-prefix <value>` String to prefix search paths with
+
+{{% admonition type="note" %}}
+Streaming over HTTP requires the `stream_over_http_enabled` flag to be set. For more information, refer to [Tempo GRPC API documentation]({{< relref "../api_docs" >}}).
+{{% /admonition %}}
+
+### Search tag values
+Call the Tempo API and search attribute values.
+
+```bash
+tempo-cli query api search-tag-values <tag> <host-port> [<start> <end>]
+```
+Arguments:
+- `host-port` A host/port combination for Tempo. The scheme will be inferred based on the options provided.
+- `tag` The fully qualified traceql tag to search for. e.g. `resource.service.name`
+- `start` Start of the time range to search: (YYYY-MM-DDThh:mm:ss)
+- `end` End of the time range to search: (YYYY-MM-DDThh:mm:ss)
+
+Options:
+- `--org-id <value>`      Organization ID (for use in multi-tenant setup).
+- `--use-grpc`            Use GRPC streaming
+- `--path-prefix <value>` String to prefix search paths with
+
+{{% admonition type="note" %}}
+Streaming over HTTP requires the `stream_over_http_enabled` flag to be set. For more information, refer to [Tempo GRPC API documentation]({{< relref "../api_docs" >}}).
+{{% /admonition %}}
 
 ## Query blocks command