-
Notifications
You must be signed in to change notification settings - Fork 77
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore(docs/examples): Adding GraphQL information #4001
Merged
Merged
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
--- | ||
id: creating-tests-graphql | ||
title: Defining GraphQL Tests as Text Files | ||
description: Tracetest enables developers to define tests as text files and run them using a CLI. Integrate the execution of tests in your existing CI pipeline. | ||
keywords: | ||
- tracetest | ||
- trace-based testing | ||
- observability | ||
- distributed tracing | ||
- testing | ||
- graphql | ||
image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg | ||
--- | ||
|
||
When defining a GraphQL trigger, you are required to define a `graphql` object containing the request Tracetest will send to your system. You also define: `body` with `query`, `variables` and `operationName`, `url`, `headers`, and `SSLVerification`, and `schema`. | ||
|
||
```yaml | ||
trigger: | ||
type: graphql | ||
graphql: | ||
url: ${env:GRAPHQL_URL} | ||
headers: | ||
- key: Content-Type | ||
value: application/json | ||
body: | ||
# path to the query file or the query itself | ||
query: ./query.graphql | ||
variables: {} | ||
operationName: "" | ||
sslVerification: false | ||
# path to the schema file or the schema itself | ||
schema: ./schema.graphql | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
319 changes: 319 additions & 0 deletions
319
...cs/examples-tutorials/recipes/running-tests-with-tracetest-graphql-pokeshop.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,319 @@ | ||||||
--- | ||||||
id: running-tests-with-tracetest-graphql-pokeshop | ||||||
title: Trace-Based Tests with the Tracetest GrapqhQL Trigger | ||||||
description: Quickstart on how to create Trace-Based Tests with the Tracetest GraphQL Trigger | ||||||
hide_table_of_contents: false | ||||||
keywords: | ||||||
- tracetest | ||||||
- trace-based testing | ||||||
- observability | ||||||
- distributed tracing | ||||||
- end-to-end testing | ||||||
- tracetest | ||||||
- graphql | ||||||
- trace-based-testing | ||||||
- trigger | ||||||
image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg | ||||||
--- | ||||||
|
||||||
:::info Version Compatibility | ||||||
The features described here are compatible with the [Tracetest CLI v1.5.2](https://github.com/kubeshop/tracetest/releases/tag/v1.5.2) and above. | ||||||
::: | ||||||
|
||||||
:::note | ||||||
[Check out the source code on GitHub here.](https://github.com/kubeshop/tracetest/tree/main/examples/tracetest-jaeger-graphql-pokeshop) | ||||||
::: | ||||||
|
||||||
[Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that permits you to test your distributed application. It allows you to use the trace data generated by your OpenTelemetry tools to check and assert if your application has the desired behavior defined by your test definitions. | ||||||
[GraphQL](https://graphql.org/) is a query language for APIs and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandable description of the data in your API, gives clients the power to ask for exactly what they need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools. | ||||||
|
||||||
## Why is this important? | ||||||
|
||||||
The Tracetest GraphQL trigger enables you to unleash the power of the trace-based testing to easily capture a full distributed trace from your OpenTelemetry instrumented GraphQL back-end system. | ||||||
|
||||||
By creating a Tracetest GraphQL test, you will be able to create trace-based assertions to be applied across the entire flow like any other Tracetest test. Not only that but it allows you to mix and match it with your existing Monitors, Test Suites and CI/CD validations. | ||||||
|
||||||
Another big benefit of using traces as test specs is that you can: | ||||||
|
||||||
- Get faster MTTR for failing performance tests | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
- Assert against the Mutiple Queries and Mutations at once from a single test execution | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
- Validate functionality of other parts of your system that may be broken, even when the initial request is passing | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
## Requirements | ||||||
|
||||||
**Tracetest Account**: | ||||||
|
||||||
- Sign up to [`app.tracetest.io`](https://app.tracetest.io) or follow the [get started](/getting-started/installation) docs. | ||||||
- Have access to the environment's [agent API key](https://app.tracetest.io/retrieve-token). | ||||||
|
||||||
**Docker**: Have [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) installed on your machine. | ||||||
|
||||||
## Run This Example | ||||||
|
||||||
The example below is provided as part of the Tracetest GitHub repo. You can download and run the example by following these steps: | ||||||
|
||||||
```bash | ||||||
git clone https://github.com/kubeshop/tracetest | ||||||
cd tracetest/examples/tracetest-jaeger-graphql-pokeshop | ||||||
``` | ||||||
|
||||||
Follow these instructions to run the quick start: | ||||||
|
||||||
1. Copy the `.env.template` file to `.env`. | ||||||
2. Fill out the [TRACETEST_TOKEN and ENVIRONMENT_ID](https://app.tracetest.io/retrieve-token) details by editing your `.env` file. | ||||||
3. Run `docker compose run tracetest-run`. | ||||||
4. Follow the links in the output to view the test results. | ||||||
|
||||||
## Project Structure | ||||||
|
||||||
The project structure for running Tracetest GraphQL tests is as follows: | ||||||
|
||||||
```bash | ||||||
.env.template | ||||||
.gitignore | ||||||
.Dockerfile | ||||||
collector.config.yaml | ||||||
docker-compose.yaml | ||||||
/resources | ||||||
apply.sh | ||||||
datastore.yaml | ||||||
test.yaml | ||||||
run.sh | ||||||
scheme.graphql | ||||||
query.graphql | ||||||
``` | ||||||
|
||||||
The [Pokeshop Demo App](/live-examples/pokeshop/overview) is a complete example of a distributed application using different back-end and front-end services. We will be launching it and running tests against it as part of this example. | ||||||
The `docker-compose.yaml` file in the root directory of the quick start runs the Pokeshop Demo app, the OpenTelemetry Collector, Jaeger, and the [Tracetest Agent](/concepts/agent) setup. | ||||||
|
||||||
The Tracetest resource definitions and scripts are defined under the `/resources` directory. The resources include tests and the tracing backend definition, while the scripts include the `apply.sh` and `run.sh` scripts to apply the resources and run the tests. | ||||||
|
||||||
## Provisioned Resources | ||||||
|
||||||
The example provisions the following resources: | ||||||
|
||||||
### Import Pokemon Test | ||||||
|
||||||
```yaml title="resources/test.yaml" | ||||||
type: Test | ||||||
spec: | ||||||
id: re9XOxqSR | ||||||
name: Pokeshop - Import | ||||||
trigger: | ||||||
type: graphql | ||||||
graphql: | ||||||
url: http://demo-api:8081/graphql | ||||||
headers: | ||||||
- key: Content-Type | ||||||
value: application/json | ||||||
auth: | ||||||
apiKey: {} | ||||||
basic: {} | ||||||
bearer: {} | ||||||
body: | ||||||
query: ./query.graphql | ||||||
variables: {} | ||||||
operationName: "" | ||||||
sslVerification: false | ||||||
schema: ./schema.graphql | ||||||
specs: | ||||||
- name: Import Pokemon Span Exists | ||||||
selector: span[tracetest.span.type="general" name="import pokemon"] | ||||||
assertions: | ||||||
- attr:tracetest.selected_spans.count = 1 | ||||||
- name: Uses Correct PokemonId | ||||||
selector: span[tracetest.span.type="http" name="GET" http.method="GET"] | ||||||
assertions: | ||||||
- attr:http.url = "https://pokeapi.co/api/v2/pokemon/6" | ||||||
- name: Matching db result with the Pokemon Name | ||||||
selector: span[tracetest.span.type="database" name="create postgres.pokemon"]:first | ||||||
assertions: | ||||||
- attr:db.result contains "charizard" | ||||||
``` | ||||||
|
||||||
### GraphQL | ||||||
|
||||||
```graphql title="resources/query.graphql" | ||||||
mutation import { | ||||||
importPokemon(id: 6) { | ||||||
id | ||||||
} | ||||||
} | ||||||
``` | ||||||
|
||||||
### GraphQL Schema | ||||||
|
||||||
```graphql title="resources/schema.graphql" | ||||||
schema { | ||||||
query: Query | ||||||
mutation: Mutation | ||||||
} | ||||||
|
||||||
type Pokemon { | ||||||
id: Int | ||||||
name: String! | ||||||
type: String! | ||||||
isFeatured: Boolean! | ||||||
imageUrl: String | ||||||
} | ||||||
|
||||||
"The `Int` scalar type represents non-fractional signed whole numeric values. Int can represent values between -(2^31) and 2^31 - 1." | ||||||
scalar Int | ||||||
|
||||||
"The `String` scalar type represents textual data, represented as UTF-8 character sequences. The String type is most often used by GraphQL to represent free-form human-readable text." | ||||||
scalar String | ||||||
|
||||||
"The `Boolean` scalar type represents `true` or `false`." | ||||||
scalar Boolean | ||||||
|
||||||
type PokemonList { | ||||||
items: [Pokemon] | ||||||
totalCount: Int | ||||||
} | ||||||
|
||||||
type ImportPokemon { | ||||||
id: Int! | ||||||
} | ||||||
|
||||||
type Query { | ||||||
getPokemonList(where: String, skip: Int, take: Int): PokemonList | ||||||
} | ||||||
|
||||||
type Mutation { | ||||||
createPokemon(name: String!, type: String!, isFeatured: Boolean!, imageUrl: String): Pokemon! | ||||||
importPokemon(id: Int!): ImportPokemon! | ||||||
} | ||||||
// ...See more in the schema.graphql file | ||||||
``` | ||||||
|
||||||
### Jaeger Tracing Backend | ||||||
|
||||||
```yaml title="resources/datastore.yaml" | ||||||
type: DataStore | ||||||
spec: | ||||||
id: current | ||||||
name: jaeger | ||||||
type: jaeger | ||||||
default: true | ||||||
jaeger: | ||||||
endpoint: jaeger:16685 | ||||||
headers: | ||||||
"": "" | ||||||
tls: | ||||||
insecure: true | ||||||
``` | ||||||
|
||||||
### The Apply Script | ||||||
|
||||||
The apply script configures and provisions the resources in the Tracetest environment: | ||||||
|
||||||
```bash title="resources/apply.sh" | ||||||
#!/bin/sh | ||||||
|
||||||
set -e | ||||||
|
||||||
TOKEN=$TRACETEST_API_KEY | ||||||
ENVIRONMENT_ID=$TRACETEST_ENVIRONMENT_ID | ||||||
|
||||||
apply() { | ||||||
echo "Configuring TraceTest" | ||||||
tracetest configure --token $TOKEN --environment $ENVIRONMENT_ID | ||||||
|
||||||
echo "Applying Resources" | ||||||
tracetest apply datastore -f /resources/datastore.yaml | ||||||
tracetest apply test -f /resources/test.yaml | ||||||
} | ||||||
|
||||||
apply | ||||||
``` | ||||||
|
||||||
### The Run Script | ||||||
|
||||||
The run script runs the test suite against the provisioned resources: | ||||||
|
||||||
```bash title="resources/run.sh" | ||||||
#!/bin/sh | ||||||
|
||||||
set -e | ||||||
|
||||||
TOKEN=$TRACETEST_API_KEY | ||||||
ENVIRONMENT_ID=$TRACETEST_ENVIRONMENT_ID | ||||||
|
||||||
run() { | ||||||
echo "Configuring Tracetest" | ||||||
tracetest configure --token $TOKEN --environment $ENVIRONMENT_ID | ||||||
|
||||||
echo "Running Trace-Based Tests..." | ||||||
tracetest run test -f /resources/test.yaml | ||||||
} | ||||||
|
||||||
run | ||||||
``` | ||||||
|
||||||
## Setting the Environment Variables | ||||||
|
||||||
Copy the `.env.template` file to `.env` and add the Tracetest API token and agent tokens to the `TRACETEST_API_TOKEN` and `TRACETEST_ENVIRONMENT_ID` variables. | ||||||
|
||||||
## Running the Full Example | ||||||
|
||||||
Everything is automated for you to only run the following command: | ||||||
|
||||||
```bash | ||||||
docker compose run tracetest-run | ||||||
``` | ||||||
|
||||||
This command will run the `apply.sh` script to provision the resources and the `run.sh` script to run the test suite. | ||||||
|
||||||
## Finding the Results | ||||||
|
||||||
The output from the Tracetest Engine script should be visible in the console log after running the test command. | ||||||
|
||||||
```bash title=Output | ||||||
WARN[0000] /Users/oscar/Documents/kubeshop/t/examples/tracetest-jaeger-graphql-pokeshop/docker-compose.yaml: `version` is obsolete | ||||||
[+] Running 2/2 | ||||||
✔ demo-api Pulled 0.9s | ||||||
✔ demo-worker Pulled 0.9s | ||||||
[+] Creating 10/9 | ||||||
✔ Network tracetest-jaeger-graphql-pokeshop_default Created 0.0s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-postgres-1 Created 0.1s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-cache-1 Created 0.1s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-tracetest-agent-1 Created 0.1s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-queue-1 Created 0.1s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-jaeger-1 Created 0.1s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-demo-worker-1 Created 0.1s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-otel-collector-1 Created 0.1s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-demo-api-1 Created 0.1s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-tracetest-apply-1 Created 0.0s | ||||||
[+] Running 9/9 | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-cache-1 Healthy 10.5s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-tracetest-agent-1 Started 0.3s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-jaeger-1 Healthy 1.9s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-postgres-1 Healthy 10.5s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-queue-1 Healthy 10.5s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-otel-collector-1 Started 0.1s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-demo-worker-1 Started 0.1s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-demo-api-1 Started 0.1s | ||||||
✔ Container tracetest-jaeger-graphql-pokeshop-tracetest-apply-1 Started 0.1s | ||||||
[+] Running 2/2 | ||||||
✔ demo-api Pulled 0.8s | ||||||
✔ demo-worker Pulled 0.8s | ||||||
Configuring Tracetest | ||||||
SUCCESS Successfully configured Tracetest CLI | ||||||
Running Trace-Based Tests... | ||||||
✔ RunGroup: #US5klbqSR (https://app-stage.tracetest.io/organizations/ttorg_c71a6b53c3709e95/environments/ttenv_bcf29b43f06a12dc/run/US5klbqSR) | ||||||
Summary: 1 passed, 0 failed, 0 pending | ||||||
✔ Pokeshop - Import (https://app-stage.tracetest.io/organizations/ttorg_c71a6b53c3709e95/environments/ttenv_bcf29b43f06a12dc/test/re9XOxqSR/run/11/test) - trace id: 6facf84ee23757eda97930c16fd1d8f9 | ||||||
✔ Import Pokemon Span Exists | ||||||
✔ Uses Correct PokemonId | ||||||
✔ Matching db result with the Pokemon Name | ||||||
``` | ||||||
|
||||||
## What's Next? | ||||||
|
||||||
After running the test, you can click the run link, update the assertions, and run the scripts once more. This flow enables complete a trace-based TDD flow. | ||||||
|
||||||
![assertions](../img/playwright-engine.gif) | ||||||
|
||||||
## Learn More | ||||||
|
||||||
Please visit our [examples in GitHub](https://github.com/kubeshop/tracetest/tree/main/examples) and join our [Slack Community](https://dub.sh/tracetest-community) for more info! |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.