Skip to content

Commit

Permalink
feat: flagd gherkin suite and flags (#54)
Browse files Browse the repository at this point in the history
Signed-off-by: Todd Baert <[email protected]>
  • Loading branch information
toddbaert authored Sep 7, 2023
1 parent a463f8f commit 81cb78b
Show file tree
Hide file tree
Showing 10 changed files with 334 additions and 51 deletions.
4 changes: 3 additions & 1 deletion .gherkin-lintrc
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@
"Feature": 0,
"Background": 2,
"Scenario": 2,
"Step": 4
"Step": 4,
"Examples": 4,
"example": 6
}
],
"no-trailing-spaces": "on",
Expand Down
25 changes: 24 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,27 @@ See the [flagd docs](https://flagd.dev/) for more information on flagd.
The _sync_-testbed_ container is a docker image built on conforming to the sync.proto - a grpc which flagd or flagd in-process providers can use as a sync-source.
It features an identical set of flags to the [flagd-testbed container](#flagd-testbed-container)

For details on the sync-testbed, see [sync/README.me](sync/README.md)
For details on the sync-testbed, see [sync/README.me](sync/README.md)

## Gherkin test suite

The [gherkin/](gherkin/) dir includes a set of [_gherkin_](https://cucumber.io/docs/gherkin/) tests that define expected behavior associated with the configurations defined in the flagd-testbed (see [flags/](flags/)).
Combined with the _flagd-provider_ for the associated SDK and the flagd-testbed, these comprise a complete integration test suite.

Included suites:

[flagd.feature](gherkin/flagd.feature) includes tests relevant to flagd and all flagd providers:
* default value (zero-value) handling (some proto3 implementations handle these inconsistently).
* basic event handling

[flagd-json-evaluator.feature](gherkin/flagd-json-evaluator.feature) includes tests relevant to flagd and in-process providers:
* custom JSONLogic operators (`starts_with`, `ends_with`, `fractional`, `sem_ver`)
* evaluator reuse via `$ref`


### Lint Gherkin files

The Gherkin files structure can be linted using [gherkin-lint](https://github.com/vsiakka/gherkin-lint). The following commands require Node.js 10 or later.

1. npm install
1. npm run gherkin-lint
12 changes: 9 additions & 3 deletions flagd/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
FROM ghcr.io/open-feature/flagd:v0.6.3 as flagd
# we NEED flagd v0.6.4 as a minimum
FROM ghcr.io/open-feature/flagd:v0.6.4 as flagd

FROM busybox:1.36

COPY --from=flagd /flagd-build /flagd
COPY flags/* .
COPY scripts/* .
LABEL org.opencontainers.image.source = "https://github.com/open-feature/test-harness"
LABEL org.opencontainers.image.source = "https://github.com/open-feature/flagd-testbed"

ENTRYPOINT ["sh", "wrapper.sh", "./flagd", "start", "-f", "file:testing-flags.json", "-f", "file:changing-flag.json"]
ENTRYPOINT ["sh", "wrapper.sh", "./flagd", "start", \
"-f", "file:testing-flags.json", \
"-f", "file:changing-flag.json", \
"-f", "file:custom-ops.json", \
"-f", "file:evaluator-refs.json", \
"-f", "file:zero-flags.json"]
104 changes: 104 additions & 0 deletions flags/custom-ops.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
{
"flags": {
"fractional-flag": {
"state": "ENABLED",
"variants": {
"clubs": "clubs",
"diamonds": "diamonds",
"hearts": "hearts",
"spades": "spades",
"wild": "wild"
},
"defaultVariant": "wild",
"targeting": {
"fractional": [
{ "var": "user.name" },
[ "clubs", 25 ],
[ "diamonds", 25 ],
[ "hearts", 25 ],
[ "spades", 25 ]
]
}
},
"starts-ends-flag": {
"state": "ENABLED",
"variants": {
"prefix": "prefix",
"postfix": "postfix",
"none": "none"
},
"defaultVariant": "none",
"targeting": {
"if": [
{
"starts_with": [{"var": "id"}, "abc"]
},
"prefix", {
"if": [
{
"ends_with": [{"var": "id"}, "xyz"]
},
"postfix", "none"
]
}
]
}
},
"equal-greater-lesser-version-flag": {
"state": "ENABLED",
"variants": {
"equal": "equal",
"greater": "greater",
"lesser": "lesser",
"none": "none"
},
"defaultVariant": "none",
"targeting": {
"if": [
{
"sem_ver": [{"var": "version"}, "=", "2.0.0"]
},
"equal", {
"if": [
{
"sem_ver": [{"var": "version"}, ">", "2.0.0"]
},
"greater", {
"if": [
{
"sem_ver": [{"var": "version"}, "<", "2.0.0"]
},
"lesser", "none"
]
}
]
}
]
}
},
"major-minor-version-flag": {
"state": "ENABLED",
"variants": {
"minor": "minor",
"major": "major",
"none": "none"
},
"defaultVariant": "none",
"targeting": {
"if": [
{
"sem_ver": [{"var": "version"}, "~", "3.0.0"]
},
"minor", {
"if": [
{
"sem_ver": [{"var": "version"}, "^", "3.0.0"]
},
"major", "none"
]
}
]
}
}
}
}
52 changes: 52 additions & 0 deletions flags/evaluator-refs.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
{
"flags": {
"some-email-targeted-flag": {
"state": "ENABLED",
"variants": {
"hi": "hi",
"bye": "bye",
"none": "none"
},
"defaultVariant": "none",
"targeting": {
"if": [
{
"$ref": "is_ballmer"
},
"hi",
"bye"
]
}
},
"some-other-email-targeted-flag": {
"state": "ENABLED",
"variants": {
"yes": "yes",
"no": "no",
"none": "none"
},
"defaultVariant": "none",
"targeting": {
"if": [
{
"$ref": "is_ballmer"
},
"yes",
"no"
]
}
}
},
"$evaluators": {
"is_ballmer": {
"==": [
"[email protected]",
{
"var": [
"email"
]
}
]
}
}
}
44 changes: 0 additions & 44 deletions flags/testing-flags.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,6 @@
},
"defaultVariant": "on"
},
"boolean-flag-copy": {
"state": "ENABLED",
"variants": {
"on": true,
"off": false
},
"defaultVariant": "on"
},
"string-flag": {
"state": "ENABLED",
"variants": {
Expand All @@ -24,14 +16,6 @@
},
"defaultVariant": "greeting"
},
"string-flag-copy": {
"state": "ENABLED",
"variants": {
"greeting": "hi",
"parting": "bye"
},
"defaultVariant": "greeting"
},
"integer-flag": {
"state": "ENABLED",
"variants": {
Expand All @@ -40,14 +24,6 @@
},
"defaultVariant": "ten"
},
"integer-flag-copy": {
"state": "ENABLED",
"variants": {
"one": 1,
"ten": 10
},
"defaultVariant": "ten"
},
"float-flag": {
"state": "ENABLED",
"variants": {
Expand All @@ -56,14 +32,6 @@
},
"defaultVariant": "half"
},
"float-flag-copy": {
"state": "ENABLED",
"variants": {
"tenth": 0.1,
"half": 0.5
},
"defaultVariant": "half"
},
"object-flag": {
"state": "ENABLED",
"variants": {
Expand All @@ -76,18 +44,6 @@
},
"defaultVariant": "template"
},
"object-flag-copy": {
"state": "ENABLED",
"variants": {
"empty": {},
"template": {
"showImages": true,
"title": "Check out these pics!",
"imagesPerPage": 100
}
},
"defaultVariant": "template"
},
"context-aware": {
"state": "ENABLED",
"variants": {
Expand Down
36 changes: 36 additions & 0 deletions flags/zero-flags.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"flags": {
"boolean-zero-flag": {
"state": "ENABLED",
"variants": {
"zero": false,
"non-zero": true
},
"defaultVariant": "zero"
},
"string-zero-flag": {
"state": "ENABLED",
"variants": {
"zero": "",
"non-zero": "str"
},
"defaultVariant": "zero"
},
"integer-zero-flag": {
"state": "ENABLED",
"variants": {
"zero": 0,
"non-zero": 1
},
"defaultVariant": "zero"
},
"float-zero-flag": {
"state": "ENABLED",
"variants": {
"zero": 0.0,
"non-zero": 1.0
},
"defaultVariant": "zero"
}
}
}
63 changes: 63 additions & 0 deletions gherkin/flagd-json-evaluator.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
Feature: flagd json evaluation

# This test suite contains scenarios to test the json-evaluation of flagd and flag-in-process providers.
# It's associated with the flags configured in flags/changing-flag.json, flags/zero-flags.json, flags/custom-ops.json and evaluator-refs.json.
# It should be used in conjunection with the suites supplied by the OpenFeature specification.

Background:
Given a flagd provider is set

# evaluator refs
Scenario Outline: Evaluator reuse
When a string flag with key <key> is evaluated with default value "fallback"
And a context containing a key "email", with value "[email protected]"
Then the returned value should be <value>
Examples:
| key | value |
| some-email-targeted-flag | hi |
| some-other-email-targeted-flag | yes |

# custom operators
Scenario Outline: Fractional operator
When a string flag with key "fractional-flag" is evaluated with default value "fallback"
And a context containing a nested property with outer key "user" and inner key "name", with value <name>
Then the returned value should be <value>
Examples:
| name | value |
| jack | clubs |
| queen | diamonds |
| ace | hearts |
| joker | spades |

Scenario Outline: Substring operators
When a string flag with key "starts-ends-flag" is evaluated with default value "fallback"
And a context containing a key "id", with value <id>
Then the returned value should be <value>
Examples:
| id | value |
| abcdef | prefix |
| uvwxyz | postfix |
| abcxyz | prefix |
| lmnopq | nomatch |

Scenario Outline: Semantic version operator numeric comparision
When a string flag with key "equal-greater-lesser-version-flag" is evaluated with default value "fallback"
And a context containing a key "version", with value <version>
Then the returned value should be <value>
Examples:
| version | value |
| 2.0.0 | equal |
| 2.1.0 | greater |
| 1.9.0 | lesser |
| 2.0.0-alpha | lesser |
| 2.0.0.0 | invalid |

Scenario Outline: Semantic version operator semantic comparision
When a string flag with key "major-minor-version-flag" is evaluated with default value "fallback"
And a context containing a key "version", with value <version>
Then the returned value should be <value>
Examples:
| version | value |
| 3.0.1 | minor |
| 3.1.0 | major |
| 4.0.0 | none |
Loading

0 comments on commit 81cb78b

Please sign in to comment.