diff --git a/README.md b/README.md index b160ff88c..0438f5667 100755 --- a/README.md +++ b/README.md @@ -2252,6 +2252,7 @@ You can adjust configuration settings for the HTTP client used by Karate using t `printEnabled` | boolean | Can be used to suppress the [`print`](#print) output when not in 'dev mode' by setting as `false` (default `true`) `report` | JSON / boolean | see [report verbosity](#report-verbosity) `afterScenario` | JS function | Will be called [after every `Scenario`](#hooks) (or `Example` within a `Scenario Outline`), refer to this example: [`hooks.feature`](karate-demo/src/test/java/demo/hooks/hooks.feature) +`afterScenarioOutline` | JS function | Will be called [after every `Scenario Outline`](#hooks). Is called after the last `afterScenario` for the last scenario in the outline. Refer to this example: [`hooks.feature`](karate-demo/src/test/java/demo/hooks/hooks.feature) `afterFeature` | JS function | Will be called [after every `Feature`](#hooks), refer to this example: [`hooks.feature`](karate-demo/src/test/java/demo/hooks/hooks.feature) `ssl` | boolean | Enable HTTPS calls without needing to configure a trusted certificate or key-store. `ssl` | string | Like above, but force the SSL algorithm to one of [these values](http://docs.oracle.com/javase/8/docs/technotes/guides/security/StandardNames.html#SSLContext). (The above form internally defaults to `TLS` if simply set to `true`). @@ -3690,6 +3691,7 @@ Operation | Description karate.response | returns the last HTTP response as a JS object that enables advanced use-cases such as getting a header ignoring case: `karate.response.header('some-header')` karate.request | returns the last HTTP request as a JS object that enables advanced use-cases such as getting a header ignoring case: `karate.request.header('some-header')`, which works [even in mocks](https://github.com/karatelabs/karate/tree/master/karate-netty#requestheaders) karate.scenario | get metadata about the currently executing `Scenario` (or `Outline` - `Example`) within a test +karate.scenarioOutline | get metadata about the currently executing scenario outline within a test karate.set(name, value) | sets the value of a variable (immediately), which may be needed in case any other routines (such as the [configured headers](#configure-headers)) depend on that variable karate.set(object) | where the single argument is expected to be a `Map` or JSON-like, and will perform the above `karate.set()` operation for all key-value pairs in one-shot karate.set(name, path, value) | only needed when you need to conditionally build payload elements, especially XML. This is best explained via [an example](karate-core/src/test/java/com/intuit/karate/core/xml/xml.feature#L211), and it behaves the same way as the [`set`](#set) keyword. Also see [`eval`](#eval). @@ -4440,6 +4442,7 @@ Before *everything* (or 'globally' once) | See [`karate.callSingle()`](#karateca Before every `Scenario` | Use the [`Background`](#script-structure). Note that [`karate-config.js`](#karate-configjs) is processed before *every* `Scenario` - so you can choose to put "global" config here, for example using [`karate.configure()`](#karate-configure). Once (or at the start of) every `Feature` | Use a [`callonce`](#callonce) in the [`Background`](#script-structure). The advantage is that you can set up variables (using [`def`](#def) if needed) which can be used in all `Scenario`-s within that `Feature`. After every `Scenario` | [`configure afterScenario`](#configure) (see [example](karate-demo/src/test/java/demo/hooks/hooks.feature)) +After every `Scenario Outline` | [`configure afterScenarioOutline`](#configure) (see [example](karate-demo/src/test/java/demo/hooks/hooks.feature)) At the end of the `Feature` | [`configure afterFeature`](#configure) (see [example](karate-demo/src/test/java/demo/hooks/hooks.feature)) > Note that for the `afterFeature` hook to work, you should be using the [`Runner` API](#parallel-execution) and not the JUnit runner. diff --git a/karate-core/src/main/java/com/intuit/karate/RuntimeHook.java b/karate-core/src/main/java/com/intuit/karate/RuntimeHook.java index a0c648d02..83f26a7cb 100644 --- a/karate-core/src/main/java/com/intuit/karate/RuntimeHook.java +++ b/karate-core/src/main/java/com/intuit/karate/RuntimeHook.java @@ -46,6 +46,10 @@ default void afterScenario(ScenarioRuntime sr) { } + default void afterScenarioOutline(ScenarioRuntime sr) { + + } + default boolean beforeFeature(FeatureRuntime fr) { return true; } diff --git a/karate-core/src/main/java/com/intuit/karate/core/AfterHookType.java b/karate-core/src/main/java/com/intuit/karate/core/AfterHookType.java new file mode 100644 index 000000000..3febf64a8 --- /dev/null +++ b/karate-core/src/main/java/com/intuit/karate/core/AfterHookType.java @@ -0,0 +1,45 @@ +/* + * The MIT License + * + * Copyright 2022 Karate Labs Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.intuit.karate.core; + +/** + * + * @author OwenK2 + */ +public enum AfterHookType { + + AFTER_SCENARIO("afterScenario"), + AFTER_OUTLINE("afterScenarioOutline"), + AFTER_FEATURE("afterFeature"); + + private String prefix; + + private AfterHookType(String prefix) { + this.prefix = prefix; + } + + public String getPrefix() { + return prefix; + } +} \ No newline at end of file diff --git a/karate-core/src/main/java/com/intuit/karate/core/Config.java b/karate-core/src/main/java/com/intuit/karate/core/Config.java index 9dbd7488f..3925933dd 100644 --- a/karate-core/src/main/java/com/intuit/karate/core/Config.java +++ b/karate-core/src/main/java/com/intuit/karate/core/Config.java @@ -93,6 +93,7 @@ public class Config { private HttpLogModifier logModifier; private Variable afterScenario = Variable.NULL; + private Variable afterScenarioOutline = Variable.NULL; private Variable afterFeature = Variable.NULL; private Variable headers = Variable.NULL; private Variable cookies = Variable.NULL; @@ -175,6 +176,9 @@ public boolean configure(String key, Variable value) { // TODO use enum case "afterScenario": afterScenario = value; return false; + case "afterScenarioOutline": + afterScenarioOutline = value; + return false; case "afterFeature": afterFeature = value; return false; @@ -382,6 +386,7 @@ public Config(Config parent) { cookies = parent.cookies; responseHeaders = parent.responseHeaders; afterScenario = parent.afterScenario; + afterScenarioOutline = parent.afterScenarioOutline; afterFeature = parent.afterFeature; continueOnStepFailureMethods = parent.continueOnStepFailureMethods; continueAfterContinueOnStepFailure = parent.continueAfterContinueOnStepFailure; @@ -538,6 +543,14 @@ public void setAfterScenario(Variable afterScenario) { this.afterScenario = afterScenario; } + public Variable getAfterScenarioOutline() { + return afterScenarioOutline; + } + + public void setAfterScenarioOutline(Variable afterScenarioOutline) { + this.afterScenarioOutline = afterScenarioOutline; + } + public Variable getAfterFeature() { return afterFeature; } diff --git a/karate-core/src/main/java/com/intuit/karate/core/ExamplesTable.java b/karate-core/src/main/java/com/intuit/karate/core/ExamplesTable.java index 14cdb90bf..92e97c58e 100644 --- a/karate-core/src/main/java/com/intuit/karate/core/ExamplesTable.java +++ b/karate-core/src/main/java/com/intuit/karate/core/ExamplesTable.java @@ -24,7 +24,9 @@ package com.intuit.karate.core; import java.util.ArrayList; +import java.util.HashMap; import java.util.List; +import java.util.Map; /** * @@ -35,6 +37,7 @@ public class ExamplesTable { private final ScenarioOutline outline; private final Table table; private List tags; + public ExamplesTable(ScenarioOutline outline, Table table) { this.outline = outline; @@ -58,4 +61,13 @@ public Table getTable() { return table; } + public Map toKarateJson() { + Map map = new HashMap(); + List tagStrings = new ArrayList(); + tags.forEach(tag -> tagStrings.add(tag.toString())); + map.put("tags", tagStrings); + map.put("data", table.getRowsAsMapsConverted()); + return map; + } + } diff --git a/karate-core/src/main/java/com/intuit/karate/core/FeatureRuntime.java b/karate-core/src/main/java/com/intuit/karate/core/FeatureRuntime.java index d4a5d7638..cf8387c4b 100644 --- a/karate-core/src/main/java/com/intuit/karate/core/FeatureRuntime.java +++ b/karate-core/src/main/java/com/intuit/karate/core/FeatureRuntime.java @@ -195,16 +195,38 @@ private void processScenario(ScenarioRuntime sr) { if (!sr.result.getStepResults().isEmpty()) { synchronized (result) { result.addResult(sr.result); + + // Execute afterScenarioOutline if applicable + // NOTE: Needs to be run after adding result, since result count is used to deterime + // if the scenario is the last in the outline + if (!sr.dryRun && isLastScenarioInOutline(sr.scenario)) { + sr.engine.invokeAfterHookIfConfigured(AfterHookType.AFTER_OUTLINE); + suite.hooks.forEach(h -> h.afterScenarioOutline(sr)); + } } } } } + private boolean isLastScenarioInOutline(Scenario scenario) { + // Check if scenario is part of an outline + if (!scenario.isOutlineExample()) return false; + + // Count the number of completed scenarios with the same section ID (in same outline) + int completedScenarios = 0; + for (ScenarioResult result : result.getScenarioResults()) { + if (result.getScenario().getSection().getIndex() == scenario.getSection().getIndex()) { + completedScenarios++; + } + } + return completedScenarios == scenario.getSection().getScenarioOutline().getNumScenarios(); + } + // extracted for junit5 public synchronized void afterFeature() { result.sortScenarioResults(); if (lastExecutedScenario != null) { - lastExecutedScenario.engine.invokeAfterHookIfConfigured(true); + lastExecutedScenario.engine.invokeAfterHookIfConfigured(AfterHookType.AFTER_FEATURE); result.setVariables(lastExecutedScenario.engine.getAllVariablesAsMap()); result.setConfig(lastExecutedScenario.engine.getConfig()); } diff --git a/karate-core/src/main/java/com/intuit/karate/core/ScenarioBridge.java b/karate-core/src/main/java/com/intuit/karate/core/ScenarioBridge.java index 19d3a4dd6..f793d5cf7 100644 --- a/karate-core/src/main/java/com/intuit/karate/core/ScenarioBridge.java +++ b/karate-core/src/main/java/com/intuit/karate/core/ScenarioBridge.java @@ -551,6 +551,10 @@ public Object getScenario() { return new JsMap(getEngine().runtime.result.toKarateJson()); } + public Object getScenarioOutline() { + return new JsMap(getEngine().runtime.outlineResult.toKarateJson()); + } + public Object getTags() { return JsValue.fromJava(getEngine().runtime.tags.getTags()); } diff --git a/karate-core/src/main/java/com/intuit/karate/core/ScenarioEngine.java b/karate-core/src/main/java/com/intuit/karate/core/ScenarioEngine.java index 24c18be84..ebd9c22f0 100644 --- a/karate-core/src/main/java/com/intuit/karate/core/ScenarioEngine.java +++ b/karate-core/src/main/java/com/intuit/karate/core/ScenarioEngine.java @@ -235,20 +235,35 @@ public void print(String exp) { evalJs("karate.log('[print]'," + exp + ")"); } - public void invokeAfterHookIfConfigured(boolean afterFeature) { + public void invokeAfterHookIfConfigured(AfterHookType hookType) { + // Do not call hooks on "called" scenarios/features if (runtime.caller.depth > 0) { return; } - Variable v = afterFeature ? config.getAfterFeature() : config.getAfterScenario(); + + // Get hook variable based on type + Variable v; + switch (hookType) { + case AFTER_SCENARIO: + v = config.getAfterScenario(); + break; + case AFTER_OUTLINE: + v = config.getAfterScenarioOutline(); + break; + case AFTER_FEATURE: + v = config.getAfterFeature(); + break; + default: return; + } + if (v.isJsOrJavaFunction()) { - if (afterFeature) { + if (hookType == AfterHookType.AFTER_FEATURE) { ScenarioEngine.set(this); // for any bridge / js to work } try { executeFunction(v); } catch (Exception e) { - String prefix = afterFeature ? "afterFeature" : "afterScenario"; - logger.warn("{} hook failed: {}", prefix, e + ""); + logger.warn("{} hook failed: {}", hookType.getPrefix(), e + ""); } } } diff --git a/karate-core/src/main/java/com/intuit/karate/core/ScenarioOutline.java b/karate-core/src/main/java/com/intuit/karate/core/ScenarioOutline.java index fb5e3ac5d..cfe376090 100644 --- a/karate-core/src/main/java/com/intuit/karate/core/ScenarioOutline.java +++ b/karate-core/src/main/java/com/intuit/karate/core/ScenarioOutline.java @@ -25,6 +25,7 @@ import java.util.ArrayList; import java.util.List; +import java.util.Map; /** * @author pthomas3 @@ -40,6 +41,7 @@ public class ScenarioOutline { private String description; private List steps; private List examplesTables; + private int numScenarios = 0; public ScenarioOutline(Feature feature, FeatureSection section) { this.feature = feature; @@ -75,6 +77,7 @@ public Scenario toScenario(String dynamicExpression, int exampleIndex, int updat step.setTable(original.getTable()); step.setComments(original.getComments()); } + numScenarios++; return s; } @@ -167,9 +170,23 @@ public void setSteps(List steps) { public List getExamplesTables() { return examplesTables; } + + public int getNumExampleTables() { + return examplesTables.size(); + } + + public List> getAllExampleData() { + List> exampleData = new ArrayList(); + examplesTables.forEach(table -> exampleData.add(table.toKarateJson())); + return exampleData; + } public void setExamplesTables(List examplesTables) { this.examplesTables = examplesTables; } + + public int getNumScenarios() { + return numScenarios; + } } diff --git a/karate-core/src/main/java/com/intuit/karate/core/ScenarioOutlineResult.java b/karate-core/src/main/java/com/intuit/karate/core/ScenarioOutlineResult.java new file mode 100644 index 000000000..50833b9fe --- /dev/null +++ b/karate-core/src/main/java/com/intuit/karate/core/ScenarioOutlineResult.java @@ -0,0 +1,82 @@ +/* + * The MIT License + * + * Copyright 2022 Karate Labs Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.intuit.karate.core; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +/** + * + * @author OwenK2 + */ +public class ScenarioOutlineResult { + + final private ScenarioOutline scenarioOutline; + final private ScenarioRuntime runtime; + + public ScenarioOutlineResult(ScenarioOutline scenarioOutline, ScenarioRuntime runtime) { + // NOTE: this value can be null, in which case the scenario is not from an outline + this.scenarioOutline = scenarioOutline; + this.runtime = runtime; + } + + public Map toKarateJson() { + if (scenarioOutline == null) return null; + Map map = new HashMap(); + map.put("name", scenarioOutline.getName()); + map.put("description", scenarioOutline.getDescription()); + map.put("line", scenarioOutline.getLine()); + map.put("sectionIndex", scenarioOutline.getSection().getIndex()); + map.put("exampleTableCount", scenarioOutline.getNumExampleTables()); + map.put("exampleTables", scenarioOutline.getAllExampleData()); + map.put("numScenariosToExecute", scenarioOutline.getNumScenarios()); + + // Get results of other examples in this outline + List> scenarioResults = new ArrayList(); + if (runtime.featureRuntime != null && runtime.featureRuntime.result != null) { + // Add all past results + boolean needToAddRecent = runtime.result != null; + for(ScenarioResult result : runtime.featureRuntime.result.getScenarioResults()) { + if (result.getScenario().getSection().getIndex() == scenarioOutline.getSection().getIndex()) { + scenarioResults.add(result.toInfoJson()); + if(result.equals(runtime.result)) { + needToAddRecent = false; + } + } + } + + // Add most recent result if we haven't already (and it's not null) + if (needToAddRecent) { + scenarioResults.add(runtime.result.toInfoJson()); + } + } + map.put("scenarioResults", scenarioResults); + map.put("numScenariosExecuted", scenarioResults.size()); + + return map; + } + +} diff --git a/karate-core/src/main/java/com/intuit/karate/core/ScenarioResult.java b/karate-core/src/main/java/com/intuit/karate/core/ScenarioResult.java index 993496caa..1a85043c4 100644 --- a/karate-core/src/main/java/com/intuit/karate/core/ScenarioResult.java +++ b/karate-core/src/main/java/com/intuit/karate/core/ScenarioResult.java @@ -61,6 +61,20 @@ public int compareTo(ScenarioResult sr) { return scenario.getExampleIndex() - sr.scenario.getExampleIndex(); } + @Override + public boolean equals(Object obj) { + if (this == obj) { + return true; + } + if (obj == null) { + return false; + } + if (getClass() != obj.getClass()) { + return false; + } + return compareTo((ScenarioResult)obj) == 0; + } + public String getFailureMessageForDisplay() { if (failedStep == null) { return null; @@ -213,6 +227,27 @@ public Map toKarateJson() { return map; } + // Paired down information for use in karate.scenarioOutline + public Map toInfoJson() { + Map map = new HashMap(); + map.put("durationMillis", getDurationMillis()); + List tags = scenario.getTagsEffective().getTags(); + if (tags != null && !tags.isEmpty()) { + map.put("tags", tags); + } + map.put("failed", isFailed()); + map.put("refId", scenario.getRefId()); + map.put("sectionIndex", scenario.getSection().getIndex()); + map.put("exampleIndex", scenario.getExampleIndex()); + map.put("name", scenario.getName()); + map.put("description", scenario.getDescription()); + map.put("line", scenario.getLine()); + map.put("executorName", executorName); + map.put("startTime", startTime); + map.put("endTime", endTime); + return map; + } + public Map toCucumberJson() { Map map = new HashMap(); map.put("name", scenario.getName()); diff --git a/karate-core/src/main/java/com/intuit/karate/core/ScenarioRuntime.java b/karate-core/src/main/java/com/intuit/karate/core/ScenarioRuntime.java index 407dda37f..096a6f330 100644 --- a/karate-core/src/main/java/com/intuit/karate/core/ScenarioRuntime.java +++ b/karate-core/src/main/java/com/intuit/karate/core/ScenarioRuntime.java @@ -53,6 +53,7 @@ public class ScenarioRuntime implements Runnable { public final Tags tags; public final ScenarioActions actions; public final ScenarioResult result; + public final ScenarioOutlineResult outlineResult; public final ScenarioEngine engine; public final boolean reportDisabled; public final Map magicVariables; @@ -91,6 +92,7 @@ public ScenarioRuntime(FeatureRuntime featureRuntime, Scenario scenario) { magicVariables = initMagicVariables(); } result = new ScenarioResult(scenario); + outlineResult = new ScenarioOutlineResult(scenario.getSection().getScenarioOutline(), this); if (featureRuntime.setupResult != null) { // TODO improve this and simplify report rendering code in report/karate-feature.html StepResult sr = result.addFakeStepResult("@setup", null); @@ -503,7 +505,7 @@ public void afterRun() { currentStepResult = result.addFakeStepResult("no steps executed", null); } if (!dryRun) { - engine.invokeAfterHookIfConfigured(false); + engine.invokeAfterHookIfConfigured(AfterHookType.AFTER_SCENARIO); featureRuntime.suite.hooks.forEach(h -> h.afterScenario(this)); engine.stop(currentStepResult); } diff --git a/karate-core/src/main/java/com/intuit/karate/core/Table.java b/karate-core/src/main/java/com/intuit/karate/core/Table.java index 0d2c543d8..85375a13c 100644 --- a/karate-core/src/main/java/com/intuit/karate/core/Table.java +++ b/karate-core/src/main/java/com/intuit/karate/core/Table.java @@ -175,6 +175,20 @@ public List> getRowsAsMaps() { return list; } + public List> getRowsAsMapsConverted() { + int rowCount = rows.size(); + List> list = new ArrayList(rowCount - 1); + for (int i = 1; i < rowCount; i++) { // don't include header row + Map map = new LinkedHashMap(cols.size()); + list.add(map); + List row = rows.get(i); + for (Column col : cols) { + map.put(col.key, convert(row.get(col.index), col)); + } + } + return list; + } + private static Object convert(String raw, Column col) { try { switch (col.type) { diff --git a/karate-core/src/test/java/com/intuit/karate/core/ScenarioOutlineResultTest.java b/karate-core/src/test/java/com/intuit/karate/core/ScenarioOutlineResultTest.java new file mode 100644 index 000000000..8119f67bb --- /dev/null +++ b/karate-core/src/test/java/com/intuit/karate/core/ScenarioOutlineResultTest.java @@ -0,0 +1,29 @@ +package com.intuit.karate.core; + +import com.intuit.karate.FileUtils; +import com.intuit.karate.TestUtils; +import static com.intuit.karate.TestUtils.*; +import com.intuit.karate.report.Report; +import com.intuit.karate.report.SuiteReports; +import java.io.File; +import java.util.Map; +import static org.junit.jupiter.api.Assertions.*; +import org.junit.jupiter.api.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * + * @author OwenK2 + */ +class ScenarioOutlineResultTest { + + static final Logger logger = LoggerFactory.getLogger(ScenarioOutlineResultTest.class); + + @Test + void testJsonConversion() { + FeatureRuntime fr = TestUtils.runFeature("classpath:com/intuit/karate/core/scenario-outline-result.feature"); + assertFalse(fr.result.isFailed()); + } + +} diff --git a/karate-core/src/test/java/com/intuit/karate/core/feature-result-cucumber.json b/karate-core/src/test/java/com/intuit/karate/core/feature-result-cucumber.json index fcd25f4de..7be40d484 100644 --- a/karate-core/src/test/java/com/intuit/karate/core/feature-result-cucumber.json +++ b/karate-core/src/test/java/com/intuit/karate/core/feature-result-cucumber.json @@ -305,6 +305,70 @@ "line": 1 } ] + }, + { + "line": 5, + "name": "", + "description": "", + "type": "background", + "keyword": "Background", + "steps": [ + { + "name": "print 'in background'", + "result": { + "duration": "#number", + "status": "passed" + }, + "match": { + "location": "karate", + "arguments": [ + ] + }, + "keyword": "*", + "line": 6, + "doc_string": { + "content_type": "", + "value": "#string", + "line": 6 + } + } + ] + }, + { + "line": 26, + "name": "hello baz", + "description": "", + "id": "hello-baz", + "type": "scenario", + "start_timestamp": "#string", + "keyword": "Scenario Outline", + "steps": [ + { + "name": "print 'name:', name", + "result": { + "duration": "#number", + "status": "passed" + }, + "match": { + "location": "karate", + "arguments": [ + ] + }, + "keyword": "*", + "line": 17, + "doc_string": { + "content_type": "", + "value": "#string", + "line": 17 + } + } + ], + "tags": [ + { + "name": "@one", + "line": 1 + } + ] } ], "name": "com\/intuit\/karate\/core\/feature-result.feature", diff --git a/karate-core/src/test/java/com/intuit/karate/core/feature-result.feature b/karate-core/src/test/java/com/intuit/karate/core/feature-result.feature index 74798aeda..cc6db4cda 100644 --- a/karate-core/src/test/java/com/intuit/karate/core/feature-result.feature +++ b/karate-core/src/test/java/com/intuit/karate/core/feature-result.feature @@ -20,3 +20,7 @@ Examples: | name | | foo | | bar | + +Examples: +| name | +| baz | \ No newline at end of file diff --git a/karate-core/src/test/java/com/intuit/karate/core/feature-result.json b/karate-core/src/test/java/com/intuit/karate/core/feature-result.json index 863ff5cad..ff8e81259 100644 --- a/karate-core/src/test/java/com/intuit/karate/core/feature-result.json +++ b/karate-core/src/test/java/com/intuit/karate/core/feature-result.json @@ -280,6 +280,60 @@ "exampleData": { "name": "bar" } + }, + { + "stepResults": [ + { + "result": { + "nanos": "#number", + "millis": "#number", + "status": "passed", + "startTime": "#number", + "endTime": "#number" + }, + "step": { + "background": true, + "line": 6, + "prefix": "*", + "index": 0, + "text": "print 'in background'" + }, + "stepLog": "#string" + }, + { + "result": { + "nanos": "#number", + "millis": "#number", + "status": "passed", + "startTime": "#number", + "endTime": "#number" + }, + "step": { + "line": 17, + "prefix": "*", + "index": 0, + "text": "print 'name:', name" + }, + "stepLog": "#string" + } + ], + "executorName": "#string", + "name": "hello baz", + "description": "", + "line": 26, + "sectionIndex": 1, + "startTime": "#number", + "endTime": "#number", + "exampleIndex": 0, + "refId": "[2.1:26]", + "durationMillis": "#number", + "failed": false, + "tags": [ + "one" + ], + "exampleData": { + "name": "baz" + } } ], "prefixedPath": "classpath:com\/intuit\/karate\/core\/feature-result.feature", @@ -289,7 +343,7 @@ "description": "my description", "durationMillis": "#number", "resultDate": "#string", - "passedCount": 3, + "passedCount": 4, "failedCount": 0, "callDepth": 0, "loopIndex": -1 diff --git a/karate-core/src/test/java/com/intuit/karate/core/parallel/parallel-outline-1.feature b/karate-core/src/test/java/com/intuit/karate/core/parallel/parallel-outline-1.feature index 225ec6165..c1ed42389 100644 --- a/karate-core/src/test/java/com/intuit/karate/core/parallel/parallel-outline-1.feature +++ b/karate-core/src/test/java/com/intuit/karate/core/parallel/parallel-outline-1.feature @@ -13,6 +13,12 @@ Background: function fn() { console.log('afterFeature'); } + """ + * configure afterScenarioOutline = + """ + function fn() { + console.log('afterScenarioOutline'); + } """ * configure afterScenario = """ diff --git a/karate-core/src/test/java/com/intuit/karate/core/scenario-outline-result.feature b/karate-core/src/test/java/com/intuit/karate/core/scenario-outline-result.feature new file mode 100644 index 000000000..72c2e05d6 --- /dev/null +++ b/karate-core/src/test/java/com/intuit/karate/core/scenario-outline-result.feature @@ -0,0 +1,54 @@ +Feature: my feature + +Scenario: single scenario + * match karate.scenarioOutline == null + +Scenario Outline: outline + description of outline + + # Confirm example index within table + * match __num == + * match karate.scenario.exampleIndex == + + # Confirm scenarioOutline result + * match karate.scenarioOutline == + """ + { + "sectionIndex": 1, + "numScenariosToExecute": 4, + "exampleTableCount": 2, + "line": 6, + "name": "outline", + "numScenariosExecuted": , + "description": "description of outline", + "exampleTables": [ + { + "data": [ + {num: 0, executedSoFar: 1}, + {num: 1, executedSoFar: 2}, + ], + "tags": ["@one", "@two"] + }, + { + "data": [ + {num: 0, executedSoFar: 3}, + {num: 1, executedSoFar: 4}, + ], + "tags": ["@three", "@four"] + }, + ], + "scenarioResults": "#[] #object" + } + """ + + @one @two + Examples: + | num! | executedSoFar! | + | 0 | 1 | + | 1 | 2 | + + @three @four + Examples: + | num! | executedSoFar! | + | 0 | 3 | + | 1 | 4 | \ No newline at end of file diff --git a/karate-demo/src/test/java/demo/hooks/called.feature b/karate-demo/src/test/java/demo/hooks/called.feature index aabbbe373..7c67ac900 100644 --- a/karate-demo/src/test/java/demo/hooks/called.feature +++ b/karate-demo/src/test/java/demo/hooks/called.feature @@ -2,7 +2,7 @@ Feature: Background: -# 'afterScenario' and 'afterFeature' are NOT supported when a feature is called +# 'afterScenario', 'afterScenarioOutline', and 'afterFeature' are NOT supported when a feature is called # so this will have no effect, UNLESS this feature is run directly * configure afterScenario = function(){ karate.log('end called scenario') } diff --git a/karate-demo/src/test/java/demo/hooks/hooks.feature b/karate-demo/src/test/java/demo/hooks/hooks.feature index 70d847e4f..12e01269a 100644 --- a/karate-demo/src/test/java/demo/hooks/hooks.feature +++ b/karate-demo/src/test/java/demo/hooks/hooks.feature @@ -1,5 +1,5 @@ Feature: demo karate's equivalent of before and after hooks - note that 'afterScenario' / 'afterFeature' if set up using 'configure' + note that 'afterScenario' / 'afterScenarioOutline' / 'afterFeature' if set up using 'configure' is not supported within features invoked using the 'call' or 'callonce' keywords Background: @@ -28,6 +28,17 @@ function(){ * configure afterFeature = function(){ karate.call('after-feature.feature'); } +# Only runs at the end of a scenario outline after all examples have been run +# This hook will be called after the last scenario in the outline is executed +# It will also run after any configured 'afterScenario's for that outline +# NOTE if using parallel, last Scenario executed may not be the last example in the outline +* configure afterScenarioOutline = +""" +function(){ + karate.log('after scenario outline:', karate.scenarioOutline.name); +} +""" + Scenario: first * print foo