[Epic] Create testing package for WebDriverIO

[StephenEsser]

Feature Request

Create a new WebDriverIO testing package using WDIO 6.

This issue will be treated as an Epic to be closed after the initial release of the package. There will be multiple issues created to move the enhancements into smaller pieces.

Description

The existing WebDriver testing infrastructure is using WDIO 4 and an unmaintained visual regression service. The new package should utilize the latest available WebDriverIO and accessibility (axe-core) packages and carry forward the necessary customization options.

An upgrade guide and transition strategy should be included with these changes.

Notes

This section will be used to document various conversations that happen during development.

When running on CI the maxInstances for the WDIO configuration needs to be set to one.

Breaking Changes

• WDIO 6 has several breaking changes for global/browser command execution
• The selenium docker images have been updated to the latest available
• The timezone is now going to default to the zone provided by the docker image. See https://github.com/cerner/terra-toolkit-boneyard/issues/7 for additional details. Related discussion can be found here.
• Axe core has been updated to the latest available version which introduced a few bug fixes that may break existing tests
• Removed the Terra.it commands
• Removed the global browser.axe command

Additions

• The Selenium Docker Service accepts a configurable version of selenium to test against various images.

Subtasks

• Initialize terra-functional-testing package
• Add Initial WDIO 6 test runner
• Add Docker Service
• Add Axe Integration
• Add Asset Server Service
• Add documentation for each of the available testing environment variable overrides - Stephen Esser
• Add Terra WDIO Reporter - Stephen Esser
• Add Terra Accessibility Reporter - Stephen Esser - Added in PR [terra-functional-testing] Add Accessibility Reporter #456
• Use latest axe core version
• Remove custom TZ on deploy WDIO grid
• Setup and test internally against the grid
• Add Terra Service
  • Add Terra viewports - PR - Ben Cai
  • Add Terra.validates.accessibility command
  • Add Terra.validates.screenshot command - Ben Cai
  • Add Terra.validates.element command - Ben Cai
  • Add API and reporter for warning on new axe-core rules - Stephen Esser
  • Add deprecation warning to terra-toolkit-boneyard for it matchers - PR - Ben Cai
• Uplift Repositories to WDIO 6
  • Terra Core
  • Terra Framework
  • Terra Clinical
  • Terra Application #125 & #126
  • Terra Dev Site
• Add Visual Regression Service - PR
  • Fail VR if new reference screenshot is created during CI run (similar to https://github.com/cerner/terra-toolkit-boneyard/issues/21 && Add isNewScreenshot to Comparison Results zinserjan/wdio-visual-regression-service#83). [terra-functional-testing] Add option to fail tests when new screenshots are generated #603 logged to terra-functional-testing.
  • Fix x / y out of bounds error commonly seen when element is larger than the viewport (similar to Add "Offset" Option for Element Scrolling when Making a Screenshot zinserjan/wdio-screenshot#87)
  • Generate a diff screenshot if the latest screenshot is larger than the reference screenshot - Emily
  • Simplify configuration - Emily
• Migration Guide - In progress on PR [terra-functional-testing] Add Migration Guide #574

Side Quests

• Terra CLI - created in PR terra-cli initial release #460
• Automatically detect IP address for use on VPN
• Improve Logger functionality
• Improve VR error log message to include the screenshot name for quicker tracing when it fails.

Spikes

• Investigate an easier API for overriding/customizing the wdio configuration
• Using Page objects
  - Using the latest selenium images
• Using the non-headless chrome
• Including a configuration argument for webpack file path
• Obsolete screenshot flags similar to jest for updating screenshots
• Visual regression services

[emilyrohrbough]

Recently there was discussion that the selenium-docker image change would not be made in this Major Version Bump.

@StephenEsser reverted the selenium-docker image change in PR #472.

I have updated the issue summary to remove the work related to this change.

[stale]

This issue has been automatically marked as inactive because it has not had recent activity. It will be closed in seven days if no further activity occurs. Thank you for your contributions.