FIO - QA and Test Automation

[ericbutz]

Overview

The Foundation for Interwallet Operability (FIO, http://fioprotocol.io) is looking for an experienced QA engineer to lead our QA effort and expand our automated API test harness.

FIO Protocol is fully open source (https://github.com/fioprotocol) with documentation and test APIs (https://developers.fioprotocol.io/). FIO Protocol is built on a blockchain platform that uses C++ as its smart contract programming language. Underlying the blockchain is a WebAssembly (WASM) virtual machine which executes smart contract code. Blockchain functionality is accessed via an API. Integration of FIO features by crypto wallets and exchanges is facilitated by Typescript, Kotlin, and iOS SDKs.

Responsibilities

• Ensure that feature deliverables meet quality objectives in functionality, performance, stability, and security
• Participate actively in functional, system, integration and regression testing activities within project timelines
• Develop, maintain, and execute automated test harness
• Configure and maintain crypto wallet manual test environments for mobile (iOS and Android), browser, and hardware-based wallets
• Build effective test cases that are used for manual and automation testing
• Create testing documentation, including test plans, test cases/scripts, and bug reports assessing quality and associated risk
• Diagnose, track and manage quality issues to resolution
• Collaborate with the broader team to identify, understand and resolve the root cause of defects and end user escalations
• Online technical support to end users on technical issues

Requirements

• Experience in QA and QA automation
• Hands-on experience with JavaScript development
• Understanding of Software QA methodologies, tools and approaches (black box and automated testing)
• Experience in writing test cases and test scenarios that cover both sanity and in-depth testing including and not limited to boundary, negative, compatibility and concurrency tests
• Ability to accurately estimate task durations and meet schedules

Initial project: QA Test Harness

The goal of the automation project is to update and expand the current javascript test code used to test the FIO API (https://developers.fioprotocol.io/pages/api/fio-api). The current implementation was developed to support high-level regression testing. The test suite requires an architectural update from an experienced javascript developer as well as a general expansion of test cases. The developer should have experience with REST API testing, general QA, and QA automation.

Requirements of this project include the updating of the test architecture, migrating existing tests to call the FIO API directly, and expanding test coverage.

The test harness uses the following libraries and tools:

• FIO Test Harness (https://github.com/fioprotocol/fio.test)
• FIO Typescript SDK (https://github.com/fioprotocol/fiosdk_typescript)
• FIOJS (https://github.com/fioprotocol/fiojs) - Utility/helper library that provides encryption, packing and signing capabilities.
• Mocha - JavaScript test framework running on nodejs.
• Chai - Assertion library for nodejs.

The project will be divided into several phases as follows:

Phase 1 (5-10 hours)

https://fioprotocol.atlassian.net/browse/BD-2095

Review the existing code and architecture and make recommendations regarding test tools and/or code architecture. All tools must be open source and widely used. Restructuring may include breaking down the tests to map more directly to the API or grouping tests into individual use cases.

Deliverables for Phase 1:

• Review of existing test suite.
• Recommendations regarding code structure, architecture, and tools.
• Estimate for Phase 2.

Phase 2 (40-80 hours)

https://fioprotocol.atlassian.net/browse/BD-2096

Phase 2 focuses on restructuring of the test harness and the initial migration of tests away from SDK-based transaction calls to API-based calls.

A majority of the existing test code calls into the FIO SDK to execute FIO transactions. The SDK has its own error checking which complicates direct testing of the API. An example of an SDK-based test:

const result = await userA1.sdk.genericAction('addPublicAddresses', {
  fioAddress: userA1.address,
  publicAddresses: [
    {
      chain_code: 'BTC',
      token_code: 'BTC',
      public_address: '1PMycacnJaSqwwJqjawXBErnLsZ7RkXUAs',
      }
  ],
  maxFee: 1000000000,
  technologyProviderId: 'address@wallet'
})

The goal in Phase 2 is to migrate all existing tests away from SDK-based calls to API-based calls. These use the FIOJS libraries to create the transactions and send them directly to the API. An example of a API-based test:

const result = await callFioApiSigned('push_transaction', {
  action: 'addaddress',
  account: 'fio.address',
  actor: userA1.account,
  privKey: userA1.privateKey,
  data: {
    "fio_address": "purse@alice",
    "public_addresses": [
        {
          "chain_code": "BTC",
          "token_code": "BTC",
          "public_address": "1PMycacnJaSqwwJqjawXBErnLsZ7RkXUAs"
        }
    ],
    "max_fee": 1000000000,
    "tpid": "rewards@wallet",
    "actor": "aftyershcu22"
  }
})

Existing tests that must be updated include: (From https://github.com/fioprotocol/fio.test/blob/develop/index.js)

• addaddress.js
• fees.js
• fio-request.js
• producer.js
• register_fio_domain.js
• vote.js
• transfer-domain.js
• paging.js
• cancel-funds-request.js
• removeaddress.js

Deliverables for Phase 2:

• Structural/architectural updates.
• Migrate all existing tests away from SDK-based calls to API-based calls.
• Time estimate for Phase 3.

Phase 3 (timing TBD)

Phase 3 focuses on expanding the coverage of existing tests. The developer will examine the FIO API from a QA perspective and expand the test harness to increase test coverage.

Existing tests to be expanded include:

• addaddress.js (tracked in: https://fioprotocol.atlassian.net/browse/BD-2003)
• fees.js
• fio-request.js
• producer.js
• register_fio_domain.js
• vote.js
• transfer-domain.js
• paging.js
• cancel-funds-request.js
• removeaddress.js

Deliverables for Phase 3:

• Submit a test plan for increasing coverage for existing tests.
• Updates to existing tests.

Phase 4 (timing TBD)

Phase 4 focuses on adding tests for FIO API interfaces not yet tested by existing tests or where there are minimal existing tests.

New tests areas include:

• Refactoring of ram2.js
• Create tests to confirm all FIO API get_ responses conform to the API spec at https://developers.fioprotocol.io/api/api-spec
• Tests for expired addresses and domains (tracked in: https://fioprotocol.atlassian.net/browse/BD-2015)
• /burn_expired
• /record_obt_data
• /renew_fio_address and /renew_fio_domain
• /transfer_tokens_pub_key

Deliverables for Phase 4:

• Submit a test plan and time estimate for development of new tests.
• Development of new tests.

Development Environment

Github

fio.test develop branch will serve as the base branch for development (https://github.com/fioprotocol/fio.test/tree/develop)

FIO

Developer should be able to install and run a FIO node in their local Linux or MacOS environment. This includes:

• Cloning, building, and installing a local copy of fio (https://github.com/fioprotocol/fio/tree/master/scripts)
• Cloning a local copy of fio.contracts (https://github.com/fioprotocol/fio.contracts)
• Running a local node using fio.devtools (https://github.com/fioprotocol/fio.devtools)