0.0.1 (#8)

# Changelog

## [0.0.1](https://github.com/NeonGeckoCom/neon-minerva/tree/0.0.1)
(2023-10-26)

[Full
Changelog](1f81d66...0.0.1)

**Merged pull requests:**

- Update GitHub release automation
[\#7](#7)
([NeonDaniel](https://github.com/NeonDaniel))
- Update GitHub release automation
[\#6](#6)
([NeonDaniel](https://github.com/NeonDaniel))
- Loosen padacioso dependency spec for neon-core compat.
[\#4](#4)
([NeonDaniel](https://github.com/NeonDaniel))
- Add integration tests
[\#3](#3)
([NeonDaniel](https://github.com/NeonDaniel))
- Initial implementation of portable skill tests
[\#1](#1)
([NeonDaniel](https://github.com/NeonDaniel))



\* *This Changelog was automatically generated by
[github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*

.github/workflows/license_tests.yml

@@ -0,0 +1,8 @@
+name: Run License Tests
+on:
+  push:
+  workflow_dispatch:
+
+jobs:
+  license_tests:
+    uses: neongeckocom/.github/.github/workflows/license_tests.yml@master

.github/workflows/propose_release.yml

@@ -0,0 +1,28 @@
+name: Propose Stable Release
+on:
+  workflow_dispatch:
+    inputs:
+      release_type:
+        type: choice
+        description: Release Type
+        options:
+          - patch
+          - minor
+          - major
+jobs:
+  update_version:
+    uses: neongeckocom/.github/.github/workflows/propose_semver_release.yml@master
+    with:
+      branch: dev
+      release_type: ${{ inputs.release_type }}
+      update_changelog: True
+      version_file: "neon_minerva/version.py"
+  pull_changes:
+    uses: neongeckocom/.github/.github/workflows/pull_master.yml@master
+    needs: update_version
+    with:
+      pr_reviewer: neonreviewers
+      pr_assignee: ${{ github.actor }}
+      pr_draft: false
+      pr_title: ${{ needs.update_version.outputs.version }}
+      pr_body: ${{ needs.update_version.outputs.changelog }}

.github/workflows/publish_release.yml

@@ -0,0 +1,12 @@
+# This workflow will generate a release distribution and upload it to PyPI
+
+name: Publish Build and GitHub Release
+on:
+  push:
+    branches:
+      - master
+
+jobs:
+  build_and_publish_pypi_and_release:
+    uses: neongeckocom/.github/.github/workflows/publish_stable_release.yml@master
+    secrets: inherit

.github/workflows/publish_test_build.yml

@@ -0,0 +1,17 @@
+# This workflow will generate a distribution and upload it to PyPI
+
+name: Publish Alpha Build
+on:
+  push:
+    branches:
+      - dev
+    paths-ignore:
+      - 'neon_minerva/version.py'
+
+jobs:
+  build_and_publish_pypi:
+    uses: neongeckocom/.github/.github/workflows/publish_alpha_release.yml@master
+    secrets: inherit
+    with:
+      version_file: "neon_minerva/version.py"
+      setup_py: "setup.py"

.github/workflows/unit_tests.yml

@@ -0,0 +1,10 @@
+# This workflow will run unit tests
+
+name: Run Unit Tests
+on:
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  py_build_tests:
+    uses: neongeckocom/.github/.github/workflows/python_build_tests.yml@master

CHANGELOG.md

@@ -0,0 +1,17 @@
+# Changelog
+
+## [0.0.1](https://github.com/NeonGeckoCom/neon-minerva/tree/0.0.1) (2023-10-26)
+
+[Full Changelog](https://github.com/NeonGeckoCom/neon-minerva/compare/1f81d6670a144b65069a882623ff3ef44ae93582...0.0.1)
+
+**Merged pull requests:**
+
+- Update GitHub release automation [\#7](https://github.com/NeonGeckoCom/neon-minerva/pull/7) ([NeonDaniel](https://github.com/NeonDaniel))
+- Update GitHub release automation [\#6](https://github.com/NeonGeckoCom/neon-minerva/pull/6) ([NeonDaniel](https://github.com/NeonDaniel))
+- Loosen padacioso dependency spec for neon-core compat. [\#4](https://github.com/NeonGeckoCom/neon-minerva/pull/4) ([NeonDaniel](https://github.com/NeonDaniel))
+- Add integration tests [\#3](https://github.com/NeonGeckoCom/neon-minerva/pull/3) ([NeonDaniel](https://github.com/NeonDaniel))
+- Initial implementation of portable skill tests [\#1](https://github.com/NeonGeckoCom/neon-minerva/pull/1) ([NeonDaniel](https://github.com/NeonDaniel))
+
+
+
+\* *This Changelog was automatically generated by [github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*

LICENSE.md

@@ -0,0 +1,21 @@
+# NEON AI (TM) SOFTWARE, Software Development Kit & Application Development System
+# All trademark and other rights reserved by their respective owners
+# Copyright 2008-2021 Neongecko.com Inc.
+# BSD-3 License
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the
+following conditions are met:
+1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following
+disclaimer.
+2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following
+disclaimer in the documentation and/or other materials provided with the distribution.
+3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products
+derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README.md

@@ -0,0 +1,38 @@
+# Neon Minerva
+Neon Minerva (Modular INtelligent Evaluation for a Reliable Voice Assistant) 
+provides tools for testing skills.
+
+Install the Minerva Python package with: `pip install neon-minerva`
+The `minerva` entrypoint is available to interact with a bus via CLI. 
+Help is available via `minerva --help`.
+
+## Installation
+Since skill intents may use Padatious, the following system packages must be 
+installed before installing this package:
+```shell
+sudo apt install swig libfann-dev
+```
+To install this package from PyPI, simply run:
+```shell
+pip install neon-minerva
+```
+
+## Usage
+This package provides a CLI for local testing of skills. Skills installed with 
+`pip` can be specified by entrypoint, or skills cloned locally can be specified
+by root directory.
+
+### Resource Tests
+To test that skill resources are defined for all supported languages,
+`minerva test-resources <skill-entrypoint> <test-file>`
+> - <skill-entrypoint\> is the string entrypoint for the skill to test as specified in `setup.py` OR the path to 
+    the skill's root directory 
+> - <test-file\> is a relative or absolute path to the resource test file, usually `test_resources.yaml`
+
+### Intent Tests
+To test that skill intents match as expected for all supported languages,
+`minerva test-intents <skill-entrypoint> <test-file>`
+> - <skill-entrypoint\> is the string entrypoint for the skill to test as specified in `setup.py` OR the path to 
+    the skill's root directory 
+> - <test-file\> is a relative or absolute path to the resource test file, usually `test_intents.yaml`
+> - The `--padacioso` flag can be added to test with Padacioso instead of Padatious for relevant intents

neon_minerva/__init__.py

@@ -0,0 +1,25 @@
+# NEON AI (TM) SOFTWARE, Software Development Kit & Application Development System
+# All trademark and other rights reserved by their respective owners
+# Copyright 2008-2021 Neongecko.com Inc.
+# BSD-3
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+#    this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+#    this list of conditions and the following disclaimer in the documentation
+#    and/or other materials provided with the distribution.
+# 3. Neither the name of the copyright holder nor the names of its
+#    contributors may be used to endorse or promote products derived from this
+#    software without specific prior written permission.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS  BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
+# OR PROFITS;  OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE,  EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

neon_minerva/cli.py

@@ -0,0 +1,150 @@
+# NEON AI (TM) SOFTWARE, Software Development Kit & Application Development System
+# All trademark and other rights reserved by their respective owners
+# Copyright 2008-2021 Neongecko.com Inc.
+# BSD-3
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+#    this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+#    this list of conditions and the following disclaimer in the documentation
+#    and/or other materials provided with the distribution.
+# 3. Neither the name of the copyright holder nor the names of its
+#    contributors may be used to endorse or promote products derived from this
+#    software without specific prior written permission.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS  BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
+# OR PROFITS;  OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE,  EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import os
+import click
+import yaml
+
+from os.path import expanduser, relpath, isfile, isdir
+from click_default_group import DefaultGroup
+from unittest.runner import TextTestRunner
+from unittest import makeSuite
+
+from neon_minerva.version import __version__
+
+
+def _init_tests(debug: bool = False):
+    from os.path import join
+    from os import makedirs
+    from tempfile import mkdtemp
+    base_dir = mkdtemp()
+    config = join(base_dir, "config")
+    data = join(base_dir, "data")
+    cache = join(base_dir, "cache")
+    makedirs(config, exist_ok=True)
+    makedirs(data, exist_ok=True)
+    makedirs(cache, exist_ok=True)
+    os.environ["XDG_CONFIG_HOME"] = config
+    os.environ["XDG_DATA_HOME"] = data
+    os.environ["XDG_CACHE_HOME"] = cache
+
+    if debug:
+        os.environ["OVOS_DEFAULT_LOG_LEVEL"] = "DEBUG"
+
+
+def _get_test_file(test_file: str) -> str:
+    """
+    Parse an input path to locate a test file that may be relative to `~` or the
+    current working directory.
+    @param test_file: test file argument
+    @returns: best guess at the desired file path (may not exist)
+    """
+    test_file = expanduser(test_file)
+    if not isfile(test_file):
+        test_file = relpath(test_file)
+    return test_file
+
+
+def _get_skill_entrypoint(skill_entrypoint: str) -> str:
+    """
+    Parse an input skill entrypoint and resolve either a locally installed skill
+    path, or an entrypoint for a plugin skill.
+    @param skill_entrypoint: Plugin entrypoint or path to skill
+    @returns: absolute file path if exists, else input entrypoint
+    """
+    skill_path = expanduser(skill_entrypoint)
+    if not isdir(skill_path):
+        skill_path = relpath(skill_path)
+    if isdir(skill_path):
+        return skill_path
+    return skill_entrypoint
+
+
+@click.group("minerva", cls=DefaultGroup,
+             no_args_is_help=True, invoke_without_command=True,
+             help="Minerva: Modular INtelligent Evaluation for a Reliable "
+                  "Voice Assistant.\n\n"
+                  "See also: minerva COMMAND --help")
+@click.option("--version", "-v", is_flag=True, required=False,
+              help="Print the current version")
+def neon_minerva_cli(version: bool = False):
+    if version:
+        click.echo(f"Minerva version {__version__}")
+
+
+@neon_minerva_cli.command
+@click.option('--debug', is_flag=True, default=False,
+              help="Flag to enable debug logging")
+@click.argument("skill_entrypoint")
+@click.argument("test_file")
+def test_resources(skill_entrypoint, test_file, debug):
+    _init_tests(debug)
+    os.environ["TEST_SKILL_ENTRYPOINT"] = _get_skill_entrypoint(skill_entrypoint)
+    test_file = _get_test_file(test_file)
+    if not isfile(test_file):
+        click.echo(f"Could not find test file: {test_file}")
+        exit(2)
+    os.environ["RESOURCE_TEST_FILE"] = test_file
+    from neon_minerva.tests.test_skill_resources import TestSkillResources
+    TextTestRunner().run(makeSuite(TestSkillResources))
+
+
+@neon_minerva_cli.command
+@click.option('--debug', is_flag=True, default=False,
+              help="Flag to enable debug logging")
+@click.option('--padacioso', is_flag=True, default=False,
+              help="Flag to enable testing with Padacioso instead of Padatious")
+@click.argument("skill_entrypoint")
+@click.argument("test_file")
+def test_intents(skill_entrypoint, test_file, debug, padacioso):
+    _init_tests(debug)
+    os.environ["TEST_PADACIOSO"] = "true" if padacioso else "false"
+    os.environ["TEST_SKILL_ENTRYPOINT"] = _get_skill_entrypoint(skill_entrypoint)
+    test_file = _get_test_file(test_file)
+    if not isfile(test_file):
+        click.echo(f"Could not find test file: {test_file}")
+        exit(2)
+    os.environ["INTENT_TEST_FILE"] = test_file
+    from neon_minerva.tests.test_skill_intents import TestSkillIntentMatching
+    TextTestRunner().run(makeSuite(TestSkillIntentMatching))
+
+
+@neon_minerva_cli.command
+@click.option('-l', '--lang', default="en-us",
+              help="Language of test_file inputs")
+@click.option('-a', '--audio', is_flag=True, default=False,
+              help="Test input as audio")
+@click.argument("test_file")
+def test_utterances(lang, audio, test_file):
+    from neon_utils.file_utils import load_commented_file
+    from neon_minerva.integration.user_utterance import UtteranceTests
+
+    test_file = _get_test_file(test_file)
+    prompts = load_commented_file(test_file).split('\n')
+    click.echo(f"Testing {len(prompts)} prompts")
+    runner = UtteranceTests(prompts, lang=lang, audio=audio)
+    results = runner.run_test()
+    click.echo(yaml.safe_dump(results))

neon_minerva/exceptions.py

@@ -0,0 +1,43 @@
+# NEON AI (TM) SOFTWARE, Software Development Kit & Application Framework
+# All trademark and other rights reserved by their respective owners
+# Copyright 2008-2022 Neongecko.com Inc.
+# Contributors: Daniel McKnight, Guy Daniels, Elon Gasper, Richard Leeds,
+# Regina Bloomstine, Casimiro Ferreira, Andrii Pernatii, Kirill Hrymailo
+# BSD-3 License
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+#    this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+#    this list of conditions and the following disclaimer in the documentation
+#    and/or other materials provided with the distribution.
+# 3. Neither the name of the copyright holder nor the names of its
+#    contributors may be used to endorse or promote products derived from this
+#    software without specific prior written permission.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
+# CONTRIBUTORS  BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
+# OR PROFITS;  OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+# SOFTWARE,  EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+class IntentException(Exception):
+    """
+    Base exception for an intent error
+    """
+
+class IntentNotMatched(IntentException):
+    """
+    Exception indicating an intent match was expected but not found
+    """
+
+
+class ConfidenceTooLow(IntentException):
+    """
+    Exception indicating an intent match confidence was below specified minimum.
+    """