Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[uss_qualifier] Add U-space MSL test scenario #413

Merged
merged 3 commits into from
Dec 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions monitoring/uss_qualifier/configurations/dev/uspace.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,8 @@ v1:
- astm.f3411.v22a.display_provider#Operator Position transmitter
- astm.f3411.v22a.dss_provider
- astm.f3548.v21.scd#Automated verification
- requirements:
- uspace.article8.MSLAltitude
participant_requirements:
uss1: uspace
uss2: uspace
Expand Down
21 changes: 21 additions & 0 deletions monitoring/uss_qualifier/requirements/uspace/article8.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# [U-space Article 8](https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32021R0664&qid=1702917443967#d1e905-161-1): Network Identification Service

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment apply for amc/gm references

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added links

## Article 8(2)(c)

> The network identification service shall allow for the authorised users to receive messages with the geographical position of the UAS, its altitude above mean sea level and its height above the surface or take-off point.

<tt>MSLAltitude</tt>: Authorised users must receive messages with UAS altitude above mean sea level.

## Article 8(4)

> The authorised users shall be:
>
> (a) the general public as regards information that is deemed public in accordance with applicable Union and national rules;
>
> (b) other U-space service providers in order to ensure the safety of operations in the U-space airspace;
>
> (c) the air traffic services providers concerned;
>
> (d) when designated, the single common information service provider;
>
> (e) the relevant competent authorities.
Empty file.
48 changes: 48 additions & 0 deletions monitoring/uss_qualifier/scenarios/uspace/netrid/msl.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# U-space MSL altitude test scenario

## Description

[Article 8](https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32021R0664&qid=1702917443967#d1e905-161-1)(2)(c) specifies (emphasis added):

> The network identification service shall allow for the authorised users to receive messages with the geographical position of the UAS, its *altitude above mean sea level* and its height above the surface or take-off point.


ASTM F3411 provides geodetic altitude above the WGS84 ellipsoidal estimate of sea level, however [AMC1](https://www.easa.europa.eu/en/document-library/acceptable-means-of-compliance-and-guidance-materials/amc-and-gm-implementing) specifies:

> USSPs should convert the heights above the WGS 84 ellipsoid exchanged with the ASTM F-3411-22A standard to height above mean sea level (MSL) before providing it to the UAS operators.


[GM1](https://www.easa.europa.eu/en/document-library/acceptable-means-of-compliance-and-guidance-materials/amc-and-gm-implementing) further clarifies the desired definition of mean sea level:

> Wherever the flight altitude above sea level is required to be determined with the use of GNSS systems, it is recommended to use the EGM2008 or at least the EGM96 geoid models as the definition of mean sea level, as agreed with the competent authority.


Therefore, to comply with Article 8(2)(c), a USSP must allow for the authorised users to receive messages with the UAS's altitude above the EGM96 geoid.

### Assumptions

This scenario assumes that [the ASTM F3411-22a nominal behavior NetRID test scenario](../../astm/netrid/v22a/nominal_behavior.md) has already been completed and determines compliance with the requirement above by examining the observations made by uss_qualifier as an automated "authorised user".

## Resources

### observers

The set of USSPs providing messages to authorised users to be evaluated for U-space MSL compliance.

## UAS observations evaluation test case

### Find nominal behavior report test step

To avoid re-running a nearly-identical test scenario, this test scenario merely examines data collected in a separate test scenario (see [Assumptions](#assumptions)). Therefore, the first step in this scenario is to find the test report for that other scenario.

If an appropriate test report cannot be found, this scenario will be discontinued.

### Evaluate UAS observations test step

#### ⚠️ Message contains MSL altitude check

If the response message for the remote identification observation made by the virtual/automated authorised user does not contain the UAS's MSL altitude, the USSP will have failed to comply with **[uspace.article8.MSLAltitude](../../../requirements/uspace/article8.md)**.

#### ⚠️ MSL altitude is correct check

In the previously-conducted test scenario, UAS altitudes were injected relative to the WGS84 ellipsoid. Since the EGM96 geoid is a standard shape that is well-defined relative to the WGS84 ellipsoid, this means the altitude relative to the EGM96 is defined by the injection. If the observed MSL altitude differs from the injected MSL altitude, then the USSP has failed to allow the automated authorised user to receive messages with the UAS's altitude above mean sea level per **[uspace.article8.MSLAltitude](../../../requirements/uspace/article8.md)** because the altitude reported was not the altitude of the UAS.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to specify an error tolerance?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since I don't think the requirements specify an error tolerance, I would expect we wouldn't mention it in the documentation and we would pick a value in implementation that doesn't cause any false positives while minimizing false negatives.

31 changes: 31 additions & 0 deletions monitoring/uss_qualifier/scenarios/uspace/netrid/msl.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
from typing import List

from monitoring.uss_qualifier.configurations.configuration import ParticipantID
from monitoring.uss_qualifier.resources.netrid import NetRIDObserversResource
from monitoring.uss_qualifier.scenarios.scenario import TestScenario


class MSLAltitude(TestScenario):
_ussps: List[ParticipantID]

def __init__(self, observers: NetRIDObserversResource):
super().__init__()
self._ussps = [obs.participant_id for obs in observers.observers]

def run(self, context):
self.begin_test_scenario(context)

self.begin_test_case("UAS observations evaluation")

self.begin_test_step("Find nominal behavior report")
# TODO: Find test report for NetRID nominal behavior scenario
self.end_test_step()

self.begin_test_step("Evaluate UAS observations")
# TODO: Examine observation queries in test report to see if MSL was present
# TODO: When MSL is present, verify that its value matches injected altitude above ellipsoid
self.end_test_step()

self.end_test_case()

self.end_test_scenario()
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
## [Actions](../README.md#actions)

1. Suite: [ASTM F3411-22a](../astm/netrid/f3411_22a.md) ([`suites.astm.netrid.f3411_22a`](../astm/netrid/f3411_22a.yaml))
2. Scenario: [U-space MSL altitude](../../scenarios/uspace/netrid/msl.md) ([`scenarios.uspace.netrid.msl.MSLAltitude`](../../scenarios/uspace/netrid/msl.py))

## [Checked requirements](../README.md#checked-requirements)

Expand Down Expand Up @@ -489,4 +490,10 @@
<td>Implemented</td>
<td><a href="../../scenarios/astm/netrid/v22a/dss/heavy_traffic_concurrent.md">ASTM NetRID DSS: Concurrent Requests</a><br><a href="../../scenarios/astm/netrid/v22a/dss/isa_expiry.md">ASTM NetRID DSS: ISA Expiry</a><br><a href="../../scenarios/astm/netrid/v22a/dss/isa_simple.md">ASTM NetRID DSS: Simple ISA</a><br><a href="../../scenarios/astm/netrid/v22a/dss/token_validation.md">ASTM NetRID DSS: Token Validation</a><br><a href="../../scenarios/astm/netrid/v22a/nominal_behavior.md">ASTM NetRID nominal behavior</a></td>
</tr>
<tr>
<td rowspan="1" style="vertical-align:top;"><a href="../../requirements/uspace/article8.md">uspace<br>.article8</a></td>
<td><a href="../../requirements/uspace/article8.md">MSLAltitude</a></td>
<td>Implemented</td>
<td><a href="../../scenarios/uspace/netrid/msl.md">U-space MSL altitude</a></td>
</tr>
</table>
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ actions:
id_generator: id_generator
service_area: service_area
problematically_big_area: problematically_big_area
on_failure: Abort
- test_scenario:
scenario_type: scenarios.uspace.netrid.msl.MSLAltitude
resources:
observers: observers
on_failure: Continue
participant_verifiable_capabilities:
- id: uspace_netrid_service_provider
Expand Down
6 changes: 6 additions & 0 deletions monitoring/uss_qualifier/suites/uspace/required_services.md
Original file line number Diff line number Diff line change
Expand Up @@ -675,6 +675,12 @@
<td>Implemented</td>
<td><a href="../../scenarios/astm/utm/data_exchange_validation/get_op_data_validation.md">Data Validation of GET operational intents by USS</a></td>
</tr>
<tr>
<td rowspan="1" style="vertical-align:top;"><a href="../../requirements/uspace/article8.md">uspace<br>.article8</a></td>
<td><a href="../../requirements/uspace/article8.md">MSLAltitude</a></td>
<td>Implemented</td>
<td><a href="../../scenarios/uspace/netrid/msl.md">U-space MSL altitude</a></td>
</tr>
<tr>
<td rowspan="1" style="vertical-align:top;"><a href="../../requirements/versioning.md">versioning</a></td>
<td><a href="../../requirements/versioning.md">ReportSystemVersion</a></td>
Expand Down
Loading