-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[uss_qualifier] netrid: nominal_behavior also covers max area requirement for SPs #821
Conversation
I'm not convinced the approach is ideal: those are different requirements for different participants, different requirements, with data coming from similar but different interfaces. However it does look like the implementation may require a refactoring as there is some mixing of DP and SP flows. I'd say that's a good topic for today's meeting? |
You're right that this is not ideal: I got confused and mixed SP and DP check logic.
Taking this to draft while I move the check to the proper place. |
4fcbe5a
to
491aa62
Compare
@mickmis after double checking and our conversation at the community meeting, the present solution seems good enough: checks for the display provider and the service provider are properly separated in the code, and the present PR does not increase the complexity of the display data evaluator. |
22ec5ef
to
56b7fe6
Compare
d3313ad
to
7f4194f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approach LGTM, however looking at implementation the check text should be updated:
monitoring/monitoring/uss_qualifier/scenarios/astm/netrid/display_data_evaluator.py
Lines 975 to 986 in 44843f8
for mapping in mappings.values(): | |
with self._test_scenario.check( | |
"Area too large", [mapping.injected_flight.uss_participant_id] | |
) as check: | |
check.record_failed( | |
summary="Flight discovered using too-large area request", | |
details=f"{mapping.injected_flight.uss_participant_id} was queried for flights in {_rect_str(rect)} with a diagonal of {diagonal} km which is larger than the maximum allowed diagonal of {self._rid_version.max_diagonal_km} km. The expected error code is 413, but instead a valid response containing the expected flight was received.", | |
severity=Severity.High, | |
query_timestamps=[ | |
mapping.observed_flight.query.query.request.timestamp | |
], | |
) |
In short, for an SP, this check is failed after the fact if a flight was discovered using a rectangle that was too large.
Additionally the check is created+failed after the failed conditions are evaluated: that means it will never show as a success. I think the implementation should be changed so that it can surface as successfully tested.
7f4194f
to
b06e18a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, however since that check can't be set successful as is, I don't think we can consider NET0250 covered yet. We could relatively easily add a scenario or a case in an existing scenario that attempts direct queries to the SP with a too large diagonal. I've put a note in the spreadsheet tracking the progress.
97f8a49
to
edab217
Compare
edab217
to
f8b1f51
Compare
The
nominal_behavior
scenario already checks offNET0430
(which requires a Display Provider to not return data if the requested diagonal is too big) within the "Service Provider polling" test step.This seems incorrect, as
NET0430
specifically concerns a display provider, it should not be present in the Service Provider pollingThe PR adds coverage for
NET0250
simply by replacingNET0430
withNET0250
Part of the effort for #754