Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: Make fct_daily_schedule_feed_validation_notices properly validator version-aware #1994

Closed
lauriemerrell opened this issue Nov 17, 2022 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@lauriemerrell
Copy link
Contributor

lauriemerrell commented Nov 17, 2022

Describe the bug
Uncovered in review of #1960.

Our fct_daily_schedule_feed_validation_notices table does not properly account for validator versioning, because the cross join does not check for which validator version we were actually running on a given date, and the way that we COALESCE the results means that we imply that if any validator ran that we can count 0 notices for all codes, even codes that did not exist in the validator version that was run on that day.

There are at least two specific symptoms of this that we need to address:

  1. gtfs_validator_version column is null for rows with zero notices. gtfs_validator_version should be populated for all rows to reflect what validator version actually ran against that feed on that date.

image

  1. Validations introduced in validator version 4 (which we only started running this week, in November) are listed for dates before that validator version started running in our pipeline. Validations should only be listed on dates for which their associated validator version was actually run.

image

I think to fix this we may need to start listing the validator version that was run in the validations outcome file, not just in the validations files themselves, to account for cases where a feed returns 0 notices. We should be recording which validator version was actually run against each feed each day. Ambivalent on whether we can/should backfill this information if we can get the right outcome without doing so.

We also need to fix the actual SQL to record this information.

Acceptance criteria:

@lauriemerrell lauriemerrell changed the title Bug: Make fct_daily_schedule_feed_validation_notices properly version-aware Bug: Make fct_daily_schedule_feed_validation_notices properly validator version-aware Nov 17, 2022
@holly-g holly-g added the bug Something isn't working label Nov 29, 2022
@atvaccaro
Copy link
Contributor

@lauriemerrell did we ever fix this?

@lauriemerrell
Copy link
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants