upgrade schedule validator to v4.0.0

[atvaccaro]

Description

See the validator release notes for the full list of changes. Per our Slack discussion we are choosing to just upgrade in-place rather than backfill and/or keep two versions of validations over the same data. We can still choose to do that in the future, and we will probably need to do something similar with the reports site so old reports are not updated.

In addition, we can't even really backfill accurately if we wanted to; it would lead to inaccurate notices that rely on the "current date" when executing, such as feed expiration. I created MobilityData/gtfs-validator#1292 to track the need for a "time of validation" input which would facilitate backfills.

Resolves #1924

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation
• agencies.yml

How has this been tested?

Locally; I've verified the validator runs and the external table can be queried.

docker-compose run airflow tasks test unzip_and_validate_gtfs_schedule validate_gtfs_schedule 2022-11-14T02:00:00
...
[2022-11-15 19:46:58,431] {pod_launcher.py:149} INFO - got 219 successes and 0 failures
[2022-11-15 19:46:58,431] {pod_launcher.py:149} INFO - saving 219 to gs://test-calitp-gtfs-schedule-validation/validation_job_results/dt=2022-11-14/results.jsonl
[2022-11-15 19:47:00,668] {pod_launcher.py:198} INFO - Event: validate-gtfs-schedule.e100b24fc3394fdcae6cb5e0d88e9a47 had an event of type Succeeded
[2022-11-15 19:47:00,668] {pod_launcher.py:311} INFO - Event with job id validate-gtfs-schedule.e100b24fc3394fdcae6cb5e0d88e9a47 Succeeded
[2022-11-15 19:47:00,772] {pod_launcher.py:198} INFO - Event: validate-gtfs-schedule.e100b24fc3394fdcae6cb5e0d88e9a47 had an event of type Succeeded
[2022-11-15 19:47:00,773] {pod_launcher.py:311} INFO - Event with job id validate-gtfs-schedule.e100b24fc3394fdcae6cb5e0d88e9a47 Succeeded
[2022-11-15 19:47:00,930] {taskinstance.py:1212} INFO - Marking task as SUCCESS. dag_id=unzip_and_validate_gtfs_schedule, task_id=validate_gtfs_schedule, execution_date=20221114T020000, start_date=20221114T212922, end_date=20221115T194700

select metadata.gtfs_validator_version
     , code
     , count(0)
from cal-itp-data-infra-staging.external_gtfs_schedule.validation_notices
group by 1, 2
order by 2
;
...
v4.0.0,block_trips_with_overlapping_stop_times,5
v3.1.1,block_trips_with_overlapping_stop_times,46
v4.0.0,decreasing_or_equal_stop_time_distance,23
v3.1.1,decreasing_or_equal_stop_time_distance,142

Screenshots (optional)

[owades]

Thank you! This +1 is just approving the business logic

[lauriemerrell]

🚀