Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate ProcessedBsm #135

Draft
wants to merge 26 commits into
base: develop
Choose a base branch
from
Draft

Integrate ProcessedBsm #135

wants to merge 26 commits into from

Conversation

iyourshaw
Copy link
Collaborator

@iyourshaw iyourshaw commented Jan 9, 2025

For Q2 Release

Fully integrates the new ProcessedBsm data structure into the Conflict Monitor's data processing.

  • Removes all dependence on the OdeBsmData class and on the ODE raw BSM Kafka topics, and instead reads from topic.ProcessedBsm. Updates all logic that formerly used raw BSMs to use ProcessedBsms instead.

  • Removes the Repartition topology which is no longer needed because the topic.ProcessedBsm stream is pre-partitioned. Changes the BSM key and partitioning to use RsuLogKey to match what the Geojson Converter produces.

Tests

This PR was tested by verifying that the unit tests pass, and by running the following integration test scripts involving BSMs and verifying that they still produce the expected events:

  • Script-BSMs.csv - BSM path Events
  • BSM_Revisions.csv - Aggregated BSM Message Count Progression Events
  • ConnectionOfTravel-connected.csv, ConnectionOfTravel-u-turn.csv - Connection of Travel, Lane Direction of Travel, and Stop Line Passage Events
  • ApproachAndStopAtLine.csv - Stop Line Stop Events

Note: This PR depends on the Geojson Converter PR: Convert ProcessedBsm to Feature #85 and requires ODE v. 4.0.0 and Geojson Converter v 2.0.0 JARs be available from Github Maven.

Temporary setting in settings.xml. Revert this when updated jars are available in this repo:

<repository>
    <id>github</id>
    <name>GitHub JPO Conflict Monitor</name>
    <!-- <url>https://maven.pkg.github.com/${env.MAVEN_GITHUB_ORG}/*</url> -->
    <url>https://maven.pkg.github.com/iyourshaw/*</url>
...
</repository>

@iyourshaw iyourshaw marked this pull request as ready for review January 10, 2025 21:52
@iyourshaw iyourshaw requested a review from John-Wiens January 10, 2025 21:53
@iyourshaw iyourshaw marked this pull request as draft January 16, 2025 00:38
@dan-du-car
Copy link
Contributor

@iyourshaw Is this PR ready for review?

@iyourshaw
Copy link
Collaborator Author

@iyourshaw Is this PR ready for review?

No, the plan is to hold this one over for the next Q2 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants