Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

arroyo-connectors: Test code-gen for JSON Schema and Avro. #408

Merged
merged 1 commit into from
Nov 15, 2023

Conversation

jacksonrnewhouse
Copy link
Contributor

This updates arroyo-sql-testing by registering two new kafka sources, one with a schema defined by JsonSchema, the other using an Avro definition. It is invoked from full_query_testing, and just confirms code is generated without issue. This required two changes:

  1. In arroyo-sql avro.rs and json_schema.rs were importing crate::*. This only works if the definitions they need are at the top level of the crate. This is true for real pipelines, but not the test infrastructure.
  2. We changed how we specify timestamp deserialization from JSON, and json_schema.rs was not migrated at that time. The tests will stop this from happening again.

@jacksonrnewhouse jacksonrnewhouse force-pushed the connectors/code_gen_schema_def_connectors branch from 27abc31 to fb4692d Compare November 14, 2023 18:36
arroyo-sql-macro/Cargo.toml Outdated Show resolved Hide resolved
@jacksonrnewhouse jacksonrnewhouse force-pushed the connectors/code_gen_schema_def_connectors branch from fb4692d to e1172a8 Compare November 15, 2023 00:39
@jacksonrnewhouse jacksonrnewhouse merged commit 0a43e08 into master Nov 15, 2023
8 checks passed
zh4ngx pushed a commit to StrikeTeamOne/arroyo that referenced this pull request Nov 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants