Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Dekaf control-plane components #1665

Merged
merged 26 commits into from
Oct 7, 2024

Commits on Oct 2, 2024

  1. dekaf: Introduce MaterializationEndpoint::Dekaf

    A materialization of this kind is purely descriptive, and cannot be directly started. It is used to convey the intent to expose a set of bindings through Dekaf, which will look it up to determine things like binding names and field selection/projections.
    jshearer committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    fd041a3 View commit details
    Browse the repository at this point in the history
  2. crates/activate: Dekaf materializations should not get shards or reco…

    …very journals, but they _should_ get ops stats+logs collections
    jshearer committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    f141de9 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    13e45c7 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    1576b90 View commit details
    Browse the repository at this point in the history
  5. dekaf: Introduce and validate DekafResourceConfig to configure a pa…

    …rticular dekaf binding
    jshearer committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    c101864 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    c9ff4df View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    b8b33d6 View commit details
    Browse the repository at this point in the history
  8. Factor out flow-client from flowctl

    Largely retain the same functionality as `flowctl::Client`, just in a new home so that it can be shared without introducing inconvenient dependencies
    jshearer committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    280195d View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    0e37c22 View commit details
    Browse the repository at this point in the history
  10. test: Update Dekaf tests

    jshearer committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    ad46121 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    c99ce10 View commit details
    Browse the repository at this point in the history
  12. Some PR feedback

    jshearer committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    705281d View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    0a95114 View commit details
    Browse the repository at this point in the history
  14. Configuration menu
    Copy the full SHA
    6b52dbd View commit details
    Browse the repository at this point in the history
  15. flow-client: Refactor to extract out refresh logic into bare function…

    …, and re-use Postgrest connection pool instead of creating a new one on each call to `pg_client()`
    jshearer committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    1ac9c90 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2024

  1. Configuration menu
    Copy the full SHA
    fdd7c1a View commit details
    Browse the repository at this point in the history
  2. flow_client: Refactor to expose refreshed credentials

    `flowctl` needs these credentials exposed in order to store them in `flowctl::Config` + on disk for the next flowctl invocation
    jshearer committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    14c50a8 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    063f286 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2024

  1. Configuration menu
    Copy the full SHA
    d55ecee View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    02db746 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. Configuration menu
    Copy the full SHA
    1ce6a58 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a98284a View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    52defc3 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    86745ef View commit details
    Browse the repository at this point in the history
  5. dekaf: Temporarily implement and use `flow_client::Client::with_fresh…

    …_gazette_client()`
    
    gRPC connections seem to lose stability after a while, this temporarily fixes that
    jshearer committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    4064be8 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    e1362fd View commit details
    Browse the repository at this point in the history