Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add basic Edge server structure #24552

Merged
merged 6 commits into from
Jan 8, 2024
Merged

feat: Add basic Edge server structure #24552

merged 6 commits into from
Jan 8, 2024

Conversation

pauldix
Copy link
Member

@pauldix pauldix commented Jan 7, 2024

This adds the basic InfluxDB Edge server structure. There are three new crates: influxdb3 (the new default binary), influxdb3_server, and influxdb3_write. Most of the action and definition can be found in the influxdb3_write crate. I don't expect this to be the final structure and we'll likely pull the query stuff out into its own crate from influxdb3_server, but I wasn't focused on that to begin with. I just wanted a basic server that can accept data, buffer in memory, and answer a query from it.

I've tried to sketch out how I think the different components for writing data should work and interact. I don't think we can use the wal crate as is as it contains a bunch of things that don't make sense in the context of the Edge project. We may just want to use parts of it as a starting point and just dump it into the influxdb3_write crate for now.

My goal is to leave all existing crates that we will use from IOx unmodified. New crates in this repo should be named starting with influxdb3_ to differentiate between the crates in here vs. what is in the IOx repo. Ideally, as we get the MVP of Edge fleshed out, we will be able to clear out any IOx code that doesn't belong in this project. I've gotten a start here.

Since this is a big change, I'm inclined to just YOLO this through and we can then use it as a starting point. But I'll wait until @mgattozzi and I get a chance to talk about it tomorrow.

@pauldix pauldix added the v3 label Jan 7, 2024
@pauldix pauldix requested a review from mgattozzi January 7, 2024 22:09
@pauldix pauldix changed the title Feat: Add basic Edge server structure FEAT: Add basic Edge server structure Jan 7, 2024
@pauldix pauldix changed the title FEAT: Add basic Edge server structure feat: Add basic Edge server structure Jan 8, 2024
This was referenced Jan 8, 2024
Copy link
Contributor

@mgattozzi mgattozzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We talked offline about this and came to the decision to move this through for now. CI is broken currently, but that will get fixed in a follow up PR. For now this will let us work on a shared main branch for now

@pauldix pauldix merged commit 5831cf8 into main Jan 8, 2024
7 of 16 checks passed
@pauldix pauldix deleted the pd/basic-server branch January 8, 2024 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants