feat(pb, ds): move to edge #1942

MasterPtato · 2025-01-25T02:27:48Z

Changes

cloudflare-workers-and-pages · 2025-01-25T02:27:51Z

Deploying rivet with Cloudflare Pages

Latest commit:	`1162ce3`
Status:	✅ Deploy successful!
Preview URL:	https://5440b11e.rivet.pages.dev
Branch Preview URL:	https://01-25-feat-pb-ds--move-to-ed.rivet.pages.dev

View logs

MasterPtato · 2025-01-25T02:28:02Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

How to use the Graphite Merge Queue

Add the label merge-queue to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

_{An organization admin has enabled the Graphite Merge Queue in this repository.} _{Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.}

This stack of pull requests is managed by Graphite. Learn more about stacking.

greptile-apps

PR Summary

This PR moves pegboard and ds services to the edge infrastructure, introducing new workflow handling capabilities and SQLite database integration.

Added new edge-monolith-workflow-worker in /packages/services/edge/monolith/standalone/workflow-worker/ with concerning error handling that unconditionally bails on worker exit
Empty test table migrations in /packages/services/edge/pegboard/src/workflows/actor/migrations.rs and /packages/services/edge/pegboard/src/workflows/client/migrations.rs need proper schema definitions
Added SQLite database access in ActivityCtx and Connection for per-workflow state storage with workflow-specific database names
Implemented comprehensive client workflow in /packages/services/edge/pegboard/src/workflows/client/mod.rs for handling registration, events, and actor management
New feature flag system in /packages/services/edge/pegboard/Cargo.toml organizes dependencies into tiers (default, workflows, chirp)

_{12 file(s) reviewed, 16 comment(s)}
_{Edit PR Review Bot Settings | Greptile}

greptile-apps · 2025-01-27T22:01:05Z

packages/services/edge/monolith/standalone/workflow-worker/Cargo.toml

+ds.workspace = true
+pegboard.workspace = true


logic: ds and pegboard dependencies should specify features needed for workflow integration, e.g. pegboard = { workspace = true, features = ["workflows"] }

greptile-apps · 2025-01-27T22:01:05Z

packages/services/edge/monolith/standalone/workflow-worker/Cargo.toml

+[dependencies]
+chirp-workflow.workspace = true
+rivet-config.workspace = true
+rivet-health-checks.workspace = true
+rivet-metrics.workspace = true
+rivet-runtime.workspace = true


style: consider adding tokio with appropriate runtime features since this is a worker service that will likely need async functionality

greptile-apps · 2025-01-27T22:01:42Z

packages/services/edge/monolith/standalone/workflow-worker/src/lib.rs

+
+	// Start worker
+	worker.wake_start(config, pools).await?;
+	bail!("worker exited unexpectedly");


logic: unconditional bail! means worker will always exit with error, even for graceful shutdowns - should check exit condition first

greptile-apps · 2025-01-27T22:01:43Z

packages/services/edge/monolith/standalone/workflow-worker/src/lib.rs

+	let reg = ds::registry()?
+		.merge(pegboard::registry()?)?;
+
+	let db = db::DatabaseFdbSqliteNats::from_pools(pools.clone())?;


style: pools is cloned here but original pools object is passed to wake_start - consider reusing the cloned pools

greptile-apps · 2025-01-27T22:01:43Z

packages/services/edge/pegboard/Cargo.toml

+workflows = ["chirp"]
+chirp = ["chirp-workflow", "sqlx", "nix", "server-spec"]


logic: workflows feature depends on chirp but chirp is not listed as a dependency in the feature definition - should be workflows = ["chirp-workflow"] or include both

greptile-apps · 2025-01-27T22:02:15Z

packages/services/edge/pegboard/src/workflows/client/migrations.rs

+	sql_execute!(
+		[ctx, pool]
+		"
+		CREATE TABLE test (


style: 'test' appears to be a placeholder name. Use a descriptive table name that reflects its purpose in production.

greptile-apps · 2025-01-27T22:02:21Z

packages/services/edge/pegboard/src/workflows/mod.rs

+pub mod client;
+pub mod actor;


style: module order should be alphabetical - 'actor' should come before 'client'

greptile-apps · 2025-01-27T22:02:56Z

packages/services/edge/pegboard/src/workflows/client/mod.rs

+	// TODO(RVT-4450): `last_event_idx < $2` and `ON CONFLICT DO NOTHING` is a workaround
+	let inserted_rows = sql_fetch_all!(


logic: The workaround for RVT-4450 using last_event_idx < $2 and ON CONFLICT DO NOTHING could potentially miss events if they arrive out of order. Consider adding additional validation or logging.

greptile-apps · 2025-01-27T22:02:56Z

packages/services/edge/pegboard/src/workflows/client/mod.rs

+	// TODO: Send as a single message
+	for (i, raw_command) in raw_commands.into_iter().enumerate() {
+		let wrapped_command = protocol::CommandWrapper {
+			index: index + i as i64,
+			inner: raw_command,
+		};
+
+		// Forward signal to ws as message
+		ctx.msg(ToWs {
+			client_id,
+			inner: protocol::ToClient::Commands(vec![wrapped_command]),
+		})
+		.send()
+		.await?;
+	}


style: Sending commands individually could cause performance issues with high command volumes. Consider implementing the TODO to batch commands into a single message.

greptile-apps · 2025-01-27T22:02:57Z

packages/services/edge/pegboard/src/workflows/client/mod.rs

+			if matches!(signal.try_into()?, Signal::SIGTERM | Signal::SIGKILL) {
+				let res = ctx


logic: The signal type check should handle conversion errors from try_into() more explicitly to avoid potential panics

cloudflare-workers-and-pages · 2025-01-28T01:43:32Z

Deploying rivet-hub with Cloudflare Pages

Latest commit:	`1162ce3`
Status:	✅ Deploy successful!
Preview URL:	https://5c8bac11.rivet-hub-7jb.pages.dev
Branch Preview URL:	https://01-25-feat-pb-ds--move-to-ed.rivet-hub-7jb.pages.dev

View logs

This was referenced Jan 25, 2025

chore: restructure server binaries #1941

Open

feat: fdb sqlite workflows driver #1850

Open

fix(workflows): allow op ctx to do all the things #1938

Open

feat(workflows): add loop state #1939

Closed

MasterPtato mentioned this pull request Jan 25, 2025

fix(cluster): create new cluster server wf with stateful loop #1940

Closed

MasterPtato marked this pull request as ready for review January 27, 2025 22:00

MasterPtato requested review from NathanFlurry and removed request for NathanFlurry January 27, 2025 22:00

MasterPtato marked this pull request as draft January 27, 2025 22:00

MasterPtato mentioned this pull request Jan 27, 2025

chore(workflows): move wf gc and metrics publish into worker #1943

Open

greptile-apps bot reviewed Jan 27, 2025

View reviewed changes

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 1023d01 to 9f75ea9 Compare January 27, 2025 22:18

MasterPtato changed the base branch from 01-25-chore_restructure_server_binaries to 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker January 27, 2025 22:18

MasterPtato force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from df8b94d to f6f8a7b Compare January 28, 2025 01:43

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 9f75ea9 to 176d520 Compare January 28, 2025 01:43

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch 9 times, most recently from 9c05edc to 0ce8234 Compare January 31, 2025 21:37

This was referenced Feb 1, 2025

chore(hub): update dev command to use turbo #1984

Closed

chore(docker/monolith): remove unneeded apt-transport-https dep #1985

Closed

chore: enable git-lfs in release workflow #1987

Closed

MasterPtato force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from f6f8a7b to d7a614b Compare February 1, 2025 02:41

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 0ce8234 to f532f05 Compare February 1, 2025 02:41

MasterPtato force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from d7a614b to cbfb3c8 Compare February 1, 2025 02:44

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch from f532f05 to 0a58d12 Compare February 1, 2025 02:44

MasterPtato force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from cbfb3c8 to f993bd7 Compare February 3, 2025 19:19

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 0a58d12 to ba14c9f Compare February 3, 2025 19:19

MasterPtato force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from f993bd7 to ca16fe8 Compare February 4, 2025 00:47

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch 2 times, most recently from e057146 to 5e1789e Compare February 4, 2025 01:58

MasterPtato force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from ca16fe8 to e8b385a Compare February 4, 2025 22:32

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch from 5e1789e to dfa1115 Compare February 4, 2025 22:32

MasterPtato added 2 commits February 5, 2025 01:42

feat(pb, ds): move to edge

182c7a6

feat(ds, pb): move to edge

1162ce3

MasterPtato force-pushed the 01-27-chore_workflows_move_wf_gc_and_metrics_publish_into_worker branch from e8b385a to 962e10c Compare February 5, 2025 01:42

MasterPtato force-pushed the 01-25-feat_pb_ds__move_to_edge branch from dfa1115 to 1162ce3 Compare February 5, 2025 01:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(pb, ds): move to edge #1942

feat(pb, ds): move to edge #1942

MasterPtato commented Jan 25, 2025

cloudflare-workers-and-pages bot commented Jan 25, 2025 •

edited

Loading

MasterPtato commented Jan 25, 2025 •

edited by NathanFlurry

Loading

greptile-apps bot left a comment

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

greptile-apps bot Jan 27, 2025

cloudflare-workers-and-pages bot commented Jan 28, 2025 •

edited

Loading

		workflows = ["chirp"]
		chirp = ["chirp-workflow", "sqlx", "nix", "server-spec"]

		// TODO(RVT-4450): `last_event_idx < $2` and `ON CONFLICT DO NOTHING` is a workaround
		let inserted_rows = sql_fetch_all!(

		if matches!(signal.try_into()?, Signal::SIGTERM \| Signal::SIGKILL) {
		let res = ctx

		ds.workspace = true
		pegboard.workspace = true

feat(pb, ds): move to edge #1942

Are you sure you want to change the base?

feat(pb, ds): move to edge #1942

Conversation

MasterPtato commented Jan 25, 2025

Changes

cloudflare-workers-and-pages bot commented Jan 25, 2025 • edited Loading

Deploying rivet with Cloudflare Pages

MasterPtato commented Jan 25, 2025 • edited by NathanFlurry Loading

How to use the Graphite Merge Queue

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

greptile-apps bot Jan 27, 2025

Choose a reason for hiding this comment

cloudflare-workers-and-pages bot commented Jan 28, 2025 • edited Loading

Deploying rivet-hub with Cloudflare Pages

cloudflare-workers-and-pages bot commented Jan 25, 2025 •

edited

Loading

MasterPtato commented Jan 25, 2025 •

edited by NathanFlurry

Loading

cloudflare-workers-and-pages bot commented Jan 28, 2025 •

edited

Loading