LogAnalysis should not rely on cell execution being recorded by LogEv…

…ents (#222)

Now that we no longer need to process RunMe logs we can simplify log
processing

* For the watermark we can just keep track of a single file and its
offset
* Only the latest file will be active. Therefore we don't need to watch
the filesystem for modifications
  we can just periodically scan the logs.

## Changes to BlockLog Proto

* We can remove ExecTraceIds
* Since the AI doesn't handle executions we don't have traces for
execution
* We should rely on the Frontend sending details of execution (e.g.
execute code) which it currently isn't
* Since LogEvents are processed in order we know that the most recent
cell execution will be the final one reported by a LogEvent
* Add a field to record suggestion status
* Right now we only get log events for cells being accepted we don't
record cells being rejected but we can reasonably infer that unaccepted
cells have been rejected

## Config Changes

* We can deprecate the logDirs field in learner since users should no
longer need to configure it to monitor RunMe Logs
* Simplifying the setup process of the learner is one of the main
motivations of this PR.

## Analyzer Changes

*. We no longer need CombineTraces functions for RunMe and Execute
traces
* We just accumulate LogEvents on the BlockLog object as we sequentially
process the logs.

## Other changes
* Add an RPC method to check the logs status.
  * This will report the watermark
* Add ZapProto function to handle logging protos that include RunMe
objects
* We can't rely on the custom go plugin to generate the MarshalObject
function because the RunMe protos aren't using that plugin

Related to #211

app/pkg/agent/agent.go

@@ -529,7 +529,8 @@ func (a *Agent) LogEvents(ctx context.Context, req *connect.Request[v1alpha1.Log
 		func() {
 			_, span := tp.Start(ctx, "LogEvent", trace.WithAttributes(attribute.String("eventType", event.Type.String()), attribute.String("contextId", event.ContextId), attribute.String("selectedCellId", event.SelectedId)))
 			defer span.End()
-			log.Info("LogEvent", "eventType", event.Type, "contextId", event.ContextId, "selectedCellId", event.SelectedId, "event", zap.Object("event", event))
+			// N.B we can't use zap.Object to log the event because it contains runme protos which don't have the zap marshaler bindings.
+			log.Info("LogEvent", "eventId", event.GetEventId(), "eventType", event.Type, "contextId", event.ContextId, "selectedCellId", event.SelectedId, logs.ZapProto("event", event))
 		}()
 	}
 	return connect.NewResponse(&v1alpha1.LogEventsResponse{}), nil