Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Event driven script orchestrator #1058

Merged
merged 36 commits into from
Jan 21, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
d470d9f
Try to make an event driven orchstrator
LukeButters Jun 20, 2024
c912d08
Drop start script type
LukeButters Jun 20, 2024
269dce8
Start to remove TScriptStatusResponse
LukeButters Jun 20, 2024
720b9c4
Drop nullable
LukeButters Jun 20, 2024
4336c4c
WIP
samdanaei Jun 20, 2024
254bd55
Use new IStructuredScriptOrchestrator interface
samdanaei Jun 20, 2024
acac7fc
Finish can return more logs
LukeButters Jun 20, 2024
3f0dfac
Change over ScriptServiceV2Orchestrator to use new interface
samdanaei Jun 20, 2024
d80bc62
Fix factory
LukeButters Jun 20, 2024
11b96d4
Change KubernetesScriptServiceV1Orchestrator to use new interface
samdanaei Jun 20, 2024
6a4b1f6
get building
samdanaei Jun 20, 2024
e99bc2a
WIP
samdanaei Jun 20, 2024
a5e0589
Revert back to using tuples
samdanaei Jun 20, 2024
e4ee43c
Update k8s and v1
LukeButters Jun 20, 2024
9c0f285
WIP
samdanaei Jun 20, 2024
4796b05
Fix ssv2
LukeButters Jun 20, 2024
33bb1b4
Implement EventDrivenScriptExecutor
samdanaei Jun 21, 2024
9003b32
minor refactor
LukeButters Jun 21, 2024
28f0ccc
ssv2 fix
LukeButters Jun 21, 2024
87ccc83
.
LukeButters Jun 21, 2024
cd5086b
Push knowledge around SSV1 into its executor
LukeButters Jun 24, 2024
d8c2f32
Create the AggregateScriptExecutor
LukeButters Jun 24, 2024
228af09
.
LukeButters Jul 5, 2024
5bd1329
Merge branch 'main' into luke/event-driven-script-orchestrator
sburmanoctopus Jan 20, 2025
0702813
Merge fixes
sburmanoctopus Jan 20, 2025
9a00c66
Small tidy ups
sburmanoctopus Jan 20, 2025
af4d603
Renaming (and fixing bug with Kubernetes cancelling)
sburmanoctopus Jan 20, 2025
e00a9d2
Tidy
sburmanoctopus Jan 20, 2025
015fdb8
Adding result object
sburmanoctopus Jan 20, 2025
a754c9a
Using the same client object for executor
sburmanoctopus Jan 20, 2025
a377da4
Keeping metrics when using TentacleClient
sburmanoctopus Jan 20, 2025
3ed6087
Dealing with cancellation issues
sburmanoctopus Jan 21, 2025
d16a942
Rename
sburmanoctopus Jan 21, 2025
a13b162
Better wording
sburmanoctopus Jan 21, 2025
051a74d
Fixing cancellation on builds
sburmanoctopus Jan 21, 2025
8ba4e0f
PR fixes
sburmanoctopus Jan 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions source/Octopus.Tentacle.Client/EventDriven/CommandContext.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
using Octopus.Tentacle.Client.Scripts;
using Octopus.Tentacle.Contracts;

namespace Octopus.Tentacle.Client.EventDriven
{
/// <summary>
/// This class holds the context of where we are up to within the script execution life cycle.
/// When executing a script, there are several stages it goes through (e.g. starting the script, periodically checking status for completion, completing the script).
/// To be able to progress through these cycles in an event-driven environment, we need to remember some state, and then keep passing that state back into the script executor.
/// </summary>
public class CommandContext
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: consider documenting what this class is for.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. Done

{
public CommandContext(ScriptTicket scriptTicket,
long nextLogSequence,
ScriptServiceVersion scripServiceVersionUsed)
{
ScriptTicket = scriptTicket;
NextLogSequence = nextLogSequence;
ScripServiceVersionUsed = scripServiceVersionUsed;
}

public ScriptTicket ScriptTicket { get; }
public long NextLogSequence { get; }
public ScriptServiceVersion ScripServiceVersionUsed { get; }
}
}
118 changes: 118 additions & 0 deletions source/Octopus.Tentacle.Client/ScriptExecutor.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
using System;
using System.Threading;
using System.Threading.Tasks;
using Octopus.Tentacle.Client.EventDriven;
using Octopus.Tentacle.Client.Execution;
using Octopus.Tentacle.Client.Observability;
using Octopus.Tentacle.Client.Scripts;
using Octopus.Tentacle.Client.Scripts.Models;
using Octopus.Tentacle.Client.ServiceHelpers;
using Octopus.Tentacle.Contracts;
using Octopus.Tentacle.Contracts.Logging;
using Octopus.Tentacle.Contracts.Observability;

namespace Octopus.Tentacle.Client
{
/// <summary>
/// Executes scripts, on the best available script service.
/// </summary>
public class ScriptExecutor : IScriptExecutor
{
readonly ITentacleClientTaskLog logger;
readonly ClientOperationMetricsBuilder operationMetricsBuilder;
readonly TentacleClientOptions clientOptions;
readonly AllClients allClients;
readonly RpcCallExecutor rpcCallExecutor;
readonly TimeSpan onCancellationAbandonCompleteScriptAfter;

public ScriptExecutor(AllClients allClients,
ITentacleClientTaskLog logger,
ITentacleClientObserver tentacleClientObserver,
TentacleClientOptions clientOptions,
TimeSpan onCancellationAbandonCompleteScriptAfter)
: this(
allClients,
logger,
tentacleClientObserver,
// For now, we do not support operation based metrics when used outside the TentacleClient. So just plug in a builder to discard.
ClientOperationMetricsBuilder.Start(),
clientOptions,
onCancellationAbandonCompleteScriptAfter)
{
}

internal ScriptExecutor(AllClients allClients,
ITentacleClientTaskLog logger,
ITentacleClientObserver tentacleClientObserver,
ClientOperationMetricsBuilder operationMetricsBuilder,
TentacleClientOptions clientOptions,
TimeSpan onCancellationAbandonCompleteScriptAfter)
{
this.allClients = allClients;
this.logger = logger;
this.clientOptions = clientOptions;
this.onCancellationAbandonCompleteScriptAfter = onCancellationAbandonCompleteScriptAfter;
this.operationMetricsBuilder = operationMetricsBuilder;
rpcCallExecutor = RpcCallExecutorFactory.Create(this.clientOptions.RpcRetrySettings.RetryDuration, tentacleClientObserver);
}

public async Task<ScriptOperationExecutionResult> StartScript(ExecuteScriptCommand executeScriptCommand,
StartScriptIsBeingReAttempted startScriptIsBeingReAttempted,
CancellationToken cancellationToken)
{
var scriptServiceVersionToUse = await DetermineScriptServiceVersionToUse(cancellationToken);

var scriptExecutorFactory = CreateScriptExecutorFactory();
var scriptExecutor = scriptExecutorFactory.CreateScriptExecutor(scriptServiceVersionToUse);

return await scriptExecutor.StartScript(executeScriptCommand, startScriptIsBeingReAttempted, cancellationToken);
}

public async Task<ScriptOperationExecutionResult> GetStatus(CommandContext ticketForNextNextStatus, CancellationToken cancellationToken)
{
var scriptExecutorFactory = CreateScriptExecutorFactory();
var scriptExecutor = scriptExecutorFactory.CreateScriptExecutor(ticketForNextNextStatus.ScripServiceVersionUsed);

return await scriptExecutor.GetStatus(ticketForNextNextStatus, cancellationToken);
}

public async Task<ScriptOperationExecutionResult> CancelScript(CommandContext ticketForNextNextStatus)
{
var scriptExecutorFactory = CreateScriptExecutorFactory();
var scriptExecutor = scriptExecutorFactory.CreateScriptExecutor(ticketForNextNextStatus.ScripServiceVersionUsed);

return await scriptExecutor.CancelScript(ticketForNextNextStatus);
}

public async Task<ScriptStatus?> CompleteScript(CommandContext ticketForNextNextStatus, CancellationToken cancellationToken)
{
var scriptExecutorFactory = CreateScriptExecutorFactory();
var scriptExecutor = scriptExecutorFactory.CreateScriptExecutor(ticketForNextNextStatus.ScripServiceVersionUsed);

return await scriptExecutor.CompleteScript(ticketForNextNextStatus, cancellationToken);
}

ScriptExecutorFactory CreateScriptExecutorFactory()
{
return new ScriptExecutorFactory(allClients,
rpcCallExecutor,
operationMetricsBuilder,
onCancellationAbandonCompleteScriptAfter,
clientOptions,
logger);
}

async Task<ScriptServiceVersion> DetermineScriptServiceVersionToUse(CancellationToken cancellationToken)
{
try
{
var scriptServiceVersionSelector = new ScriptServiceVersionSelector(allClients.CapabilitiesServiceV2, logger, rpcCallExecutor, clientOptions, operationMetricsBuilder);
return await scriptServiceVersionSelector.DetermineScriptServiceVersionToUse(cancellationToken);
}
catch (Exception ex) when (cancellationToken.IsCancellationRequested)
{
throw new OperationCanceledException("Script execution was cancelled", ex);
}
}
}
}
42 changes: 42 additions & 0 deletions source/Octopus.Tentacle.Client/Scripts/IScriptExecutor.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
using System;
using System.Threading;
using System.Threading.Tasks;
using Octopus.Tentacle.Client.EventDriven;
using Octopus.Tentacle.Client.Scripts.Models;
using Octopus.Tentacle.Contracts;

namespace Octopus.Tentacle.Client.Scripts
{
public interface IScriptExecutor
{
/// <summary>
/// Start the script.
/// </summary>
/// <returns>The result, which includes the CommandContext for the next command</returns>
Task<ScriptOperationExecutionResult> StartScript(ExecuteScriptCommand command,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might make sense to document that the resulting CommandContext should be used for the next call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

StartScriptIsBeingReAttempted startScriptIsBeingReAttempted,
CancellationToken scriptExecutionCancellationToken);

/// <summary>
/// Get the status.
/// </summary>
/// <param name="commandContext">The CommandContext from the previous command</param>
/// <param name="scriptExecutionCancellationToken"></param>
/// <returns>The result, which includes the CommandContext for the next command</returns>
Task<ScriptOperationExecutionResult> GetStatus(CommandContext commandContext, CancellationToken scriptExecutionCancellationToken);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think I actually preferred the tuple because the responses where not something that are really coupled AND I think it improved readability. ie it was clear each method was returning a CommandContext which might be enough to hint to the user to use that CommandContext for the next call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think I actually preferred the tuple because the responses where not something that are really coupled
🤔 I feel that the fact we return both implies that they are related.

I think it improved readability. ie it was clear each method was returning a CommandContext which might be enough to hint to the user to use that CommandContext for the next call.

🤔 I would have thought the property named ContextForNextCommand would be good enough. Plus, adding the documentation from the previous comment should be more than enough.

I personally don't like tuples at all, and feel this is much easier to understand. For now, let's leave it the way it is, and if we find it unhelpful we can address it then.


/// <summary>
/// Cancel the script.
/// </summary>
/// <param name="commandContext">The CommandContext from the previous command</param>
/// <returns>The result, which includes the CommandContext for the next command</returns>
Task<ScriptOperationExecutionResult> CancelScript(CommandContext commandContext);

/// <summary>
/// Complete the script.
/// </summary>
/// <param name="commandContext">The CommandContext from the previous command</param>
/// <param name="scriptExecutionCancellationToken"></param>
Task<ScriptStatus?> CompleteScript(CommandContext commandContext, CancellationToken scriptExecutionCancellationToken);
}
}
12 changes: 0 additions & 12 deletions source/Octopus.Tentacle.Client/Scripts/IScriptOrchestrator.cs

This file was deleted.

This file was deleted.

Loading