Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graphql: Submit Workflow Execution #838

Open
wants to merge 30 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
e2d0546
more fields
JeffreyThiessen Jan 5, 2024
6afe1db
cleanup, comments, fixes
JeffreyThiessen Jan 12, 2024
b26367c
add test cases
JeffreyThiessen Jan 12, 2024
188e366
add test for workflow executions samples resolver
JeffreyThiessen Jan 12, 2024
24fb9d9
update tests
JeffreyThiessen May 10, 2024
cd147ba
add new tests for new resolver
JeffreyThiessen May 10, 2024
3652d13
add cleaned to type
JeffreyThiessen May 28, 2024
0dcbfa3
add workflow namespace type connection
JeffreyThiessen Oct 29, 2024
52c63c1
attempt fixing namespace access
JeffreyThiessen Nov 1, 2024
f281eb9
fix namespace fields
JeffreyThiessen Nov 1, 2024
7954352
add tests for workflow executions on groups and on projects
JeffreyThiessen Nov 1, 2024
6ed75d5
get samples through SamplesWorkflowExecutions instead of directly
JeffreyThiessen Nov 26, 2024
3efda81
dump schema
JeffreyThiessen Dec 19, 2024
61ff6f9
files added, not working yet
JeffreyThiessen Oct 29, 2024
ef744d2
working logic
JeffreyThiessen Nov 6, 2024
a5cc96e
Restructure code flow. no error handling yet.
JeffreyThiessen Nov 7, 2024
a09635d
error handling and attachment validation for workflow execution
JeffreyThiessen Nov 8, 2024
e14a5ff
todo
JeffreyThiessen Nov 22, 2024
499c4ad
restructure samples workflow execution attributes
JeffreyThiessen Dec 10, 2024
2cd89a9
add test cases
JeffreyThiessen Dec 12, 2024
89f4204
fix schema
JeffreyThiessen Dec 19, 2024
712cbf1
fix rebase errors
JeffreyThiessen Dec 19, 2024
e3cffe9
fix rebase error duplicating tests
JeffreyThiessen Dec 19, 2024
2a05aab
fix one"
JeffreyThiessen Jan 7, 2025
6c95ea5
fix tests hopefully
JeffreyThiessen Jan 7, 2025
ba57036
linting
JeffreyThiessen Jan 8, 2025
5581613
fix seeds not passing validation
JeffreyThiessen Jan 10, 2025
e5bc4b1
fix new test case
JeffreyThiessen Jan 10, 2025
e7fff0f
requested changes
JeffreyThiessen Jan 13, 2025
658bd98
verify samples match internally and that attachments match samples
JeffreyThiessen Jan 15, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion app/controllers/workflow_executions_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ def workflow_execution_params_attributes

def samples_workflow_execution_params_attributes
[
:id,
:id, # index, increment for each one, not necissary for functionality
:sample_id,
{ samplesheet_params: {} }
]
Expand Down
134 changes: 134 additions & 0 deletions app/graphql/mutations/submit_workflow_execution.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# frozen_string_literal: true

module Mutations
# Base Mutation
class SubmitWorkflowExecution < BaseMutation # rubocop:disable Metrics/ClassLength
null true
description 'Create a new workflow execution..'

argument :email_notification,
Boolean,
required: false,
default_value: false,
description: 'Set to true to enable email notifications from this workflow execution'
argument :name, String, required: false, description: 'Name for the new workflow.'
argument :samples_workflow_executions_attributes, [GraphQL::Types::JSON], description: "A list of hashes containing a 'sample_id', and a hash of `samplesheet_params`." # rubocop:disable GraphQL/ExtractInputType,Layout/LineLength
argument :update_samples, # rubocop:disable GraphQL/ExtractInputType
Boolean,
required: false,
default_value: false,
description: 'Set true for samples to be updated from this workflow execution'
argument :workflow_engine, String, description: '' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_engine_parameters, [GraphQL::Types::JSON], description: 'List of Hashes containing `key` and `value` to be passed to the workflow engine.' # rubocop:disable GraphQL/ExtractInputType,Layout/LineLength
argument :workflow_engine_version, String, description: 'Workflow Engine Version' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_name, String, description: 'Name of the pipeline to be run on this workflow execution' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_params, GraphQL::Types::JSON, description: 'Parameters to be passed to the pipeline.' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_type, String, description: 'Type of pipelines workflow.' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_type_version, String, description: 'Version of the pipelines workflow type.' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_url, String, description: 'Url for the pipeline.' # rubocop:disable GraphQL/ExtractInputType
argument :workflow_version, String, description: 'Version of the pipeline to be run on this workflow execution' # rubocop:disable GraphQL/ExtractInputType

# one of project/group, to use as the namespace
argument :project_id, ID, # rubocop:disable GraphQL/ExtractInputType
required: false,
description: 'The Node ID of the project to run workflow in. For example, `gid://irida/Project/a84cd757-dedb-4c64-8b01-097020163077`.' # rubocop:disable Layout/LineLength
argument :group_id, ID, # rubocop:disable GraphQL/OrderedArguments,GraphQL/ExtractInputType
required: false,
description: 'The Node ID of the group to run workflow in. For example, `gid://irida/Group/a84cd757-dedb-4c64-8b01-097020163077`.' # rubocop:disable Layout/LineLength
validates required: { one_of: %i[project_id group_id] }

field :errors, [Types::UserErrorType], null: false, description: 'A list of errors that prevented the mutation.'
field :workflow_execution, Types::WorkflowExecutionType, description: 'The newly created workflow execution.'

def resolve(args)
create_workflow_execution(args)
end

def ready?(**_args)
authorize!(to: :mutate?, with: GraphqlPolicy, context: { user: context[:current_user], token: context[:token] })
end

private

def create_workflow_execution(args) # rubocop:disable Metrics/MethodLength,Metrics/AbcSize
workflow_engine_parameters = build_workflow_engine_parameters(args[:workflow_engine_parameters])

update_samples = args[:update_samples] ? '1' : '0'
email_notification = args[:email_notification] ? '1' : '0'

namespace_id = namespace(args[:project_id], args[:group_id])

samples_workflow_executions_attributes = build_samples_workflow_executions_attributes(args[:samples_workflow_executions_attributes]) # rubocop:disable Layout/LineLength

workflow_execution_params = {
name: args[:name],
metadata: { workflow_name: args[:workflow_name],
workflow_version: args[:workflow_version] },
namespace_id:,
workflow_params: args[:workflow_params],
workflow_type: args[:workflow_type],
workflow_type_version: args[:workflow_type_version],
workflow_engine: args[:workflow_engine],
workflow_engine_version: args[:workflow_engine_version],
workflow_engine_parameters:,
workflow_url: args[:workflow_url],
update_samples:,
email_notification:,
samples_workflow_executions_attributes:
}

workflow_execution = WorkflowExecutions::CreateService.new(
current_user, workflow_execution_params
).execute

if workflow_execution.persisted?
{
workflow_execution:,
errors: []
}
else
user_errors = workflow_execution.errors.map do |error|
{
path: ['workflow_execution', error.attribute.to_s.camelize(:lower)],
message: error.message
}
end
{
workflow_execution: nil,
errors: user_errors
}
end
end

def namespace(project_id, group_id)
if project_id
IridaSchema.object_from_id(project_id, { expected_type: Project }).namespace.id
else # group_id
IridaSchema.object_from_id(group_id, { expected_type: Group }).id
end
end

# workflow engine parameters can have keys that start with `-` which is not allowed in graphql,
# so we parse a list of key value pairs into a hash that can be used.
def build_workflow_engine_parameters(parameter_list)
result = {}
parameter_list.each do |params|
result[params['key']] = params['value']
end
result
end

def build_samples_workflow_executions_attributes(samples_workflow_executions_attributes)
result = {}
samples_workflow_executions_attributes.each_with_index do |data, index|
sample_id = IridaSchema.object_from_id(data['sample_id'], { expected_type: Sample }).id
result[index.to_s] = {
'sample_id' => sample_id,
'samplesheet_params' => data['samplesheet_params']
}
end

result
end
end
end
115 changes: 115 additions & 0 deletions app/graphql/schema.graphql
Original file line number Diff line number Diff line change
Expand Up @@ -1416,6 +1416,16 @@ type Mutation {
input: CreateSampleInput!
): CreateSamplePayload

"""
Create a new workflow execution..
"""
submitWorkflowExecution(
"""
Parameters for SubmitWorkflowExecution
"""
input: SubmitWorkflowExecutionInput!
): SubmitWorkflowExecutionPayload

"""
Transfer a list of sample to another project.
"""
Expand Down Expand Up @@ -2857,6 +2867,111 @@ type SamplesWorkflowExecution implements Node {
workflowExecution: WorkflowExecution
}

"""
Autogenerated input type of SubmitWorkflowExecution
"""
input SubmitWorkflowExecutionInput {
"""
A unique identifier for the client performing the mutation.
"""
clientMutationId: String

"""
Set to true to enable email notifications from this workflow execution
"""
emailNotification: Boolean = false

"""
The Node ID of the group to run workflow in. For example, `gid://irida/Group/a84cd757-dedb-4c64-8b01-097020163077`.
"""
groupId: ID

"""
Name for the new workflow.
"""
name: String

"""
The Node ID of the project to run workflow in. For example, `gid://irida/Project/a84cd757-dedb-4c64-8b01-097020163077`.
"""
projectId: ID

"""
A list of hashes containing a 'sample_id', and a hash of `samplesheet_params`.
"""
samplesWorkflowExecutionsAttributes: [JSON!]!

"""
Set true for samples to be updated from this workflow execution
"""
updateSamples: Boolean = false

"""
"""
workflowEngine: String!

"""
List of Hashes containing `key` and `value` to be passed to the workflow engine.
"""
workflowEngineParameters: [JSON!]!

"""
Workflow Engine Version
"""
workflowEngineVersion: String!

"""
Name of the pipeline to be run on this workflow execution
"""
workflowName: String!

"""
Parameters to be passed to the pipeline.
"""
workflowParams: JSON!

"""
Type of pipelines workflow.
"""
workflowType: String!

"""
Version of the pipelines workflow type.
"""
workflowTypeVersion: String!

"""
Url for the pipeline.
"""
workflowUrl: String!

"""
Version of the pipeline to be run on this workflow execution
"""
workflowVersion: String!
}

"""
Autogenerated return type of SubmitWorkflowExecution.
"""
type SubmitWorkflowExecutionPayload {
"""
A unique identifier for the client performing the mutation.
"""
clientMutationId: String

"""
A list of errors that prevented the mutation.
"""
errors: [UserError!]!

"""
The newly created workflow execution.
"""
workflowExecution: WorkflowExecution
}

"""
Autogenerated input type of TransferSamples
"""
Expand Down
1 change: 1 addition & 0 deletions app/graphql/types/mutation_type.rb
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ class MutationType < Types::BaseObject
field :create_group, mutation: Mutations::CreateGroup # rubocop:disable GraphQL/FieldDescription
field :create_project, mutation: Mutations::CreateProject # rubocop:disable GraphQL/FieldDescription
field :create_sample, mutation: Mutations::CreateSample # rubocop:disable GraphQL/FieldDescription,GraphQL/ExtractType
field :submit_workflow_execution, mutation: Mutations::SubmitWorkflowExecution # rubocop:disable GraphQL/FieldDescription
field :transfer_samples, mutation: Mutations::TransferSamples # rubocop:disable GraphQL/FieldDescription
field :update_sample_metadata, mutation: Mutations::UpdateSampleMetadata # rubocop:disable GraphQL/FieldDescription
end
Expand Down
3 changes: 2 additions & 1 deletion app/models/workflow_execution.rb
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ class WorkflowExecution < ApplicationRecord

validates :metadata, presence: true, json: { message: ->(errors) { errors }, schema: METADATA_JSON_SCHEMA }
validate :validate_namespace
validates_with WorkflowExecutionSamplesheetParamsValidator

enum :state, %i[initial prepared submitted running completing completed error canceling canceled]

Expand All @@ -40,7 +41,7 @@ def send_email
end

def cancellable?
%w[submitted running prepared initial].include?(state)
%w[submitted running prepared initial].include?(state)
end

def deletable?
Expand Down
35 changes: 35 additions & 0 deletions app/validators/workflow_execution_samplesheet_params_validator.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# frozen_string_literal: true

# Validator for Workflow Execution Samplesheet Params
# This will cause the validation to fail if any of the attachment ids cannot be resolved
class WorkflowExecutionSamplesheetParamsValidator < ActiveModel::Validator
def validate(record) # rubocop:disable Metrics/MethodLength,Metrics/AbcSize,Metrics/CyclomaticComplexity,Metrics/PerceivedComplexity
if record.samples_workflow_executions.empty?
record.errors.add :base, 'Missing samplesheet params'
else
record.samples_workflow_executions.each do |sample_workflow_execution|
sample_workflow_execution.samplesheet_params.each do |key, value|
next if value == ''

if key == 'sample'
unless value == (sample_workflow_execution.sample.puid)
record.errors.add :sample, 'Provided Sample PUID does not match SampleWorkflowExecution Sample PUID'
end
next
end

begin
# Attempt to parse an object from the id provided
attachment = IridaSchema.object_from_id(value, { expected_type: Attachment })
unless attachment.attachable == sample_workflow_execution.sample
record.errors.add :attachment, 'Attachment does not belong to Sample.'
end
rescue StandardError => e
record.errors.add :attachment, e.message
next
end
end
end
end
end
end
21 changes: 18 additions & 3 deletions db/seeds.rb
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,15 @@
workflow_engine_version: '23.10.0',
workflow_engine_parameters: { '-r': 'dev' },
workflow_url: 'https://github.com/phac-nml/iridanextexample',
submitter: User.find_by(email: '[email protected]')
submitter: User.find_by(email: '[email protected]'),
samples_workflow_executions_attributes: {
'0': {
sample_id: Sample.first.id,

Check warning on line 172 in db/seeds.rb

View workflow job for this annotation

GitHub Actions / runner / rubocop

[rubocop] reported by reviewdog 🐶 Use 2 spaces for indentation in a hash, relative to the start of the line where the left curly brace is. Raw Output: db/seeds.rb:172:11: C: Layout/FirstHashElementIndentation: Use 2 spaces for indentation in a hash, relative to the start of the line where the left curly brace is.
samplesheet_params: {
sample: Sample.first.puid
}
}

Check warning on line 176 in db/seeds.rb

View workflow job for this annotation

GitHub Actions / runner / rubocop

[rubocop] reported by reviewdog 🐶 Indent the right brace the same as the start of the line where the left brace is. Raw Output: db/seeds.rb:176:9: C: Layout/FirstHashElementIndentation: Indent the right brace the same as the start of the line where the left brace is.
}
)

SamplesWorkflowExecution.create(
Expand All @@ -188,7 +196,15 @@
workflow_url: 'https://github.com/phac-nml/iridanextexample',
submitter: User.find_by(email: '[email protected]'),
blob_run_directory: 'this should be a generated key',
state: :completed
state: :completed,
samples_workflow_executions_attributes: {
'0': {
sample_id: Sample.first.id,

Check warning on line 202 in db/seeds.rb

View workflow job for this annotation

GitHub Actions / runner / rubocop

[rubocop] reported by reviewdog 🐶 Use 2 spaces for indentation in a hash, relative to the start of the line where the left curly brace is. Raw Output: db/seeds.rb:202:11: C: Layout/FirstHashElementIndentation: Use 2 spaces for indentation in a hash, relative to the start of the line where the left curly brace is.
samplesheet_params: {
sample: Sample.first.puid
}
}

Check warning on line 206 in db/seeds.rb

View workflow job for this annotation

GitHub Actions / runner / rubocop

[rubocop] reported by reviewdog 🐶 Indent the right brace the same as the start of the line where the left brace is. Raw Output: db/seeds.rb:206:9: C: Layout/FirstHashElementIndentation: Indent the right brace the same as the start of the line where the left brace is.
}
)

filename = 'summary.txt'
Expand All @@ -197,7 +213,6 @@
attachment.save!

SamplesWorkflowExecution.create(
samplesheet_params: { my_key1: 'my_value_2', my_key2: 'my_value_3' },
sample: Sample.first,
workflow_execution: workflow_execution_completed
)
Expand Down
Loading
Loading