Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document dynamic directive syntax with named arguments #5758

Merged
merged 3 commits into from
Feb 13, 2025
Merged

Document dynamic directive syntax with named arguments #5758

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1da5f53
Document dynamic directive syntax with named arguments
bentsherman Feb 6, 2025
e7fd011
Apply suggestions from code review
bentsherman Feb 6, 2025
776c23e
Merge branch 'master' into docs-dynamic-directive-syntax
bentsherman Feb 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 17 additions & 4 deletions docs/process.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1293,13 +1293,26 @@ process foo {
}
```

In the above example the {ref}`process-memory` and execution {ref}`process-time` limits are defined dynamically. The first time the process is executed the `task.attempt` is set to `1`, thus it will request a two GB of memory and one hour of maximum execution time.
In the above example the {ref}`process-memory` and execution {ref}`process-time` limits are defined dynamically. The first time the process is executed the `task.attempt` is set to `1`, thus it will request 2 GB of memory and 1 hour of walltime.

If the task execution fail reporting an exit status in the range between 137 and 140, the task is re-submitted (otherwise terminates immediately). This time the value of `task.attempt` is `2`, thus increasing the amount of the memory to four GB and the time to 2 hours, and so on.
If the task execution fails with an exit status between 137 and 140, the task is re-executed; otherwise, the run is terminated immediately. The re-executed task will have `task.attempt` set to `2`, and will request 4 GB of memory and 2 hours of walltime.

The directive {ref}`process-maxretries` set the maximum number of time the same task can be re-executed.
The {ref}`process-maxretries` directive sets the maximum number of times the same task can be re-executed.

:::{tip}
Directives with named arguments, such as `accelerator` and `disk`, must use a more verbose syntax when they are dynamic. For example:

```nextflow
// static request
disk 375.GB, type: 'local-ssd'

// dynamic request
disk { [request: 375.GB * task.attempt, type: 'local-ssd'] }
```
:::

### Dynamic task resources with previous execution trace

:::{versionadded} 24.10.0
:::

Expand All @@ -1317,8 +1330,8 @@ process foo {
"""
}
```
In the above example, the {ref}`process-memory` is set according to previous trace record metrics. In the first attempt, when no trace metrics are available, it is set to one GB. In the subsequent attempts, it doubles the previously allocated memory. See {ref}`trace-report` for more information about trace records.

In the above example, the {ref}`process-memory` is set according to previous trace record metrics. In the first attempt, when no trace metrics are available, it is set to 1 GB. In each subsequent attempt, the requested memory is doubled. See {ref}`trace-report` for more information about trace records.

### Dynamic retry with backoff

Expand Down