Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gh-24311] Expand on documentation about jobs that are both parameterised and periodic #24384

Merged
merged 19 commits into from
Nov 8, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
bcb344b
docs: expand on documentation about jobs that are both parameterized …
Juanadelacuesta Nov 7, 2024
e57bbf4
fix: typo
Juanadelacuesta Nov 7, 2024
4095fc5
docs: expand on the example
Juanadelacuesta Nov 7, 2024
4aaba64
Update website/content/docs/job-specification/parameterized.mdx
Juanadelacuesta Nov 8, 2024
961ef58
Update website/content/docs/job-specification/parameterized.mdx
Juanadelacuesta Nov 8, 2024
389ceb9
Update website/content/docs/job-specification/parameterized.mdx
Juanadelacuesta Nov 8, 2024
7a937bf
Update website/content/docs/job-specification/parameterized.mdx
Juanadelacuesta Nov 8, 2024
72bf22f
Update website/content/docs/job-specification/periodic.mdx
Juanadelacuesta Nov 8, 2024
1053a53
Update website/content/docs/job-specification/parameterized.mdx
Juanadelacuesta Nov 8, 2024
0a1d152
Update website/content/docs/job-specification/periodic.mdx
Juanadelacuesta Nov 8, 2024
cd239bd
Update website/content/docs/job-specification/parameterized.mdx
Juanadelacuesta Nov 8, 2024
eb53a62
style: improve the content with PR suggestions
Juanadelacuesta Nov 8, 2024
ea4e464
periodic.mdx fix link to parameterized
aimeeu Nov 8, 2024
3125809
Update website/content/docs/job-specification/parameterized.mdx
Juanadelacuesta Nov 8, 2024
fa0849e
Update parameterized.mdx
Juanadelacuesta Nov 8, 2024
e233bbe
Update website/content/docs/job-specification/parameterized.mdx
Juanadelacuesta Nov 8, 2024
bbdea38
Update website/content/docs/job-specification/parameterized.mdx
Juanadelacuesta Nov 8, 2024
4301c57
Update website/content/docs/job-specification/parameterized.mdx
Juanadelacuesta Nov 8, 2024
f278259
Update parameterized.mdx
Juanadelacuesta Nov 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions website/content/docs/job-specification/parameterized.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -152,9 +152,59 @@ job "email-blast" {
}
```

### Interactions with [`periodic`][periodic]
Juanadelacuesta marked this conversation as resolved.
Show resolved Hide resolved

When trying to schedule a job that is both `parameterized` and `periodic`, an internal
hierarchy comes into play: A parametrized job should not be dispatched without
parameters by the periodic configuration, forcing the parameterized option to take
precedence: Once the job is dispatched and given parameters, the
periodic configuration comes into play and a new jobs will be dispatched according
to the periodic configurations and with the parameters of the triggering job.
Juanadelacuesta marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When trying to schedule a job that is both `parameterized` and `periodic`, an internal
hierarchy comes into play: A parametrized job should not be dispatched without
parameters by the periodic configuration, forcing the parameterized option to take
precedence: Once the job is dispatched and given parameters, the
periodic configuration comes into play and a new jobs will be dispatched according
to the periodic configurations and with the parameters of the triggering job.
Nomad uses an internal hierarchy when scheduling a job that is both `parameterized` and [`periodic`][periodic].
```plaintext
parameterized => periodic => batch
```
1. Nomad does not dispatch a periodic job with null parameters in the periodic configuration. This forces the parameterized job to take precedence over the periodic job.
1. After Nomad dispatches the parameterized job and gives it parameters, Nomad uses the periodic configuration.
1. Nomad dispatches new jobs according to the periodic configuration that uses thee parameters from the triggering parameterized job.

I reorganized based on Daniel's L164 suggestion. The opening paragraph contains a lot of information, and I had to read it a couple of times to understand it all. I tried to reorganize into a basic sentence structure and numbered list so the content is easier to scan.

Style guide -> present tense, active voice, direct language (see https://github.com/hashicorp/team-nomad/blob/main/wiki/docs.md#sentence-structure)

Copy link
Contributor

@aimeeu aimeeu Nov 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Juanadelacuesta do you want to include this suggestion? After your most recent push, this suggestion would replace L157-164.


Juanadelacuesta marked this conversation as resolved.
Show resolved Hide resolved
For example a job with the following configuration will not trigger any new jobs
until it is dispatched at least once, and after that the dispatched child will
periodically trigger more children with the given parameters.
Juanadelacuesta marked this conversation as resolved.
Show resolved Hide resolved

```hcl
periodic {
crons = [
"*/40 * * * * * *"
]
}
parameterized {
payload = "required"
meta_required = ["dispatcher_email"]
meta_optional = ["pager_email"]
}
```

The first job `batch/periodic/parameterized` corresponds to the parameterized and
periodic job, but it won't result in any new allocations until a `dispatch` with the
necessary parameters is submitted. The second job `batch/periodic` will trigger
the subsequent jobs `batch`. Any new dispatch will trigger a new flow of new
periodic jobs with the corresponding parameters.
Juanadelacuesta marked this conversation as resolved.
Show resolved Hide resolved

```
$ nomad job status
ID Type Submit Date
sync batch/periodic/parameterized 2024-11-07T10:43:30+01:00 // Original submitted job
sync/dispatch-1730972650-247c6e97 batch/periodic 2024-11-07T10:44:10+01:00 // First dispatched job with parameters A
sync/dispatch-1730972650-247c6e97/periodic-1730972680 batch 2024-11-07T10:44:40+01:00 // Cron job with parameters A
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whoa! I didn't notice these great // comments in the preview, because I didn't see that the little code block could scroll to the right; I only saw to the Type column. I'm not sure what to suggest for that... maybe say above this to "Scroll to the right for descriptions of each job."?

sync/dispatch-1730972650-247c6e97/periodic-1730972860 batch 2024-11-07T10:47:40+01:00 // Cron job with parameters A
sync/dispatch-1730972760-f79a96e1 batch/periodic 2024-11-07T10:46:00+01:00 // Second dispatched job with parameters B
sync/dispatch-1730972760-f79a96e1/periodic-1730972800 batch 2024-11-07T10:46:40+01:00 // Cron job with parameters B
sync/dispatch-1730972760-f79a96e1/periodic-1730972860 batch 2024-11-07T10:47:40+01:00 // Cron job with parameters B
```
If a periodic job needs to be forced by any reason, the corresponding parameterized job needs to be used:
Juanadelacuesta marked this conversation as resolved.
Show resolved Hide resolved

```
$ nomad job periodic force sync/dispatch-1730972650-247c6e97
```

[batch-type]: /nomad/docs/job-specification/job#type 'Batch scheduler type'
[dispatch command]: /nomad/docs/commands/job/dispatch 'Nomad Job Dispatch Command'
[resources]: /nomad/docs/job-specification/resources 'Nomad resources Job Specification'
[interpolation]: /nomad/docs/runtime/interpolation 'Nomad Runtime Interpolation'
[dispatch_payload]: /nomad/docs/job-specification/dispatch_payload 'Nomad dispatch_payload Job Specification'
[multiregion]: /nomad/docs/job-specification/multiregion#parameterized-dispatch
[periodic]: /nomad/docs/job-specification/periodic
4 changes: 4 additions & 0 deletions website/content/docs/job-specification/periodic.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,9 @@ consistent evaluation when Nomad spans multiple time zones.
prevents this job from running on the `cron` schedule but prevents force
launches.

See the [parameterized] documentation for additional considerations when
using in conjunction with parameterized options.
Juanadelacuesta marked this conversation as resolved.
Show resolved Hide resolved

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems a little buried under these Parameters. maybe move it up under Requirements?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this periodic page doesn't talk about parameterized options at all, what about moving this line right before the # periodic Requirements section?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a bit of a corner case, I didn't want to put it in the general description

## `periodic` Examples

The following examples only show the `periodic` blocks. Remember that the
Expand Down Expand Up @@ -116,3 +119,4 @@ configuring time zones for periodic jobs.
[cron]: https://github.com/hashicorp/cronexpr#implementation 'List of cron expressions'
[dst]: #daylight-saving-time
[multiregion]: /nomad/docs/job-specification/multiregion#periodic-time-zones
[parameterized]: /nomad/docs/job-specification/parameterized
Juanadelacuesta marked this conversation as resolved.
Show resolved Hide resolved