From 69a6035ff64427231299e71b16ff5a5061092f35 Mon Sep 17 00:00:00 2001 From: Oliver Sanders Date: Fri, 2 Feb 2024 15:06:49 +0000 Subject: [PATCH 1/5] tidy --- _layouts/default.html | 549 +++++++++++++++++++++--------------------- 1 file changed, 272 insertions(+), 277 deletions(-) diff --git a/_layouts/default.html b/_layouts/default.html index 3729c59..5c47dbe 100644 --- a/_layouts/default.html +++ b/_layouts/default.html @@ -7,307 +7,302 @@ {%- include header.html -%} -
-
-
-
-
- - NIWA - Met Office - US Air Force - US Navy - AGBOM - CSIRO - MOES - Altair - ICM - Korea - COECSS - METS - SAWS - ISENES - NCFAS - ESIWACE - -
-
+
+
+
+
+
+ + NIWA + Met Office + US Air Force + US Navy + AGBOM + CSIRO + MOES + Altair + ICM + Korea + COECSS + METS + SAWS + ISENES + NCFAS + ESIWACE +
+
+
+ + +
+
+
+

+ Cylc is a general purpose workflow engine that also + orchestrates cycling systems very efficiently. + It is used in production weather, climate, and environmental + forecasting on HPC, but is not specialized to those domains. +

+ +

+ Cylc is an active Open Source project + on GitHub. +
+ See Cylc Publications and Reports below. +

+
+
+
+
+
+ + + + + +
-
-
- -
- -

- Cylc is a general purpose workflow engine that also - orchestrates cycling systems very efficiently. - It is used in production weather, climate, and environmental - forecasting on HPC, but is not specialized to those domains. -

- -

- Cylc is an active Open Source project - on GitHub. -
- See Cylc Publications and Reports below. -

+
+ + + + + +
-
-
-
- -
- - - - - -
- -
- - - - - -
-
- -

Next Release:  - () -

-
-
-
-
- 0% - 100% -
+

Next Release:  + () +

+
+
+
+
+ 0% + 100%
+
-
-
-
-
-

Cylc Does Cycling Properly

-
-

- Cylc unwinds the cycle loop to get a dependency graph with - no barrier between cycles. This allows interleaved cycles - for: -

    -
  • Fast catch-up from delays
    • -
  • Sustained high throughput off the clock
    • -
  • Seamless transitions between fast catch-up and - clock-limited real-time scheduling
    • -
-

-
-
-
- - - +
+
+
+
+

Cylc Does Cycling Properly

+
+

+ Cylc unwinds the cycle loop to get a dependency graph with + no barrier between cycles. This allows interleaved cycles + for: +

    +
  • Fast catch-up from delays
    • +
  • Sustained high throughput off the clock
    • +
  • Seamless transitions between fast catch-up and + clock-limited real-time scheduling
    • +
+

+
+ + + +
- -
-
- -

Why Use Cylc?

- - Production Ready -
    -
  • Cylc is used 24/7 in production weather forecasting, - notorious for the size and complexity of its workflows.
    • -
- - Scaling -
    -
  • Each workflow gets a scheduler that runs as the user - and relies only on the filesystem.
    • -
  • Schedulers can be distributed over a pool of hosts, and - jobs over job platforms.
    • -
  • Individual workflows scale to thousands of tasks per cycle.
    • -
- - Powerful UIs -
    -
  • A CLI that can operate on multiple workflows and tasks at once.
    • -
  • A web GUI, served by a - Jupyter - Server extension.
    • -
  • A terminal UI with control capability.
    • -
- - Ease of Use -
    -
  • Cylc installs from - Conda Forge - or PyPI and works - "out of the box" on a Linux host.
    • -
  • Small workflows are easy. Many features to help with large, - complex workflows.
    • -
  • With no central scheduler or database server, Cylc has low - admin overheads and a small security footprint.
    • -
  • Built in support for HPC resource managers like PBS and - Slurm.
    • -
- - Multi-User Installations -
    -
  • Cylc leverages - JupyterHub for - authentication and to launch Cylc UI Servers.
    • -
  • Fine-grained authorization of control by other users.
    • -
- - Advanced Scheduling -
    -
  • Graph branching from optional task outputs.
    • -
  • Supports multiple flows in the same workflow, e.g. to - re-run a past subgraph while the main flow continues.
    • -
  • Start a flow from anywhere in the depedency graph.
    • -
  • Cycle interleaving: this cut catch-up from delays from ~24 - hours to ~30 minutes at NIWA.
    • +
+ +
+
+ +

Why Use Cylc?

+ + Production Ready +
    +
  • Cylc is used 24/7 in production weather forecasting, + notorious for the size and complexity of its workflows.
    • +
+ + Scaling +
    +
  • Each workflow gets a scheduler that runs as the user + and relies only on the filesystem.
    • +
  • Schedulers can be distributed over a pool of hosts, and + jobs over job platforms.
    • +
  • Individual workflows scale to thousands of tasks per cycle.
    • +
+ + Powerful UIs +
    +
  • A CLI that can operate on multiple workflows and tasks at once.
    • +
  • A web GUI, served by a + Jupyter + Server extension.
    • +
  • A terminal UI with control capability.
    • +
+ + Ease of Use +
    +
  • Cylc installs from + Conda Forge + or PyPI and works + "out of the box" on a Linux host.
    • +
  • Small workflows are easy. Many features to help with large, + complex workflows.
    • +
  • With no central scheduler or database server, Cylc has low + admin overheads and a small security footprint.
    • +
  • Built in support for HPC resource managers like PBS and + Slurm.
    • +
+ + Multi-User Installations +
    +
  • Cylc leverages + JupyterHub for + authentication and to launch Cylc UI Servers.
    • +
  • Fine-grained authorization of control by other users.
- Resilience -
    -
  • Configurable task retries.
    • -
  • Triggering off of all task states.
    • -
  • Comprehensive event handling.
    • -
  • Manual job control: poll, kill, or retrigger jobs.
    • -
  • Restart from latest state even if the scheduler was killed, - and follow up on orphaned jobs.
    • -
  • Cylc is cluster-aware: it can interact with jobs via multiple platform hosts.
    • -
+ Advanced Scheduling +
    +
  • Graph branching from optional task outputs.
    • +
  • Supports multiple flows in the same workflow, e.g. to + re-run a past subgraph while the main flow continues.
    • +
  • Start a flow from anywhere in the depedency graph.
    • +
  • Cycle interleaving: this cut catch-up from delays from ~24 + hours to ~30 minutes at NIWA.
    • +
+ + Resilience +
    +
  • Configurable task retries.
    • +
  • Triggering off of all task states.
    • +
  • Comprehensive event handling.
    • +
  • Manual job control: poll, kill, or retrigger jobs.
    • +
  • Restart from latest state even if the scheduler was killed, + and follow up on orphaned jobs.
    • +
  • Cylc is cluster-aware: it can interact with jobs via multiple platform hosts.
    • +
-
- -
-
-

Publications and Reports

-

If you use Cylc please cite it in your - publications and let us - know if your site can be - listed on this website. -

- -

- - Workflow Automation for Cycling Systems: The - Cylc Workflow Engine - -
- H. Oliver et al -
- - Computing in Science & Engineering Vol - 21, Issue 4, July/Aug 2019. DOI: 10.1109/MCSE.2019.2906593 - +

+ +
+
+

Publications and Reports

+

If you use Cylc please cite it in your + publications and let us + know if your site can be + listed on this website. +

+ +

+ + Workflow Automation for Cycling Systems: The + Cylc Workflow Engine + +
+ H. Oliver et al +
+ + Computing in Science & Engineering Vol + 21, Issue 4, July/Aug 2019. DOI: 10.1109/MCSE.2019.2906593 + +
+ + + + DOI: 10.1109/MCSE.2019.2906593 + + + + + + + + DOI + + 10.1109/MCSE.2019.2906593 + + +

+ +

+ + Cylc: A Workflow Engine for Cycling Systems + +
+ Oliver et al., (2018). +
+ + Journal of Open Source Software, 3(27), 737. + DOI: 10.21105/joss.00737 + +
+ + +
- - - - DOI: 10.1109/MCSE.2019.2906593 - - - - - - - - DOI - - 10.1109/MCSE.2019.2906593 - - -

- -

- - Cylc: A Workflow Engine for Cycling Systems - -
- Oliver et al., (2018). -
- - Journal of Open Source Software, 3(27), 737. - DOI: 10.21105/joss.00737 - -
- - - -
-

- -

- - Citable DOI for Cylc code releases. - -
- - - -

-

- Old (2017): - - Cylc response to Assessment Report on Autosubmit, Cylc and ecFlow - -

-
+

+ +

+ + Citable DOI for Cylc code releases. + +
+ + + +

+

+ Old (2017): + + Cylc response to Assessment Report on Autosubmit, Cylc and ecFlow + +

+
{%- include header.html -%}