diff --git a/src/img/cylc-cli.png b/src/img/cylc-cli.png new file mode 100644 index 0000000000..575286f2f3 Binary files /dev/null and b/src/img/cylc-cli.png differ diff --git a/src/installation.rst b/src/installation.rst index e23884deb6..c35cd68909 100644 --- a/src/installation.rst +++ b/src/installation.rst @@ -19,19 +19,6 @@ Cylc runs on Unix-like systems including Linux and Mac OS. Via Conda (recommended) ^^^^^^^^^^^^^^^^^^^^^^^ -.. tip:: - - We recommend using Mamba to install Cylc. - Mamba (or Micromamba) can be used as - `a faster, drop-in replacement for the conda command - `_. - If using Conda, make sure to use the libmamba-solver by updating to - Conda 23.10 or using the `conda-libmamba-solver plugin for conda - `_. - The classic conda environment solver may be too slow for a complex package - like Cylc. - - .. code-block:: sub $ conda install -c conda-forge cylc-flow @@ -67,7 +54,7 @@ Via Pip (+npm) $ pip install cylc-rose metomi-rose There are also certain optional extra requirements which you may choose to -install: +install (already included if installing via Conda): .. code-block:: sub @@ -92,7 +79,8 @@ Non-Python Requirements .. _Graphviz: https://graphviz.org/download/ .. _configurable-http-proxy: https://anaconda.org/conda-forge/configurable-http-proxy -These dependencies are not installed by Conda or pip: +These dependencies are not installed by Conda or pip and may not be available on your +system by default: * ``bash`` * GNU `coreutils`_ @@ -100,7 +88,7 @@ These dependencies are not installed by Conda or pip: * ``rsync`` * ``mail`` (optional - for automated email functionality) -These dependencies are installed by Conda but not by pip: +The following dependencies are installed automatically by Conda but not by pip: * `Graphviz`_ (optional - used by ``cylc graph`` for displaying workflow graphs) @@ -290,8 +278,8 @@ can be found: :ref:`UI_Server_config`. Bash Profile ^^^^^^^^^^^^ -Cylc :term:`job scripts ` are bash scripts, which is good for -manipulating files and processes, They invoke ``bash -l`` to allow environment +Cylc :term:`job scripts ` are bash scripts, which are good for +manipulating files and processes. They invoke ``bash -l`` to allow environment configuration in login scripts. .. warning:: diff --git a/src/tutorial/furthertopics/clock-triggered-tasks.rst b/src/tutorial/furthertopics/clock-triggered-tasks.rst index 0da5e7ec92..958fcc72d4 100644 --- a/src/tutorial/furthertopics/clock-triggered-tasks.rst +++ b/src/tutorial/furthertopics/clock-triggered-tasks.rst @@ -60,7 +60,7 @@ Paste the following code into a ``flow.cylc`` file: [[[events]]] mail events = failed [[bell]] - script = printf 'bong%.02d\n' $(seq 1 $(cylc cyclepoint --print-hour)) + script = printf 'bong%.02d\n' $(seq 1 $(cylc cyclepoint --print-hour)); sleep 5 Change the initial cycle point to 00:00 this morning (e.g. if it was the first of January 2000 we would write ``2000-01-01T00Z``). @@ -78,7 +78,6 @@ Stop the workflow after a few cycles using ``cylc stop --now --now clock-trigger Notice how the tasks run as soon as possible rather than waiting for the actual time to be equal to the cycle point. - Clock-Triggering Tasks ---------------------- diff --git a/src/tutorial/img/cylc-tools.png b/src/tutorial/img/cylc-tools.png deleted file mode 100644 index 298445dff3..0000000000 Binary files a/src/tutorial/img/cylc-tools.png and /dev/null differ diff --git a/src/tutorial/introduction.rst b/src/tutorial/introduction.rst index ebaada27d8..ced905cdbe 100644 --- a/src/tutorial/introduction.rst +++ b/src/tutorial/introduction.rst @@ -88,10 +88,24 @@ Cylc can automatically: Cylc provides a variety of command line and GUI tools for visualising, monitoring, and controlling workflows. The Cylc TUI (Terminal -User Interface), web GUI, and ``cylc scan`` (bottom left) are shown below. +User Interface), web GUI, and CLI (Command Line Interface) are shown below. -.. image:: /tutorial/img/cylc-tools.png - :alt: A screenshot of several Cylc tools. +.. tab-set:: + + .. tab-item:: GUI + + .. image:: ../../src/img/cylc-ui-dash.png + :alt: A screenshot of the Cylc GUI dashboard. + + .. tab-item:: Tui + + .. image:: ../../src/img/tui-1.png + :alt: A screenshot of the Cylc TUI. + + .. tab-item:: CLI + + .. image:: ../../src/img/cylc-cli.png + :alt: A screenshot of the Cylc CLI. .. nextslide:: diff --git a/src/tutorial/runtime/configuration-consolidation/families.rst b/src/tutorial/runtime/configuration-consolidation/families.rst index b092369c48..f2bd6cce8a 100644 --- a/src/tutorial/runtime/configuration-consolidation/families.rst +++ b/src/tutorial/runtime/configuration-consolidation/families.rst @@ -6,7 +6,8 @@ Families :term:`Families ` provide a way of grouping tasks together so they can be treated as one. -This example adds a new environment variable to the configuration. ``GET_NEARBY`` shows how families can make workflows simpler. We are going to add it to the ``get_observations`` script. If the script cannot get data from the named site this setting will allow it to try nearby sites. +They can be used to consolidate tasks runtime configuration and environment variables, +as well as to simplify the workflow's :term:`graph` by grouping together related tasks. Runtime ------- @@ -14,23 +15,22 @@ Runtime .. ifnotslides:: :term:`Families ` are groups of tasks which share a common - configuration. In the present example the common configuration is: + configuration. In this example we have multiple tasks sharing this configuration: .. code-block:: cylc - script = get-observations - [[[environment]]] - GET_NEARBY = true + [[consolidate_observations]] + [[[environment]]] + DOMAIN = -12,46,12,61 We define a family as a new task consisting of the common configuration. By convention families are named in upper case: .. code-block:: cylc - [[GET_OBSERVATIONS]] - script = get-observations + [[PROCESS_DATA]] [[[environment]]] - GET_NEARBY = true + DOMAIN = -12,46,12,61 .. ifnotslides:: @@ -38,10 +38,8 @@ Runtime .. code-block:: cylc - [[get_observations_heathrow]] - inherit = GET_OBSERVATIONS - [[[environment]]] - SITE_ID = 3772 + [[consolidate_observations]] + inherit = PROCESS_DATA .. ifnotslides:: @@ -51,67 +49,28 @@ Runtime .. code-block:: cylc - [[get_observations_heathrow]] - script = get-observations + [[consolidate_observations]] [[[environment]]] - SITE_ID = 3772 - GET_NEARBY = true + DOMAIN = -12,46,12,61 .. nextslide:: .. ifnotslides:: It is possible to override inherited configuration within the task. For - example if we wanted the ``get_observations_heathrow`` task to fail rather - than use a nearby alternative: + example if we wanted the ``consolidate_observations`` task to use a different + domain compared to the other members of the family we could do: .. code-block:: cylc :emphasize-lines: 4 - [[get_observations_heathrow]] - inherit = GET_OBSERVATIONS + [[consolidate_observations]] + inherit = PROCESS_DATA [[[environment]]] - SITE_ID = 3772 - GET_NEARBY = false + DOMAIN = -10,40,10,60 .. nextslide:: -.. ifnotslides:: - - Using families the ``get_observations`` tasks could be written in a - shorter form: - -.. code-block:: diff - - [runtime] - [[GET_OBSERVATIONS]] - script = get-observations - + [[[environment]]] - + GET_NEARBY = true - - [[get_observations_heathrow]] - inherit = GET_OBSERVATIONS - script = get_metar_observation - [[[environment]]] - SITE_ID = EGLL - - GET_NEARBY = true - [[get_observations_camborne]] - inherit = GET_OBSERVATIONS - [[[environment]]] - SITE_ID = 3808 - - GET_NEARBY = true - [[get_observations_shetland]] - inherit = GET_OBSERVATIONS - [[[environment]]] - - GET_NEARBY = true - SITE_ID = 3005 - [[get_observations_aldergrove]] - inherit = GET_OBSERVATIONS - [[[environment]]] - SITE_ID = 3917 - - GET_NEARBY = true - - Graphing -------- @@ -233,8 +192,9 @@ The ``root`` Family .. practical:: - .. rubric:: This practical continues on from the - :ref:`jinja2 practical `. + .. rubric:: In this practical we will consolidate the configuration of the + :ref:`weather-forecasting workflow ` + from the previous section. 1. **Create A New Workflow.** @@ -251,7 +211,7 @@ The ``root`` Family .. code-block:: none - RESOLUTION = {{ RESOLUTION }} + RESOLUTION = 0.2 DOMAIN = -12,46,12,61 # Do not change! Rather than manually adding them to each task individually we could put @@ -266,7 +226,7 @@ The ``root`` Family + [[root]] + [[[environment]]] + # The dimensions of each grid cell in degrees. - + RESOLUTION = {{ RESOLUTION }} + + RESOLUTION = 0.2 + # The area to generate forecasts for (lng1, lat1, lng2, lat2). + DOMAIN = -12,46,12,61 # Do not change! @@ -276,15 +236,15 @@ The ``root`` Family script = consolidate-observations - [[[environment]]] - # The dimensions of each grid cell in degrees. - - RESOLUTION = {{ RESOLUTION }} + - RESOLUTION = 0.2 - # The area to generate forecasts for (lng1, lat1, lng2, lat2). - DOMAIN = -12,46,12,61 # Do not change! [[get_rainfall]] script = get-rainfall - [[[environment]]] + - [[[environment]]] - # The dimensions of each grid cell in degrees. - - RESOLUTION = {{ RESOLUTION }} + - RESOLUTION = 0.2 - # The area to generate forecasts for (lng1, lat1, lng2, lat2). - DOMAIN = -12,46,12,61 # Do not change! @@ -292,7 +252,7 @@ The ``root`` Family script = forecast 60 5 # Generate 5 forecasts at 60 minute intervals. [[[environment]]] - # The dimensions of each grid cell in degrees. - - RESOLUTION = {{ RESOLUTION }} + - RESOLUTION = 0.2 - # The area to generate forecasts for (lng1, lat1, lng2, lat2) - DOMAIN = -12,46,12,61 # Do not change! # The path to the files containing wind data (the {variables} will @@ -309,11 +269,11 @@ The ``root`` Family MAP_TEMPLATE = "$CYLC_WORKFLOW_RUN_DIR/lib/template/map.html" [[post_process_exeter]] - # Generate a forecast for Exeter 60 minutes into the future. - script = post-process exeter 60 + # Generate a forecast for Exeter 300 minutes into the future. + script = post-process exeter 300 - [[[environment]]] - # The dimensions of each grid cell in degrees. - - RESOLUTION = {{ RESOLUTION }} + - RESOLUTION = 0.2 - # The area to generate forecasts for (lng1, lat1, lng2, lat2). - DOMAIN = -12,46,12,61 # Do not change! diff --git a/src/tutorial/runtime/configuration-consolidation/index.rst b/src/tutorial/runtime/configuration-consolidation/index.rst index 7ad00b8f9c..4103c25860 100644 --- a/src/tutorial/runtime/configuration-consolidation/index.rst +++ b/src/tutorial/runtime/configuration-consolidation/index.rst @@ -73,8 +73,8 @@ structure of a workflow and avoid duplication. .. toctree:: :maxdepth: 1 - jinja2 families + jinja2 parameters @@ -139,10 +139,12 @@ The Three Approaches could use them to simplify the workflow from the previous tutorial. *Work through them in order!* -* :ref:`families ` -* :ref:`jinja2 ` -* :ref:`parameters ` +.. toctree:: + :maxdepth: 1 + families + jinja2 + parameters .. _cylc-tutorial-consolidation-conclusion: diff --git a/src/tutorial/runtime/configuration-consolidation/jinja2.rst b/src/tutorial/runtime/configuration-consolidation/jinja2.rst index 2681a412d4..3dc31f88e9 100644 --- a/src/tutorial/runtime/configuration-consolidation/jinja2.rst +++ b/src/tutorial/runtime/configuration-consolidation/jinja2.rst @@ -165,7 +165,7 @@ This would result in: .. ifslides:: - Next section: :ref:`tutorial-cylc-families` + Next section: :ref:`tutorial-cylc-parameters` @@ -173,49 +173,48 @@ This would result in: .. practical:: - .. rubric:: In this practical we will consolidate the configuration of the - :ref:`weather-forecasting workflow ` - from the previous section. + .. rubric:: This practical continues on from the + :ref:`Families practical `. 3. **Use Jinja2 To Avoid Duplication.** - The ``RESOLUTION`` environment variable is used by multiple tasks. - Rather than writing it out multiple times we will use Jinja2 - to centralise this configuration. + We have seen how families can be used to avoid duplication in task definitions. + They are, however, limited to definitions, and only where properties exactly match. + In contrast, Jinja2 can be used in any part of a workflow, and can be used + for simple programmatic logic. Here, we will make use of Jinja2 for some + maths and string manipulation. - At the top of the :cylc:conf:`flow.cylc` file add the Jinja2 shebang line. Then - copy the value of the ``RESOLUTION`` environment variable and use it to - define an ``RESOLUTION`` Jinja2 variable: + At the top of the :cylc:conf:`flow.cylc` file you should see the Jinja2 + shebang line has been included for you. Create some new Jinja2 variables + for ``FORECAST_LENGTH`` and ``FORECAST_COUNT``: .. code-block:: cylc - #!Jinja2 + #!Jinja2 - {% set RESOLUTION = 0.2 %} + {% set FORECAST_LENGTH = 60 %} + {% set FORECAST_COUNT = 5 %} - Next replace the key, where it appears in the workflow, with - ``{{ RESOLUTION }}``: + Next replace the parameters for the forecast script within the ``forecast`` + task with these variables: .. code-block:: diff - [[get_rainfall]] - script = get-rainfall - [[[environment]]] - - RESOLUTION = 0.2 - + RESOLUTION = {{ RESOLUTION }} - - [[forecast]] - script = forecast 60 5 # Generate 5 forecasts at 60 minute intervals. - [[[environment]]] - - RESOLUTION = 0.2 - + RESOLUTION = {{ RESOLUTION }} - - [[post_process_exeter]] - # Generate a forecast for Exeter 60 minutes in the future. - script = post-process exeter 60 - [[[environment]]] - - RESOLUTION = 0.2 - + RESOLUTION = {{ RESOLUTION }} + [[forecast]] + - script = forecast 60 5 # Generate 5 forecasts at 60 minute intervals. + + script = forecast {{ FORECAST_LENGTH }} {{ FORECAST_COUNT }} + + Then, in the ``post-processing`` task, replace the timestep parameter for the + post-processing script with the a simple multiplication of the two variables. + You can also use Jinja2 within a comment allowing it to match the dynamic value: + + .. code-block:: diff + + [[post_process_exeter]] + - # Generate a forecast for Exeter 300 minutes in the future. + - script = post-process exeter 300 + + # Generate a forecast [length * count] minutes in the future. + + script = post-process exeter {{ FORECAST_LENGTH * FORECAST_COUNT }} Check the result with ``cylc config``. The Jinja2 will be processed so you should not see any difference after making these changes. diff --git a/src/tutorial/runtime/configuration-consolidation/parameters.rst b/src/tutorial/runtime/configuration-consolidation/parameters.rst index d6d916c044..114a3791cb 100644 --- a/src/tutorial/runtime/configuration-consolidation/parameters.rst +++ b/src/tutorial/runtime/configuration-consolidation/parameters.rst @@ -182,7 +182,7 @@ Parameters can be either strings or integers: .. ifslides:: .. rubric:: This practical continues on from the - :ref:`Families practical `. + :ref:`Jinja2 practical `. Next section: :ref:`Which approach to use ` @@ -314,7 +314,7 @@ Parameters can be either strings or integers: [task parameters] station = aldergrove, camborne, heathrow, shetland - + site = exeter, edinburgh + + site = exeter, edinburgh Next we parameterize the task in the graph: @@ -330,6 +330,6 @@ Parameters can be either strings or integers: -[[post_process_exeter]] +[[post_process]] - # Generate a forecast for Exeter 60 minutes in the future. - - script = post-process exeter 60 - + script = post-process $CYLC_TASK_PARAM_site 60 + # Generate a forecast [length * count] minutes in the future. + - script = post-process exeter {{ FORECAST_LENGTH * FORECAST_COUNT }} + + script = post-process "$CYLC_TASK_PARAM_site" {{ FORECAST_LENGTH * FORECAST_COUNT }} diff --git a/src/tutorial/scheduling/integer-cycling.rst b/src/tutorial/scheduling/integer-cycling.rst index 26d0af6c1d..97fdd3dc47 100644 --- a/src/tutorial/scheduling/integer-cycling.rst +++ b/src/tutorial/scheduling/integer-cycling.rst @@ -70,8 +70,9 @@ To make a workflow repeat we must tell Cylc three things: ` integer labels. * ``initial cycle point = 1`` tells Cylc to start counting cycle points from 1. - * ``P1`` is the :term:`recurrence`; a ``P1`` :term:`graph string` - repeats at every integer :term:`cycle point`. + * ``P1`` is the :term:`recurrence`. The tasks in a ``P1`` + :term:`graph string` repeat at every integer :term:`cycle point` + (whereas ``R1`` means "run once"). The first three :term:`cycles` look like this, with the entire workflow repeated at each cycle point: diff --git a/src/user-guide/interventions/index.rst b/src/user-guide/interventions/index.rst index d89a5e75aa..dc4b796e8e 100644 --- a/src/user-guide/interventions/index.rst +++ b/src/user-guide/interventions/index.rst @@ -113,7 +113,7 @@ Re-Run Multiple Tasks :width: 75% .. tab-item:: CLI - :sync: gui + :sync: cli .. code-block:: console