-
Notifications
You must be signed in to change notification settings - Fork 24
Various small proposed improvements #909
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
aa2399d
465f46e
13b6832
4016765
f41651c
af9a92c
7c2e36f
fb49049
609cff9
a8572f0
c9d07d0
3a5972d
e7ad72c
9a16acf
f6affb5
5f29081
d0af77a
9b6a004
7f25cfb
ec2f4df
fd8f02b
24fd350
9cd0717
2b6a4e9
33eea11
9c2d534
e91c81b
be7c9ad
9a71138
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,19 +19,6 @@ Cylc runs on Unix-like systems including Linux and Mac OS. | |
| Via Conda (recommended) | ||
| ^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| .. tip:: | ||
|
|
||
| We recommend using Mamba to install Cylc. | ||
| Mamba (or Micromamba) can be used as | ||
| `a faster, drop-in replacement for the conda command | ||
| <https://mamba.readthedocs.io/en/latest/index.html>`_. | ||
| If using Conda, make sure to use the libmamba-solver by updating to | ||
| Conda 23.10 or using the `conda-libmamba-solver plugin for conda | ||
| <https://conda.github.io/conda-libmamba-solver/getting-started/>`_. | ||
| The classic conda environment solver may be too slow for a complex package | ||
| like Cylc. | ||
|
Comment on lines
-22
to
-32
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This tip is no longer relevant thanks to conda adopting the Libmamba dependency resolver |
||
|
|
||
|
|
||
| .. code-block:: sub | ||
|
|
||
| $ conda install -c conda-forge cylc-flow | ||
|
|
@@ -67,7 +54,7 @@ Via Pip (+npm) | |
| $ pip install cylc-rose metomi-rose | ||
|
|
||
| There are also certain optional extra requirements which you may choose to | ||
| install: | ||
| install (already included if installing via Conda): | ||
|
|
||
| .. code-block:: sub | ||
|
|
||
|
|
@@ -92,15 +79,16 @@ Non-Python Requirements | |
| .. _Graphviz: https://graphviz.org/download/ | ||
| .. _configurable-http-proxy: https://anaconda.org/conda-forge/configurable-http-proxy | ||
|
|
||
| These dependencies are not installed by Conda or pip: | ||
| These dependencies are not installed by Conda or pip and may not be available on your | ||
| system by default: | ||
|
|
||
| * ``bash`` | ||
| * GNU `coreutils`_ | ||
| * ``ssh`` | ||
| * ``rsync`` | ||
| * ``mail`` (optional - for automated email functionality) | ||
|
|
||
| These dependencies are installed by Conda but not by pip: | ||
| The following dependencies are installed automatically by Conda but not by pip: | ||
|
|
||
| * `Graphviz`_ (optional - used by ``cylc graph`` for displaying workflow | ||
| graphs) | ||
|
|
@@ -290,8 +278,8 @@ can be found: :ref:`UI_Server_config`. | |
| Bash Profile | ||
| ^^^^^^^^^^^^ | ||
|
|
||
| Cylc :term:`job scripts <job script>` are bash scripts, which is good for | ||
| manipulating files and processes, They invoke ``bash -l`` to allow environment | ||
| Cylc :term:`job scripts <job script>` are bash scripts, which are good for | ||
| manipulating files and processes. They invoke ``bash -l`` to allow environment | ||
| configuration in login scripts. | ||
|
|
||
| .. warning:: | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -60,7 +60,7 @@ Paste the following code into a ``flow.cylc`` file: | |
| [[[events]]] | ||
| mail events = failed | ||
| [[bell]] | ||
| script = printf 'bong%.02d\n' $(seq 1 $(cylc cyclepoint --print-hour)) | ||
| script = printf 'bong%.02d\n' $(seq 1 $(cylc cyclepoint --print-hour)); sleep 5 | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added sleep 5 to tutorial to allow users time to see the workflow running. A previous update to cylc sped up the running of this workflow significantly. |
||
|
|
||
| Change the initial cycle point to 00:00 this morning (e.g. if it was | ||
| the first of January 2000 we would write ``2000-01-01T00Z``). | ||
|
|
@@ -78,7 +78,6 @@ Stop the workflow after a few cycles using ``cylc stop --now --now clock-trigger | |
| Notice how the tasks run as soon as possible rather than | ||
| waiting for the actual time to be equal to the cycle point. | ||
|
|
||
|
|
||
| Clock-Triggering Tasks | ||
| ---------------------- | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,42 +6,40 @@ Families | |
| :term:`Families <family>` provide a way of grouping tasks together so they can | ||
| be treated as one. | ||
|
|
||
| This example adds a new environment variable to the configuration. ``GET_NEARBY`` shows how families can make workflows simpler. We are going to add it to the ``get_observations`` script. If the script cannot get data from the named site this setting will allow it to try nearby sites. | ||
| They can be used to consolidate tasks runtime configuration and environment variables, | ||
| as well as to simplify the workflow's :term:`graph` by grouping together related tasks. | ||
|
|
||
| Runtime | ||
| ------- | ||
|
|
||
| .. ifnotslides:: | ||
|
|
||
| :term:`Families <family>` are groups of tasks which share a common | ||
| configuration. In the present example the common configuration is: | ||
| configuration. In this example we have multiple tasks sharing this configuration: | ||
|
|
||
| .. code-block:: cylc | ||
|
|
||
| script = get-observations | ||
| [[[environment]]] | ||
| GET_NEARBY = true | ||
| [[consolidate_observations]] | ||
| [[[environment]]] | ||
| DOMAIN = -12,46,12,61 | ||
|
|
||
| We define a family as a new task consisting of the common configuration. By | ||
| convention families are named in upper case: | ||
|
|
||
| .. code-block:: cylc | ||
|
|
||
| [[GET_OBSERVATIONS]] | ||
| script = get-observations | ||
| [[PROCESS_DATA]] | ||
| [[[environment]]] | ||
| GET_NEARBY = true | ||
| DOMAIN = -12,46,12,61 | ||
|
|
||
| .. ifnotslides:: | ||
|
|
||
| We "add" tasks to a family using the ``inherit`` setting: | ||
|
|
||
| .. code-block:: cylc | ||
|
|
||
| [[get_observations_heathrow]] | ||
| inherit = GET_OBSERVATIONS | ||
| [[[environment]]] | ||
| SITE_ID = 3772 | ||
| [[consolidate_observations]] | ||
| inherit = PROCESS_DATA | ||
|
|
||
| .. ifnotslides:: | ||
|
|
||
|
|
@@ -51,67 +49,28 @@ Runtime | |
|
|
||
| .. code-block:: cylc | ||
|
|
||
| [[get_observations_heathrow]] | ||
| script = get-observations | ||
| [[consolidate_observations]] | ||
| [[[environment]]] | ||
| SITE_ID = 3772 | ||
| GET_NEARBY = true | ||
| DOMAIN = -12,46,12,61 | ||
|
|
||
| .. nextslide:: | ||
|
|
||
| .. ifnotslides:: | ||
|
|
||
| It is possible to override inherited configuration within the task. For | ||
| example if we wanted the ``get_observations_heathrow`` task to fail rather | ||
| than use a nearby alternative: | ||
| example if we wanted the ``consolidate_observations`` task to use a different | ||
| domain compared to the other members of the family we could do: | ||
|
|
||
| .. code-block:: cylc | ||
| :emphasize-lines: 4 | ||
|
|
||
| [[get_observations_heathrow]] | ||
| inherit = GET_OBSERVATIONS | ||
| [[consolidate_observations]] | ||
| inherit = PROCESS_DATA | ||
| [[[environment]]] | ||
| SITE_ID = 3772 | ||
| GET_NEARBY = false | ||
| DOMAIN = -10,40,10,60 | ||
|
|
||
| .. nextslide:: | ||
|
|
||
| .. ifnotslides:: | ||
|
|
||
| Using families the ``get_observations`` tasks could be written in a | ||
| shorter form: | ||
|
|
||
| .. code-block:: diff | ||
|
|
||
| [runtime] | ||
| [[GET_OBSERVATIONS]] | ||
| script = get-observations | ||
| + [[[environment]]] | ||
| + GET_NEARBY = true | ||
|
|
||
| [[get_observations_heathrow]] | ||
| inherit = GET_OBSERVATIONS | ||
| script = get_metar_observation | ||
| [[[environment]]] | ||
| SITE_ID = EGLL | ||
| - GET_NEARBY = true | ||
| [[get_observations_camborne]] | ||
| inherit = GET_OBSERVATIONS | ||
| [[[environment]]] | ||
| SITE_ID = 3808 | ||
| - GET_NEARBY = true | ||
| [[get_observations_shetland]] | ||
| inherit = GET_OBSERVATIONS | ||
| [[[environment]]] | ||
| - GET_NEARBY = true | ||
| SITE_ID = 3005 | ||
| [[get_observations_aldergrove]] | ||
| inherit = GET_OBSERVATIONS | ||
| [[[environment]]] | ||
| SITE_ID = 3917 | ||
| - GET_NEARBY = true | ||
|
|
||
|
|
||
| Graphing | ||
| -------- | ||
|
|
||
|
|
@@ -233,8 +192,9 @@ The ``root`` Family | |
|
|
||
| .. practical:: | ||
|
|
||
| .. rubric:: This practical continues on from the | ||
| :ref:`jinja2 practical <cylc-tutorial-jinja2-practical>`. | ||
| .. rubric:: In this practical we will consolidate the configuration of the | ||
| :ref:`weather-forecasting workflow <tutorial-cylc-runtime-forecasting-workflow>` | ||
| from the previous section. | ||
|
|
||
| 1. **Create A New Workflow.** | ||
|
|
||
|
|
@@ -251,7 +211,7 @@ The ``root`` Family | |
|
|
||
| .. code-block:: none | ||
|
|
||
| RESOLUTION = {{ RESOLUTION }} | ||
| RESOLUTION = 0.2 | ||
| DOMAIN = -12,46,12,61 # Do not change! | ||
|
|
||
| Rather than manually adding them to each task individually we could put | ||
|
|
@@ -266,7 +226,7 @@ The ``root`` Family | |
| + [[root]] | ||
| + [[[environment]]] | ||
| + # The dimensions of each grid cell in degrees. | ||
| + RESOLUTION = {{ RESOLUTION }} | ||
| + RESOLUTION = 0.2 | ||
| + # The area to generate forecasts for (lng1, lat1, lng2, lat2). | ||
| + DOMAIN = -12,46,12,61 # Do not change! | ||
|
|
||
|
|
@@ -276,23 +236,23 @@ The ``root`` Family | |
| script = consolidate-observations | ||
| - [[[environment]]] | ||
| - # The dimensions of each grid cell in degrees. | ||
| - RESOLUTION = {{ RESOLUTION }} | ||
| - RESOLUTION = 0.2 | ||
| - # The area to generate forecasts for (lng1, lat1, lng2, lat2). | ||
| - DOMAIN = -12,46,12,61 # Do not change! | ||
|
|
||
| [[get_rainfall]] | ||
| script = get-rainfall | ||
| [[[environment]]] | ||
| - [[[environment]]] | ||
| - # The dimensions of each grid cell in degrees. | ||
| - RESOLUTION = {{ RESOLUTION }} | ||
| - RESOLUTION = 0.2 | ||
| - # The area to generate forecasts for (lng1, lat1, lng2, lat2). | ||
| - DOMAIN = -12,46,12,61 # Do not change! | ||
|
|
||
| [[forecast]] | ||
| script = forecast 60 5 # Generate 5 forecasts at 60 minute intervals. | ||
| [[[environment]]] | ||
| - # The dimensions of each grid cell in degrees. | ||
| - RESOLUTION = {{ RESOLUTION }} | ||
| - RESOLUTION = 0.2 | ||
| - # The area to generate forecasts for (lng1, lat1, lng2, lat2) | ||
| - DOMAIN = -12,46,12,61 # Do not change! | ||
| # The path to the files containing wind data (the {variables} will | ||
|
|
@@ -309,11 +269,11 @@ The ``root`` Family | |
| MAP_TEMPLATE = "$CYLC_WORKFLOW_RUN_DIR/lib/template/map.html" | ||
|
|
||
| [[post_process_exeter]] | ||
| # Generate a forecast for Exeter 60 minutes into the future. | ||
| script = post-process exeter 60 | ||
| # Generate a forecast for Exeter 300 minutes into the future. | ||
| script = post-process exeter 300 | ||
|
Comment on lines
+272
to
+273
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is to match the change in the Jinja2 tutorial. |
||
| - [[[environment]]] | ||
| - # The dimensions of each grid cell in degrees. | ||
| - RESOLUTION = {{ RESOLUTION }} | ||
| - RESOLUTION = 0.2 | ||
| - # The area to generate forecasts for (lng1, lat1, lng2, lat2). | ||
| - DOMAIN = -12,46,12,61 # Do not change! | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lots of small wording changes to clarify what is and is not installed via pip vs conda.