-
Notifications
You must be signed in to change notification settings - Fork 24
Various small proposed improvements #909
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 18 commits
aa2399d
465f46e
13b6832
4016765
f41651c
af9a92c
7c2e36f
fb49049
609cff9
a8572f0
c9d07d0
3a5972d
e7ad72c
9a16acf
f6affb5
5f29081
d0af77a
9b6a004
7f25cfb
ec2f4df
fd8f02b
24fd350
9cd0717
2b6a4e9
33eea11
9c2d534
e91c81b
be7c9ad
9a71138
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,19 +19,6 @@ Cylc runs on Unix-like systems including Linux and Mac OS. | |
| Via Conda (recommended) | ||
| ^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| .. tip:: | ||
|
|
||
| We recommend using Mamba to install Cylc. | ||
| Mamba (or Micromamba) can be used as | ||
| `a faster, drop-in replacement for the conda command | ||
| <https://mamba.readthedocs.io/en/latest/index.html>`_. | ||
| If using Conda, make sure to use the libmamba-solver by updating to | ||
| Conda 23.10 or using the `conda-libmamba-solver plugin for conda | ||
| <https://conda.github.io/conda-libmamba-solver/getting-started/>`_. | ||
| The classic conda environment solver may be too slow for a complex package | ||
| like Cylc. | ||
|
Comment on lines
-22
to
-32
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This tip is no longer relevant thanks to conda adopting the Libmamba dependency resolver |
||
|
|
||
|
|
||
| .. code-block:: sub | ||
|
|
||
| $ conda install -c conda-forge cylc-flow | ||
|
|
@@ -67,7 +54,7 @@ Via Pip (+npm) | |
| $ pip install cylc-rose metomi-rose | ||
|
|
||
| There are also certain optional extra requirements which you may choose to | ||
| install: | ||
| install (already included if installing via Conda): | ||
|
|
||
| .. code-block:: sub | ||
|
|
||
|
|
@@ -92,15 +79,16 @@ Non-Python Requirements | |
| .. _Graphviz: https://graphviz.org/download/ | ||
| .. _configurable-http-proxy: https://anaconda.org/conda-forge/configurable-http-proxy | ||
|
|
||
| These dependencies are not installed by Conda or pip: | ||
| These dependencies are not installed by Conda or pip and may not be available on your | ||
| system by default: | ||
|
|
||
| * ``bash`` | ||
| * GNU `coreutils`_ | ||
| * ``ssh`` | ||
| * ``rsync`` | ||
| * ``mail`` (optional - for automated email functionality) | ||
|
|
||
| These dependencies are installed by Conda but not by pip: | ||
| The following dependencies are installed automatically by Conda but not by pip: | ||
|
|
||
| * `Graphviz`_ (optional - used by ``cylc graph`` for displaying workflow | ||
| graphs) | ||
|
|
@@ -290,8 +278,8 @@ can be found: :ref:`UI_Server_config`. | |
| Bash Profile | ||
| ^^^^^^^^^^^^ | ||
|
|
||
| Cylc :term:`job scripts <job script>` are bash scripts, which is good for | ||
| manipulating files and processes, They invoke ``bash -l`` to allow environment | ||
| Cylc :term:`job scripts <job script>` are bash scripts, which are good for | ||
| manipulating files and processes. They invoke ``bash -l`` to allow environment | ||
| configuration in login scripts. | ||
|
|
||
| .. warning:: | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -60,7 +60,7 @@ Paste the following code into a ``flow.cylc`` file: | |
| [[[events]]] | ||
| mail events = failed | ||
| [[bell]] | ||
| script = printf 'bong%.02d\n' $(seq 1 $(cylc cyclepoint --print-hour)) | ||
| script = printf 'bong%.02d\n' $(seq 1 $(cylc cyclepoint --print-hour)); sleep 5 | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added sleep 5 to tutorial to allow users time to see the workflow running. A previous update to cylc sped up the running of this workflow significantly. |
||
|
|
||
| Change the initial cycle point to 00:00 this morning (e.g. if it was | ||
| the first of January 2000 we would write ``2000-01-01T00Z``). | ||
|
|
@@ -78,7 +78,6 @@ Stop the workflow after a few cycles using ``cylc stop --now --now clock-trigger | |
| Notice how the tasks run as soon as possible rather than | ||
| waiting for the actual time to be equal to the cycle point. | ||
|
|
||
|
|
||
| Clock-Triggering Tasks | ||
| ---------------------- | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -233,8 +233,9 @@ The ``root`` Family | |
|
|
||
| .. practical:: | ||
|
|
||
| .. rubric:: This practical continues on from the | ||
| :ref:`jinja2 practical <cylc-tutorial-jinja2-practical>`. | ||
| .. rubric:: In this practical we will consolidate the configuration of the | ||
| :ref:`weather-forecasting workflow <tutorial-cylc-runtime-forecasting-workflow>` | ||
| from the previous section. | ||
|
|
||
| 1. **Create A New Workflow.** | ||
|
|
||
|
|
@@ -251,7 +252,7 @@ The ``root`` Family | |
|
|
||
| .. code-block:: none | ||
|
|
||
| RESOLUTION = {{ RESOLUTION }} | ||
| RESOLUTION = 0.2 | ||
| DOMAIN = -12,46,12,61 # Do not change! | ||
|
|
||
| Rather than manually adding them to each task individually we could put | ||
|
|
@@ -266,7 +267,7 @@ The ``root`` Family | |
| + [[root]] | ||
| + [[[environment]]] | ||
| + # The dimensions of each grid cell in degrees. | ||
| + RESOLUTION = {{ RESOLUTION }} | ||
| + RESOLUTION = 0.2 | ||
| + # The area to generate forecasts for (lng1, lat1, lng2, lat2). | ||
| + DOMAIN = -12,46,12,61 # Do not change! | ||
|
|
||
|
|
@@ -276,23 +277,23 @@ The ``root`` Family | |
| script = consolidate-observations | ||
| - [[[environment]]] | ||
| - # The dimensions of each grid cell in degrees. | ||
| - RESOLUTION = {{ RESOLUTION }} | ||
| - RESOLUTION = 0.2 | ||
| - # The area to generate forecasts for (lng1, lat1, lng2, lat2). | ||
| - DOMAIN = -12,46,12,61 # Do not change! | ||
|
|
||
| [[get_rainfall]] | ||
| script = get-rainfall | ||
| [[[environment]]] | ||
| - [[[environment]]] | ||
| - # The dimensions of each grid cell in degrees. | ||
| - RESOLUTION = {{ RESOLUTION }} | ||
| - RESOLUTION = 0.2 | ||
| - # The area to generate forecasts for (lng1, lat1, lng2, lat2). | ||
| - DOMAIN = -12,46,12,61 # Do not change! | ||
|
|
||
| [[forecast]] | ||
| script = forecast 60 5 # Generate 5 forecasts at 60 minute intervals. | ||
| [[[environment]]] | ||
| - # The dimensions of each grid cell in degrees. | ||
| - RESOLUTION = {{ RESOLUTION }} | ||
| - RESOLUTION = 0.2 | ||
| - # The area to generate forecasts for (lng1, lat1, lng2, lat2) | ||
| - DOMAIN = -12,46,12,61 # Do not change! | ||
| # The path to the files containing wind data (the {variables} will | ||
|
|
@@ -309,11 +310,11 @@ The ``root`` Family | |
| MAP_TEMPLATE = "$CYLC_WORKFLOW_RUN_DIR/lib/template/map.html" | ||
|
|
||
| [[post_process_exeter]] | ||
| # Generate a forecast for Exeter 60 minutes into the future. | ||
| script = post-process exeter 60 | ||
| # Generate a forecast for Exeter 300 minutes into the future. | ||
| script = post-process exeter 300 | ||
|
Comment on lines
+272
to
+273
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is to match the change in the Jinja2 tutorial. |
||
| - [[[environment]]] | ||
| - # The dimensions of each grid cell in degrees. | ||
| - RESOLUTION = {{ RESOLUTION }} | ||
| - RESOLUTION = 0.2 | ||
| - # The area to generate forecasts for (lng1, lat1, lng2, lat2). | ||
| - DOMAIN = -12,46,12,61 # Do not change! | ||
|
|
||
|
|
||
|
samuel-denton marked this conversation as resolved.
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -165,57 +165,57 @@ This would result in: | |
|
|
||
| .. ifslides:: | ||
|
|
||
| Next section: :ref:`tutorial-cylc-families` | ||
| Next section: :ref:`tutorial-cylc-parameters` | ||
|
|
||
|
|
||
|
|
||
| .. _cylc-tutorial-jinja2-practical: | ||
|
|
||
| .. practical:: | ||
|
|
||
| .. rubric:: In this practical we will consolidate the configuration of the | ||
| :ref:`weather-forecasting workflow <tutorial-cylc-runtime-forecasting-workflow>` | ||
| from the previous section. | ||
| .. rubric:: This practical continues on from the | ||
| :ref:`Families practical <cylc-tutorial-families-practical>`. | ||
|
|
||
| 3. **Use Jinja2 To Avoid Duplication.** | ||
|
|
||
| The ``RESOLUTION`` environment variable is used by multiple tasks. | ||
| Rather than writing it out multiple times we will use Jinja2 | ||
| to centralise this configuration. | ||
| We have seen how families can be used to avoid duplication in task definitions. | ||
| They are, however, limited to definitions, and only where properties exactly match. | ||
| In contrast, Jinja2 can be used in any part of a workflow, and can be used | ||
| for simple programmatic logic. Here, we will make use of Jinja2 for some | ||
| maths and string manipulation. | ||
|
|
||
| At the top of the :cylc:conf:`flow.cylc` file add the Jinja2 shebang line. Then | ||
| copy the value of the ``RESOLUTION`` environment variable and use it to | ||
| define an ``RESOLUTION`` Jinja2 variable: | ||
| At the top of the :cylc:conf:`flow.cylc` file you should see the Jinja2 | ||
| shebang line has been included for you. Create some new Jinja2 variables | ||
| for ``FORECAST_LENGTH`` and ``FORECAST_COUNT```: | ||
|
samuel-denton marked this conversation as resolved.
Outdated
|
||
|
|
||
| .. code-block:: cylc | ||
|
|
||
| #!Jinja2 | ||
| #!Jinja2 | ||
|
|
||
| {% set RESOLUTION = 0.2 %} | ||
| {% set FORECAST_LENGTH = 60 %} | ||
| {% set FORECAST_COUNT = 5 %} | ||
|
|
||
| Next replace the key, where it appears in the workflow, with | ||
| ``{{ RESOLUTION }}``: | ||
| Next replace the parameters for the forecast script within the ``forecast`` | ||
| task with these variables: | ||
|
|
||
| .. code-block:: diff | ||
|
|
||
| [[get_rainfall]] | ||
| script = get-rainfall | ||
| [[[environment]]] | ||
| - RESOLUTION = 0.2 | ||
| + RESOLUTION = {{ RESOLUTION }} | ||
|
|
||
| [[forecast]] | ||
| script = forecast 60 5 # Generate 5 forecasts at 60 minute intervals. | ||
| [[[environment]]] | ||
| - RESOLUTION = 0.2 | ||
| + RESOLUTION = {{ RESOLUTION }} | ||
|
|
||
| [[post_process_exeter]] | ||
| # Generate a forecast for Exeter 60 minutes in the future. | ||
| script = post-process exeter 60 | ||
| [[[environment]]] | ||
| - RESOLUTION = 0.2 | ||
| + RESOLUTION = {{ RESOLUTION }} | ||
| [[forecast]] | ||
| - script = forecast 60 5 | ||
|
samuel-denton marked this conversation as resolved.
Outdated
|
||
| + script = forecast {{ FORECAST_LENGTH }} {{ FORECAST_COUNT }} | ||
|
|
||
| Then, in the ``post-processing`` task, replace the timestep parameter for the | ||
| post-processing script with the a simple multiplication of the two variables. | ||
| You can also use Jinja2 within a comment allowing it to match the dynamic value: | ||
|
|
||
| .. code-block:: diff | ||
|
|
||
| [[post_process_exeter]] | ||
| - # Generate a forecast for Exeter 300 minutes in the future. | ||
| - script = post-process exeter 300 | ||
| + # Generate a forecast {{ FORECAST_LENGTH * FORECAST_COUNT }} minutes in the future. | ||
| + script = post-process exeter {{ FORECAST_LENGTH * FORECAST_COUNT }} | ||
|
|
||
| Check the result with ``cylc config``. The Jinja2 will be processed | ||
| so you should not see any difference after making these changes. | ||
| so you should not see any difference after making these changes, other than | ||
| the MAP_FILE filename now including the resolution. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think the filename change is no longer present in the PR?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep... I've changed this text so many times now I'm loosing track! |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -182,7 +182,7 @@ Parameters can be either strings or integers: | |
| .. ifslides:: | ||
|
|
||
| .. rubric:: This practical continues on from the | ||
| :ref:`Families practical <cylc-tutorial-families-practical>`. | ||
| :ref:`Jinja2 practical <cylc-tutorial-jinja2-practical>`. | ||
|
|
||
| Next section: :ref:`Which approach to use | ||
| <cylc-tutorial-consolidation-conclusion>` | ||
|
Comment on lines
182
to
188
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's currently inconsistent which pages make use of the ifslides / ifnotslides blocks. Are these still used for anything? I understood they were previously used for allowing the docs to generate in other formats but not sure if anything still uses them @oliver-sanders ? I can make an effort to clean them up if not, or add any missing if they are need. The JinJa2 page for example is missing some compared to the parametrization or families pages. |
||
|
|
@@ -314,7 +314,7 @@ Parameters can be either strings or integers: | |
|
|
||
| [task parameters] | ||
| station = aldergrove, camborne, heathrow, shetland | ||
| + site = exeter, edinburgh | ||
| + site = exeter, edinburgh | ||
|
|
||
| Next we parameterize the task in the graph: | ||
|
|
||
|
|
@@ -330,6 +330,7 @@ Parameters can be either strings or integers: | |
|
|
||
| -[[post_process_exeter]] | ||
| +[[post_process<site>]] | ||
| # Generate a forecast for Exeter 60 minutes in the future. | ||
| - script = post-process exeter 60 | ||
| + script = post-process $CYLC_TASK_PARAM_site 60 | ||
| - # Generate a forecast for Exeter 300 minutes in the future. | ||
| - script = post-process exeter 300 | ||
| + # Generate a forecast {{ FORECAST_LENGTH * FORECAST_COUNT }} minutes in the future for <site> | ||
| + script = post-process $CYLC_TASK_PARAM_site {{ FORECAST_LENGTH * FORECAST_COUNT }} | ||
|
samuel-denton marked this conversation as resolved.
Outdated
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -113,7 +113,7 @@ Re-Run Multiple Tasks | |
| :width: 75% | ||
|
|
||
| .. tab-item:: CLI | ||
| :sync: gui | ||
| :sync: cli | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Small mistake here caused tabs to sync incorrectly. |
||
|
|
||
| .. code-block:: console | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lots of small wording changes to clarify what is and is not installed via pip vs conda.