Update spack-packages from spack-packages/develop 2026/06/04

[climbfuji]

Description

This PR pulls in the upstream spack-packages develop branch as of 2026/06/04.

Additional changes:

1. Fix dependencies for py-pybtex, see Update dependencies for py-pybtex spack/spack-packages#5141 for the upstream PR
2. Compiler flags for py-lxml, see py-lxml@5: flags to compile with gcc@14 / oneapi@2026 spack/spack-packages#5307 for the upstream PR
3. cylc-related updates: Restore py-cylc-uiserver (new versions) + dependency updates spack/spack-packages#5174
4. Bug fix for py-xarray: versions 2024 and older only work with py-pandas 2.1 and older: xarray@:2024 requires pandas@:2.1 spack/spack-packages#5313

Packages with merge conflicts that should be examined by the reviewers (all others were straightforward):

CDO
GDAL
HARFBUZZ
HDF_EOS2
MAPL
NETCDF_C

Note that we have to keep using the old installer, because some packages simply don't build with the new one.

Testing and Issues

See JCSDA/spack-stack#2039