[Bug] Management utilities print SyntaxWarning: invalid escape sequence on Python 3.12+ (e.g. Rocky Linux 10)

[tuhaihe]

Apache Cloudberry version

main (89baa48) — also present in 2.x releases.

What happened

On a Rocky Linux 10 cluster (Python 3.12), every invocation of gpsync /
gpssh / gpssh-exkeys prints SyntaxWarning messages to stderr before it
does any work. It surfaced while installing the Cloudberry backup utilities,
whose make install calls gpsync to distribute binaries to the segments:

$ make install
cp /home/gpadmin/go/bin/gpbackup ... /usr/local/cloudberry-db/bin
/usr/local/cloudberry-db/lib/python/gppylib/util/ssh_utils.py:268: SyntaxWarning: invalid escape sequence '\ '
  '''Escape occurrences of \ and $ as needed and package the line as an "eval" shell command'''
/usr/local/cloudberry-db/bin/lib/pexpect/pxssh.py:105: SyntaxWarning: invalid escape sequence '\['
  self.UNIQUE_PROMPT = "\[PEXPECT\][\$\#] "
/usr/local/cloudberry-db/bin/lib/pexpect/pxssh.py:109: SyntaxWarning: invalid escape sequence '\$'
  self.PROMPT_SET_SH = "PS1='[PEXPECT]\$ '"
/usr/local/cloudberry-db/bin/lib/pexpect/pxssh.py:110: SyntaxWarning: invalid escape sequence '\$'
  self.PROMPT_SET_CSH = "set prompt='[PEXPECT]\$ '"
Successfully copied gpbackup_helper and gpbackup_s3_plugin to /usr/local/cloudberry-db on all segments

The operation itself succeeds — this is cosmetic — but it makes utility output
noisy and looks like a failure to anyone reading a deployment log.

Why it appears now: string literals such as "\[" or "\$" contain escape
sequences Python does not recognise. Until 3.11 that was a DeprecationWarning,
hidden by default; Python 3.12 promoted it to a SyntaxWarning, which is
printed (What's New in 3.12).
So the same code is quiet on Rocky 8/9 (Python 3.6/3.9) and noisy on Rocky 10,
Fedora 40+, and Ubuntu 24.10+. The Python docs also state these sequences
will become a SyntaxError in a future release, so this is not purely
cosmetic in the long run.

Note the warnings can repeat on every run: if $GPHOME is root-owned and the
utility runs as gpadmin, CPython cannot write __pycache__, so the modules
are recompiled — and re-warn — each time.

What you think should happen instead

gpsync/gpssh should produce no Python warnings. The literals in question are
regexes and shell snippets that were meant to be raw strings; they should be
written as such (r"\[PEXPECT\]..."), which leaves their value unchanged.

Compiling everything under gpMgmt/ shows the problem is broader than the four
warnings above — 41 literals in 16 files:

File	Lines
gpMgmt/bin/gpload.py	720, 2530, 2675, 2689
gpMgmt/bin/gpload_test/gpload/TEST.py	322 (×2), 329, 331, 333, 335, 353, 355, 410
gpMgmt/bin/gppylib/programs/test/unit/test_cluster_clsrecoversegment_triples.py	297, 304, 332
gpMgmt/bin/gppylib/test/unit/test_unit_database_segment_guc.py	26
gpMgmt/bin/gppylib/test/unit/test_unit_file_segment_guc.py	31
gpMgmt/bin/gppylib/test/unit/test_unit_gppkg.py	72
gpMgmt/bin/gppylib/test/unit/test_unit_gpsegrecovery.py	304, 333
gpMgmt/bin/gppylib/test/unit/test_unit_gpsegsetuprecovery.py	262, 287
gpMgmt/bin/gppylib/test/unit/test_unit_package.py	122
gpMgmt/bin/gppylib/util/ssh_utils.py	268
gpMgmt/bin/lib/pexpect/pxssh.py	105, 109, 110
gpMgmt/sbin/seg_update_pg_hba.py	41
gpMgmt/test/behave/mgmt_utils/steps/gpstate_utils.py	79
gpMgmt/test/behave/mgmt_utils/steps/mgmt_utils.py	683, 908, 2007, 2223, 3346, 3351, 3849, 3980
gpMgmt/test/behave/mgmt_utils/steps/replication_slots_utils.py	27, 90
gpMgmt/test/behave_utils/utils.py	682

A further 67 literals in 11 files under src/ and contrib/ have the same
problem (src/backend/gporca/scripts/, contrib/try_convert/scripts/). Those
are developer scripts rather than shipped utilities, so they are lower priority.

Two side findings while auditing:

1. gpMgmt/bin/lib/pexpect is a vendored copy of pexpect 3.3 (2013).
   Upstream pexpect made these literals raw years ago; refreshing the vendored
   copy — or dropping it in favour of the packaged pexpect — would be worth
   considering separately.
2. gpMgmt/test/behave/mgmt_utils/steps/replication_slots_utils.py:31 reads
   export WITH_MIRRORS={with_mirrors} && \A, in the middle of a run of shell
   line continuations. The stray A looks like a keystroke slip: the shell
   receives && A ./demo_cluster.sh -d && ... and tries to run a command named
   A. Pre-existing, unrelated to the encoding of the escape, and probably
   deserves its own fix.

How to reproduce

On any host with Python 3.12 or newer (Rocky Linux 10, Fedora 40+,
Ubuntu 24.10+):

$ gpsync --help          # or: gpssh --help

Or without a cluster, straight from a source checkout:

$ python3 --version
Python 3.12.4
$ python3 -W error::SyntaxWarning -m py_compile \
    gpMgmt/bin/gppylib/util/ssh_utils.py gpMgmt/bin/lib/pexpect/pxssh.py

To enumerate every occurrence in the tree:

$ python3 - <<'PY'
import pathlib, warnings
for p in sorted(pathlib.Path('gpMgmt').rglob('*.py')):
    with warnings.catch_warnings(record=True) as w:
        warnings.simplefilter('always')
        try:
            compile(p.read_text(encoding='utf-8', errors='replace'), str(p), 'exec')
        except SyntaxError:
            continue
        for i in w:
            if issubclass(i.category, SyntaxWarning):
                print(f'{p}:{i.lineno}: {i.message}')
PY

Operating System

Rocky Linux 10 (Python 3.12); reproduced on macOS with Python 3.12.4.

Anything else

Happens every time on Python 3.12+, never on Python ≤ 3.11.

Workaround for operators until this is fixed:

$ PYTHONWARNINGS=ignore::SyntaxWarning make install

I am preparing a PR that fixes the 41 literals under gpMgmt/, with the
src/+contrib/ remainder as a follow-up.

Are you willing to submit PR?

• Yes, I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct.