Conversation
Wires the perf framework's micro scenarios (M1-M7) and four representative macro scenarios (X1-1, X2-10k, X4, X6) into CodSpeed's walltime measurement on bare-metal Macro Runners. What it gives us: - per-PR regression detection on Python protocol code paths (M5-M7: encode int/string/float, decode int/string, escape/unescape) - macro-level regression detection on the round-trip-heavy hot paths (X2 javalist iteration, X4 callback round-trips, X6 pool saturation) - per-commit perf trend on master - bare-metal measurement environment (<1% variance vs. shared CI noise) Triggers are intentionally minimal: push to master tracks the trend, workflow_dispatch lets maintainers run on a PR when a change claims a perf impact. No PR auto-trigger to keep macro-runner minutes lean. Macros are exposed via a small pytest-codspeed adapter at src/py4j/tests/perf/scenarios/codspeed_macros.py - re-uses the existing MacroScenario classes from scenarios/macro.py, one parametrize entry per scenario, so each becomes its own tracked benchmark on the CodSpeed dashboard. Prerequisite (maintainer action, not in this PR): - Install the CodSpeed GitHub App on the py4j org: https://github.com/apps/codspeed - Set the CODSPEED_TOKEN repository secret (provided by the App). Until the App is installed, this workflow's job won't pick up a runner (codspeed-macro label) and CI on this workflow will queue. No effect on existing workflows or test runs. Co-authored-by: Isaac
) Adds a second trigger path so maintainers can opt a specific PR into CodSpeed feedback by applying the ``perf-check`` label. CodSpeed posts its standard PR-comment with the perf delta, the same as it would for an always-on ``pull_request`` trigger — except this stays off by default so the 600 min/month OSS macro-runner budget isn't consumed by routine PRs. Existing triggers (push to master, workflow_dispatch) are unchanged. Mechanism: - ``pull_request`` event type subscription is narrowed to ``[labeled, synchronize]``. - A job-level ``if:`` ensures the workflow only actually runs the benchmarks when (a) the labelled event added the ``perf-check`` label specifically, or (b) a synchronize event arrives on a PR that already carries the label. - Unrelated label additions show as a "Skipped" job in the Actions UI but consume no runner time. To use on a PR: a maintainer applies the ``perf-check`` label. CodSpeed runs and comments. Subsequent pushes to the PR branch re-fire the workflow automatically as long as the label is still present. Co-authored-by: Isaac
py4j has supported Python 3+ for some time (CI tests py3.9-py3.13)
but the codebase still carried full py2/3 dual-support machinery:
- ``from __future__`` imports in every module
- ``# -*- coding: UTF-8 -*-`` declarations
- a 130-line ``py4j.compat`` module exporting py2/3 shims
- ``sys.version_info.major < 3`` conditional templates
- ``long(x)``, ``unicode(s)``, ``basestring``, ``bytearray2``,
``bytestr``, ``isbytestr``, ``ispython3bytestr``, ``isbytearray``,
``bytetoint``, ``bytetostr``, ``strtobyte``, ``iteritems``,
``CompatThread``, ``range`` (from ``compat``)
- packaging metadata (setup.py classifiers, ``setup.cfg``'s
``universal=1`` wheel marker, tox.ini's ``envlist=py27,py34,py35``
+ ``nosetests``)
- IDE config (``.pydevproject`` files specifying ``python 2.6``)
- docs referencing py2/3 behaviour differences
This change is strictly mechanical: replaces each py2/3 alias with
its py3 equivalent inline, drops the unconditional ``from __future__``
imports, removes the encoding declarations (PEP 3120 makes UTF-8 the
default in py3), removes the ``try/except ImportError`` for
``collections.abc`` (always present on py3.3+), updates packaging
metadata to py3.9+ and drops the universal-wheel marker.
Function bodies are unchanged where possible. ``smart_decode`` calls
in source code are preserved (they will continue to work — the
function itself is just slimmed to its py3 body). ``encode_float``,
``decode_bytearray``, ``encode_bytearray``, ``get_command_part``, and
``escape_new_line`` are translated alias-by-alias with no behaviour
change.
``py4j.compat`` is kept around as a soft-deprecated shim. Its py2
branch is removed and a module docstring documents the migration path
for downstream callers, but every exported name still resolves so
external code that imports from it (``from py4j.compat import
unicode``, etc.) continues to work. The module is a candidate for
removal in a future major release.
## Files changed
Source modules:
- ``__init__.py``, ``clientserver.py``, ``finalizer.py``,
``java_collections.py``, ``java_gateway.py``, ``protocol.py``,
``signals.py`` — dropped ``from __future__``, ``# -*- coding ...``,
``from py4j.compat import (...)`` for dead names; replaced
``basestring`` → ``str``, ``long`` → ``int``, ``unicode`` → ``str``,
``bytestr`` → ``bytes``, ``bytearray2`` → ``bytes``, ``CompatThread``
→ ``threading.Thread``, ``items(d)`` → ``list(d.items())``,
``iteritems(self)`` → ``self.items()``, ``isbytestr`` /
``ispython3bytestr`` / ``isbytearray`` → direct ``isinstance(...)``,
``from py4j.compat import Queue`` → ``from queue import Queue``.
- ``java_collections.py`` — collapsed the ``try/except ImportError``
for ``collections.abc``; dropped the ``sys.version_info.major < 3``
ternary on ``__EMPTY_SET`` / ``__SET_TEMPLATE``.
- ``protocol.py`` — slimmed ``smart_decode`` body to py3 types;
``encode_bytearray`` / ``decode_bytearray`` / OUTPUT_CONVERTER
``LONG_TYPE`` lambda use native ``bytes`` / ``int``;
``get_command_part`` uses ``isinstance(..., str/int/bytes/
bytearray)`` directly.
Test modules:
- 17 test files — dropped ``from __future__``, ``# -*- coding ...``,
``from py4j.compat import`` of py2-only names. ``long(N)`` → ``N``;
``bytearray2`` → ``bytes``; ``isbytearray(a) or
ispython3bytestr(a)`` → ``isinstance(a, (bytearray, bytes))``;
``unicode(i)`` → ``str(i)``.
- ``java_gateway_test.py`` — dropped two
``if sys.version_info < (3,):`` branches (py2-only sleep paths;
the py3 ``proc.wait(5)`` branch is the only one that ran on
supported Pythons).
- New ``protocol_test.py`` (30 pure-protocol unit tests, no JVM
required): round-trip ``encode_bytearray``/``decode_bytearray``
for both ``bytes`` and ``bytearray``; ``escape_new_line``
str-passthrough and round-trip with ``unescape_new_line``;
``encode_float`` precision / inf / -inf / NaN; each branch of
``get_command_part`` including JAVA_MAX_INT boundary and bool-vs-int
dispatch; ``smart_decode`` semantics on str / bytes / non-string;
``py4j.compat`` back-compat (every export name still resolves to
its py3 equivalent, including ``CompatThread is threading.Thread``
and ``Queue is queue.Queue``).
``py4j.compat``:
- Slimmed: removed the ``if version_info.major < 3:`` branch
entirely; kept the py3 form as the only definition. Module-level
docstring documents the migration path. ``hasattr2``,
``CompatThread``, ``Queue``, ``Empty`` and every alias still
resolve so downstream imports continue to work.
Packaging:
- ``setup.py``: removed ``"Programming Language :: Python :: 2"`` and
``"... :: 3.8"`` (EOL) classifiers. Added ``... :: 3 :: Only`` and
3.9-3.13. Set ``python_requires=">=3.9"`` to match CI. Removed
``distutils`` fallback and 2to3-era comments.
- ``setup.cfg``: removed — only contained ``[bdist_wheel]
universal=1``, which is a py2/3 universal-wheel marker that
produces an incorrectly-tagged wheel on a py3-only project.
- ``tox.ini``: was ``envlist=py27,py34,py35`` calling ``nosetests``.
Now ``envlist=py{39,310,311,312,313}`` calling ``pytest``, matching
what CI runs. ``nose.cfg`` deleted alongside.
IDE config:
- ``py4j-python/.pydevproject`` and ``py4j-web/.pydevproject``:
``python 2.6`` → ``python 3.9``.
Docs:
- ``py4j-web/advanced_topics.rst``: dropped the parenthetical
``(Python 2.x) or bytes (Python 3.x)`` qualifier on the byte-array
return-type description; now reads simply ``bytes``.
## Validation
Local pytest on macOS arm64 + JDK 17:
**205 passed, 104 deselected, 0 failed** (skipping TLS / perf-framework
/ memory-leak / multithread suites that need extra setup, matching
the main CI workflow's filter).
Co-authored-by: Isaac
GitHub Actions doesn't allow `paths` and `paths-ignore` together in the same trigger block — having both causes a workflow startup failure (run ends with conclusion=failure but zero jobs spawned). Replace the paths-ignore block with a `!`-prefixed negation pattern inside the existing paths: list, which is the documented supported way to combine include + exclude in one filter. Same behavior: trigger on perf-framework code changes, skip doc-only (*.md) changes within the perf-framework directory. Co-authored-by: Isaac
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Test PR to validate the workflow fix before merging the equivalent upstream PR (py4j#587). If GitHub Actions accepts this workflow file (job parses and either runs or is filtered), the fix is good.