Skip to content

Python 2 desupport and removal - #2

Merged
Tagar merged 1 commit into
masterfrom
py2-desupport
May 20, 2026
Merged

Tagar merged 1 commit into
masterfrom
py2-desupport

Conversation

@Tagar

@Tagar Tagar commented May 19, 2026 •

Copy link
Copy Markdown
Member

Python 2 desupport / removal.

py4j has supported Python 3+ for some time (CI tests py3.9 – py3.13) but the codebase still carried the full py2 / py3 dual-support machinery: from __future__ imports in every module, a 130-line py4j.compat module exporting py2/3 shims, # -*- coding: UTF-8 -*- declarations, sys.version_info conditionals, the long(x) / unicode(s) / basestring / CompatThread aliases, packaging metadata (setup.cfg's universal=1, tox.ini's envlist=py27,py34,py35 + nosetests), IDE config (.pydevproject files specifying python 2.6), and a couple of docs references. None of that does anything on Python 3.

This change is strictly mechanical — function bodies are unchanged, no perf optimizations included. Each py2 / py3 alias is replaced inline with its py3 equivalent.

Scope

Area Change
Source modules Dropped from __future__, # -*- coding ..., dead from py4j.compat import (...) lines. Replaced basestring → str, long → int, unicode → str, bytestr → bytes, bytearray2 → bytes, CompatThread → threading.Thread, items(d) → list(d.items()), iteritems(self) → self.items(), isbytestr / ispython3bytestr / isbytearray → direct isinstance(...), from py4j.compat import Queue → from queue import Queue.
java_collections.py Collapsed try/except ImportError for collections.abc (always present on py3.3+). Dropped sys.version_info.major < 3 ternary on __EMPTY_SET / __SET_TEMPLATE.
protocol.py Slimmed smart_decode body to py3 types (isinstance(s, str) instead of isinstance(s, unicode), etc.). encode_bytearray / decode_bytearray / OUTPUT_CONVERTER LONG_TYPE lambda use native bytes / int. get_command_part uses isinstance(..., str/int/bytes/bytearray) directly. All smart_decode(...) call sites in source are preserved — the function itself still exists and behaves identically.
py4j.compat Slimmed: removed the if version_info.major < 3: branch entirely; kept the py3 form as the only definition. Module-level docstring documents the migration path. Every export still resolves (unicode, basestring, long, range, bytestr, bytearray2, unichr, iteritems, items, isbytestr, ispython3bytestr, isbytearray, bytetoint, bytetostr, strtobyte, Queue, Empty, hasattr2, CompatThread, version_info, tobytestr) so downstream code that imports from it (from py4j.compat import unicode, etc.) continues to work. The module is a candidate for removal in a future major release.
Tests (17 files + 1 new) Same header cleanup. long(N) → N; bytearray2 → bytes; isbytearray(a) or ispython3bytestr(a) → isinstance(a, (bytearray, bytes)); unicode(i) → str(i). Dropped two if sys.version_info < (3,): branches in java_gateway_test.py (py2-only sleep paths; the py3 proc.wait(5) branch is the only one that ran on supported Pythons). New protocol_test.py (30 unit tests, no JVM required) pinning the bytes/str boundary and the py4j.compat back-compat surface.
setup.py Removed "Programming Language :: Python :: 2" and "... :: 3.8" (EOL) classifiers. Added ... :: 3 :: Only and 3.9 – 3.13. Set python_requires=">=3.9" to match CI. Removed distutils fallback + 2to3-era comments.
setup.cfg Removed — only contained [bdist_wheel] universal=1, the py2/3 universal-wheel marker, which produces an incorrectly-tagged wheel on a py3-only project.
tox.ini Was envlist=py27,py34,py35 calling nosetests. Now envlist=py{39,310,311,312,313} calling pytest, matching CI. nose.cfg deleted alongside.
.pydevproject (py4j-python and py4j-web) python 2.6 → python 3.9.
py4j-web/advanced_topics.rst Dropped the parenthetical (Python 2.x) or bytes (Python 3.x) qualifier on the byte-array return-type description; now reads simply bytes.

Java side

No changes. The Java side speaks pure UTF-8 over the wire and has zero Python-version-specific code (confirmed by grep -rn -i "python 2\|python2\|py2" py4j-java/src/). The clean separation between protocol-level and Python-version-level is one of py4j's good properties — the JVM side is oblivious to which Python is on the other end.

Validation

Local pytest on macOS arm64 + JDK 17: 205 passed, 104 deselected, 0 failed (skipping TLS / perf-framework / memory-leak / multithread suites that need extra setup, matching the main CI workflow's filter).

The CodSpeed workflow will pick this PR up automatically (path filter matches py4j-python/src/py4j/**/*.py) and post per-scenario perf deltas vs master. Expected deltas: near-zero, since this is mechanical alias substitution, not optimization.

Co-authored-by: Isaac

@byteoak byteoak deleted a comment from codspeed Bot May 19, 2026
@byteoak byteoak deleted a comment from codspeed Bot May 19, 2026
py4j has supported Python 3+ for some time (CI tests py3.9-py3.13)
but the codebase still carried full py2/3 dual-support machinery:

- ``from __future__`` imports in every module
- ``# -*- coding: UTF-8 -*-`` declarations
- a 130-line ``py4j.compat`` module exporting py2/3 shims
- ``sys.version_info.major < 3`` conditional templates
- ``long(x)``, ``unicode(s)``, ``basestring``, ``bytearray2``,
  ``bytestr``, ``isbytestr``, ``ispython3bytestr``, ``isbytearray``,
  ``bytetoint``, ``bytetostr``, ``strtobyte``, ``iteritems``,
  ``CompatThread``, ``range`` (from ``compat``)
- packaging metadata (setup.py classifiers, ``setup.cfg``'s
  ``universal=1`` wheel marker, tox.ini's ``envlist=py27,py34,py35``
  + ``nosetests``)
- IDE config (``.pydevproject`` files specifying ``python 2.6``)
- docs referencing py2/3 behaviour differences

This change is strictly mechanical: replaces each py2/3 alias with
its py3 equivalent inline, drops the unconditional ``from __future__``
imports, removes the encoding declarations (PEP 3120 makes UTF-8 the
default in py3), removes the ``try/except ImportError`` for
``collections.abc`` (always present on py3.3+), updates packaging
metadata to py3.9+ and drops the universal-wheel marker.

Function bodies are unchanged where possible. ``smart_decode`` calls
in source code are preserved (they will continue to work — the
function itself is just slimmed to its py3 body). ``encode_float``,
``decode_bytearray``, ``encode_bytearray``, ``get_command_part``, and
``escape_new_line`` are translated alias-by-alias with no behaviour
change.

``py4j.compat`` is kept around as a soft-deprecated shim. Its py2
branch is removed and a module docstring documents the migration path
for downstream callers, but every exported name still resolves so
external code that imports from it (``from py4j.compat import
unicode``, etc.) continues to work. The module is a candidate for
removal in a future major release.

## Files changed

Source modules:
- ``__init__.py``, ``clientserver.py``, ``finalizer.py``,
  ``java_collections.py``, ``java_gateway.py``, ``protocol.py``,
  ``signals.py`` — dropped ``from __future__``, ``# -*- coding ...``,
  ``from py4j.compat import (...)`` for dead names; replaced
  ``basestring`` → ``str``, ``long`` → ``int``, ``unicode`` → ``str``,
  ``bytestr`` → ``bytes``, ``bytearray2`` → ``bytes``, ``CompatThread``
  → ``threading.Thread``, ``items(d)`` → ``list(d.items())``,
  ``iteritems(self)`` → ``self.items()``, ``isbytestr`` /
  ``ispython3bytestr`` / ``isbytearray`` → direct ``isinstance(...)``,
  ``from py4j.compat import Queue`` → ``from queue import Queue``.
- ``java_collections.py`` — collapsed the ``try/except ImportError``
  for ``collections.abc``; dropped the ``sys.version_info.major < 3``
  ternary on ``__EMPTY_SET`` / ``__SET_TEMPLATE``.
- ``protocol.py`` — slimmed ``smart_decode`` body to py3 types;
  ``encode_bytearray`` / ``decode_bytearray`` / OUTPUT_CONVERTER
  ``LONG_TYPE`` lambda use native ``bytes`` / ``int``;
  ``get_command_part`` uses ``isinstance(..., str/int/bytes/
  bytearray)`` directly.

Test modules:
- 17 test files — dropped ``from __future__``, ``# -*- coding ...``,
  ``from py4j.compat import`` of py2-only names. ``long(N)`` → ``N``;
  ``bytearray2`` → ``bytes``; ``isbytearray(a) or
  ispython3bytestr(a)`` → ``isinstance(a, (bytearray, bytes))``;
  ``unicode(i)`` → ``str(i)``.
- ``java_gateway_test.py`` — dropped two
  ``if sys.version_info < (3,):`` branches (py2-only sleep paths;
  the py3 ``proc.wait(5)`` branch is the only one that ran on
  supported Pythons).
- New ``protocol_test.py`` (30 pure-protocol unit tests, no JVM
  required): round-trip ``encode_bytearray``/``decode_bytearray``
  for both ``bytes`` and ``bytearray``; ``escape_new_line``
  str-passthrough and round-trip with ``unescape_new_line``;
  ``encode_float`` precision / inf / -inf / NaN; each branch of
  ``get_command_part`` including JAVA_MAX_INT boundary and bool-vs-int
  dispatch; ``smart_decode`` semantics on str / bytes / non-string;
  ``py4j.compat`` back-compat (every export name still resolves to
  its py3 equivalent, including ``CompatThread is threading.Thread``
  and ``Queue is queue.Queue``).

``py4j.compat``:
- Slimmed: removed the ``if version_info.major < 3:`` branch
  entirely; kept the py3 form as the only definition. Module-level
  docstring documents the migration path. ``hasattr2``,
  ``CompatThread``, ``Queue``, ``Empty`` and every alias still
  resolve so downstream imports continue to work.

Packaging:
- ``setup.py``: removed ``"Programming Language :: Python :: 2"`` and
  ``"... :: 3.8"`` (EOL) classifiers. Added ``... :: 3 :: Only`` and
  3.9-3.13. Set ``python_requires=">=3.9"`` to match CI. Removed
  ``distutils`` fallback and 2to3-era comments.
- ``setup.cfg``: removed — only contained ``[bdist_wheel]
  universal=1``, which is a py2/3 universal-wheel marker that
  produces an incorrectly-tagged wheel on a py3-only project.
- ``tox.ini``: was ``envlist=py27,py34,py35`` calling ``nosetests``.
  Now ``envlist=py{39,310,311,312,313}`` calling ``pytest``, matching
  what CI runs. ``nose.cfg`` deleted alongside.

IDE config:
- ``py4j-python/.pydevproject`` and ``py4j-web/.pydevproject``:
  ``python 2.6`` → ``python 3.9``.

Docs:
- ``py4j-web/advanced_topics.rst``: dropped the parenthetical
  ``(Python 2.x) or bytes (Python 3.x)`` qualifier on the byte-array
  return-type description; now reads simply ``bytes``.

## Validation

Local pytest on macOS arm64 + JDK 17:
**205 passed, 104 deselected, 0 failed** (skipping TLS / perf-framework
/ memory-leak / multithread suites that need extra setup, matching
the main CI workflow's filter).

Co-authored-by: Isaac
@Tagar Tagar changed the title Comprehensive Python 2 desupport and removal Python 2 desupport and removal May 19, 2026
@codspeed

codspeed Bot commented May 19, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 20.22%

⚡ 1 improved benchmark
✅ 15 untouched benchmarks
⏩ 7 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ WallTime test_m5b_encode_string 3.2 µs 2.7 µs +20.22%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing py2-desupport (bee1e43) with master (0bde5ca)2

Open in CodSpeed

Footnotes

  1. 7 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on master (2ac0f8a) during the generation of this report, so 0bde5ca was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@Tagar

Tagar commented May 19, 2026

Copy link
Copy Markdown
Member Author

py4j#585

@Tagar
Tagar merged commit b23e257 into master May 20, 2026
58 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant