pyo3: upgrade to v0.28.3

[tdyas]

Upgrade to PyO3 v0.28.3. The main change affecting Pants source is the need to now explicitly opt-in to the generation of FromPyObject on pyclass-types which are Clone.

[sureshjoshi]

Wasn’t there also a change in gil vs no-gil?
https://pyo3.rs/main/migration#default-to-supporting-free-threaded-python

I’ll be honest, I’ve read that migration guide like 12 times and there is just something about it that doesn’t compute in my brain.

My latest understanding is that IFF you’re using a free-threaded Python, you’re automatically supported - rather than needing to add a feature flag. But, if you’re using non-free-threaded, then it’s status quo.

[tdyas]

My latest understanding is that IFF you’re using a free-threaded Python, you’re automatically supported - rather than needing to add a feature flag. But, if you’re using non-free-threaded, then it’s status quo.

Yes, no need for a feature flag. And the particular change in 0.28.x for users of the pymodule declarative module attribute is that there is no longer a need to opt-in to those modules being compatible with the free-threaded build; it's now the default for it to work with the free-threaded build. (Pants uses pymodule for declaring its native_engine module.)

We already ported the Pants PyO3 interface code for the most part to use explicit Mutex locks (so the types are Sync) and no longer rely on the GIL for synchronization for those types to serialize access to their data. For example, PyGeneratorResponseNativeCall is defined as:

pub struct PyGeneratorResponseNativeCall(Mutex<Option<NativeCall>>);

We may still rely on the GIL elsewhere though. It'd be interesting to see what happens if we tried using the free-threaded build.