Add support for the free-threaded build and upload cp313t wheels

[ngoldbaum]

I'm working full-time on ecosystem support for free-threaded Python. See the guide we've written at https://py-free-threading.github.io if you want to learn more about this effort and supporting free-threaded Python in general.

I saw the proposed PEP about possibly incorporating pyzstd into the Python standard library and noticed there wasn't yet free-threaded support.

It looks like everything builds correctly. When I force-disable the GIL by setting PYTHON_GIL=0, the tests all pass on my Mac.

That said, it doesn't look like there are any explicitly multithreaded Python tests using e.g. concurrent.futures or threading. It'd probably be a good idea to add some tests of the python-level locks in the CFFI bindings and the C-level locking in the C extension.

Finally, I notice you're not using PyMutex on CPython 3.13 and newer. Not only can you avoid heap allocations using PyMutex, but it's also (at least as I understand it) unconditionally faster than PyThread_type_lock.

[Rogdham]

Hello @ngoldbaum , thank you for your message.

I believe that some code was committed about this by Ma Lin before he removed himself from the project. This could explain why things are going smoothly here.

For the context, I took over the pyzstd project after discussing with the original author when they deleted their Github account. So far I do some minimal work for maintenance, but I have not really the motivation to do such chances that have potentially a huge impact.

I strongly suggest to focus all efforts on the inclusion on Zstandard on the standard library.

I saw you wrote about this on the Discourse thread about PEP 784 already. I would suggest to talk with @emmatyping to help her include the necessary changes in her PR on CPython (or decide to do it on a second wave).

As far as pyzstd is concerned, my plan is first to help Emma as much as I can with her initiative, and revisit the future of pyzstd afterwards.

[emmatyping]

Following up here (since I don't have an issue tracker for my branch), I pushed a commit to my branch that
a) replaces the existing locking with PyMutex and
b) adds two simple tests verifying the compress/decompress methods are threadsafe

I'll definitely want to add more, but since these are the main entrypoints of contention, I wanted to verify they are, in fact, threadsafe.

Edit: I just enabled issues on my fork, so perhaps this should be moved there?

[ngoldbaum]

Closing in favor of emmatyping/cpython#9

[biblicabeebli]

Request: I'm trying to find documentation on the pyzstd's current behavior with the GIL, I'm not seeing any comments about it in the documentation. Would one of you mind commenting on that? (or adding it to the decumentation?)

[emmatyping]

The docs state that the classes are "Thread-safe at method level." which is correct; there is a lock for each class that is used to ensure that there aren't any races on the zstd compression context. I haven't gotten to test the behavior of concurrent access to a ZstdDict, but it only has read only attributes so there shouldn't be issues there. So in short, each instance can only be used in one thread at a time.

[emmatyping]

I would say people who want free-threaded wheels should probably use backports.zstd instead. What do you think @Rogdham ?

[Rogdham]

I would say people who want free-threaded wheels should probably use backports.zstd instead.

Absolutely!

This is in line with my plan for the future of pyzstd with respect to stdlib inclusion, so I'm closing the issue.