[BUG] Can't build wheels on CPython 3.13t (no GIL / PEP 703) when bdist_wheel's py-limited-api option is set

[bastimeyer]

setuptools version

latest commit from the main branch

Python version

3.13t

OS

Arch Linux

Additional environment information

CPython 3.13 built without global interpreter lock (--disable-gil) - PEP 703.

Description

I've just been redirected here to this issue tracker from
pypa/wheel#624
because the bdist_wheel command has just been vendored into setuptools. There hasn't been a new setuptools releasee since, so I'm actually reporting a bug that currently only affects the wheel package, but will affect setuptools soon unless fixed.

Original report (with adjusted code links):

---

I ran into this issue while trying to build a pycryptodome wheel on CPython 3.13t.
See Legrandin/pycryptodome#813

The pycryptodome project sets the py-limited-api option to cp35 in order to limit its abi3 wheels to cp35 and above.

However, when using CPython 3.13t (built without global interpreter lock - PEP 703), this option raises an AssertionError when trying to build the wheel.

The reason for this is this if-else-block:

setuptools/setuptools/command/bdist_wheel.py

Lines 351 to 355 in f91fa3d

	if self.py_limited_api and (impl_name + impl_ver).startswith("cp3"):
	impl = self.py_limited_api
	abi_tag = "abi3"
	else:
	abi_tag = str(get_abi_tag()).lower()

Building CPython 3.13 without the GIL adds the t ABI flag, so the (impl_name + impl_ver).startswith("cp3") check is incorrect where it sets abi_tag = "abi3" if the condition is true. The AssertionError is then raised afterwards when it checks for supported tags:

setuptools/setuptools/command/bdist_wheel.py

Lines 356 to 363 in f91fa3d

	tag = (impl, abi_tag, plat_name)
	# issue gh-374: allow overriding plat_name
	supported_tags = [
	(t.interpreter, t.abi, plat_name) for t in tags.sys_tags()
	]
	assert (
	tag in supported_tags
	), f"would build wheel with unsupported tag {tag}"

$ python3.13 -m venv /tmp/venv-pycryptodome-313t
$ source /tmp/venv-pycryptodome-313t/bin/activate

$ python -c 'import sys;print(f"{sys.version_info=}\n{sys.abiflags=}")'
sys.version_info=sys.version_info(major=3, minor=13, micro=0, releaselevel='beta', serial=2)
sys.abiflags='t'

$ pip install build
$ pip install git+https://github.com/pypa/setuptools.git
$ pip list --format=freeze
build==1.2.1
packaging==24.1
pip==24.0
pyproject_hooks==1.1.0
setuptools==70.0.0.post20240615

$ git clone https://github.com/Legrandin/pycryptodome /tmp/venv-pycryptodome-313t/pycryptodome
$ cd /tmp/venv-pycryptodome-313t/pycryptodome
$ python -m build --wheel --no-isolation
...
running install_scripts
Traceback (most recent call last):
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in <module>
    main()
    ~~~~^^
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 357, in main
    json_out["return_val"] = hook(**hook_input["kwargs"])
                             ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 271, in build_wheel
    return _build_backend().build_wheel(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        wheel_directory, config_settings, metadata_directory
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/setuptools/build_meta.py", line 410, in build_wheel
    return self._build_with_temp_dir(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        ['bdist_wheel'],
        ^^^^^^^^^^^^^^^^
    ...<3 lines>...
        self._arbitrary_args(config_settings),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/setuptools/build_meta.py", line 395, in _build_with_temp_dir
    self.run_setup()
    ~~~~~~~~~~~~~~^^
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/setuptools/build_meta.py", line 311, in run_setup
    exec(code, locals())
    ~~~~^^^^^^^^^^^^^^^^
  File "<string>", line 500, in <module>
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/setuptools/__init__.py", line 103, in setup
    return distutils.core.setup(**attrs)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/setuptools/_distutils/core.py", line 184, in setup
    return run_commands(dist)
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
    dist.run_commands()
    ~~~~~~~~~~~~~~~~~^^
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
    self.run_command(cmd)
    ~~~~~~~~~~~~~~~~^^^^^
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/setuptools/dist.py", line 974, in run_command
    super().run_command(command)
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
    cmd_obj.run()
    ~~~~~~~~~~~^^
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/setuptools/command/bdist_wheel.py", line 412, in run
    impl_tag, abi_tag, plat_tag = self.get_tag()
                                  ~~~~~~~~~~~~^^
  File "/tmp/venv-pycryptodome-313t/lib/python3.13/site-packages/setuptools/command/bdist_wheel.py", line 362, in get_tag
    tag in supported_tags
AssertionError: would build wheel with unsupported tag ('cp35', 'abi3', 'linux_x86_64')

ERROR Backend subprocess exited when trying to invoke build_wheel

---

pypa/wheel#624 (comment)

This isn't a packaging issue, because if I remove the option from pycryptodome's setup.cfg, then a cp313t wheel is built just fine.

This is not supposed to be an abi3 wheel. abi3 is incompatible with no-GIL builds (t):

• https://github.com/pypa/setuptools/blob/main/setuptools/_vendor/packaging/tags.py#L148-L149
• https://github.com/pypa/setuptools/blob/main/setuptools/_vendor/packaging/tags.py#L232-L236

Expected behavior

.

How to Reproduce

1. Build CPython 3.13 with --disable-gil as build option
2. Create new venv and activate it
3. Try to build pycryptodome wheel (which sets the py-limited-api option to cp35 in order to limit its abi3 wheels to cp35 and above, but which is irrelevant, as we can't build abi3 wheels due to the no GIL CPython build option)

As said in the beginning, I've been redirected here, so this issue currently doesn't apply to setuptools yet, since the vendored bdist_wheel command hasn't arrived in a new release yet.

Output

.

[abravalheri]

Hi @bastimeyer, thank you very much for reporting this.

If I understood correctly, you mention that the following piece of code is the main problem, right? Do you have a suggestion on how to fix the checks/abi assignment?

setuptools/setuptools/command/bdist_wheel.py

Lines 351 to 355 in f91fa3d

	if self.py_limited_api and (impl_name + impl_ver).startswith("cp3"):
	impl = self.py_limited_api
	abi_tag = "abi3"
	else:
	abi_tag = str(get_abi_tag()).lower()

which sets the py-limited-api option to cp35 in order to limit its abi3 wheels to cp35 and above, but which is irrelevant, as we can't build abi3 wheels due to the no GIL CPython build option

This is curious... If the developer sets a configuration option that cannot be satisfied, isn't it the right thing to raise an exception? Maybe we need to refine the exception...

[bastimeyer]

The (impl_name + impl_ver).startswith("cp3") part of the if-condition in bdist_wheel is simply wrong. This is how packaging determines whether abi3 applies (I've already linked it):

setuptools/setuptools/_vendor/packaging/tags.py

Lines 144 to 151 in f91fa3d

	def _abi3_applies(python_version: PythonVersion, threading: bool) -> bool:
	"""
	Determine if the Python version supports abi3.
	PEP 384 was first implemented in Python 3.2. The threaded (`--disable-gil`)
	builds do not support abi3.
	"""
	return len(python_version) > 1 and tuple(python_version) >= (3, 2) and not threading

The py-limited-api configuration option (which apparently is still undocumented - #4741) just limits abi3 wheels to a specific min-version of CPython. This doesn't apply here because we can't build an abi3 wheel in the used environment (no GIL), so the option is irrelevant.

If you want to prevent (or allow) building non-abi3 wheels while the option is set, then I guess another option needs to be implemented which package authors need to set in addition to py-limited-api.

edit:
But I don't know much about Python's C-extensions, so I can't give any real suggestions here. I'm just trying to build the pycryptodome wheel in my 313t env (as an end-user) in order to test my application which has pycryptodome as a dependency.

[abravalheri]

Thank you very much @bastimeyer for the information.

I'm just trying to build the pycryptodome wheel in my 313t env (as an end-user) in order to test my application which has pycryptodome as a dependency.

Have you consider the following workaround while we figure this out?

For setuptools>=69, it might be possible to do use config_settings to overwrite that configuration during build time:

# using build to build from source code
python -m build -C "--build-option=--py-limited-api ''"
# using pip>=23.1 to install from source-code (may also work with sdist)
pip install . -C "--build-option=--py-limited-api ''"

Note however that the support for --config-settings as a whole is not considered stable/final and may change in the future.

[abravalheri]

The py-limited-api configuration option (which apparently is still undocumented - #4741) just limits abi3 wheels to a specific min-version of CPython. This doesn't apply here because we can't build an abi3 wheel in the used environment (no GIL), so the option is irrelevant.

If you want to prevent (or allow) building non-abi3 wheels while the option is set, then I guess another option needs to be implemented which package auther need to set in addition to py-limited-api.

This is how py_limited_api is described in the source code:

Python tag (cp32|cp33|cpNN) for abi3 wheel tag (default: false)

There is some comments on the original commit.

So I guess that the original intent was that py_limited_api would only work in tandem with abi3?

---

Hi @dholth, do you know if it would make sense to separate the py_limited_api and abi3 behaviours in bdist_wheel as suggested by @bastimeyer. Or do they only make sense together in the existing implementation?

Do you have any suggestions or hints regarding the original issue? Or do you think that an exception is the best when trying to build py_limited_api on 3.13t (no GIL)?

[dholth]

Change the check from startswith("cp3") to match("^cp\d*$")?

[bastimeyer]

Change the check from startswith("cp3") to match("^cp\d*$")?

That won't have any effect (apart from accepting non-cp3x versions).

(impl_name + impl_ver).startswith("cp3") is supposed to mean "is-abi3-compatible". cp313t however is not compatible. impl_name + impl_ver doesn't include the abi flag. As linked above, packaging has a proper implementation for checking abi3 compatibility where it includes and not threading.

[dholth]

I mean "match cp3(digit)" but "don't match cp3(digit)(letter)"