UTF8 authors, broken metadata?

[pawamoy]

Opened here: pawamoy/markdown-exec#64.

Check the Author metadata on this release on PyPI: https://pypi.org/project/markdown-exec/1.9.3/.

Building locally results in the same metadata:

authors = [{name = "Timothée Mazzucotelli", email = "dev@pawamoy.fr"}]

Author-Email: =?utf-8?q?Timoth=C3=A9e_Mazzucotelli?= <dev@pawamoy.fr>

Not sure this actually comes from pdm-backend.

[pawamoy]

build==1.2.2.post1

% make run pyproject-build -v
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - pdm-backend
> /media/data/dev/markdown-exec/.venv/bin/python3 -m pip --python /tmp/build-
  env-x86kpiji/bin/python install --use-pep517 --no-warn-script-location --no-compile -r
  /tmp/build-reqs-3p17ugcs.txt
< Looking in indexes: http://localhost:31411/simple/
< Collecting pdm-backend (from -r /tmp/build-reqs-3p17ugcs.txt (line 1))
<   Using cached pdm_backend-2.4.3-py3-none-any.whl.metadata (2.8 kB)
< Using cached pdm_backend-2.4.3-py3-none-any.whl (114 kB)
< Installing collected packages: pdm-backend
< Successfully installed pdm-backend-2.4.3
* Getting build dependencies for sdist...
* Building sdist...
 - Adding CHANGELOG.md -> CHANGELOG.md
 - Adding CODE_OF_CONDUCT.md -> CODE_OF_CONDUCT.md
 - Adding CONTRIBUTING.md -> CONTRIBUTING.md
 - Adding LICENSE -> LICENSE
 - Adding README.md -> README.md
 - Adding config/coverage.ini -> config/coverage.ini
 - Adding config/git-changelog.toml -> config/git-changelog.toml
 - Adding config/mypy.ini -> config/mypy.ini
 - Adding config/pytest.ini -> config/pytest.ini
 - Adding config/ruff.toml -> config/ruff.toml
 - Adding config/vscode/launch.json -> config/vscode/launch.json
 - Adding config/vscode/settings.json -> config/vscode/settings.json
 - Adding config/vscode/tasks.json -> config/vscode/tasks.json
 - Adding docs/.overrides/main.html -> docs/.overrides/main.html
 - Adding docs/.overrides/partials/comments.html -> docs/.overrides/partials/comments.html
 - Adding docs/changelog.md -> docs/changelog.md
 - Adding docs/code_of_conduct.md -> docs/code_of_conduct.md
 - Adding docs/contributing.md -> docs/contributing.md
 - Adding docs/credits.md -> docs/credits.md
 - Adding docs/css/insiders.css -> docs/css/insiders.css
 - Adding docs/css/material.css -> docs/css/material.css
 - Adding docs/css/mkdocstrings.css -> docs/css/mkdocstrings.css
 - Adding docs/gallery.md -> docs/gallery.md
 - Adding docs/index.md -> docs/index.md
 - Adding docs/insiders/changelog.md -> docs/insiders/changelog.md
 - Adding docs/insiders/goals.yml -> docs/insiders/goals.yml
 - Adding docs/insiders/index.md -> docs/insiders/index.md
 - Adding docs/insiders/installation.md -> docs/insiders/installation.md
 - Adding docs/js/feedback.js -> docs/js/feedback.js
 - Adding docs/js/insiders.js -> docs/js/insiders.js
 - Adding docs/license.md -> docs/license.md
 - Adding docs/schema.json -> docs/schema.json
 - Adding docs/snippets/gallery/ansi.sh -> docs/snippets/gallery/ansi.sh
 - Adding docs/snippets/gallery/argparse.py -> docs/snippets/gallery/argparse.py
 - Adding docs/snippets/gallery/argparse_format.py -> docs/snippets/gallery/argparse_format.py
 - Adding docs/snippets/gallery/chalk.py -> docs/snippets/gallery/chalk.py
 - Adding docs/snippets/gallery/d2.py -> docs/snippets/gallery/d2.py
 - Adding docs/snippets/gallery/diagrams.py -> docs/snippets/gallery/diagrams.py
 - Adding docs/snippets/gallery/drawsvg.py -> docs/snippets/gallery/drawsvg.py
 - Adding docs/snippets/gallery/expandable_filetree.py -> docs/snippets/gallery/expandable_filetree.py
 - Adding docs/snippets/gallery/hyperbolic.py -> docs/snippets/gallery/hyperbolic.py
 - Adding docs/snippets/gallery/matplotlib.py -> docs/snippets/gallery/matplotlib.py
 - Adding docs/snippets/gallery/plotly.py -> docs/snippets/gallery/plotly.py
 - Adding docs/snippets/gallery/pydeps.py -> docs/snippets/gallery/pydeps.py
 - Adding docs/snippets/gallery/pytermgui.py -> docs/snippets/gallery/pytermgui.py
 - Adding docs/snippets/gallery/qrcode.py -> docs/snippets/gallery/qrcode.py
 - Adding docs/snippets/gallery/rich.py -> docs/snippets/gallery/rich.py
 - Adding docs/snippets/gallery/rich_terminal.py -> docs/snippets/gallery/rich_terminal.py
 - Adding docs/snippets/gallery/runpy.py -> docs/snippets/gallery/runpy.py
 - Adding docs/snippets/gallery/textual.py -> docs/snippets/gallery/textual.py
 - Adding docs/snippets/usage/boolean_matrix.py -> docs/snippets/usage/boolean_matrix.py
 - Adding docs/snippets/usage/hide.py -> docs/snippets/usage/hide.py
 - Adding docs/snippets/usage/multiple.pycon -> docs/snippets/usage/multiple.pycon
 - Adding docs/snippets/usage/platform_html.py -> docs/snippets/usage/platform_html.py
 - Adding docs/snippets/usage/platform_md.py -> docs/snippets/usage/platform_md.py
 - Adding docs/snippets/usage/source.py -> docs/snippets/usage/source.py
 - Adding docs/snippets/usage/source.pycon -> docs/snippets/usage/source.pycon
 - Adding docs/usage/index.md -> docs/usage/index.md
 - Adding docs/usage/pyodide.md -> docs/usage/pyodide.md
 - Adding docs/usage/python.md -> docs/usage/python.md
 - Adding docs/usage/shell.md -> docs/usage/shell.md
 - Adding docs/usage/tree.md -> docs/usage/tree.md
 - Adding duties.py -> duties.py
 - Adding mkdocs.yml -> mkdocs.yml
 - Adding .pdm-build/pyproject.toml -> pyproject.toml
 - Adding scripts/gen_credits.py -> scripts/gen_credits.py
 - Adding scripts/gen_ref_nav.py -> scripts/gen_ref_nav.py
 - Adding scripts/insiders.py -> scripts/insiders.py
 - Adding scripts/make -> scripts/make
 - Adding scripts/make.py -> scripts/make.py
 - Adding src/markdown_exec/__init__.py -> src/markdown_exec/__init__.py
 - Adding src/markdown_exec/ansi.css -> src/markdown_exec/ansi.css
 - Adding src/markdown_exec/debug.py -> src/markdown_exec/debug.py
 - Adding src/markdown_exec/formatters/__init__.py -> src/markdown_exec/formatters/__init__.py
 - Adding src/markdown_exec/formatters/_exec_python.py -> src/markdown_exec/formatters/_exec_python.py
 - Adding src/markdown_exec/formatters/base.py -> src/markdown_exec/formatters/base.py
 - Adding src/markdown_exec/formatters/bash.py -> src/markdown_exec/formatters/bash.py
 - Adding src/markdown_exec/formatters/console.py -> src/markdown_exec/formatters/console.py
 - Adding src/markdown_exec/formatters/markdown.py -> src/markdown_exec/formatters/markdown.py
 - Adding src/markdown_exec/formatters/pycon.py -> src/markdown_exec/formatters/pycon.py
 - Adding src/markdown_exec/formatters/pyodide.py -> src/markdown_exec/formatters/pyodide.py
 - Adding src/markdown_exec/formatters/python.py -> src/markdown_exec/formatters/python.py
 - Adding src/markdown_exec/formatters/sh.py -> src/markdown_exec/formatters/sh.py
 - Adding src/markdown_exec/formatters/tree.py -> src/markdown_exec/formatters/tree.py
 - Adding src/markdown_exec/logger.py -> src/markdown_exec/logger.py
 - Adding src/markdown_exec/mkdocs_plugin.py -> src/markdown_exec/mkdocs_plugin.py
 - Adding src/markdown_exec/processors.py -> src/markdown_exec/processors.py
 - Adding src/markdown_exec/py.typed -> src/markdown_exec/py.typed
 - Adding src/markdown_exec/pyodide.css -> src/markdown_exec/pyodide.css
 - Adding src/markdown_exec/pyodide.js -> src/markdown_exec/pyodide.js
 - Adding src/markdown_exec/rendering.py -> src/markdown_exec/rendering.py
 - Adding tests/__init__.py -> tests/__init__.py
 - Adding tests/conftest.py -> tests/conftest.py
 - Adding tests/test_base_formatter.py -> tests/test_base_formatter.py
 - Adding tests/test_converter.py -> tests/test_converter.py
 - Adding tests/test_headings.py -> tests/test_headings.py
 - Adding tests/test_python.py -> tests/test_python.py
 - Adding tests/test_shell.py -> tests/test_shell.py
 - Adding tests/test_toc.py -> tests/test_toc.py
 - Adding tests/test_tree.py -> tests/test_tree.py
 - Adding tests/test_validator.py -> tests/test_validator.py
 - Adding PKG-INFO -> PKG-INFO
* Building wheel from sdist
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - pdm-backend
> /media/data/dev/markdown-exec/.venv/bin/python3 -m pip --python /tmp/build-env-
  jnozgzar/bin/python install --use-pep517 --no-warn-script-location --no-compile -r
  /tmp/build-reqs-1fb269ef.txt
< Looking in indexes: http://localhost:31411/simple/
< Collecting pdm-backend (from -r /tmp/build-reqs-1fb269ef.txt (line 1))
<   Using cached pdm_backend-2.4.3-py3-none-any.whl.metadata (2.8 kB)
< Using cached pdm_backend-2.4.3-py3-none-any.whl (114 kB)
< Installing collected packages: pdm-backend
< Successfully installed pdm-backend-2.4.3
* Getting build dependencies for wheel...
* Building wheel...
 - Adding .pdm-build/markdown_exec-1.9.4.dev6+g3562fbb.dist-info/METADATA -> markdown_exec-1.9.4.dev6+g3562fbb.dist-info/METADATA
 - Adding .pdm-build/markdown_exec-1.9.4.dev6+g3562fbb.dist-info/WHEEL -> markdown_exec-1.9.4.dev6+g3562fbb.dist-info/WHEEL
 - Adding .pdm-build/markdown_exec-1.9.4.dev6+g3562fbb.dist-info/entry_points.txt -> markdown_exec-1.9.4.dev6+g3562fbb.dist-info/entry_points.txt
 - Adding .pdm-build/markdown_exec-1.9.4.dev6+g3562fbb.dist-info/licenses/LICENSE -> markdown_exec-1.9.4.dev6+g3562fbb.dist-info/licenses/LICENSE
 - Adding src/markdown_exec/__init__.py -> markdown_exec/__init__.py
 - Adding src/markdown_exec/ansi.css -> markdown_exec/ansi.css
 - Adding src/markdown_exec/debug.py -> markdown_exec/debug.py
 - Adding src/markdown_exec/formatters/__init__.py -> markdown_exec/formatters/__init__.py
 - Adding src/markdown_exec/formatters/_exec_python.py -> markdown_exec/formatters/_exec_python.py
 - Adding src/markdown_exec/formatters/base.py -> markdown_exec/formatters/base.py
 - Adding src/markdown_exec/formatters/bash.py -> markdown_exec/formatters/bash.py
 - Adding src/markdown_exec/formatters/console.py -> markdown_exec/formatters/console.py
 - Adding src/markdown_exec/formatters/markdown.py -> markdown_exec/formatters/markdown.py
 - Adding src/markdown_exec/formatters/pycon.py -> markdown_exec/formatters/pycon.py
 - Adding src/markdown_exec/formatters/pyodide.py -> markdown_exec/formatters/pyodide.py
 - Adding src/markdown_exec/formatters/python.py -> markdown_exec/formatters/python.py
 - Adding src/markdown_exec/formatters/sh.py -> markdown_exec/formatters/sh.py
 - Adding src/markdown_exec/formatters/tree.py -> markdown_exec/formatters/tree.py
 - Adding src/markdown_exec/logger.py -> markdown_exec/logger.py
 - Adding src/markdown_exec/mkdocs_plugin.py -> markdown_exec/mkdocs_plugin.py
 - Adding src/markdown_exec/processors.py -> markdown_exec/processors.py
 - Adding src/markdown_exec/py.typed -> markdown_exec/py.typed
 - Adding src/markdown_exec/pyodide.css -> markdown_exec/pyodide.css
 - Adding src/markdown_exec/pyodide.js -> markdown_exec/pyodide.js
 - Adding src/markdown_exec/rendering.py -> markdown_exec/rendering.py
 - Adding markdown_exec-1.9.4.dev6+g3562fbb.dist-info/RECORD -> markdown_exec-1.9.4.dev6+g3562fbb.dist-info/RECORD
Successfully built markdown_exec-1.9.4.dev6+g3562fbb.tar.gz and markdown_exec-1.9.4.dev6+g3562fbb-py3-none-any.whl

[frostming]

I think it's https://github.com/pypa/pyproject-metadata

[frostming]

After some investigation it seems pyproject-metadata is not doing wrong: https://github.com/pypa/pyproject-metadata/blob/dd4ae0cefbd5bcdb37cdb8283f7f40d24ca2cad7/pyproject_metadata/__init__.py#L623-L632

The string is RFC822-compliant, and it can be correctly decoded by email.message.Message in Python. The problem may lie in the uploader not decoding the value correctly. Did you change the upload tool recently?

---

The PDM publish seems not working properly, if you use it to upload, here is the fix: pdm-project/pdm@349b17d

[pawamoy]

Thanks @frostming! I still use Twine to upload packages to PyPI.

[pawamoy]

Created pypa/twine#1197.

[pawamoy]

Actual issue in warehouse: pypi/warehouse#16496.

[pawamoy]

Interesting comment in the discussion:

I had another look at the documentation and email.utils.formataddr() is deprecated. The recommended new way is to use email.headerregistry.Address. It doesn’t encode non-ASCII characters in names but correctly quotes names that need it according to the RFCs.

>>> str(email.headerregistry.Address("Zoë", addr_spec="zoe@example.org"))
'Zoë <zoe@example.org>'
>>> str(email.headerregistry.Address("<\"Hack\"er>man", addr_spec="hacker@example.org"))
'"<\\"Hack\\"er>man" <hacker@example.org>'
>>> str(email.headerregistry.Address("᚛ᚈᚑᚋ ᚄᚉᚑᚈᚈ᚜", addr_spec="tomscott@example.org"))
'᚛ᚈᚑᚋ ᚄᚉᚑᚈᚈ᚜ <tomscott@example.org>'

https://discuss.python.org/t/core-metadata-email-fields-unicode/7421/4

(that would for pyproject-metadata anyway, still nothing actionable in pdm-backend)