Unable to generate POT for Python documentation

[rffontenelle]

Bug report

I coordinate Brazilian Portuguese translation of the Python documentation. As part of the translation update process, I run sphinx-build's gettext builder to generate POT (translation files template) and update translations.

Since 3 days ago I'm unable to successfully generate POT files because the gettext builder fails for me with the following output:

 ...
reading sources... [ 14%] distutils/apiref                                                                          
Exception occurred:
  File "/home/rffontenelle/cpython/Doc/venv/lib/python3.11/site-packages/sphinx/domains/python.py", line 1043, in run
    indextext = '%s; %s' % (pairindextypes['module'], modname)
                            ~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: 'module'
The full traceback has been saved in /tmp/sphinx-err-ds0bs43h.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make: *** [Makefile:53: build] Error 2

The log file containing the full traceback: sphinx-err-ds0bs43h.log

This might be related to #104000.

Steps to reproduce:

• cd cpython/Doc
• git switch 3.11 # current target of translation efforts
• make venv
• make ALLSPHINXOPTS='-b gettext -d build/doctrees . locales/pot' build

Your environment

• CPython versions tested on: 3.11 and 3.10
• Operating system and architecture: Arch Linux x86_64 and Ubuntu 22.04 (GitHub runner image)
• Packages installed by make venv: Jinja2-3.1.2 MarkupSafe-2.1.2 Pygments-2.15.1 alabaster-0.7.13 babel-2.12.1 blurb-1.1.0 certifi-2023.5.7 charset-normalizer-3.1.0 contourpy-1.0.7 cycler-0.11.0 docutils-0.17.1 fonttools-4.39.3 idna-3.4 imagesize-1.4.1 kiwisolver-1.4.4 matplotlib-3.7.1 numpy-1.24.3 packaging-23.1 pillow-9.5.0 polib-1.2.0 pyparsing-3.0.9 python-dateutil-2.8.2 python-docs-theme-2023.3.1 regex-2023.5.5 requests-2.30.0 six-1.16.0 snowballstemmer-2.2.0 sphinx-4.5.0 sphinx-lint-0.6.7 sphinxcontrib-applehelp-1.0.4 sphinxcontrib-devhelp-1.0.2 sphinxcontrib-htmlhelp-2.0.1 sphinxcontrib-jsmath-1.0.1 sphinxcontrib-qthelp-1.0.3 sphinxcontrib-serializinghtml-1.1.5 sphinxext-opengraph-0.8.2 urllib3-2.0.2

Linked PRs

• GH-104284: Fix documentation gettext build #104296
• [3.11] GH-104284: Fix documentation gettext build (GH-104296) #104299

[AlexWaygood]

This might be related to #104000.

Cc. @AA-Turner

[hugovk]

Checking with the excellent git bisect tool (https://thoughtbot.com/blog/git-bisect):

Starting with 0d40264 from 1st May as a good starting commit and current 3.11 (19abf69) as the bad one:

git bisect start
git bisect good 0d4026432591d43185568dd31cef6a034c4b9261
git bisect bad 3.11

and automating the bisect (from repo root):

git bisect run make -C Doc clean venv ALLSPHINXOPTS='-b gettext -d build/doctrees . locales/pot' build

Points to commit 0056369 from backport PR #104154, main PR #103996 and issue #97950:

00563694bb94f61ed4d8d36646a881ed6f028367 is the first bad commit
commit 00563694bb94f61ed4d8d36646a881ed6f028367
Author: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Date:   Thu May 4 09:39:15 2023 +0100

    [3.11] GH-97950: Use new-style index directive ('module') (GH-103996) (#104154)

 Doc/c-api/exceptions.rst              |  6 +++---
 Doc/c-api/init.rst                    | 14 +++++++-------
 Doc/c-api/intro.rst                   |  6 +++---
 Doc/library/_thread.rst               |  2 +-
 Doc/library/binascii.rst              |  4 ++--
 Doc/library/cmath.rst                 |  2 +-
 Doc/library/copy.rst                  |  2 +-
 Doc/library/copyreg.rst               |  4 ++--
 Doc/library/exceptions.rst            |  2 +-
 Doc/library/fnmatch.rst               |  4 ++--
 Doc/library/functions.rst             |  4 ++--
 Doc/library/http.client.rst           |  2 +-
 Doc/library/internet.rst              |  2 +-
 Doc/library/locale.rst                |  4 ++--
 Doc/library/marshal.rst               |  4 ++--
 Doc/library/os.path.rst               |  2 +-
 Doc/library/os.rst                    |  4 ++--
 Doc/library/pdb.rst                   |  4 ++--
 Doc/library/posix.rst                 |  2 +-
 Doc/library/pwd.rst                   |  2 +-
 Doc/library/pyexpat.rst               |  2 +-
 Doc/library/runpy.rst                 |  4 ++--
 Doc/library/shelve.rst                |  6 +++---
 Doc/library/site.rst                  |  4 ++--
 Doc/library/socket.rst                |  2 +-
 Doc/library/stdtypes.rst              |  8 ++++----
 Doc/reference/compound_stmts.rst      |  2 +-
 Doc/reference/datamodel.rst           |  8 ++++----
 Doc/reference/executionmodel.rst      |  2 +-
 Doc/reference/toplevel_components.rst |  8 ++++----
 Doc/tools/extensions/pyspecific.py    |  6 +-----
 Doc/tutorial/inputoutput.rst          |  2 +-
 Doc/tutorial/modules.rst              |  4 ++--
 33 files changed, 65 insertions(+), 69 deletions(-)
bisect found first bad commit

And confirming the previous commit 8f94c94 does build.

@AA-Turner, please could you check this?

[AA-Turner]

Will look later this afternoon, thank you for the notes!

A

[AA-Turner]

I've put up a PR that I believe fixes this (@rffontenelle would you be able to test, please?).

As analyis, I believe my previous local testing passed as I had a cached set of doctrees in my build folder, and seemingly didn't use the -E option when testing. The fix changes the patching period to come just before writing the POT files.

A

[hugovk]

Fixed in main and backported to 3.11. Thanks both!