Python 3.13.0, 3.13.1 python_d.exe cannot import numpy, python.exe works fine.

[prniii]

Bug report

Bug description:

import numpy

Using default Python.org installers installed for all users. Running Windows 11 (intel) using the python 3.13 debug executable, python_d.exe numpy gives an ImportError. While the import is successful if using python.exe. My suspicion is the changes that 3.13 militantly enforces making everything debug need an _d suffix. Libraries, python313.zip needs to be python313_d.zip so I'm assuming it's looking for _d libraries in the numpy dirctories as well? I'm seeing a failure on numpy._core._multiarray_umath which is a .pyd file.

This is broken for python_d.exe and embedding the python313_d.dll gives rise to the same issues. It makes a debug build worthless for packages with compiled executables since the pip installs aren't downloading debug builds of the packages.

Below is the output from running in a cmd window on Windows

c:\Program Files\Python313>.\python_d.exe
Python 3.13.1 (tags/v3.13.1:0671451, Dec  3 2024, 18:47:37) [MSC v.1942 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
  File "C:\Users\panels\AppData\Roaming\Python\Python313\site-packages\numpy\_core\__init__.py", line 23, in <module>
    from . import multiarray
  File "C:\Users\panels\AppData\Roaming\Python\Python313\site-packages\numpy\_core\multiarray.py", line 10, in <module>
    from . import overrides
  File "C:\Users\panels\AppData\Roaming\Python\Python313\site-packages\numpy\_core\overrides.py", line 8, in <module>
    from numpy._core._multiarray_umath import (
        add_docstring,  _get_implementing_args, _ArrayFunctionDispatcher)
ModuleNotFoundError: No module named 'numpy._core._multiarray_umath'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\panels\AppData\Roaming\Python\Python313\site-packages\numpy\__init__.py", line 127, in <module>
    from numpy.__config__ import show as show_config
  File "C:\Users\panels\AppData\Roaming\Python\Python313\site-packages\numpy\__config__.py", line 4, in <module>
    from numpy._core._multiarray_umath import (
    ...<3 lines>...
    )
  File "C:\Users\panels\AppData\Roaming\Python\Python313\site-packages\numpy\_core\__init__.py", line 49, in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.13 from "c:\Program Files\Python313\python_d.exe"
  * The NumPy version is: "2.1.3"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy._core._multiarray_umath'


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    import numpy
  File "C:\Users\panels\AppData\Roaming\Python\Python313\site-packages\numpy\__init__.py", line 132, in <module>
    raise ImportError(msg) from e
ImportError: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.
>>>

CPython versions tested on:

3.13

Operating systems tested on:

Windows

[ZeroIntensity]

How did you install numpy? Looking at the tracebacks, my guess is that while you have the Python source, you have extensions built for release 3.13, not debug 3.13. I'm not sure we support using debug builds with extensions built for release builds, because the ABI is different. Does this still occur if you go into a venv using python_d, and then install numpy inside it?

[zooba]

The debug binaries for CPython require debug binaries for extension modules, which are practically never published on PyPI.

It makes a debug build worthless ...

Yeah, pretty much. They're intended for debugging Python itself, not for debugging other projects.

We have a pending issue somewhere to write up generic extension module compilation steps (so that build backends have a reference), and one of the things it ought to include is how to build debuggable extensions against release ABI CPython.

But that doesn't seem like it would help your scenario here at all? What are you hoping to achieve by using a debug build of CPython?

[prniii]

The specific issue I ran into relates to python embedded in a C++ application. I found the command line behavior of python_d.exe is the same for 3.11 and 3.13. However, in my case the embedded Python behaves differently... My debug build with Python 3.11 uses python311_d.lib, .dll but does seem to properly import numpy in the debug build. Where using the same Visual Studio project file, just updated to pickup python313_d.lib, .dll I am unable to load numpy. Toying around with the project, and modifying the pyconfig.h in Python 3.13, I was able to convince my project to build a debug version of my application where _DEBUG is set, and link against the release build of Python 3.13. But I had to dig into pyconfig.h to do 2 things. Make all the conditionals depend on Py_DEBUG, Move the #if _DEBUG #define Py_DEBUG before the usage of then now updated _DEBUG checks and comment out the #define Py_DEBUG so when my app includes Python.h it's looking for a release python313.dll instead of the debug version.

What I would really like to be able to do is what embedded 311 seems to do, use the python311_d.dll but still be able to import non-debug packages. If not, I have at least found the work around by modifying the pyconfig.h on windows.

My embedding app in 3.11 just uses PyPreConfig to set utf-8 =1, isolated = 0, use_environment = 0 and Py_NoSiteFlag =1, before a plain Py_Initialize(); Because, while an embedded app I still am providing users the ability to use pip and install packages, but into a user specific, app specific directory for site-packages.

With Python 3.13 I had to change my Python initialization, removing the Py_NoSiteFlag in favor of Py_Config options. PyPreConfig only setting utf-8=1, and setting PyConfig site_import=0, isolated=0, use_environment =0, running PyConfig_Read(), then Py_InitializeFromConfig() Code I run after startup sets site.USER_BASE = the path to my isolated directory.

I thought when Python313's python_d exhibited the same behavior as the python313.dll I had found the simple test case. Until today I found Python311's python_d.exe behaves the same way even thought the embedded python31x_d.dlls seem to behave differently with respect to package imports.

So in my case being able to step from the C++ application into the Python DLL is desired, but being able to load release packages is what I want most of the time, because I am debugging the behavior of my app embedding Python, not really debugging Python or the loaded modules.

[ZeroIntensity]

I suspect you probably installed numpy using the pip from your 3.11 debug build and it just built 3.11d wheels for you locally. But that doesn't answer the question: why are you trying to embed a debug interpreter?

[zooba]

What I would really like to be able to do is what embedded 311 seems to do, use the python311_d.dll but still be able to import non-debug packages.

It shouldn't be able to do this, the binary is no different and the extension module suffixes are built in. I would guess something different was going on, perhaps as ZeroIntensity says you managed to get a local debug build of numpy for 3.11?

So in my case being able to step from the C++ application into the Python DLL is desired, but being able to load release packages is what I want most of the time

Then you will probably be satisfied with Python's release binaries and debug symbols. I'd suggest #undef _DEBUG//#define _DEBUG around the Python.h include as the easiest way to deal with the preprocessor, and be aware that global state handled by the C runtime (locale, stdio, FILE*, descriptors, etc.) will likely not be shared with Python until you're on matching builds.

[ZeroIntensity]

After some discussion elsewhere with @godlygeek, and based on what the documentation says about ABI compatibility between debug and release builds, I think this is safe to write off as misuse of the embedding APIs rather than an interpreter problem.