runpy.run_path mishandles Path objects

[nedbat]

Bug report

runpy.run_path will accept a Path object, and run it, but various code inspection tools will be unhappy with it because it is not a string. Should run_path convert its argument to a true string before execution?

Coverage.py is one of those tools; this report is based on nedbat/coveragepy#1417.

Create debugme.py:

from pathlib import Path
import runpy, sys

runme = Path(__file__).resolve().parent / sys.argv[1]
print(f"Running {runme!r}")
runpy.run_path(runme)

Create inspectme.py:

import inspect

print(inspect.stack())
print('here')

Run it:

% python3.12 debugme.py inspectme.py
Running PosixPath('/System/Volumes/Data/root/src/bugs/bug1417/inspectme.py')
Traceback (most recent call last):
  File "/System/Volumes/Data/root/src/bugs/bug1417/debugme.py", line 6, in <module>
    runpy.run_path(runme)
  File "<frozen runpy>", line 291, in run_path
  File "<frozen runpy>", line 98, in _run_module_code
  File "<frozen runpy>", line 88, in _run_code
  File "/System/Volumes/Data/root/src/bugs/bug1417/inspectme.py", line 3, in <module>
    print(inspect.stack())
          ^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 1748, in stack
    return getouterframes(sys._getframe(1), context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 1723, in getouterframes
    traceback_info = getframeinfo(frame, context)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 1685, in getframeinfo
    lines, lnum = findsource(frame)
                  ^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 1060, in findsource
    module = getmodule(object, file)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 983, in getmodule
    f = getabsfile(module)
        ^^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 952, in getabsfile
    _filename = getsourcefile(object) or getfile(object)
                ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 928, in getsourcefile
    if any(filename.endswith(s) for s in all_bytecode_suffixes):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 928, in <genexpr>
    if any(filename.endswith(s) for s in all_bytecode_suffixes):
           ^^^^^^^^^^^^^^^^^
AttributeError: 'PosixPath' object has no attribute 'endswith'

Coverage.py fails earlier, but for a similar reason:

% coverage run debugme.py inspectme.py
Running PosixPath('/System/Volumes/Data/root/src/bugs/bug1417/inspectme.py')
Traceback (most recent call last):
  File "/System/Volumes/Data/root/src/bugs/bug1417/debugme.py", line 6, in <module>
    runpy.run_path(runme)
  File "/usr/local/pyenv/pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 288, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/usr/local/pyenv/pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/local/pyenv/pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/control.py", line 344, in _should_trace
    disp = self._inorout.should_trace(filename, frame)
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/inorout.py", line 335, in should_trace
    filename = source_for_file(dunder_file)
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/python.py", line 105, in source_for_file
    if filename.endswith(".py"):
AttributeError: 'PosixPath' object has no attribute 'endswith'

Using coverage.py's --timid flag uses a Python trace function and also shows the error:

% coverage run --timid debugme.py inspectme.py
Running PosixPath('/System/Volumes/Data/root/src/bugs/bug1417/inspectme.py')
Traceback (most recent call last):
  File "/System/Volumes/Data/root/src/bugs/bug1417/debugme.py", line 6, in <module>
    runpy.run_path(runme)
  File "/usr/local/pyenv/pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 288, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/usr/local/pyenv/pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/local/pyenv/pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/System/Volumes/Data/root/src/bugs/bug1417/inspectme.py", line 1, in <module>
    import inspect
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/pytracer.py", line 166, in _trace
    disp = self.should_trace(filename, frame)
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/control.py", line 344, in _should_trace
    disp = self._inorout.should_trace(filename, frame)
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/inorout.py", line 335, in should_trace
    filename = source_for_file(dunder_file)
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/python.py", line 105, in source_for_file
    if filename.endswith(".py"):
AttributeError: 'PosixPath' object has no attribute 'endswith'

I don't understand why a simpler trace example gets a string for co_filename? Create traceme.py:

import sys

def foo():
    print("FOO")

def trace(frame, event, arg):
    if "traceme" in str(frame.f_code.co_filename):
        print(type(frame.f_code.co_filename))
        print(f"{frame.f_code.co_filename=}")

sys.settrace(trace)

foo()

print('here')

and run it:

% python debugme.py traceme.py
Running PosixPath('/System/Volumes/Data/root/src/bugs/bug1417/traceme.py')
<class 'str'>
frame.f_code.co_filename='/System/Volumes/Data/root/src/bugs/bug1417/traceme.py'
FOO
here

Your environment

I'm on a Mac, and this behavior seems consistent on versions 3.7 through 3.12.0a1.

Linked PRs

• gh-99437: runpy: decode path-like objects before setting globals #107644
• [3.12] gh-99437: runpy: decode path-like objects before setting globals #114838

[nedbat]

A bit more detail... Create modname.py:

import sys

def func(): pass

print(f"{func.__module__=}")
print(f"{sys.modules[func.__module__].__file__=}")

print('here')

and run it:

% python3.12 debugme.py modname.py
Running PosixPath('/System/Volumes/Data/root/src/bugs/bug1417/modname.py')
func.__module__='<run_path>'
sys.modules[func.__module__].__file__=PosixPath('/System/Volumes/Data/root/src/bugs/bug1417/modname.py')
here

The executed file has a module with __file__ as a Path object instead of a string.

[Jacob-Stevens-Haas]

Some of this is a bit above my level, but FWIW verified the same errors and output on Ubuntu 22.04 and python 3.10.6

[brettcannon]

There's probably an implicit os.fsdecode() call missing to convert whatever gets passed in as a path to a string.

[FFY00]

The runpy.run_path documentation does not mention support for passing path-like objects, but since it somewhat works, I think it's better to not break it and instead fix this issue by properly supporting this use-case.