DictWriter is not subscriptable

[clo-vis]

mypy complains that DictWriter is missing the type parameter in the code:

from csv import DictWriter
from typing import IO
def f_is_for_fun(file: IO[str]) -> DictWriter:
    return DictWriter(file, ["id", "name"])

which is reasonable, because DictWriter is defined as class DictWriter(Generic[_T]) (https://github.com/python/typeshed/blob/master/stdlib/csv.pyi)

But if I change the code to make mypy happy:

from csv import DictWriter
from typing import IO
def f_is_for_fun(file: IO[str]) -> DictWriter[str]:
    return DictWriter(file, ["id", "name"])

then I become an error at runtime:

…
def f_is_for_fun(file: IO[str]) -> DictWriter[str]:
TypeError: 'type' object is not subscriptable

or, with python 3.11
TypeError: type 'DictWriter' is not subscriptable

And if I add
from __future__ import annotations
then mypy and the python interpreter are happy, but not pylint.

[clo-vis]

#4800

[AlexWaygood]

This is a very commonly encountered problem -- there's some useful mypy docs on tackling it here: https://mypy.readthedocs.io/en/stable/runtime_troubles.html#using-classes-that-are-generic-in-stubs-but-not-at-runtime.

For this specific situation, you can get round it by just using quoted annotations, e.g.

from csv import DictWriter
from typing import IO
def f_is_for_fun(file: IO[str]) -> "DictWriter[str]":
    return DictWriter(file, ["id", "name"])

CPython is also pretty receptive to feature requests adding __class_getitem__ to classes which are generic in typeshed (e.g. python/cpython#92129).

[AlexWaygood]

And if I add from __future__ import annotations then mypy and the python interpreter are happy, but not pylint.

Wait, why does pylint complain about __future__ annotations??

[clo-vis]

pylint has no problem with __future__, it just complains that DictWriter is not subscriptable (with or without __future__)

[AlexWaygood]

pylint has no problem with __future__, it just complains that DictWriter is not subscriptable (with or without __future__)

That seems like a problem with pylint, do you know if there any bug reports tracking that issue?

[clo-vis]

Coincidentally I know there is a bug report a few minutes old :)
pylint-dev/pylint#6523

[AlexWaygood]

Coincidentally I know there is a bug report a few minutes old :) PyCQA/pylint#6523

Great. I'm going to close this for now, as I don't think there's anything actionable from the typeshed end on this one. There are some pretty easy workarounds for this specific situation (either use explicit quotes, or use from __future__ import annotations); the pylint bug is a pylint bug; and if we want to add __class_getitem__ to the runtime, that should be discussed over at the CPython issue tracker rather than here.

I don't know that I'm 100% confident that the stubs for csv.DictReader and csv.DictWriter are perfect right now, but I'll need to some experimenting before I have fully formed thoughts on that. And it's not really the issue under discussion here, anyway.

[cdce8p]

After reading the pylint issue, I opened an issue in cpython python/cpython#92391 to add __class_getitem__. Happy to provide a PR, not sure though if that's the way to go now though after your last comment @AlexWaygood.

I'll have a look at the other pylint issue with from __future__ import annotations though. That seems like a bug.

[AlexWaygood]

Happy to provide a PR, not sure though if that's the way to go now though after your last comment @AlexWaygood.

DictReader seems pretty clearly like it should be generic -- the type that's returned from DictReader.__next__ depends very clearly on the type that you stick into DictReader.__init__.

In fact, DictReader seems like it could possibly be typed more correctly if it took another parameter: see #5994 (comment). (But that's irrelevant to the runtime; adding __class_getitem__ will allow it to take any number of parameters at runtime.)

I was confused about the implicit conversion to string in DictWriter, which made me wonder if DictWriter should be generic. But DictWriter does some runtime typechecking in the _dict_to_list method before it does the implicit conversion to string. So I think it makes sense for DictWriter to be generic as well (as it is in the stub currently).

My nervousness here is that these haven't been generic in typeshed for that long, and they're not "simple" classes to type. So the stubs might still change in the future.

But I think they'll probably stay generic. So I think this is okay.

[Akuli]

I made DictWriter DictReader generic in #5366 to support writing enums to reading enums from csv files easily.

[AlexWaygood]

I made DictWriter generic in #5366 to support writing enums to csv files easily.

I think that was a good change to make. Just took me a little bit of time to get my head around some bits of the stub :)