`at` and `loc` don't accept index of the dataframe itself

[gothicVI]

Describe the bug
The following code works but mypy complains with

Invalid index type "tuple[Hashable, str]" for "_AtIndexerFrame"; expected type "Series[builtins.bool] | ndarray[tuple[Any, ...], dtype[numpy.bool[builtins.bool]]] | list[builtins.bool] | str | str_ | <9 more items>"  [index]

and pyright comlains with:

error: Argument of type "tuple[Hashable, Literal['uuid']]" cannot be assigned to parameter "idx" of type "Series[bool] | np_ndarray_bool | list[bool] | str | str_ | _IndexSliceTuple[Unknown] | list[ScalarT@__setitem__] | int | int_ | integer[Any] | unsignedinteger[Any] | signedinteger[Any] | int8 | slice[Any, Any, Any]" in function "__setitem__"
    Type "tuple[Hashable, Literal['uuid']]" is not assignable to type "Series[bool] | np_ndarray_bool | list[bool] | str | str_ | _IndexSliceTuple[Unknown] | list[ScalarT@__setitem__] | int | int_ | integer[Any] | unsignedinteger[Any] | signedinteger[Any] | int8 | slice[Any, Any, Any]"
      "tuple[Hashable, Literal['uuid']]" is not assignable to "int"
      "tuple[Hashable, Literal['uuid']]" is not assignable to "str"
      "tuple[Hashable, Literal['uuid']]" is not assignable to "str_"
      "tuple[Hashable, Literal['uuid']]" is not assignable to "Series[bool]"
      "tuple[Hashable, Literal['uuid']]" is not assignable to "integer[Any]"
      "tuple[Hashable, Literal['uuid']]" is not assignable to "list[ScalarT@__setitem__]"
      "tuple[Hashable, Literal['uuid']]" is not assignable to "list[bool]" (reportArgumentType)

To Reproduce

import pandas as pd
df = pd.DataFrame({"A": ["B", "B"]})
print(df)
#    A
# 0  B
# 1  B
for index, row in df.iterrows():
    df.at[index, "A"] = "C"
print(df)
#    A
# 0  C
# 1  C

The same happens if loc is used. Adding

assert isinstance(index, int)

before assigning solves it.

Please complete the following information:

• OS: Linux
• OS Version: Debian 12
• python 3.11
• version of type checker: mypy & pyright
• version of installed pandas-stubs: pandas 2.3.2 and pandas-stubs 2.3.2.250827

[Dr-Irv]

The issue here is that iterrows() is typed as returning a Iterator[tuple[Hashable, Series]], but we don't accept general Hashable in DataFrame.at or DataFrame.loc, because that would be too wide for those methods. And if we did narrow the return type of iterrows(), you would have to do a cast anyway. Here is a workaround that I have used in our code:

from typing import TypedDict, cast, Iterator
import pandas as pd
df = pd.DataFrame({"A": ["B", "B"]})

class MyRows(TypedDict):
    index: int
    row:   pd.Series

for index, row in cast(Iterator[MyRows], df.iterrows()):
    df.at[index, "A"] = "C"

[gothicVI]

Adding assert isinstance(index, int) is quicker imho but neither is an acceptable soluition.
It can not be that due to type constraints a dataframe can not be indexed with its own index!
If accepting Hashable is too broad (why would it be that is what's allowed after all) then maybe iterrows should not return it.

[Dr-Irv]

Adding assert isinstance(index, int) is quicker imho but neither is an acceptable soluition. It can not be that due to type constraints a dataframe can not be indexed with its own index! If accepting Hashable is too broad (why would it be that is what's allowed after all) then maybe iterrows should not return it.

The issue here is that index could be int, str, float, Timestamp, Interval, and, the hard one is tuple (due to a MultiIndex). With static typing, we can't track which kind of index is in the DataFrame. And because a tuple can be returned, that makes the calls to .loc and .at ambiguous.

It's a tradeoff of trying to support the most common usage and inflicting the least amount of pain across the entire set of stubs.