Tracking: Drop support for python `3.8`

[dangotbanned]

Had quite the ordeal trying to avoid the bugs of Protocol in (#2064 (comment)).
If that compatibility code gets merged, I'd really like to remove it ASAP.
It adds a lot of complexity for a version almost 5 months past EOL

Note

Collected more info on Protocol issues in (#2294 (comment))

I collected a lot of info a while back in a similar issue for altair

• Preparing for Python 3.8 EOL vega/altair#3497

Downstream "<=3.8"

All projects listed in README.md#used-by have dropped 3.8 support, except:

• plotly
  • drop support for Python 3.8 plotly/plotly.py#5054
• scikit-lego
  • Consider dropping python 3.8 tests koaning/scikit-lego#734

Upstream

• Drop support for Python 3.8 python/typing_extensions#568
  • Dev dependency only

[FBruzzesi]

As mentioned in Discord, timebasedcv will keep working with the latest Narwhals version that supports python 3.8 (for python 3.8), thus let's not consider that a blocker for dropping support for 3.8.
The library is quite feature complete for now and I am not planning many developments that depends on new narwhals features

[dangotbanned]

#2084 (comment)

Got it, thanks @FBruzzesi - just edited the description

[dangotbanned]

Might be useful to decide on something like this moving forward

• Python version support policy python/typing_extensions#571

[dangotbanned]

Minimum version bumps

Dropping support for 3.8 will trigger these bumps

Package	Minimum	Source
pandas	1.1.3	https://pandas.pydata.org/docs/whatsnew/v1.1.3.html#added-support-for-new-python-version
modin	0.8.2	modin-project/modin#2371

narwhals/narwhals/utils.py

Lines 639 to 651 in a0d82cd

	MIN_VERSIONS: dict[Implementation, tuple[int, ...]] = {
	Implementation.PANDAS: (0, 25, 3),
	Implementation.MODIN: (0, 25, 3),
	Implementation.CUDF: (24, 10),
	Implementation.PYARROW: (11,),
	Implementation.PYSPARK: (3, 5),
	Implementation.PYSPARK_CONNECT: (3, 5),
	Implementation.POLARS: (0, 20, 3),
	Implementation.DASK: (2024, 8),
	Implementation.DUCKDB: (1,),
	Implementation.IBIS: (6,),
	Implementation.SQLFRAME: (3, 22, 0),
	}

[dangotbanned]

Update

• We're now 4-5 months away from 3.9 EOL
• The next release of typing_extensions drops support for 3.8
• mypy already dropped support, but apparently will soon drop more support?
• The downstream issue in plotly hasn't had any activity, so I'm not sure if there's any blocking issues on their end
  • (drop support for Python 3.8 plotly/plotly.py#5054)

Looking forward

Bumping our minimum version to >=3.10 has quite a few more benefits (What’s New In Python 3.10).
The one's I see having the biggest impact for us are:

• PEP 634: Structural Pattern Matching
  • I'm sorely missing this in (feat(RFC): A richer Expr IR #2572) 😔
  • As it is a syntax feature, there's no alternative
• PEP 618 – Add Optional Length-Checking To zip
  • We're using zip everywhere
  • Can be worked around like we do for a test utility
  • But in >=3.10, we can use a ruff autofix zip-without-explicit-strict

Backends

Important

There's no pressure from these for now

But, the next pandas release (soon?) and Python 3.9 EOL (2025-10) will likely be the events we should watch out for.

Package	Minimum	Notes	Source
pandas	2.3.*	Next minor introduces >=3.10	pandas-dev/pandas#60792
modin	?	Follows pandas	https://github.com/modin-project/modin/blob/cc38a16aab45b7c5cd1315c14a6cc046390d2fe7/setup.py#L53-L54
cudf	24.10.0	Already our minimum	https://github.com/rapidsai/cudf/releases/tag/v24.10.00
pyarrow	?	20.0.0 supports >=3.9	https://github.com/apache/arrow/blob/f1c6619332a280625a2381b03b2fd4a0722db82f/python/pyproject.toml#L37
pyspark	?	4.1.0 supports >=3.9	https://github.com/apache/spark/blob/b0c2ba357bf080dd328b95e4a6402b134a641a1a/python/packaging/classic/setup.py#L372
pyspark-connect	?	4.1.0 supports >=3.9	https://github.com/apache/spark/blob/b0c2ba357bf080dd328b95e4a6402b134a641a1a/python/packaging/connect/setup.py#L127
polars	?	1.30.0 supports >=3.9	https://github.com/pola-rs/polars/blob/78b0cdc7116d16d35afbfcc5d7a014c14da9b5b8/py-polars/pyproject.toml#L13
dask	2024.8.1	Bumps our minimum	dask/dask#11245
duckdb	?	1.3.0 supports >=3.7	https://github.com/duckdb/duckdb/blob/4dc61eb5e38809554afa39b4b7c97547c0dee8c6/tools/pythonpkg/pyproject.toml#L15
ibis	?	10.5.0 supports >=3.9	https://github.com/ibis-project/ibis/blob/87814c6f69230ced51bfc6436161867c87a2925f/pyproject.toml#L4
sqlframe	?	3.35.0 supports >=3.9	https://github.com/eakmanrq/sqlframe/blob/f25f4096ecb3f8daacf9d19459d9c6f579032c45/setup.py#L20

Thoughts

Personally, I'd like to drop 3.8 support now and at the same time introduce a Python version support policy (e.g. python/typing_extensions#571).
My suggestion for that is to keep it simple and stick to EOL support - which is what most of the (upstream) backends do anyway.
We can always extend this to avoid breaking downstream but I think documenting it 5 months ahead of 3.9 EOL will leave plenty of time to prepare CI workflows 🤞.

I'm open to any other ideas, just hoping to move this issue along if possible 🙂

[MarcoGorelli]

Sure let's do it and drop 38, it's time