Narwhals tools for working with and generating datetime indices?

[jessegrabowski]

Hi all --

I have a few packages for doing statistics on time series(here, here, here), and I'm currently all-in on pandas DateTimeIndex objects for getting and validating frequency, generating new date ranges, and so on. I'm very interesting in diversifying away from Pandas, and Narwhals is a natural path forward for me.

I looked through your docs and online talks, but didn't see anything about this specific use case. Does Narwhals offer tools for these kinds of time-series tasks in a backend-agnostic way?

One specific test case I'm in is generating a companion high-frequency date range from a provided low-frequency series, as in this example (the Denton one). So if you give start-of-year annual data like 2000, 2001, 2002 and ask for start-of-quarter data out, I can generate a date range like 2001-01-01, 2001-04-01, 2001-07-01, 2001-10-01, ... etc.

This also comes up when generating forecasts; it would be nice to be able to automatically spit out the same kind of dataframe the user brought to the model with h steps forward, using the same frequency.

I'd be interested in contributing to narwhals if any of these features are on the roadmap. Thanks in advance for any advice/suggestions!

[FBruzzesi]

Hey @jessegrabowski thanks for considering adopting Narwhals in multiple projects.

We currently don't have support to generate date(time) ranges. The closest we got is #2895 (comment) but int_range is under discussion, and date_range would benefit from it. That said, while for int_range there are other ways to achieve the same outcome, for date_range it does not seem to be the case.

1. I think this discussion might make it into an issue for better tracking
2. Are there other specific time series features you might require?

[jessegrabowski]

I'm happy to make an issue if we can zero in on a specific set of features that make sense.

I'm mostly interested in:

• Validating user inputs
• Frequency detection / handling (including business calendars etc)
• Generating date ranges based on frequencies

The types of stuff I want to do allow the user to give me a dataframe of any kind, do some stats on it, then let her call .forecast and get back the same kind of dataframe with a date column that makes sense given the input dates.

I'm hesitant to jump straight to "let's do x" because I worry this kind of calendar math is a deep rabbit hole, and it might not belong in Narwhals. For example, I could always use Nawhals as a front-end, use pandas to do the datetime logic internally, then convert that result with narwhals to whatever format the user gave and ship it all back to_native.

[FBruzzesi]

The general philosophy is:

• if it's a polars feature, then it's in scope for narwhals (e.g. the equivalent of pl.date_range)
• if it's a utility function to make something pandas compatible, it might be in scope (e.g. we have maybe_get_index, maybe_set_index, maybe_reset_index)
• if it's something else, i.e. logic on top of the core functionalities, then it would not belong here

[MarcoGorelli]

if it's a polars feature, then it's in scope for narwhals

then it might be in-scope 😉

[dangotbanned]

@jessegrabowski I've had a quick skim through these links:

I have a few packages for doing statistics on time series(here, here, here),

I'd say that besides echoing @FBruzzesi's advice:

if it's a polars feature, then it's in scope for narwhals

I would recommend reading through the following to see if polars can cover your pandas (+ friends) use-case.

Not everything listed here is in narwhals yet.
But a request for a specific polars feature to solve a problem in adopting narwhals is the kind of thing we can work best with 🙂

User Guide

• time-series/resampling
• time-series/rolling
• time-series/parsing
• time-series/filter

API Reference

• expressions/temporal
• polars.date_range
• polars.date_ranges
• polars.datetime_range
• polars.datetime_ranges

[MarcoGorelli]

Thanks for your request!

I'll address some points

Validating user inputs

What kind of validation are you looking to do?

Frequency detection / handling (including business calendars etc)

This depends on what kind of frequency detection you're looking to do.

Fixed interval ones like "hour" should be easy, use .diff() and then check the unique elements

Calendar intervals like "day" - especially if you want to include business calendars - are trickier. I didn't even know that pandas did this - which pandas functionality do you use for this?

Generating date ranges based on frequencies

My issue with pl.date_range / pl.int_range and those is that they're expressions but we'd only be able to support them in the eager case. I'm OK with adding this functionality, but under the guide of nw.Series.from_int_range or something like that