Handle PROGRAM-DATE-TIME across discontinuities and introduce timeline change event

[miyaji27]

Have you read the FAQ and checked for duplicate open issues?

Yes, I have. This specific issue of getPlayheadTimeAsDate inaccuracy across discontinuities with a proposal for a corresponding event does not appear to be covered in existing open issues.

Is your feature request related to a problem? Please describe.

Yes. The player.getPlayheadTimeAsDate() method currently returns an incorrect wall-clock time for HLS streams where the #EXT-X-PROGRAM-DATE-TIME (PDT) tag is not contiguous across an #EXT-X-DISCONTINUITY boundary.

The current implementation calculates the date by simply adding the playhead's elapsed time (in the presentation timeline) to the initial PDT found at the start of the manifest. This assumption fails in common real-world scenarios:

1. Live Stream Gaps: When a live stream stops and restarts, a discontinuity tag is often inserted, and the new segments start with a new PDT that reflects the new wall-clock time. The player, however, continues to calculate from the original PDT, causing the reported time to drift significantly from reality.
2. Server-Side Ad Insertion (SSAI): Ads are often stitched into the main content with discontinuity tags. If an ad's duration does not perfectly match the content it replaces, the PDT after the ad break will jump forward or backward to align with the live timeline. The current getPlayheadTimeAsDate() does not account for this jump, leading to inaccurate time reporting during and after ad breaks.

This makes it impossible for applications to build reliable features based on wall-clock time, such as displaying an accurate program clock or synchronizing secondary events with the stream.

Describe the solution you'd like

I propose a two-part solution to make Shaka Player's time reporting robust and transparent:

1. Make getPlayheadTimeAsDate() Discontinuity-Aware:
   The core logic of getPlayheadTimeAsDate() should be enhanced. Instead of relying on a single initial PDT, the player should:

   • During HLS manifest parsing, build a map of "timeline regions", where each region (demarcated by a discontinuity) is associated with its starting presentation time and its starting PDT.
   • When getPlayheadTimeAsDate() is called, it should first identify which timeline region the current playhead position falls into.
   • It should then calculate the date by adding the playhead's offset within that specific region to that region's starting PDT. This will ensure the returned date is always relative to the currently active timeline.

2. Introduce a timelinechange Event:
   To allow applications to react to these timeline shifts, I propose adding a new event, timelinechange, to the shaka.Player instance.

   • This event would be dispatched whenever the playhead crosses a discontinuity boundary that has a new #EXT-X-PROGRAM-DATE-TIME.
   • The event payload should include essential information about the change:
     • programDateTime: The new Date object that is now the basis for the timeline.
     • timestamp: The presentation time (in seconds) at which this change occurred.

   An example of how an application would use this:

   player.addEventListener('timelinechange', (event) => {
     console.log(`Timeline changed at ${event.timestamp}s.`);
     console.log(`New base Program Date Time is ${event.programDateTime.toISOString()}`);
     // Application can now update its UI clock or other time-sensitive elements.
   });

This dual approach not only fixes the immediate bug but also empowers developers with the tools to build more robust applications on top of Shaka Player.

Describe alternatives you've considered

1. Only Fix getPlayheadTimeAsDate(): We could just fix the method without adding a new event. This would solve the accuracy problem, but it would be a "silent" fix. Applications would have no way of knowing when the underlying time base has changed, which can be critical for UI updates or logging. It is less transparent.

2. Only Add an Event: We could add the timelinechange event and leave it to the application developer to calculate the correct time. This is not ideal as it would shift a complex burden from the player (which has all the necessary context from parsing the manifest) to every application developer, leading to duplicated and potentially incorrect implementations.

The proposed solution of fixing the core method and adding a notification event is the most comprehensive and developer-friendly approach.

Additional context

This feature would be immensely valuable not only for HLS but could also establish a pattern for DASH Multi-Period streams. A Period transition in DASH is conceptually similar to a discontinuity in HLS. The timelinechange event could be generalized to fire on Period changes as well, providing a unified API for developers to handle timeline shifts regardless of the streaming format.

Are you planning to send a PR to add it?

N/A

[tykus160]

Interesting idea! @theRealRobG what do you think about it?

[theRealRobG]

Yes, makes sense to me; I even asked for something similar in a Feedback ticket to Apple for AVPlayer on iOS (as an aside, for another view on API style, Apple introduced a somewhat similar idea with AVPlayerItemIntegratedTimeline, though it is focused on daterange and specifically interstitials).

One thing to bear in mind is that discontinuities are, unfortunately, not the only place where the timeline can technically change. This came out of this thread in hls-interest, and was subsequently clarified in the specification in draft 15 with the following addition:

The Server MUST NOT add any EXT-X-PROGRAM-DATE-TIME tag to a Playlist
that would cause the mapping between program date and Media Segment
to become ambiguous. One exception is permitted: the Server MAY
introduce small (sub-second) overlaps to acount for drift between the
encoder clock and some independently produced date/time reference.
The later segment MUST only partially overlap the preceeding segment.
The client MUST resolve these ambiguous date/times in favor of the
later segment.

For me, this was actually an upsetting conclusion, because it means that you technically can't just rely on discontinuity boundaries to defined date-time mapping regions (though in practice, that may be good enough). This means that to do something like this in a technically correct fashion, it may be more complicated and process intensive then anticipated, especially if the stream has EXT-X-PROGRAM-DATE-TIME tags on every segment (as do our live streams).

I'm thinking of some outcomes of this definition and these points come to mind:

• Define the region with every PDT tag.
  • This could potentially be a performance issue (though I have no benchmarks to verify that). For example, we have live streams that have 10h windows of 3.8s segments, with each one having a PDT tag, meaning about 9,474 Date objects being created on manifest parse, and then a dictionary to hold each of those. This would then be a very noisy event system as potentially every segment triggers a timelinechange event, though those could be smoothed by a function to check if there was actually a mapping change, but then that is more complexity. Perhaps this could be done upfront on parsing to simplify the dictionary holding this mapping.
• Define the region with discontinuities only.
  • This is as the suggestion in the PR; however, this would risk being technically incorrect. For a business specific project where we know our streams don't introduce these small corrections, that would be fine. But given Shaka is a widely used generic player, I wouldn't feel comfortable about that.

Is there a best of both worlds in the middle?

Also, as an aside, I believe that Safari native player does not give you the ability to do this, because you only have access to getStartDate method for a mapping (I don't think anything else exists). I suppose technically, if we're pulling the playlist alongside the native player, we could locate the segment corresponding to getStartDate and then recreate the mapping from that... I can't remember of Shaka still does playlist polling on native Safari playback.

[theRealRobG]

As a note on DASH, the program-date-time mapping is accomplished via the ProducerReferenceTime element, described in section 5.12. I'm not sure if a similar result is possible in DASH without creating a new Period, because the element is in the CommonAttributesElements, which only goes as deep as the Representation/SubRepresentation, so does not give access to a per segment mapping (e.g. within SegmentTimeline)... I suppose it is possible given that the element notifies the presence of prft within the segments and provides only an initial mapping for the period, so segment prft can subsequently change (though that is even more annoying because then there is nothing at the manifest level that can indicate that).

[tykus160]

@theRealRobG thanks for your input!
We won't take Safari Native into account here as I don't see anything else than getStartDate in Apple documentation. Shaka doesn't fetch playlists at all during native playback. We have this feature in internal fork only, but we won't contribute it back as we will likely switch to MSE and it adds too much unnecessary complexity for general purpose project IMO.

I'm personally more pro of defining regions only for discontinuities, as I'm worried about storing Date objects for every PDT (as we have them for every segment, as you pointed out already). Something to consider though.

[gkatsev]

For me, this was actually an upsetting conclusion, because it means that you technically can't just rely on discontinuity boundaries to defined date-time mapping regions (though in practice, that may be good enough).

Since it says to resolve overlaps based on the later segments, wouldn't that imply that at the disco, you can reliably just ignore the previous times?

[gkatsev]

If I remember correctly, getStartDate() accounts for discos and will return a different value after playback has crossed a disco with a PDT gap. So, fixing getPlayheadTimeAsDate() to account for these gaps makes sense to me as, even if that's the only fix.

For the event, how useful is it in practice? Since most of the time, it's supposed to be transparent to the user? Also, we'd want to make sure that this works with both HLS and DASH. Would the DASH version trigger this event on every period?

1. Live Stream Gaps: When a live stream stops and restarts, a discontinuity tag is often inserted, and the new segments start with a new PDT that reflects the new wall-clock time

Just as an extra fyi, I think HLS spec now recommends using #EXT-X-GAP for these instead, though, your point still stands as the disco way is still very common.

[theRealRobG]

Since it says to resolve overlaps based on the later segments, wouldn't that imply that at the disco, you can reliably just ignore the previous times?

Yep, so the only concern is clock drift in between discontinuities that makes the region mapping inaccurate (after the drift correction)... But, in fairness, that should really never be large, so in practice I'd imagine it would only ever be at most a few fractions of a second out (if it ever happens)... That's probably good enough to accept the "region by discontinuity" approach 👍