HLS DAI TS demuxed live: segment mis-positioned in sequence mode when DTS isn't continuous with previous segment

[saintberry]

FAQ

• I have read the FAQ and checked for duplicate open issues

If the problem is related to FairPlay, have you read the tutorial?

Not applicable

What version of Shaka Player are you using?

v4.15, v4.16, v5.2

Can you reproduce the issue with our latest release version?

Yes

Can you reproduce the issue with the latest code from main?

Not tested

Are you using the demo app or your own custom app?

Both

What browser and OS are you using?

macOS Chrome 151.0.7922.170, CAF v3 latest

For embedded devices (smart TVs, etc.), what model and firmware version are you using?

System firmware version: 30.20260121.103.1604600 Cast firmware: 3.78.533433

What are the manifest and license server URIs?

DAI asset key k7m3kY2bQm-tx9upVP5YTw

What configuration are you using? What is the output of player.getNonDefaultConfiguration()?

{
  "cmcd": {
    "sessionId": "331f9c11-a286-4669-8491-78f49e0cf856"
  }
}

What did you do?

Load the DAI asset key k7m3kY2bQm-tx9upVP5YTw which is a HLS TS Demuxed live stream. This stream has a 30min DVR window. Use the scrub bar to navigate through the timeline until you encounter an ad period which will contain slate because no ad tags have been provided in the DAI request. Let the slate play into content.

On 5.x you will notice the player log to the console that it has detected a possible encoding error and then make a large jump (up to a full segment length, in this example 6.4sec) cutting out the first segment of content after the ad period

On 4.x you will notice the player play the first frame of content after the ad period at the correct time however audio will not start until about 6.4 sec after the content starts and video / audio will remain out of sync until a discontinuity is passed.

I am logging this bug for 4.x as it's impacting our Chromecast application. However the bug still exists in 5.x, although 5.x does a better job at recovering.

• I have created a JSFiddle for 4.x that demonstrates the issue: https://jsfiddle.net/q58zawer/
• For 5.x just load k7m3kY2bQm-tx9upVP5YTw into the Ads Asset Key section

A while back I logged a similar bug here: #9154 it was marked as fixed and closed. The changes there have improved the situation but not resolved it. Sorry for raising it again. I reached out to our Google DAI account manager and they suggested raising here again is the best approach.

I spent a lot of time with Claude trying to understand what could be the cause of the issue and found a patch in 4.x that smooths over it somewhat - it's not perfect but better than the audio being out of sync! I tested by compiling my own 4.15.5 and running on CAF device (Nest Hub). I'm not sure if this will be useful, however will share in case it is.

# Corrective resync for cross-segment position collapse (Shaka Player v4.15.55,
# reproduced against latest 4.x too)
#
# Applies to: shaka-player v4.15.55 tag, lib/media/media_source_engine.js
#
# Root cause: within one discontinuity group, MSE `sequence` append mode
# only gets a fresh timestampOffset for the FIRST segment (resync() is
# called once per discontinuitySequence change). Every later segment in
# the group is positioned by rebasing its own embedded decode timestamp
# against that one reference point — correct only if every segment's
# embedded DTS forms one continuous clock across physical file boundaries.
# Google DAI ad-break slate/filler segments (and some real-content
# segments observed on our own live streams) break that assumption, so the
# segment gets appended on top of the PREVIOUS segment's position instead
# of its own, and every subsequent segment inherits the same error until
# a discontinuity boundary or a much later gap-jump stall-recovery.
#
# Fix: Shaka already detects this after the fact via the goog.DEBUG-gated
# "Possible encoding problem detected!" check in appendBuffer() — it just
# never acted on it. This patch reuses the same detection (buffered end vs.
# reference.endTime, tolerance: half the segment duration, or a 30ms added-
# range-size mismatch) and calls resync(contentType, reference.endTime) on
# mismatch, wiring the existing diagnostic to a corrective action. Runs in
# both debug and release builds, since the bug isn't debug-build-specific.
#

--- a/lib/media/media_source_engine.js
+++ b/lib/media/media_source_engine.js
@@ -1390,6 +1390,31 @@
         }
       }
     }
+
+    // Fix: within one discontinuity group, only the first segment gets
+    // a fresh timestampOffset (see resync() / initSourceBuffer_'s
+    // discontinuitySequence gating). If a segment's embedded decode
+    // timestamps aren't actually continuous with the previous segment's
+    // (e.g. independently encoded/spliced content within one nominal
+    // continuity group, as happens with Google DAI ad-break slate/filler
+    // segments), sequence-mode placement collapses it onto the previous
+    // segment's position instead of its own - and every subsequent segment
+    // inherits that same error until a discontinuity boundary or gap-jump
+    // recovery. Detect the mismatch immediately after append and resync so
+    // only the *next* segment (not the whole rest of the group) needs a
+    // gap-jump. Runs in both debug and release builds, since the
+    // underlying bug is not debug-build-specific (unlike the goog.DEBUG
+    // diagnostic above, which only logs, never corrects).
+    if (reference && !reference.isPreload() && !isChunkedData) {
+      const bufferEndAfter = this.bufferEnd(contentType);
+      if (bufferEndAfter != null) {
+        const segmentDuration = reference.endTime - reference.startTime;
+        const offset = Math.abs(bufferEndAfter - reference.endTime);
+        if (segmentDuration > 0.100 && offset > segmentDuration / 2) {
+          await this.resync(contentType, reference.endTime);
+        }
+      }
+    }
   }

   /**

What did you expect to happen?

Slate ends and video / audio of content starts at the correct time and is in sync for duration of stream.

What actually happened?

On 5.x you will notice the player log to the console that it has detected a possible encoding error and then make a large jump (up to a full segment length, in this example 6.4sec) cutting the first segment of content after the ad period

On 4.x you will notice the player play the first frame of content after the ad period at the correct time however audio will not start until about 6.4 sec until after the content starts and video / audio will remain out of sync until a discontinuity is passed.

Are you planning to send a PR to add it?

Yes