**Separate HLS audio attached externally progresses differently from the main HLS video**

[collinscoder1]

I’m seeing a sync issue when using separate HLS video and HLS audio playlists instead of a single HLS master.

Setup

The source shape is:

• one video HLS media playlist
• one audio HLS media playlist
• subtitles may also be present externally, but they are not the source of the issue

The audio is attached separately rather than coming from a shared HLS master.

Expected behavior

I expect:

• audio and video to stay on the same playback timeline
• buffering/seeking to keep them tightly synchronized
• external HLS audio to behave like a normal selectable audio source, not like a separate playback path

Actual behavior

What happens in practice is:

• the external HLS audio does load and play
• but it behaves like it is on another playback graph / another clock
• during buffering/seeking, the audio progresses differently from the main video
• the result is repeatable desynchronization, not just a one-off drift

The important observation is that this is not simply “audio failed to load”.
It loads and plays, but it does not behave like one tightly shared A/V graph.

What I tried

I tried several approaches before filing this issue:

• attached separate HLS audio with setMedia(url, MediaType.audio) instead of relying on a shared HLS master
• switched audio through the normal MDK/FVP track-selection flow where possible
• tried Dart-side onSync clocking to keep playback anchored during buffering/seeking
• reverted that and tried native-side onSync clocking in the FVP C++ layer
• verified that onSync is only useful for video timing and does not solve the audio side
• tried probing audio timing through onAudio / audio frame callbacks, but that path was not usable in this setup
• logged video timing/status and compared behavior during buffering/seeking
• tried FFmpeg/MDK property tuning, including:
  • audio.avfilter=aresample=async=1:first_pts=0
  • stronger aresample variants
  • avformat.fflags combinations such as +genpts+discardcorrupt
• tested disabling/deselecting audio to isolate whether the issue was coming from the audio pipeline
• compared behavior with external subtitles; subtitles were not the source of the desync
• built a synthetic local HLS master that combines:
  • one video playlist
  • one selected audio playlist
  • subtitles kept external

In other words:

• combined/master-based HLS playback works
• separate external HLS audio progresses differently from the main video

Conclusion

The problem appears to be:

• external split HLS audio is not equivalent to embedded/master-managed audio

When video and audio are normalized into a single HLS master, playback behaves correctly.
When audio is attached separately, it can play, but it does not remain tightly synchronized with the main video during buffering/seeking.

If this is expected behavior, it would be useful to document that separate external HLS audio is a lower-sync-capability path than a shared/master-based HLS source.

If it is not expected behavior, then this looks like a real sync/runtime issue in the external split-audio path.

[wang-bin]

setMedia(url, MediaType.audio) was designed for local files. the clock should be correct, but audio and video are buffered and demuxed seperately like 2 players.

only 1 demuxer is used for an hls with multiple tracks.

[collinscoder1]

setMedia(url, MediaType.audio) was designed for local files. the clock should be correct, but audio and video are buffered and demuxed seperately like 2 players.

only 1 demuxer is used for an hls with multiple tracks.

Thanks, that explains the behavior clearly.

The remaining point on my side is broader than HLS specifically.

From your explanation, setMedia(url, MediaType.audio) creates an external audio path that is separate from the main playback graph. That helps explain the observed sync behavior.

The other limitation is the API surface after attach.

Once an external audio track is attached, there does not seem to be much public API to manage that track as a first-class playback component. In practice, what seems to be missing is:

• a post-attach control model for external audio tracks
• a way to influence or recover sync behavior after attach
• runtime hooks to inspect or manage the attached audio track more directly
• behavior equivalent to integrated multi-track playback where audio is part of the same main graph

So the main follow-up point is not only about sync quality, but also about the lack of runtime control after an external audio track has been attached.

[wang-bin]

what api do you need? an external track is expected to be the same as an integrated track, you can attach and detach, and use the same api as a single video.