Adding trick play to a DASH or HLS stream

Trick play, or trick mode as it is sometimes called, is a feature that gives viewers visual feedback while they are rewinding or fast-forwarding a stream. Adding this feature to your streams can be done in different ways:

  • For DASH, a sync-samples only track can be added to enable trick play support (reference: [1]). Alternatively, a track with tiled thumbnails in JPEG format can be added (reference: [2]).
  • For HLS, an I-frame Playlist is required for trick play. Such a playlist can reference either the original video track or a sync-samples only track.
    • For HLS sync-samples only tracks are supported for fMP4 HLS, but not for HLS with Transport Streams (HLS TS).
    • Using a sync-samples only track can make trick play’s visual feedback more smooth, as it may offer a higher density of frames compared to an I-frame Playlist that merely references the original video track.

Note

Working with sync-samples only tracks, or tiled thumbnails tracks, is currently only supported for static packaging. Adding such files to a Origin VOD workflow has no use at the moment.

How to package a sync-samples only MP4

–trickplay

You can package a CMAF compliant MP4 that contains only sync-samples with proper signaling using mp4split with the --trickplay option:

mp4split --trickplay -o tos-1000k-sync-samples.cmfv \
  tears-of-steel-avc1-1000k.cmfv

Note

The --trickplay option is only supported for CMAF output, which is specified using the .cmfv file extension and can only contain one track per file.

How to package a tiled thumbnails MP4

–trickplay –fourcc=jpeg

You can package a CMAF compliant MP4 that contains tiled thumbnails encoded in JPEG format with proper signaling using mp4split with the --trickplay and --fourcc=jpeg options:

mp4split --trickplay --fourcc=jpeg -o tos-1000k-tiled-thumbnails.cmfv \
  tears-of-steel-avc1-1000k.cmfv

The tiled thumbnails are generated by decoding a frame from the source video track every 5 seconds, tiling the frames into a 3 by 5 grid, and encoding the result as a JPEG picture. These pictures are stored in the CMAF output file, in a track with special signaling for DASH packaging.

Important

The --trickplay --fourcc=jpeg option combination:

  • Requires Intel Media SDK installation
  • Is only supported for CMAF output, which can only contain one track per file and is specified using the .cmfv file extension

How to add trick play to DASH

For DASH, trick play can added with a sync-samples only track, or with a tiled thumbnails track. The first can be packaged using the –trickplay option, while the second can be packaged using –trickplay –fourcc=jpeg options. How you can use these tracks to add trick play to your DASH stream is explained below.

With a sync-samples only track

After you have created a sync-samples only file (using –trickplay), you can simply add that file to the other media files that should become part of your stream and, based on these files, create a MPD:

mp4split --package_mpd -o tos-with-trickplay.mpd
  tears-of-steel-avc1-1000k.cmfv
  tos-1000k-sync-samples.cmfv
  tears-of-steel-aac-128k.cmfa

With a tiled thumbnails track

After you have created a CMAF compliant MP4 that contains tiled thumbnails (using –trickplay –fourcc=jpeg), you can add that file to the other media files that should become part of your stream and, based on these files, create a MPD:

mp4split --package_mpd -o tos-with-tiled-thumbnails.mpd
  tears-of-steel-avc1-1000k.cmfv
  tos-1000k-tiled-thumbnails.cmfv
  tears-of-steel-aac-128k.cmfa

Note

It is currently not possible to add trick play with tiled thumbnails to HLS playlists, as there is no defined signaling for it yet.

How to add trick play to fMP4 HLS

The I-frame Playlists necessary for adding trick play to HLS need to be generated manually for each video track when streaming fMP4 HLS. Depending on whether or not you are working with a dedicated sync-samples only track, this process is slightly different.

Note

Currently, trick play for fMP4 HLS is only supported for static packaging.

Without a sync-samples only track

Use the –create_iframe_playlist option to create an I-frame Playlist that references the original video track:

mp4split -o iframe.m3u8 --create_iframe_playlist \
  tears-of-steel-avc1-1000k.cmfv
mp4split -o video.m3u8 --fragment_duration=6000 \
  tears-of-steel-avc1-1000k.cmfv
mp4split -o audio.m3u8 --fragment_duration=6000 \
  tears-of-steel-aac-128k.cmfa

mp4split -o master_playlist.m3u8 \
  iframe.m3u8 \
  video.m3u8 \
  audio.m3u8

With a sync-samples only track

When a track contains only sync-samples, mp4split automatically detects this and will create an I-frame Playlist for it, instead of a regular Media Playlist. Using the –create_iframe_playlist option is not necessary.

mp4split -o iframe-sync-samples.m3u8 \
  tos-1000k-sync-samples.cmfv
mp4split -o video.m3u8 \
  tears-of-steel-avc1-1000k.cmfv
mp4split -o audio.m3u8 \
  tears-of-steel-aac-128k.cmfa

mp4split -o master_playlist.m3u8 \
  iframe-sync-samples.m3u8 \
  video.m3u8 \
  audio.m3u8

Note

When using a sync-samples only file to add trick play to HLS, a common error reported by Apple’s MediaStreamValidator is the following must fix issue:

Error: Framerate change without discontinuity tag detected

This problem occurs when the source of sync-samples only file has variable GOP size (which is reflected in the sync-samples only file). To solve it, make sure that your source has a constant GOP size.

How to add trick play to HLS TS

The I-frame Playlists necessary for adding trick play to HLS are automatically generated for each video track in a HLS TS stream. This is true for static as well as dynamic packaging. In the case of the latter, you must specify a client manifest version of 4 or higher in order for Origin to add the necessary I-frame Playlists, using –hls.client_manifest_version.

Footnote

[1]See ‘Trick Mode support’ in the DASH-IF Interoperability Guidelines 4.2
[2]See ‘Tiles of thumbnail images’ in the DASH-IF Interoperability Guidelines 4.2