Playout Control

Using different manifests

Sometimes devices cannot play certain types of video or audio, mostly because there is no support for a certain codec. However, you do want the codec for other devices, so packaging all the video in one server manifest may not the best approach.

It is possible to create more than one server manifest to cover a broad range of devices. When creating several server manifests you can select the video and audio supported for the targeted platform. Each manifest can be addressed as a separate URL, so you can present each device the content it can play.

Let's say we are targeting the following devices to deliver the same presentation, but with different technical specifications.

The targeted platforms for this example are:

Target platform Specifications
Smartphones (iPhone, Android, Windows Phone) Low-end devices supporting AVC Baseline profile and AAC-LC audio
Tablets (iPad, Android, Windows) Devices supporting AVC Main profile
Gaming consoles (Xbox, Playstation) Higher quality video and support for Dolby Digital audio
STB, HbbTV, SmartTV's Highest quality audio and video plus support for DTS audio

Now that we have defined our target platforms, we can start packaging for these platforms. Shown below are a couple of example commands selecting certain video or audio tracks for the targets.

These are the files we are going to use in our examples:

Files Video Audio
video01.ismv H.264 video 350 kbps Baseline profile AAC-LC audio 96kbps
video02.ismv H.264 video 700 kbps Baseline profile  
video03.ismv H.264 video 1000 kbps Baseline profile AAC-LC audio 128kbps
video04.ismv H.264 video 1500 kbps Main profile  
video05.ismv H.264 video 2000 kbps Main profile  
video06.ismv H.264 video 3000 kbps Main profile AAC-LC audio 256kbps
audio01.isma   Dolby Digital audio 384 kbps
audio02.isma   DTS Audio audio 384 kbps

Scenario 1 (Smartphones)

For this targeted platform we use the first three videos with a maximum bandwidth of 1000 kbps. These devices only support the AVC Baseline profile. For audio we pick the audio of the first video, and only select the video track from the third video file.

#!/bin/bash

mp4split -o smartphone.ism \
  video01.ismv \
  video02.ismv \
  video03.ismv --track_type=video

Scenario 2 (Tablets)

For this targeted platform we use the first four videos with a maximum bandwidth of 1500 kbps. For audio we pick the audio of the 3rd video, and only select the video track from the 1st video.

#!/bin/bash

mp4split -o tablet.ism \
  video01.ismv --track_type=video \
  video02.ismv \
  video03.ismv \
  video04.ismv

Scenario 3 (Gaming consoles)

For these targeted platforms we add all the video streams except for the video with the lowest bitrate. For audio we pick the audio of the third video, and only select the video track from the sixth video file. Also since there is support for Dolby Digital, we also add these to our manifest.

#!/bin/bash

mp4split -o console.ism \
  video02.ismv \
  video03.ismv \
  video04.ismv \
  video05.ismv \
  video06.ismv --track_type=video \
  audio01.isma

Scenario 4 (STBs, HbbTV, SmartTVs)

Similar to the third scenario, but we also add DTS audio to our manifest.

#!/bin/bash

mp4split -o stb.ism \
  video02.ismv \
  video03.ismv --track_type=video \
  video04.ismv \
  video05.ismv \
  video06.ismv \
  audio01.isma \
  audio02.isma

Using dynamic track selection

New in version 1.6.1.

Requesting a manifest/playlist by default returns all the available streams. It is the player that can make the best decision on which stream to play. Sometimes it is useful to limit the number of streams, or cap the minimum/maximum bitrates returned in the manifest file. E.g. for testing, specific devices, or to allow more users to watch the same video but at an average lower bitrate.

A generic way to filter the tracks is to write an expression. The expression is evaluated for each track specified in the server manifest file. Only when it evaluates to true the track is added to the returned manifest/playlist.

The expression is passed as a query parameter using the syntax filter=EXPRESSION.

Some examples:

Filer Expression Description
true The client manifest contains all the tracks.
type != "video" || systemBitrate < 400000 Include all the tracks, where the video tracks must be at least 400 Kbps.
systemLanguage == "eng" Only include audio/video/text tracks with the English language.
FourCC != "AVC1" || AVC_PROFILE == AVC_PROFILE_BASELINE Include all the tracks, where the video tracks must use the BASELINE profile.
(FourCC == "AACL" && SampleRate == 48000) || (FourCC == "AVC1" && AVC_LEVEL >= 31) Include all AAC audio tracks with a samplerate of 48KHz and all AVC video tracks with a minimum level of 3.1.

The following constants are available:

Constant variable name Value
AVC_PROFILE_BASELINE 66
AVC_PROFILE_MAIN 77
AVC_PROFILE_HIGH 100

The following track variables are available:

Track variable name Value
type The type of the track: "audio", "video", "textstream", "data"
FourCC The FourCC code of the track: AVC1, AACL, TTML, JPEG, dtse, ac-3, hvc1
trackID  
AudioTag The format tag of the audio
Channels The number of audio channels
MaxWidth The coded width (in pixels)
trackName  
MaxHeight The coded height (in pixels)
TimeScale The timescale
avc_level The AVC level
avc_profile The AVC profile (AVC_PROFILE_BASELINE, AVC_PROFILE_MAIN, AVC_PROFILE_HIGH)
DisplayWidth The display width (in pixels)
SamplingRate Audio sampling rate (in Hz)
BitsPerSample The resolution of the audio samples
DisplayHeight The display height (in pixels)
systemBitrate  
systemLanguage  

Dynamic Track selection example

In this Tears of Steel Manifest example we have two audio (64 and 128 Kbps) and five video (405, 814, 1209, 1989 and 2997 Kbps) tracks.

Here are some example scenarios (and demo URLs) to show you what you can do with the Dynamic Track Selection feature.

Scenario 1: Select the lowest audio track and the lowest two video tracks.

?filter=(type=="audio"&&systemBitrate<100000)||(type=="video"&&systemBitrate<1024000)

Scenario 2: Select the highest audio track and the two highest video tracks.

?filter=(type=="audio"&&systemBitrate>100000)||(type=="video"&&systemBitrate>1300000)

Attention

The filter expression is passed as a query parameter, e.g. ?filter=, and must be properly escaped (preferably using a standard function from your favourite framework). Note that when testing the URL with a command-line tool like cURL you have to take into account the escaping rules for your command-line as well.

So for curl the '&' needs to be escaped in the bash shell:

#!/bin/bash

curl -v 'http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/Manifest?filter=(type=="audio"%26%26systemBitrate<100000)||(type=="video"%26%26systemBitrate<1024000)'

Limiting bandwidth use example

The dynamic track selection feature can be used to cap the minimum/maximum bitrates returned in the manifest file as well.

Include all the video tracks with a bitrate higher than 800Kbps:

#!/bin/bash

curl -v 'http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/Manifest?filter=(systemBitrate>800000)'

Similarly, include all the tracks where the bitrate is smaller than 800Kbps:

#!/bin/bash

curl -v 'http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/Manifest?filter=(systemBitrate<800000)'

Dynamic Track selection capture example

Sometimes not all bitrates should be captured, but a selection. The following examples outline how to do that, combined with a start-end time.

#!/bin/bash

curl -v 'http://live.example.com/channel01/channel01.isml/manifest?filter=((trackName=="video"%26%26systemBitrate==800000)||(trackName=="audio_1"%26%26systemBitrate==64000))&t=2015-12-08T15:20:00-2015-12-08T15:21:00'

This captures only the 800k video bitrate (and the 64k audio bitrate).

The timeline in the manifest is this:

<c t="14495879990000000" d="36000000" r="17" />

which is about a minute (17 * 3.6 = 61.2, nearest gop) and 14495879990000000 is the start time, Tue, 08 Dec 2015 15:19:59 GMT.

The following example shows how to capture a single audio and video bitrate from a certain point in time until the end (the current time):

#!/bin/bash

curl -v 'http://live.example.com/channel01/channel01.isml/manifest?filter=((trackName=="video"%26%26systemBitrate==800000)||(trackName=="audio_1"%26%26systemBitrate==64000))&t=2015-12-08T15:20:00'

The timeline in the manifest is this:

<c t="14495879990000000" d="36000000" r="141" />

which reads as from the time indicated until the end.

The url used for the curl command in above examples can be used for capture as well:

#!/bin/bash

unified_capture -o test.ismv 'http://live.example.com/channel01/channel01.isml/manifest?filter=((trackName=="video"%26%26systemBitrate==800000)||(trackName=="audio_1"%26%26systemBitrate==64000))&t=2015-12-08T15:20:00'

Dynamic Track selection using count

New in version 1.7.27.

Sometimes tracks need to be selected by knowing information of other tracks in the playlist. This can help in cases when the underlying manifest may not be known in advance, or when using the same filters for different manifests.

For example, there are two different manifests, which have either:
  • Video, EC-3 224 Kbps, AAC 192Kbps, AAC 64Kbps
  • Video, AAC 192Kbps, AAC 64Kbps

Suppose that you would need to write one filter that can achieve the following: Take all video tracks and, if there is a EC-3 track, take that, otherwise the highest AAC bitrate track.

With the new count operator it is possible to achieve this:

filter=(type=="video"||fourcc=="EC-3"||(count(fourcc=="EC-3")==0 && systembitrate==192000))

The expression count(fourcc=="EC-3")==0 checks whether if we count the number of tracks with EC-3 is exactly zero then evaluate && systembitrate==192000, select one AAC audio track. If there are EC-3 tracks, the && systembitrate==192000 will not be evaluated, skipping the other AAC audio track.

Usage: count(expr)
  • count: returns an integer value for the number of tracks in the playlist that match
  • expr: the expression to match all the tracks with

Using Rewrite Map for manifest aliases

It is possible to create dynamic manifests per device using filters and a Rewrite Map with key:value pairs.

By creating filters with it is possible to easily create profiles for devices without having to prepare different manifests.

This has a number of benifits:

  1. Have one base manifest for all content used between devices
  2. Share video content urls between manifests for caching in CDN's
  3. Flexible filter rules for different types of content

In the Apache configuration file:

RewriteEngine on
RewriteMap profiles "txt:/path/to/profiles.txt"

# match an alias and append profile filter
RewriteRule "(.*\.ism/)(.*)(\.m3u8|\.mpd|\.f4m)$" "$1$2$3${profiles:$2}" [PT]
RewriteRule "(.*\.ism/)(.*)\.(Manifest)$" "$1$3${profiles:$2}" [PT]

With the rewrite rule the name of a manifest is extracted and passed to the profiles RewriteMap.

If there is a match a filter is appended, denoted by ${profiles:$2}.

A passthrough [PT] flag is added for correct rewrite handling in mod_smooth_streaming.

In the profiles.txt:

mobile ?filter=(trackName=="aud1"||trackName=="vid1)&hls_client_manifest_version=2

tablet ?filter=(trackName=="aud2"||trackName=="vid2"||trackName=="vid3")

tv ?filter=(type=="audio")||(type=="video"%26%26systemBitrate>500000)

desktop ?filter=(type=="audio"||systemBitrate==1228000||systemBitrate==813000)

Here tracks are chosen based on name, bitrates, type, for a full list of available filters see: Using dynamic track selection. Note that %26%26 is && properly escaped for the URL request.

For a working example the following HLS playlist request is used:

http://example.com/manifest.ism/tablet.m3u8

This is internally written to:

http://example.com/manifest.ism/tablet.m3u8?filter=(trackName=="aud2"||trackName=="vid2"||trackName=="vid3")

Where an appropriate manifest with only track names of aud2, vid2, and vid3 are returned.

For smooth streaming, a special rewrite rule is made due to the naming convention:

http://example.com/manifest.ism/tablet.Manifest

This is internally written to:

http://example.com/manifest.ism/Manifest?filter=(trackName=="aud2"||trackName=="vid2"||trackName=="vid3")

HTTP Dynamic Streaming (HDS) 'stream level' manifests

In HDS the configuration information about multi-bitrate streams has been divided into multiple levels: set-level F4M and stream-level F4M files. For HDS, the files are F4M manifest files.

--hds.multi_level

Set-level F4M files contain the URL to the stream-level manifest file and the bitrate information for each stream in a multi-bitrate set. For HDS, the set-level F4M file can also contain information about a DVR rolling window.

Name Description
Set-Level Manifest A manifest file that groups individual stream-level manifest files
Stream-Level Manifest A manifest file describing a single <media> instance
#!/bin/bash

mp4split --hds.multi_level \
  --hds.client_manifest_version=2

Note

For HLS, there is a similar feature, called --variant-set.

Configuration and ordering of HLS playlists

Apple's HTTP Live Streaming (HLS) uses media groups and variant streams to sort the media contents of an ABR stream so that a client will know between which combinations of content it should switch. Media groups and stream variants are defined in a stream's Master Playlist.

A media group is the most basic sorting mechanism defined by HLS. It is used to group together different representations of content in media of an identical type (such as video or audio) with similar properties (like average bandwith).

Variant streams represent the different varieties of a stream between which a client should switch for playout. Each variant is a unique combination of the stream's media, in which every type of media available in a stream is represented once:

  • As a group, if the particular type of media is sorted as such within the stream
  • As an individual track, if the track's media type is not sorted in a group(s)

Important

The variety of variant streams that is available in a stream is restricted by the way in which the different media in a stream are grouped.

How HLS groups and variants are defined

Definition of HLS media groups

The default behaviour is that tracks of the same media type are grouped together if their encoding is the same (i.e., if codecs are identical and, for audio and video, bitrate as well). This default configuration should suffice for almost all use cases.

In short, audio tracks that are similarly encoded but that represent different languages will be grouped, and this is true for subtitles in different languages as well, but video tracks that contain different bitrates will remain individual tracks.

Definition of HLS variant streams

By default, each variant will contain at least one representation of each media type that is represented in a stream. Thus, if a stream contains one group of subtitle tracks, this group will be part of each variant.

Other than that, the behaviour that defines the available variants is best understood starting from a stream that contains several audio groups and several video tracks (the logic is the same if the video tracks are also grouped, or if the audio tracks are not):

  1. Associate lowest bitrate audio group with lowest bitrate video track to generate a variant.
  2. Re-apply the first step, based on the remaining audio groups and video tracks.
  3. Continue this process until all audio groups and/or all video tracks are part of a variant stream.
  4. If either audio groups or video tracks remain, take the highest bitrate group/ track of the other media type and associate it with each of the remaining groups/tracks.

Please find in the table below find an example that shows the result of the above process. In this example, two audio groups are associated with five video tracks to create five variant streams. Notice that the audio group with the lowest bitrate is part of only one variant stream:

Audio tracks Audio groups Video tracks Variant streams
English 32kHz @ 64k AAC-64 360p @ 256k AAC-64 + 256k
Dutch 32kHz @ 64k 360p @ 512k AAC-192 + 512k
Spanish, 32kHz @ 64k 720p @ 1024k AAC-192 + 1024k
English, 48kHz @ 192k AAC-192 1080p @ 2048k AAC-192 + 2048k
Dutch, 48kHz @ 192k 1080p @ 4096k AAC-192 + 4096k
Spanish, 48kHz @ 192k

If text tracks are also part of the stream, an additional step will define the final set of variant streams. Depending on whether the text tracks are grouped or not, each individual track or each group of tracks will be associated with each of the of the variants that were the result of the process described above.

Select variant stream that HLS clients should start playout with

It is possible to specify an initial bitrate that suits your client device best. With this option you can set the bitrate that the client starts playback with instead of leaving the intial selection to the client. This feature can be used when you want to skip the lowest bandwidth at start, for example.

It only influences the variant selection at the start of playback and does not influence playback after that. That is, it does not filter out any of the variants or media tracks that are available within a stream.

URL to the media presentation Description
http://localhost/video/video.ism/video.m3u8?start_index=0 The client manifest start at the first indexed bitrate.
http://localhost/video/video.ism/video.m3u8?start_index=4 The client manifest starts at the fifth indexed bitrate.

If you would like to apply some kind of filter, you can use the above method in combination with Using dynamic track selection to actually filter out tracks before setting the start index for the client. This is done like so:

#!/bin/bash

curl -v 'http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/.m3u8?filter=(systemBitrate<1200000)&start_index=1'

In this example the two lowest video tracks are selected and the highest of them is set to be the initial bitrate.

Configure HLS variant streams

--variant-set

New in version 1.7.15.

Note

Before using this option, please read through Configuration and ordering of HLS playlists.

By default all the variants are listed (in no particular order) and it is for the player to decide which variant to play (although Apple does define the first variant that is listed as the 'default variant').

When the default behaviour does not give you the result you want, you can define a subset of tracks based on which the variants will be generated. If a track in such a subset is part of a group, the group will represent the track in the subset. This is true even if some tracks in the group were not part of the subset that was defined.

The syntax of the expression that defines a subset is the same as for Using dynamic track selection. You can add the expression using the variant-set-option when generating a server or HLS client manifest with mp4split.

It is possbile to use the --variant-set-option more than once, to define several subsets. When doing so, the order in which you add the expressions will define the order of the variants in the Master Playlist. This makes the --variant-set-option useful for when you want to list a specific variant at the top of the Master Playlist.

If you only want to use the option to list a certain variant at the top of the Master Playlist, you should define two subsets that are jointly exhaustive, with the first subset containing a single track or group per media type so that the only variant that it will generate is the one that you want listed at the top.

For example, if we have a multi-bitrate stream like in the table above and we want to make sure that a variant stream with the 1024k video track is listed at the top, a subset that will generate this variant should be defined by the expression of the first --variant-set-argument in your mp4split command-line. The second subset that is defined should then cover all the other variants.

--variant_set='(systemBitrate=1024000 && type="video") || (systemBitrate=64000 && type="audio")'
--variant_set='systemBitrate!=1024000 || type!="video"'

Configure HLS media groups

New in version 1.7.2.

Note

Before using this option, please read through Configuration and ordering of HLS playlists.

It is possible to change the default behaviour and configure the HLS media groups yourself. Keep in mind that this will also influence the variant streams that are generated and that media groups are only supported in HLS client manifests of version 4 and onwards (i.e., specify --hls.client_manifest_version=4 or higher on your mp4split command-line when you want to work with media groups).

An example command-line that shows how the --track_groupid-option can be used, is shown below. Note that the grouping defined in this command-line is identical to the default behaviour:

#!/bin/bash

mp4split -o presentation.ism --hls.client_manifest_version=4
  audio_eng.isma --track_groupid=aac --track_description="English" \
  audio_deu.isma --track_groupid=aac --track_description="Deutsch" \
  commentary.isma --track_groupid=aac \
  --track_description="Commentary" --track_name="audio_commentary" \
  video_128k.ismv video_256k.ismv video_768k.ismv

Warning

We strongly advise against configuring HLS media groups yourself. Doing so can easily create a stream that does not follow Apple's recommendations and break playout.

Ordering HTTP Smooth Streaming (HSS), MPEG-DASH tracks

For Smooth Streaming and DASH, ordering the tracks in the manifest is not possible. Both specifications dictate that there should be no such ordering and that the players should make the appropriate choices. We follow the specification in that regard. However, if you want to make a certain selection of a server manifest's tracks available for play-out, you can make use of a filter, as described in Using dynamic track selection.

IIS passthrough

Sometimes, pre-encoded VOD content is not encoded correctly. This can happen especially with older versions of Expression Encoder or Transform Manager.

When using IIS this might go unnoticed becasue IIS doesn't look at the actual content it's serving, it simply extracts a byte range and sends it along to the client. USP actually parses the sample data (for various reasons, for example converting to other formats, applying different DRM schemes).

When the content cannot be re-encoded, which is the preferred way to handle this problem as it is the content that is at fault, ISS passthrough mode may be used.

This will allow USP to still serve the content to VOD - as IIS does. However, as the content is no longer parsed transmuxing capabilities are lost: only ISS will play as it is 'passed through' - there is no transmuxing to HLS, HDS or DASH.

Apache

For Apache the directive looks like the following:

<Directory "/var/www/example">
  UspIssPassThrough On
  UspPreferStatic On
</Directory>

Please note that the UspPreferStatic is added as well. This is needed to ensure client manifests are not generated by USP (by reading the content), but instead are read from disk as well.

Both UspIssPassThrough and UspPreferStatic are used in the Directory context within the VirtualHost. Multiple directories can be used in a single virtual host, so this option can be used on a per directory basis.

Nginx

For Nginx the directive looks like the following:

location ^~ /example/ {
  root /var/www/usp_test;
  usp_iss_pass_through;
  usp_prefer_static;
  usp_handle_ism;
}

Both usp_iss_pass_through and usp_prefer_static are used in the location context. Multiple locations can be used a single virtual host, so this option can be used on a per location basis.