Playout Control

Using different manifests

Sometimes devices cannot play certain types of video or audio, mostly because there is no support for a certain codec. However, you do want the codec for other devices, so packaging all the video in one server manifest may not the best approach.

It is possible to create more than one server manifest to cover a broad range of devices. When creating several server manifests you can select the video and audio supported for the targeted platform. Each manifest can be addressed as a separate URL, so you can present each device the content it can play.

Let's say we are targeting the following devices to deliver the same presentation, but with different technical specifications.

The targeted platforms for this example are:

Target platform Specifications
Smartphones (iPhone, Android, Windows Phone) Low-end devices supporting AVC Baseline profile and AAC-LC audio
Tablets (iPad, Android, Windows) Devices supporting AVC Main profile
Gaming consoles (Xbox, Playstation) Higher quality video and support for Dolby Digital audio
STB, HbbTV, SmartTV's Highest quality audio and video plus support for DTS audio

Now that we have defined our target platforms, we can start packaging for these platforms. Shown below are a couple of example commands selecting certain video or audio tracks for the targets.

These are the files we are going to use in our examples:

Files Video Audio
video01.ismv H.264 video 350 kbps Baseline profile AAC-LC audio 96kbps
video02.ismv H.264 video 700 kbps Baseline profile  
video03.ismv H.264 video 1000 kbps Baseline profile AAC-LC audio 128kbps
video04.ismv H.264 video 1500 kbps Main profile  
video05.ismv H.264 video 2000 kbps Main profile  
video06.ismv H.264 video 3000 kbps Main profile AAC-LC audio 256kbps
audio01.isma   Dolby Digital audio 384 kbps
audio02.isma   DTS Audio audio 384 kbps

Scenario 1 (Smartphones)

For this targeted platform we use the first three videos with a maximum bandwidth of 1000 kbps. These devices only support the AVC Baseline profile. For audio we pick the audio of the first video, and only select the video track from the third video file.

#!/bin/bash

mp4split -o smartphone.ism \
  video01.ismv \
  video02.ismv \
  video03.ismv --track_type=video

Scenario 2 (Tablets)

For this targeted platform we use the first four videos with a maximum bandwidth of 1500 kbps. For audio we pick the audio of the 3rd video, and only select the video track from the 1st video.

#!/bin/bash

mp4split -o tablet.ism \
  video01.ismv --track_type=video \
  video02.ismv \
  video03.ismv \
  video04.ismv

Scenario 3 (Gaming consoles)

For these targeted platforms we add all the video streams except for the video with the lowest bitrate. For audio we pick the audio of the third video, and only select the video track from the sixth video file. Also since there is support for Dolby Digital, we also add these to our manifest.

#!/bin/bash

mp4split -o console.ism \
  video02.ismv \
  video03.ismv \
  video04.ismv \
  video05.ismv \
  video06.ismv --track_type=video \
  audio01.isma

Scenario 4 (STBs, HbbTV, SmartTVs)

Similar to the third scenario, but we also add DTS audio to our manifest.

#!/bin/bash

mp4split -o stb.ism \
  video02.ismv \
  video03.ismv --track_type=video \
  video04.ismv \
  video05.ismv \
  video06.ismv \
  audio01.isma \
  audio02.isma

Using dynamic track selection

New in version 1.6.1.

Requesting a manifest/playlist by default returns all the available streams. It is the player that can make the best decision on which stream to play. Sometimes it is useful to limit the number of streams, or cap the minimum/maximum bitrates returned in the manifest file. E.g. for testing, specific devices, or to allow more users to watch the same video but at an average lower bitrate.

A generic way to filter the tracks is to write an expression. The expression is evaluated for each track specified in the server manifest file. Only when it evaluates to true the track is added to the returned manifest/playlist.

The expression is passed as a query parameter using the syntax filter=EXPRESSION.

Some examples:

Filer Expression Description
true The client manifest contains all the tracks.
type != "video" || systemBitrate < 400000 Include all the tracks, where the video tracks must be at least 400 Kbps.
systemLanguage == "eng" Only include audio/video/text tracks with the English language.
FourCC != "AVC1" || AVC_PROFILE == AVC_PROFILE_BASELINE Include all the tracks, where the video tracks must use the BASELINE profile.
(FourCC == "AACL" && SampleRate == 48000) || (FourCC == "AVC1" && AVC_LEVEL >= 31) Include all AAC audio tracks with a samplerate of 48KHz and all AVC video tracks with a minimum level of 3.1.

The following constants are available:

Constant variable name Value
AVC_PROFILE_BASELINE 66
AVC_PROFILE_MAIN 77
AVC_PROFILE_HIGH 100

The following track variables are available:

Track variable name Value
type The type of the track: "audio", "video", "textstream", "data"
FourCC The FourCC code of the track: AVC1, AACL, TTML, JPEG, dtse, ac-3, hvc1
trackID  
AudioTag The format tag of the audio
Channels The number of audio channels
MaxWidth The coded width (in pixels)
trackName  
MaxHeight The coded height (in pixels)
TimeScale The timescale
avc_level The AVC level
avc_profile The AVC profile (AVC_PROFILE_BASELINE, AVC_PROFILE_MAIN, AVC_PROFILE_HIGH)
DisplayWidth The display width (in pixels)
SamplingRate Audio sampling rate (in Hz)
BitsPerSample The resolution of the audio samples
DisplayHeight The display height (in pixels)
systemBitrate  
systemLanguage  

Dynamic Track selection example

In this Tears of Steel Manifest example we have two audio (64 and 128 Kbps) and five video (405, 814, 1209, 1989 and 2997 Kbps) tracks.

Here are some example scenarios (and demo URLs) to show you what you can do with the Dynamic Track Selection feature.

Scenario 1: Select the lowest audio track and the lowest two video tracks.

?filter=(type=="audio"&&systemBitrate<100000)||(type=="video"&&systemBitrate<1024000)

Scenario 2: Select the highest audio track and the two highest video tracks.

?filter=(type=="audio"&&systemBitrate>100000)||(type=="video"&&systemBitrate>1300000)

Attention

The filter expression is passed as a query parameter, e.g. ?filter=, and must be properly escaped (preferably using a standard function from your favourite framework). Note that when testing the URL with a command-line tool like cURL you have to take into account the escaping rules for your command-line as well.

So for curl the '&' needs to be escaped in the bash shell:

#!/bin/bash

curl -v 'http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/Manifest?filter=(type=="audio"%26%26systemBitrate<100000)||(type=="video"%26%26systemBitrate<1024000)'

Limiting bandwidth use example

The dynamic track selection feature can be used to cap the minimum/maximum bitrates returned in the manifest file as well.

Include all the video tracks with a bitrate higher than 800Kbps:

#!/bin/bash

curl -v 'http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/Manifest?filter=(systemBitrate>800000)'

Similarly, include all the tracks where the bitrate is smaller than 800Kbps:

#!/bin/bash

curl -v 'http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/Manifest?filter=(systemBitrate<800000)'

Dynamic Track selection capture example

Sometimes not all bitrates should be captured, but a selection. The following examples outline how to do that, combined with a start-end time.

#!/bin/bash

curl -v 'http://live.example.com/channel01/channel01.isml/manifest?filter=((trackName=="video"%26%26systemBitrate==800000)||(trackName=="audio_1"%26%26systemBitrate==64000))&t=2015-12-08T15:20:00-2015-12-08T15:21:00'

This captures only the 800k video bitrate (and the 64k audio bitrate).

The timeline in the manifest is this:

<c t="14495879990000000" d="36000000" r="17" />

which is about a minute (17 * 3.6 = 61.2, nearest gop) and 14495879990000000 is the start time, Tue, 08 Dec 2015 15:19:59 GMT.

The following example shows how to capture a single audio and video bitrate from a certain point in time until the end (the current time):

#!/bin/bash

curl -v 'http://live.example.com/channel01/channel01.isml/manifest?filter=((trackName=="video"%26%26systemBitrate==800000)||(trackName=="audio_1"%26%26systemBitrate==64000))&t=2015-12-08T15:20:00'

The timeline in the manifest is this:

<c t="14495879990000000" d="36000000" r="141" />

which reads as from the time indicated until the end.

The url used for the curl command in above examples can be used for capture as well:

#!/bin/bash

unified_capture -o test.ismv 'http://live.example.com/channel01/channel01.isml/manifest?filter=((trackName=="video"%26%26systemBitrate==800000)||(trackName=="audio_1"%26%26systemBitrate==64000))&t=2015-12-08T15:20:00'

Using Rewrite Map for manifest aliases

It is possible to create dynamic manifests per device using filters and a Rewrite Map with key:value pairs.

By creating filters with it is possible to easily create profiles for devices without having to prepare different manifests.

This has a number of benifits:

  1. Have one base manifest for all content used between devices
  2. Share video content urls between manifests for caching in CDN's
  3. Flexible filter rules for different types of content

In the Apache configuration file:

RewriteEngine on
RewriteMap profiles "txt:/path/to/profiles.txt"

# match an alias and append profile filter
RewriteRule "(.*\.ism/)(.*)(\.m3u8|\.mpd|\.f4m)$" "$1$2$3${profiles:$2}" [PT]
RewriteRule "(.*\.ism/)(.*)\.(Manifest)$" "$1$3${profiles:$2}" [PT]

With the rewrite rule the name of a manifest is extracted and passed to the profiles RewriteMap.

If there is a match a filter is appended, denoted by ${profiles:$2}.

A passthrough [PT] flag is added for correct rewrite handling in mod_smooth_streaming.

In the profiles.txt:

mobile ?filter=(trackName=="aud1"||trackName=="vid1)&hls_client_manifest_version=2

tablet ?filter=(trackName=="aud2"||trackName=="vid2"||trackName=="vid3")

tv ?filter=(type=="audio")||(type=="video"%26%26systemBitrate>500000)

desktop ?filter=(type=="audio"||systemBitrate==1228000||systemBitrate==813000)

Here tracks are chosen based on name, bitrates, type, for a full list of available filters see: Using dynamic track selection. Note that %26%26 is && properly escaped for the URL request.

For a working example the following HLS playlist request is used:

http://example.com/manifest.ism/tablet.m3u8

This is internally written to:

http://example.com/manifest.ism/tablet.m3u8?filter=(trackName=="aud2"||trackName=="vid2"||trackName=="vid3")

Where an appropriate manifest with only track names of aud2, vid2, and vid3 are returned.

For smooth streaming, a special rewrite rule is made due to the naming convention.

http://example.com/manifest.ism/tablet.Manifest

This is internally written to:

http://example.com/manifest.ism/Manifest?filter=(trackName=="aud2"||trackName=="vid2"||trackName=="vid3")

HTTP Dynamic Streaming (HDS) 'stream level' manifests

In HDS the configuration information about multi-bitrate streams has been divided into multiple levels: set-level F4M and stream-level F4M files. For HDS, the files are F4M manifest files.

--hds.multi_level

Set-level F4M files contain the URL to the stream-level manifest file and the bitrate information for each stream in a multi-bitrate set. For HDS, the set-level F4M file can also contain information about a DVR rolling window.

Name Description
Set-Level Manifest A manifest file that groups individual stream-level manifest files
Stream-Level Manifest A manifest file describing a single <media> instance
#!/bin/bash

mp4split --hds.multi_level \
  --hds.client_manifest_version=2

Note

For HLS, there is a similar feature, called HTTP Live Streaming (HLS) variant sets.

Selecting HTTP Live Streaming (HLS) tracks (at startup)

For HLS you can specify a certain initial bitrate that suits your client device best. With this option you can set the bitrate the client starts to play instead of leaving the intial selection to the client. This feature can be used when you want to skip the lowest bandwidth at start, and is only for starting playback purposes. Not for filtering out bandwidth during complete playback.

URL to the media presentation Description
http://localhost/video/video.ism/video.m3u8?start_index=0 The client manifest start at the first indexed bitrate.
http://localhost/video/video.ism/video.m3u8?start_index=4 The client manifest starts at the fifth indexed bitrate.

The above can also be used in combination with Using dynamic track selection mentioned before to filter tracks before setting the start index for the client.

#!/bin/bash

curl -v 'http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/.m3u8?filter=(systemBitrate<1200000)&start_index=1'

In this example the two lowest video tracks are selected and the highest of them is set to be the initial bitrate.

HTTP Live Streaming (HLS) variant sets

New in version 1.7.15.

By default all the variants are listed (in no particular order) and it is for the player to decide which variant to play.

You can however select the variants that are generated. This is useful when you want to force listing a specific variant at the top of the master playlist. Creating variant sets with mp4split is done using the following option:

--variant_set

The expression is in the same format used for Using dynamic track selection.

HLS variant sets: Selecting the default video variant

In this case we want to list the 1.2MBit variant at the top. This is the first set. The second set is to list all the other variants.

--variant_set='systemBitrate>=1200000 || type!="video"'
--variant_set='systemBitrate<1200000 || type!="video"'

Please download the hls-variant-sets.sh sample script which creates the server manifest as discussed above. The sample content is Tears of Steel.

IIS passthrough

Sometimes, pre-encoded VOD content is not encoded correctly. This can happen especially with older versions of Expression Encoder or Transform Manager.

When using IIS this might go unnoticed becasue IIS doesn't look at the actual content it's serving, it simply extracts a byte range and sends it along to the client. USP actually parses the sample data (for various reasons, for example converting to other formats, applying different DRM schemes).

When the content cannot be re-encoded, which is the preferred way to handle this problem as it is the content that is at fault, ISS passthrough mode may be used.

This will allow USP to still serve the content to VOD - as IIS does. However, as the content is no longer parsed transmuxing capabilities are lost: only ISS will play as it is 'passed through' - there is no transmuxing to HLS, HDS or DASH.

Apache

For Apache the directive looks like the following:

<Directory "/var/www/example">
  UspIssPassThrough On
  UspPreferStatic On
</Directory>

Please note that the UspPreferStatic is added as well. This is needed to ensure client manifests are not generated by USP (by reading the content), but instead are read from disk as well.

Both UspIssPassThrough and UspPreferStatic are used in the Directory context within the VirtualHost. Multiple directories can be used in a single virtual host, so this option can be used on a per directory basis.

Nginx

For Nginx the directive looks like the following:

location ^~ /example/ {
  root /var/www/usp_test;
  usp_iss_pass_through;
  usp_prefer_static;
  usp_handle_ism;
}

Both usp_iss_pass_through and usp_prefer_static are used in the location context. Multiple locations can be used a single virtual host, so this option can be used on a per location basis.