Advanced LIVE Streaming Options

Redundancy

Using two encoders will ensure that the ingest will continue even if one of the encoders becomes unavailable. For this to work the encoders should be time-aligned, so they insert the same timestamp into the chunks posted to the ingest. The chunk that arrives first is ingested, the second one discarded. As both chunks have the same timestamp, one will always be ingested: the timeline continues.

This can be combined with a dual origin setup where the encoders cross-post: a 2x2 setup. Both origin and encoder will have max resilience:

 origin1   origin2
    |   \ /   |
    |   / \   |
encoder1  encoder2

Dual ingest setup (Failover)

When creating a duplicate origin setup it is important that the options for the publishing points are identical. E.g. you might use the following options:

--archiving=1
--archive_length=3600
--archive_segment_length=60
--dvr_window_length=600
--hls.minimum_fragment_length=9
--iss.minimum_fragment_length=3
--restart_on_encoder_reconnect

That is we turn archiving on, store an archive of one hour, write one minute segments to disk and use a 10 minute DVR window. The fragment durations are chosen so that the HSS fragments are the same as the (fixed) GOP used by the Encoder, and the HLS fragments are exactly a concatenation of 3 GOPs.

Spinning up a second origin (Failover)

First we create a publishing point on the second origin:

#!/bin/bash

MP4SPLIT_OPTIONS=your-options

mp4split -o http://live2.example.com/channel1/channel1.isml \
  ${MP4SPLIT_OPTIONS}

Now that the publishing point is created we notify the encoder to also start pushing to the second origin. At this point the origin is ingesting the live stream, but it is still missing the archived content from the first origin.

Synchronizing the archive from the first origin with the second

This is straightforward. Since the origin handles duplicate fragments (by simply ignoring any fragments that are already ingested) and the archive on disk is already in a format that the origin can ingest, we can simply POST the archived files from the first origin to the second origin. For example with cURL:

#!/bin/bash

curl --data-binary @/var/www/chan1/chan1-33.ismv \
  -X POST "http://origin2/chan1.isml/Streams(chan1)"

curl --data-binary @/var/www/chan1/chan1-34.ismv \
  -X POST "http://origin2/chan1.isml/Streams(chan1)"

curl --data-binary @/var/www/chan1/chan1-35.ismv \
  -X POST "http://origin2/chan1.isml/Streams(chan1)"

Note that the last archived fragment on the second origin may not be identical to the .ismv file stored on the first origin. This may happen when the encoder is restarted at a time which is not a multiple of the archive_segment_length and is caused by the fact that the fragments are ingested out-of-order.

This makes no difference for the functionality of the two origins, but it is something to be aware of that the resulting .ismv and .db3 files aren't binary identical. If possible, it is recommended that the encoder (re)restarts at the beginning of an archive_segment_length.

Coordinated Universal Time (UTC)

New in version 1.6.6.

For Live ingest the preferred setup is to configure the encoder to use (wall-clock) time in UTC format.

This is an informative mapping of the ingested media to a known clock (e.g. the same clock that is used by your EPG (Electronic Program Guide)).

For options like 'restart-on-encoder-reconnect' (see Options for LIVE ingest), Virtual subclips or Capturing LIVE the encoder must be set to use (wallclock) Coordinated Universal Time (UTC).

Without it the encoder most likely starts with time 0, which will make it difficult to use options like Virtual subclips and will make --restart_on_encoder_reconnect impossible.

See Encoder settings on how to set (wallclock) Coordinated Universal Time (UTC).

Alignment of sequence numbers from UTC

New in version 1.6.0.

Sequence numbers are now derived from the (UTC) timestamps of the fragments. Deriving them this way makes it possible that two origin servers use the same sequence numbers when generating e.g. M3U8 playlists.

The fragment_index is calculated as floor(timestamp of fragment (in seconds) / fragment_duration (in seconds))

Attention

Care must be taken when selecting the value of the different fragment_durations to avoid introducing discontinuities in the sequence numbers. The preferred value is a multiple of the fixed GOP duration. You should set --iss.minimum_fragment_length and --hls.minimum_fragment_length to at least the duration of the GOP size.

Here is an overview of which GOP size and 'minimum_fragment_length' options to use depending on the FPS (Frames Per Second):

FPS GOP size iss/hds hls
23.976 48 4000 / 1001 8000 / 1001
24 48 4 8
25 50 4 8
29.976 60 4000 / 1001 8000 / 1001
30 60 4 8

Some combinations of FPS, GOP size and AAC samplerate have the special property that the audio and video durations match exactly.

The following tables is for AAC-LC audio with a samplerate of 48 KHz:

FPS GOP size iss/hds hls
24 64 8 / 3 16 / 3
25 48 24 / 25 48 / 25

Ingest F4M

The webserver module also ingests live F4M streams. This is the playlist format used by HTTP Dynamic Streaming. The webserver module uses the F4M playlists (and bootstrap and fragments) as its source format and makes the live presentation available in the different supported formats (HSS, HLS, DASH).

Create a server manifest file with the URL to the F4M stream as input:

#!/bin/bash

mp4split -o http://live.example.com/f4m/live.isml \
  http://live.example.com/loop/loop.isml/loop.f4m

MP4Split fetches the F4M manifest and extracts all the information necessary to create the server manifest file.

By default the DVR window settings are taken from the bootstrap. You can adjust the DVR window by specifying the following server manifest options:

--f4m_dvr_offset_begin

The number of fragments to skip from the beginning of the DVR window. (Defaults to 0)

--f4m_dvr_offset_end

The number of fragments to skip before the end of the DVR window. (Defaults to 0)

For example, say you are using a rolling DVR window and the fragments older than the DVR window are being purged. In that case you may want to set the f4m_dvr_offset_begin to an initial value of 2. This make sure that client manifests generated reference only fragments/segments that are still available from the F4M source.

Normally there is no need to adjust the ending of the DVR offset, but some players may be requesting new fragments quite aggressively, while other players may need additional information stored in a fragment about subsequent fragments. The latter is e.g. the case for HTTP Smooth Streaming and you may set the f4m_dvr_offset_end to 2 for some additional headroom.

Example command line:

#!/bin/bash

mp4split -o http://live.example.com/f4m/live.isml \
  --f4m_dvr_offset_begin=2 \
  --f4m_dvr_offset_end=2 \
  http://live.example.com/loop/loop.isml/loop.f4m