Unified Origin - Recommendations for Live¶
Table of Contents
- Req.: Use of Smooth Streaming or CMAF ingest
- Req.: Use fast, local storage (remote storage is not supported for Live)
- Req.: Set up a publishing point (including configuration)
- Rec.: Add Origin and encoder redundancy
In Live workflows Unified Origin is placed between one or more Live encoders and a Content Delivery Network (CDN), with a shield cache between Origin and CDN being recommended to ensure robustness and increase performance:
player --> CDN --> Unified Origin <-- Live encoder
For VOD the use of a CDN is recommended, but for streaming Live at scale it is a must. Through efficient caching, dynamically packaged livestreams from Origin can be delivered to the largest of audiences. Typically, a minimum of 10 channels can be server from a single Origin. To achieve this you need to take the following into account:
- Configuring Live encoders to output streams compliant with Live ingest specifications supported by Origin
- Configuring Origin to adjust output streams to use case, adding DRM if needed
- Fine tuning Apache (on which Origin runs) to your environment and use case
- Adding redundancy and failover to support 24x7 streaming
- Configuring CDN to properly cache expiring live content
- Integrating Digital Rights Management (DRM) providers, if needed
- Integrating 3rd-party Dynamic Ad Insertion into the workflow, if needed
A Live encoder needs to POST (or PUT) an fMP4 packaged livestream over HTTP to a publishing point on Unified Origin so that Origin can ingest the livestream and dynamically package and deliver it based on incoming requests from clients.
There are two specifications that define how a Live encoder should do this:
- Microsoft Smooth Streaming ingest, as defined in Azure Media Services fragmented MP4 live ingest specification
- CMAF ingest, as defined in Interface 1 of the DASH-IF Live Media Ingest specification
Smooth Streaming ingest is supported by a lot of encoders, but isn't actively developed anymore and can therefore not be considered future proof. CMAF ingest is its spiritual successor, as it builds on the foundation of Smooth ingest with Microsoft having contributed significantly to the development of this newer specification, alongside a wide variety of industry partners. CMAF ingest fully aligns with the CMAF specification, hence its name. For these reasons, CMAF ingest is the preferred choice, if your Live encoder supports it.
In addition to media and subtitles, Origin can also ingest Timed Metadata. For this, Interface 1 of the DASH-IF Live Media Ingest specification recommends the use of a separate track, where the Timed Metadata like SCTE 35 and ID3 can be carried DashEventMessageBoxes. The advantage of using a separate track for this purpose is that it allows for faster and more efficient processing (because not all media tracks have to scanned to determine whether they also carry Timed Metadata).
Unified Origin may be able to ingest live DASH or fMP4 HLS content directly, given some additional configuration. This is not a recommended approach, but it may be an acceptable solution in certain circumstances, if the CMAF headers and fragments of the live DASH or fMP4 HLS content are formatted correctly. However, we strongly recommend to ask your encoder vendor to produce output that is compliant with one of the two aforementioned ingest protocols, as Origin's compatibility with these protocols is guaranteed.
Overall, the ingest should comply to to the requirements and recommendations detailed in the Content Preparation section of the Best Practices. In addition, the following three points are important:
For Live ingest, it is a requirement to configure the encoder and the server to use (wall-clock) time in UTC format, otherwise you might face timestamp issues when playing the live stream.
Without it the encoder most likely starts with time
0, which will make it
difficult to use options like Virtual subclips and will make
--restart_on_encoder_reconnect impossible. UTC timestamps are crucial for
Ingest one livestream on two Origins (failover) as well, as this requires that the segment contents
(including timing information) generated by each encoder to be identical.
Req.: Live encoder should use UTC timing¶
Unified Origin accepts timestamps in either "tfxd" or "tfdt" box.
You can verify your configuration through a GET request to:
The end time attribute in each stream should be (close to) the actual current UTC time.
Req.: Unified Origin should use UTC timing¶
Not only should the encoder output the livestream with UTC timestamps, the server running Unified Origin should use UTC timing, too. To configure your server to use UTC timing, please run the following command and follow the instructions:
#!/bin/bash sudo dpkg-reconfigure tzdata
Representation of UTC timing in DASH MPD¶
MPD@availabilityStartTime indicates the zero point on the MPD timeline
of a dynamic MPD. When using Origin for Live it defaults to UNIX epoch.
MPD@publishTime specifies the wall-clock time when Origin
generated the MPD (based on an incoming request).
To comply with the requirement described in paragraph 4.7.2 of version 4.0 of
the DASH-IF Interoperability Guidelines (IOP), we add the
to the MPD when streaming Live (as of release 1.7.27):
<UTCTiming schemeIdUri="urn:mpeg:dash:utc:http-iso:2014" value="https://time.akamai.com/?iso"/>
Note that Origin itself simply relies on the time of the server that it runs on
(as explained above) and does not synchronize its time with the
time.akamai.com server (although you may synchronize your Origin server's
time with Akamai's).
Representation of UTC timing in HLS playlists¶
When the Origin recognizes the use of UTC timestamps it adds the 'EXT-X-PROGRAM-DATE-TIME' tag to the HLS playlists it generates. A player can then use it as a basis for its timeline presentation.
The date/time representation is ISO 8601 and always indicates the UTC timezone:
As specified in Interface 1 of the DASH-IF Live Media Ingest specification (and Azure Media Services fragmented MP4 live ingest specification), a Live encoder must push the live stream to a publishing point using HTTP POST (or PUT). It may do so by POSTing each segment individually or through a long-running POST (using chunked transfer encoding).
It is recommended that an encoder sends an initial HTTP 1.1 handshake (
100-Continue header + empty body) before sending (chunked) payload.
A live encoder must push to a publishing point using the following URL syntax:
Where the <stream-identifier> should be unique for each stream that is pushes (note that a live stream may consists of several streams, with each stream containing one or more tracks).
The main advantage to posting multiple (multiplexed) tracks in a single stream is that they are implicitly synchronized. i.e., either all bitrates are there for some interval or none are.
Track bit rate hack (for testing purposes only)¶
Track bit rate may be signaled (as a last resort hack) in the stream identifier, where a decimal number following a dash is interpreted as kilobits associated with each (first, second etc.) track found in the stream, for instance:
This brittle setup is not recommended and is intended for testing purposes only. It should not be used in any production setup.
At the end of a Live streaming event the encoder should send an EOS signal for all of the livestream's tracks. This will properly signify the end of a Live broadcast and make the publishing point switch to a 'stopped' state. An EOS signal consists of an empty "mfra" box with no embedded sample entries in the "tfra" box and no "mfro" box following, as specified by ISO/IEC-14496-12.
The empty "mfra" box containing the 8 byte sequence looks like the following:
00 00 00 08 6d 66 72 61
Stop your encoder and query the REST API of the Live Publishing point to check its state, which should be 'stopped' (see Retrieving additional publishing point information for more information).
To achieve optimal results, fast local storage should be used (e.g., SSD's). Shared storage is not recommended, as it might impact Origin's reading from and writing to the Live archive's SQLite database. The use of remote storage for Live streaming is not supported at all.
To create publishing points some sort of script should be created. This needs to take into account the following:
- An unused publishing points is a directory that contains a Live server
.imsl), which should be create with 'mp4split'
- The publishing point is hosted by Apache, in a location that the encoder can successfully access through HTTP POST (or PUT)
- Apache should be the owner of the publishing point and have read and write access to it (the directory itself, as well as the Live server manifest)
- The publishing point's directory and the Live server manifest inside should
have the same name (e.g.,
The script that creates the publishing point should rely on mp4split to create the Live server manifest. This Live server manifest contains the configuration of the publishing point.
Certain options for Origin should be configured explicitly instead of relying on the preset defaults, because many options have evolved over time while their defaults have often remained the same for backwards compatibility reasons. Some requirements and recommendations are listed below.
The most important options to configure explicitly are related to the archive that Origin keeps on disk and the window of the stream's timeline that it makes accessible through the client manifests that it generates. The longer the archive, the further back in time Origin's Restart and Catch-up TV options can reach. The longer the DVR window, the further back a viewer can rewind the stream. When setting these options, the following should be kept in mind:
- Archiving is a legacy switch and should always be enabled (
archive_lengthshould be a multiple of
dvr_window_lengthmust be shorter than the
The exact configuration will depend on your use case, but the archive length should not exceed 7 days and the DVR window length should not exceed several hours. Regarding the DVR window, take into consideration that:
- Having a DVR window is required because a player must be able to download a few media segments to use as buffer (e.g., the DVR window must be at least three times the length of your HLS media segments for your stream to be compliant with the HLS specification.
- The longer the DVR window the longer HLS Media Playlists will be, with the same being true for Smooth and DASH client manifests in many cases as well (depending how exactly the timelines are represented in these cases).
- The longer the DVR window the more content you will to keep cached on your CDN in order to guarantee efficient delivery at scale.
- A DVR window of a considerable size will impact the time it takes for Origin to generate the client manifests for each output format.
Having said that, the following table may offer some guidance
all configured in seconds):
|Use cases / configuration||Pure live||Archiving, no 'DVR'||Archiving and 'DVR'|
|archive_length||120||86400 (24h)||86400 (24h)|
|archive_segment_length||60||600 (10m)||600 (10m)|
To offer your viewers the possibility to record and rewatch content using an efficient and reliable solution, we recommend Remix nPVR.
--restart_on_encoder_reconnect is highly recommended because it
allows an encoder to reconnect and continue pushing a stream after the stream
was (accidentally) switched to a 'stopped' state.
To verify that a publishing point is ready to ingest from a Live encoder you can make a GET request to:
The response should indicate the 'state' of the publishing to be 'idle'.
To verify that ingest was started successfully you can make a GET request to the same 'state' endpoint:
Although the response should now indicate 'started'
If the publishing point has successfully started ingest, the Live server
manifest will now list all the stream's tracks, a database file (db3) will have
been been created, and Origin will have started storing the ingested media in
one or more
More detailed information about individual tracks can be obtained through a GET request to the 'statistics' and 'archive' endpoints:
A 'started' state only indicates that ingest started at some point. It does not indicate that a stream is currently being ingested. Therefore, it is not a good means to check the health of your publishing point, or Origin in general.
Using two encoders will ensure that ingest continues even if one of the encoders becomes unavailable. This can be combined with a dual Origin setup where the encoders cross-post: a 2x2 setup. Both Origin and encoder will have maximum resilience:
origin1 origin2 | \ / | | / \ | encoder1 encoder2
Guidelines for failover of live encoders and origins are defined in the ingest specification Interface 1 of the DASH-IF Live Media Ingest specification.
To let two encoders successfully push the same livestream to a publishing point on Origin (or, in a fully resilient setup, cross-post to two Origins), both encoders should be time-aligned. They should be configured to output segments of the same, constant length, too. This will ensure that:
- Both encoders insert the same timestamps into the segments they are pushing
- Segments can be aligned across both encoders (i.e., have the same length and start at the same time).
With the same content being pushed from two sources, Origin will simply ingest whichever segment arrives first, and discard the second one.
This setup can be achieved by making sure the two encoders share the exact same time source for their output's timestamps. One way to guarantee this is to have both encoders pass on the timestamps of the shared source they rely on for the livestream, if the timing information in that source is reliable and makes sense (e.g., is Unix Epoch based).
To make sure the output of both encoders is not only time aligned but aligned across segment boundaries as well, it is important to have the encoders start (and restart) at exact multiples of the configured segment length, counting from a well-defined point in the past (e.g., Unix Epoch). This is described in a little more detail below.
Perfectly align start of two encoders¶
Assume that it is 20:00 on October 6th of 2020, UTC time, and your have configured your encoders to output segments of 1.92 seconds. If you convert the current time to a timestamp of seconds since Epoch you will get 1602014400, and it turns out that is exactly 834382500 x 1,92. In other words, if one of your encoders had just dropped out and needed to rejoin and start pushing the livestream again, it should do so at the next multiple at 20:00:01.920, or any other multiple after that (e.g., 20:00:03.840).
Or, as a rudimentary calculation using standard command-line tools to calculate the offset on a 10M timescale (which is used by Azure Media Services fragmented MP4 live ingest specification):
#!/bin/bash segments_on_10m_timescale=192000000 seconds_since_epoch_10m_ts=$(echo "( `date +%s%N` / 10 )" | bc) echo "( ( $seconds_since_epoch_10m_ts + $segments_on_10m_timescale - 1 ) \ / $segments_on_10m_timescale ) * $segments_on_10m_timescale" | bc
If your encoder supports it you can push the same livestream to two Origins for redundancy purposes. This is commonly referred to as dual ingest.
When creating a duplicate Origin setup it is important that the options for both publishing points are identical, so that a client can seamlessly switch between the output of both Origins because it will be identical.
When you do not start pushing the livestream to both Origins at the exact same time, it is possible to synchronize the part of the Live archive that the second Origin is missing with that of the first.
Synchronizing archive from one Origin with another¶
Since the Live archive that Origin keeps on disk is already in a format that Origin can ingest, and because Origin handles duplicate segments without issue (by simply ignoring any that have already been ingested), it is possible to simply POST the archived files from the first Origin to the second Origin.
For example with cURL:
#!/bin/bash curl --data-binary @/var/www/chan1/chan1-33.ismv \ -X POST "http://origin2/chan1/chan1.isml/Streams(chan1)" curl --data-binary @/var/www/chan1/chan1-34.ismv \ -X POST "http://origin2/chan1/chan1.isml/Streams(chan1)" curl --data-binary @/var/www/chan1/chan1-35.ismv \ -X POST "http://origin2/chan1/chan1.isml/Streams(chan1)"
To make dual ingest work in the best possible way, let the encoder start pushing
to the second Origin at the exact same time as the first, or have it start at a
multiple of the configured
archive_segment_length (counting from when it
start pushing to the first Origin). This approach is similar to
Perfectly align start of two encoders and will ensure that the start and end times of
all archive segments that both Origins write to disk will be perfectly aligned
(although this alignment is not as crucial as with a dual encoder setup, and may