Troubleshooting LIVE Streaming

Log files and setup

The webserver log file is the place to look if the Live stream is not ingesting properly or does not stream properly, for Apache on Ubuntu there are two log files:

  • the general error log (/var/log/apache2/error.log on Ubuntu/Debian)
  • the virtual host error log (configured in the virtual host config).

The log level of the virtual host log should be set to debug.

The first thing is to check is if the license is properly installed: License installation check and that there are no Error messages.

Secondly, a detailed check of the installation is advised, making sure Apache and Nginx are installed and configured as detailed in their respective installation documentation.

A working publishing point will return the following from the /state REST call (see Retrieving additional publishing point information):

<?xml version="1.0" encoding="utf-8"?>
<!-- Created with Unified Streaming Platform(version=1.7.10) -->
<smil
  xmlns="http://www.w3.org/2001/SMIL20/Language">
  <head>
    <meta
      name="state"
      content="idle">
    </meta>
  </head>
</smil>

'409 stream closed'

The 'Stream Closed' error occurs when the timestamps sent by the encoder are not increasing.

The webserver log will show entries like the following:

[Mon Mar 09 19:19:32 2015] [error] [client 192.168.12.100] pubpoint_push_input_stream(Streams(Video1))
  returned: 409 Stream is closed, cannot restart (track_name=video stream.time=41863154666 fragment.time=41598557000)

For an encoder to be able to start, stop and restart (using restart-on-encoder-reconnect, see Options for LIVE ingest) the encoder needs to be configured to use UTC time as the time it uses. Please refer to the encoder manual on how to configure this (or see Encoder settings).

Discontinuities

The StreamIndex element in the client manifest contains the timestamps, next to the media type (for instance video) or the QualityLevel element indicating the bitrate.

<StreamIndex>
  <QualityLevel />
  <c t="0" d="20000000" />
  <c d="20000000" />
  <c d="20000000" />
</StreamIndex>

Sometimes you might notice a @t attribute in the client manifest, which indicates a gap in the stream. In most cases, this is caused by ingest data interruption on the server but sometimes it could also be caused by improper encoder implementations.

<c t="0" d="20000000" />
<c d="20000000" />
<c t="60000000" d="20000000" />

More information can be found on the IIS blog.

Discontinuities can also be found by looking at the /archive REST call i available in the Publishing Point API.

Background

On the origin we see a discontinuity when the timestamp on a next fragment doesn't equal the timestamp of the previous fragment + its duration. Even if it's only off by 1/10th of a microsecond.

Although the difference may be very small, this triggers a discontinuity and is noticed in playlist generation and eventually by a player implementation that triggers a decoder reset (introducing ticks/clicks/pops) whenever it sees a timestamp discontinuity. This is especially true for MPEG-TS devices (iOS and Android).

So it's important that the timestamp and duration signaled in the "tfxd" box (the one that the encoder is inserting for every fragment) is accurate.

In a more mathematical way:

tfxd[fragment N].timestamp = tfxd[fragment N - 1].timestamp + tfxd[fragment N - 1].duration.

Also worth noting is that the "tfxd[fragment N].duration" should be equal to the sum of the sample durations specified in the "trun" box.