The Pipeline Configuration File

In this section, you will learn the generic structure of a .yaml Pipeline Configuration file and the meaning of each section.

We are going to refer to the following example configuration file:

mpd:

- manifest_edit.plugins.mpd.utc_remove:
    - '*'

- manifest_edit.plugins.mpd.utc_add:
    ntp:
      https://time.akamai.com/?ntp

The Pipeline Configuration File is written in yaml format. It is a human-readable data representation format. If you're not familiar with its syntax, you can have a look at this basic introduction.

If you're totally new to .yaml, you most likely will be able to follow this chapter by keeping in mind the following basic facts about yaml:

  • indentation is important and is what defines hierarchy

  • always indent using spaces and not TABs

  • a .yaml file is essentially a collection of key and value pairs (as in name: Joe).

Let's breakout the example file section by section:

Manifest format

At the root of the Pipeline Configuration File is a section specifying the specific manifest format that this pipeline will handle.

mpd:

In this specific case, this line specifies that the pipeline will only handle mpd (Dash) and will completely ignore any manifest it may receive having a different format.

The two supported format at the moment are mpd and m3u8_main. Further manifest formats may be introduced in future versions of Manifest Edit.

Pipeline definition

At the next level of hierarchy in the Pipeline Configuration File comes the pipeline definition section.

mpd:
- manifest_edit.plugins.mpd.utc_remove:
    [...]

- manifest_edit.plugins.mpd.utc_add:
    [...]

For each manifest format, in fact, you must specify the list of plugins you want to activate in the processing pipeline. This example describes a two-stages pipeline: the first stage will apply the logic implemented by the utc_remove plugin; the second stage will apply the logic implemented by the utc_add plugin.

This is a visual representation of the pipeline built by Manifest Edit when provided this configuration file: two plugins will be picked from the Library (utc_remove and utc_add) and will be put in the pipeline one after the other

../../../_images/utc_change_pipeline.svg

When building your Pipeline Configuration File, keep in mind the following two important aspects:

  1. the pipeline is ordered. Each plugin in fact will operate on the manifest resulting from the modifications already applied by previous plugins in the pipeline. This means that changing the order of appearance of plugins in the pipeline configuration file will in general lead to different results.

  2. strings such as manifest_edit.plugins.mpd.utc_remove are directly related to the name and path of the python module of the specified plugins. Refer to the Plugins Library documentation to know the correct string for each plugin.

Plugins configuration

The last element in the Pipeline Configuration File hierarchy is dedicated to the individual plugin configuration. A plugin in fact can implement a quite generic logic (i.e. "add an element"); in order to apply this logic to a real-world use case, you need to provide to the plugin additional information (i.e. "what" should be added, "where" should it be added). This is the section responsible for that.

Let's focus on the utc_add part of the example Pipeline Configuration File:

mpd:
- manifest_edit.plugins.mpd.utc_add:
    ntp:
      https://time.akamai.com/?ntp

In this case the entire section

ntp:
  https://time.akamai.com/?ntp

represents the configuration of the utc_add plugin. As such, it is specific to it and, in general, you have to consult the plugin documentation to understand what needs to be specified in this section.

This part of the Pipeline Configuration File is thus very variable and its content depends on the particular plugin you have activated.

Example Pipeline Configuration files

Manifest Edit comes with a list of example Pipeline Configuration files, covering the usage of every plugin from the Library. You can find these files in the /usr/share/manifest-edit folder in the supported *nix-based operating systems, or in C:\Program Files\Unified Streaming\ManifestEditConf on Windows. Comments and examples embedded in the example files will drive you trough the individual configuration options available for each plugin. We also encourage you to check the chapter Included Use Cases for additional use cases.

Tip

When you are ready to create your own .yaml file, it is recommended that you create a copy of the example file you want to start from and edit the copy. It is not recommended to modify the provided example files as they will be overwritten when a new manifest-edit package is installed, and you will lose all your local modifications!