Signals & Filters - Filter Pipelines

The Tobii Ocumen Filter Pipeline is currently in alpha and prone to change.

The eye movement classifiers and filters are available through the Filters package, a Unity package part of Tobii Ocumen Signals & Filters.

This page will give you an overview of how to use the pipeline supporting eye movement classifiers and filters.

Table of Contents


Overview of Pipeline Stages

  • Frame Processors: used on HMD tracking data to ensure timestamps are correctly ordered, regularly-intervaled, and time-shifted, if needed.
  • Eye Tracking Processors: used on eye tracking data to remove outliers, remove blinks, ensure that timestamps are correctly ordered, regularly-intervaled, and time-shifted, if needed.
  • Fusion Stage: gives you options for how to temporally fuse frame data and eye tracking data which often have different refresh rates.
  • Fused Processors: another chance to ensure that fused data timestamps are correctly sorted and regularly-intervaled.
  • Filters: algorithms used to detect eye tracking events such as saccades, fixations, smooth pursuits, vestibulo-ocular reflexes, etc. Ö

Step 1: Installation

Make sure you follow our installation instructions for Tobii Ocumen.


Step 2: Initialize the Pipeline

The pipeline requires a JSON5 pipeline definition file.

In order to generate one, open a command prompt in the /Tools/ folder containing the ocumen_cli.exe file and run:

ocumen_cli pipeline init

This will create a default pipeline.json5 file which is used to configure your pipeline. You can have several JSON5 pipeline definition files.

For more information about commands, you can always run:

ocumen_cli help

You can add help at the end of any command for more information.


Step 3: Customize the Pipeline

The JSON5 pipeline definition file determines how the pipeline works and which algorithms and filters it uses.

The basic idea is that eye tracking and frame data (e.g, HMD transform data) is recorded, optionally pre-processed and then fused together. Algorithms are then applied on the fused data, generating for example fixation or saccade data.

Open the JSON5 file and customize the pipeline by for example adding a second algorithm called “saccade_simple” which uses the built-in SaccadeSimple() filter. This will allow you to query for saccade results using this filter in this pipeline. It’s a good idea to rename this file to something like “myPipeline.json5”.

{
    "information": {
        "version": "0.2.2705"
    },
    "common_settings": {
        "element_capacity": 1024,
        "temp_memory": 8306688
    },
    "fusion": {
        "TakeEyeTrackingClosestFrame": {}
    },
    "algorithms": {
        "fixation": {
            "Fixation": {
                "FixationSimple": {}
            }
        },
        "saccade_simple": {
            "Saccade": {
                "SaccadeSimple": {}
            }
        }
    }
}

Running below command will generate a JSON pipeline definition file with as many options as possible which could be useful to get a good understanding of how you can customize your own file.

ocumen_cli pipeline init --max

Step 4: Validate the Pipeline

To validate and make sure that the pipeline will execute, run:

ocumen_cli pipeline check --pipeline myPipeline.json5

This will validate that your JSON pipeline definition file is set up correctly and run some additional tests on it using pre-recorded data.


Step 5: Run the Pipeline

Once you have set up your pipeline, you can now record eye tracking data and feed it into the pipeline and then query for results.

Below is some example code using the myPipeline.json5 file we just set up.

public class SamplePipelineCode : MonoBehaviour
{
    private Pipeline _pipeline;
    private DataWindow _dataWindow;
    private int windowSizeUs = 200000; // Size of time window in microseconds.
    private TimeSeries<bool> _saccadeOutputLeftEye = new TimeSeries<bool>(100);

    private void Start()
    {
        // Read pipeline definition file from StreamingAssets folder on PC.
        var filePath = Path.Combine(Application.streamingAssetsPath, "myPipeline.json5");
        var config = File.ReadAllText(filePath);

        // Create a pipeline and data aggregator.
        _pipeline = Pipeline.Create(config);
        _dataWindow = new DataWindow(windowSizeUs);
    }

    private void Update()
    {
        // Refresh data window with new frame and eye tracking data and feed that data into the pipeline.
        _dataWindow.Tick();
        _pipeline.Execute(_dataWindow.FrameData.ToArray(), _dataWindow.EyeTrackingData.ToArray());

        // Query the pipeline for results and log them.
        _pipeline.QueryResult("saccade_simple", "is_saccade_left", _saccadeOutputLeftEye);
        if (_saccadeOutputLeftEye.Count > 0)
        {
            Debug.Log(string.Join(Environment.NewLine, _saccadeOutputLeftEye.Values));
        }
    }
}

Step 6: Run the Unity Sample

We have created a Unity Sample showcasing how you can interface with the filter pipeline and how you could output the signals in graph format.

This sample can be imported from the Filters Unity package in the /Packages/ folder.

Check out our Tobii Ocumen Signals & Filters showcase page for an overview of how this scene looks like.


Documentation

The full documentation for the pipeline and filters can be generated by running:

ocumen_cli.exe pipeline document -f myDocumentation.md

You can also find the documentation here.