Tobii Ocumen I/O for Unity

Record or stream live gaze and scene elements directly to Tobii Ocumen Studio from Unity.

Once you enable recordings in your app you can use Tobii Ocumen Studio to inspect what happened.


Table of Contents

Introduction

You can attach recorders to your Unity project and they will search for relevant objects in the scene and record them, such as gaze data or colliders. Recorders are orchestrated by the Tobii Recording Manager. The manager collects data from all enabled recorders, and either

  • writes them to a .ocumen container file, or
  • streams them over network, or
  • gives stream access for custom I/O (e.g., Bluetooth).

The .ocumen container is a highly optimized self-contained streaming format that includes everything you recorded and can subsequently be used without scene access. You can think of an .ocumen stream as a .mkv but for interactive, analytical content.

Internally an Ocumen recording consists of multiple tracks, each stored with microsecond resolution. Some tracks are recorded for you (see below), and you can also create your own.

At the time of writing the .ocumen format is not stabilized yet and only compatible with a matching Tobii Ocumen Studio version at the time of release. We therefore recommend you archive your current Tobii Ocumen Studio version along with any recordings you have made.


Getting Started

Make sure you follow our installation instructions for Tobii Ocumen.

To get started, you can either run the sample scene found in the accompanying Recording sample, try the Interaction Recording project or do following in your scene:

Step 1: Add the TobiiXR_Initializer script to your scene

Make sure you add your license to the script.

Step 2: Add the Tobii Recording Manager script to your scene

It can be set to record to file or stream over network. If recording to file and the path is not used defined, it will store the recordings in the application’s persistent data path.

Autostart recording will automatically start recording at the start of the app and end when closed. If you want to manually control starting and stopping recordings, you can add the TobiiRecordingToggler to your scene or create your own script to handle this.

Step 3: Add recorders

The recorders will determine what data will be recorded. These have to be added to the same game object as the Tobii Recording Manager. There are several recorders, and their naming might differ slightly from the picture above. Not all recorders are available in all versions or packages.

Step 4: Run Your App!

Once a Tobii Recording Manager and recorders have been added to your scene, you can compile or launch your app!

Step 5: Open Tobii Ocumen Studio

Either with your application running in streaming mode or after creating a recording, you can launch Tobii Ocumen Studio to analyze your data.


Predefined Recorders

A list of pre-defined recorders and their specifications. Unless otherwise stated, the following properties are true for all recorders:

  • Recorders are invoked once per Unity frame.
  • The timestamp used will be updated in a coroutine right after Update has run on all game objects.
  • Timestamps are sampled from the system clock.

Headset Pose Recorder

Records the camera pose and the view and projection matrices of the left and right virtual camera.

  • Many headsets do not deliver valid matrices when they are not rendering so this recorder only records data when the application is rendering.

Advanced Eye Tracking Recorder

Records native Tobii Eye Tracking data.

  • This recorder records the output of TobiiXR.Advanced.QueuedData
  • The eye tracking clock is by default synchronized to the system clock.
  • If multiple eye tracking samples have arrived since the last frame, all samples will be written with their original time stamp. In other words, eye tracking data keep its system synchronized timestamp in the recording so you won’t have to worry about if the eye tracker runs at a higher or lower frame rate than your application.
  • Requires Headset Pose Recorder to be able to calculate the gaze direction in world space, in order to for example visualize the gaze rays.

Frame Data Recorder

Records the frame number given by UnityEngine.Time.frameCount.


Collider Recorder

Records any colliders attached to game objects in your scene.

  • Sphere and box colliders are recorded as is.
  • Mesh and capsule colliders are recorded as boxes.
  • Collider type is currently only recorded the first time the collider is encountered.
  • Only records one collider per object.
  • Duplicated objects that contain children colliders in a Unity scene will only allow for one recorded collider in the duplicated object.

Mesh Recorder

Records vertices and world position for all visible meshes in the scene belonging to a MeshRenderer or a SkinnedMeshRenderer.

  • For models you want to record, make sure Read/Write Enabled is checked in the inspector window. Read more at Unity’s documentation.
  • If you want to record meshes for static objects you may have to turn off Static Batching in Player Settings since Static Batching creates a combined mesh that is not always readable from CPU.

Controller Recorder

Records pose and input from VR controllers

  • Does not record axis values such as joystick or touchpad.
  • The controller input is recorded using Unity’s XR Input.

Rendered Video (Alpha)

Records a video capture of the left eye’s camera view in the HMD.

  • Supports OpenGL ES 3.0 and higher on Android platforms.
  • Currently only supports NVIDIA graphics cards using DirectX 11 on Windows platforms.
  • The Video Recorder is currently in Alpha state. It does not support all platforms and may exhibit a temporal offset when played back. When replaying video in Ocumen Studio, there is a slider for manually adjusting the offset to match the video with the rest of the recording.
  • Video data averages about 1 MB per second and usually dominates recording size by up to 2 orders of magnitude. While enabling video can give you authentic insights in highly dynamic (e.g., shader driven) scenes, disabling video can yield minute-long recordings that are just a few kilobytes long.