Object Mapping

Here, you can find two basic examples of object mapping using Tobii G2OM (Gaze-2-Object-Mapping). G2OM is a machine learned object selection algorithm which solves many of the common challenges of determining which object a person is looking at. These examples show how G2OM selects the object a person is looking at, in the case of 3d objects, Unity UI, dynamic objects, and tightly spaced objects.

There are two example scenes which can be found in TobiiXR > Examples:

  • Example_ObjectMapping
  • Example_ObjectMapping_UI

Compatibility

This particular Unity Example is compatible with the following headsets:

HMD Compatible Instructions
Tobii HTC VIVE Devkit Yes Getting Started
HTC VIVE Pro Eye Yes, follow instructions Getting Started
Pico Neo 2 Eye Yes, follow instructions Getting Started and Special Setup.

Table of Contents


G2OM Modules

The G2OM has modules that can be modified to suit your needs. On initialization these modules are set and cannot be changed during the lifetime of the G2OM instance. The modules intended use are:

  • IG2OM_ObjectFinder - Finds game objects based on gaze data in the scene.
  • IG2OM_ObjectDistinguisher - Determines if a game object is gaze focusable at any given moment.
  • IG2OM_ColliderDataProvider - Calculates the bounds for a game object.
  • IG2OM_PostTicker - Executes after G2OM has produced a result.

When initializing G2OM using the G2OM_Description you can also set values for:

  • HowLongToKeepCandidatesInSeconds - How long to keep game objects in memory.
  • LayerMask - What layer mask should be used by the IG2OM_ObjectFinder.

Example_ObjectMapping

The Example_ObjectMapping scene is a basic example of usage with G2OM, and should fit most object selection needs.

Scene Contents

The scene consists of the TobiiXR Initializer, Main Camera with G2OM Debugger and multiple Gaze Focusable Objects.

  • TobiiXR Initialier initializes the Tobii XR SDK and Tobii G2OM.
  • Main Camera with G2OM Debugger has a G2OM_DebugVisualization and a G2OM_DebugTool component attached for debugging G2OM.
  • Gaze Focusable Objects all have a Collider and a HighlightAtGaze component attached.

Gaze Focusable Objects

For an object to be considered by the G2OM, it has to have a Collider and a component which implements the interface IGazeFocusable, which in this scene is HighlightAtGaze.

Debug Visualization

Tobii G2OM can be debugged more easily with the G2OM_DebugVisualization component. In this example scene, the debugging visualization is toggled by the G2OM_DebugTool component and defaults to the keys Space for on/off and Left Control for freezing the visualization.

Pressing Space overlays a debug visualization showing what gaze focusable objects are being considered and what internal scoring they got from G2OM. The greener the cube, the higher internal scoring of being focused.

Pressing Left Control before showing the visualization freezes the debug info in place so you can visualize what happened at that frame. The red ray is the left eye, the blue ray is the right eye and the yellow is the G2OM ray.

The Main Camera with G2OM Debugger can be found in TobiiXR > DevTools > Prefabs and is a good tool to debug G2OM in your scene.


Example_ObjectMapping_UI

The Example_ObjectMapping_UI scene is a basic example of usage with G2OM for UI.

Scene Contents

The scene consists of the TobiiXR Initializer, Main Camera with G2OM Debugger and the Gaze Focusable UI.

  • TobiiXR Initialier initializes the Tobii XR SDK and Tobii G2OM.
  • Main Camera with G2OM Debugger has a G2OM_DebugVisualization and a G2OM_DebugTool component attached for debugging G2OM.
  • Gaze Focusable UI is a Unity canvas with elements, which have UIHighlightAtGaze and UIGazeCollider components attached.

UI Gaze Collider

For G2OM to be able to find potential targets, the targets need to have a Collider attached. The UIGazeCollider is a utility script to automatically set up a BoxCollider with the correct boundaries and z-values for each UI element.

The UIGazeCollider component is not required. You can manually set up a Collider for each UI element, but it is easier to do this with the UIGazeCollider component, especially for more complex user interfaces.

You can customize the BoxCollider values by changing the padding, offset or the adjust to depth values of the UIGazeCollider component or by changing the values directly on the BoxCollider.

Keep in mind that the UIGazeCollider will not update the BoxCollider in runtime.