User Interfaces

Here, you can read more about what is included in our UI Example, how it works and how you can customize it.

The UI Example contains two scenes that have similar UI elements but fundamentally different ways of interacting with them:

  • UI Touchpad Menu: requires maintaining gaze to interact with the UI elements while clicking or swiping the Touchpad button on the VIVE controller. We refer to this as a “non-locking” menu.
  • UI Trigger Menu: allows for locking onto UI elements and we refer to it as a “locking” menu. UI elements are interacted with by gazing at them and pressing down the Trigger button on the VIVE controller. As long as the trigger button is held down, the user is free to look around while still being able to manipulate the selected object by releasing the Trigger button or moving the controller left or right. Other UI elements can not be interacted with or highlighted when the Trigger button is pressed down.

Each menu have their own scripts for each UI element to handle the different interaction paradigms described above.

The scenes can be found in TobiiXR > Examples > UI_Example > Scenes.

If you want to learn more about the non-locking type menu (Touchpad) and the locking type menu (Trigger) and their differences, you can read more about it here. You can also check out our design section to learn about how to design for user interfaces or eye tracking in general.

Compatibility

This particular Unity Example is compatible with the following headsets:

HMD Compatible Instructions
Tobii HTC VIVE Devkit Yes Getting Started
HTC VIVE Pro Eye Yes, follow instructions Getting Started
Pico Neo 2 Eye Yes, follow instructions Getting Started and Special Setup.

Table of Contents


Scenes

Both scenes have a menu with 4 interactable UI elements: a button, a toggle button, a discrete slider and a continuous slider. Additionally, there is an image describing which input to use for the current scene.

The scenes consists of:

  • TobiiXR Initializer has the TobiiXR_Initializer component attached, which is required to initiate the Tobii G2OM.
  • Tobii XR Player UI handles the input and visualization of the controller with the ControllerVisualizer and ControllerManager components.
  • UI contains a Touchpad menu or a Trigger menu and the corresponding input image to know which controls are used in the current scene.
  • Environment contains the light and floor of the scene.

UI Gaze Collider

For the Tobii G2OM to be able to find potential targets, the targets need to have a Collider attached. The UIGazeCollider is a utility script to automatically set up a BoxCollider with the correct boundaries and z-values for each UI element.

The UIGazeCollider component is not required. You can manually set up a Collider for each UI element, but it is easier to do this with the UIGazeCollider component, especially for more complex user interfaces.

You can customize the BoxCollider values by changing the padding, offset or the adjust to depth values of the UIGazeCollider component or by changing the values directly on the BoxCollider.

Keep in mind that the UIGazeCollider will not update the BoxCollider in runtime.


UI Gaze Button

The UITouchpadGazeButton and UITriggerGazeButton are a gaze aware buttons.

  • To click the UITouchpadGazeButton, gaze at it and click the Touchpad button on the VIVE controller.
  • To click the UITriggerGazeButton, gaze at it and click the Trigger button on the VIVE controller.

The UITouchpadGazeButton and UITriggerGazeButton components handle the functionality of the button and require a UIGazeButtonGraphics component to handle the graphics and animations. Both components implement the IGazeFocusable interface which calls the method GazeFocusChanged(bool hasFocus) when the game object’s focus has been changed.

In the editor you can customize what will happen when the OnButtonClicked event is invoked, and in the UIGazeButtonGraphics component, you can customize how the button should behave when it is Focused (user is gazing at the button), PressedDown (user is gazing at the button and pressing down the Touchpad or Trigger button) or Idle state (user is not looking at the button) is determined by the scale and color values set for the Button Image and Label.

On Button Clicked

This event is invoked when the button is clicked. It calls The PrintButtonClick method in the UIPrinter component and prints a message in the console window.

Components

The components are animated depending on the button state and consists of the button background image and the label.

Focused

These values set how the button looks like when the user focuses on it.

  • Background Focus Color is the color of the background when the button is focused. It is set to make the button become a little brighter.
  • Label Focus Color is the color of the text label when the button is focused. It is set to make the label text white (from a grayish tone by default).
  • Button Focus Scale is the scale of the button when focused. It is set to increase the size of the button by 5%.
  • Visual Feedback Duration is the time it takes to lerp from the current colors and scale to the focus colors and scale. Current value is set to 200 ms to give the user enough time to perceive the change.
  • Visual Feedback Animation Curve describes how the visual feedback is animated. The current curve has an ease-in, ease-out behavior, meaning it will start slowly, speed up, and then slow down at the end. You can try different curves for different effects.

Pressed

These values set how the button looks like when the user focuses on it and presses the touchpad button.

  • Background Press Color is the color of the background when the button is pressed down. It becomes a tiny bit brighter than the focus color to make it shine during a press.
  • Label Press Color is the color of the label text when the button is pressed down. It is set to the same color as the focus color, to not display a change. This can be customized if you want another effect.
  • Button Scale On Press is the size of the button when it is pressed down. This is currently set to decrease the size of the button from the focused scale by 10% (1.05 -> 0.95).
  • Button Press Duration is the time it takes to press down the button. It is currently set to 100 ms to roughly match the experienced speed of pressing down the touchpad.
  • Button Press Animation Curve is how the press is being animated. It is currently set to continuously slow down to amplify the pressing down effect.

UI Gaze Toggle Button

The UITouchpadGazeToggleButton and UITriggerGazeToggleButton toggle between an Off and On state when the user looks at it and clicks, with a PressedDown state in-between. The user can also swipe the Touchpad to toggle the UITouchpadGazeToggleButton or press down the Trigger button and move the controller to toggle the UITriggerGazeToggleButton.

The UITouchpadGazeToggleButton and UITriggerGazeToggleButton components handle the functionality of the toggle button and require a UIGazeToggleButtonGraphics component to handle the graphics and animations. Both components implement the IGazeFocusable interface which calls the method GazeFocusChanged(bool hasFocus) when the game object’s focus has been changed.

In the editor you can customize what will happen when the OnButtonToggled event is invoked, and in the UIGazeToggleButtonGraphics component, you can customize how the toggle button will look like when it is Toggled Off, Toggled On or PressedDown.

On Button Toggled

This event is invoked when the button is toggled. It calls The PrintToggleButtonToggled method in the UIPrinter component and prints a message in the console window.

Components

The components are animated depending on the button state and consists of the components making up the toggle button.

Colors

Here, all the colors of the different button states can be customized, each with colors for when the button is focused and not.

  • Knob Default Color is the color of the knob when not focused. Currently set to a grayish color.
  • Knob Focus Color is the color of the knob when focused. It is set to become white.
  • Label Default Color is the color of the label text when not focused. Currently set to a grayish color.
  • Label Focus Color is the color of the label text when focused. It is set to become white.
  • Toggle Off Colors contains the default (no focus) and focus colors of the background and outline images when the toggle button is toggled off.
  • Toggle On Colors contains the default (no focus) and focus colors of the background and outline images when the toggle button is toggled on.
  • Pressed Down Color contains the default (no focus) and focus color of the background and outline images when the toggle button is pressed down.

Focused

  • Button Focus Scale is the scale of the toggle button when focused. Currently set to increase 5%.
  • Knob Focus Scale is the scale of the knob within the button when focused. It increases 10% when focused.
  • Focus Animation Duration is the time it takes to animate the colors when the user focuses or stops to focus the button. Current value is set to 200 ms to give the user enough time to perceive the change.
  • Focus Animation Curve is how the toggle button is animated when the user focuses or stops to focus the button. The current curve has an ease-in, ease-out behavior, meaning it will start slowly, speed up, and then slow down at the end. You can try different curves for different effects.

Clicked

  • Toggle Animation Duration is the duration for the toggle button to update its state when clicked. This is currently set to 100 ms to feel responsive.
  • Toggle Animation Curve is how the toggle animates switching between states. It is currently set to continuously slow down to amplify the toggle effect.

A UITouchpadSwipeDetector component is attached to allow swiping to toggle the UITouchpadGazeToggleButton and a UITriggerDragDetector component is attached to allow for dragging the UITriggerGazeToggleButton.


UI Gaze Slider

The UITouchpadGazeSlider and UITriggerGazeSlider are gaze aware sliders with a minimum and maximum value.

  • To update the UITouchpadGazeSlider, gaze at the slider and swipe the Touchpad left or right.
  • To update the UITriggerGazeSlider, gaze at the slider, press down the Trigger button and move the controller left or right.

With a big enough difference in the minimum and maximum value of the slider, it will behave as a continuous slider, whereas less steps will make it behave as a discrete slider. For the continuous sliders in the scenes, the value label has also been hidden.

The UITouchpadGazeSlider and UITriggerGazeSlider components handle the functionality of the sliders and require a UIGazeSliderGraphics component to handle the graphics and animations. Both components implement the IGazeFocusable interface which calls the method GazeFocusChanged(bool hasFocus) when the game object’s focus has been changed.

In the editor you can customize what will happen when the OnSliderValueChanged event is invoked, and in the UIGazeSliderGraphics component, you can customize how the slider will look like in its different states.

On Slider Value Changed

This event is invoked when the slider value has been updated. It calls The PrintSliderValueHasChanged method in the UIPrinter component which prints a message in the console window.

Functionality

  • Touch Pad Multiplier and Controller Movement Multiplier determines how much you need to swipe the touchpad or move the controller to update the slider. You can customize these values to get different sensitivities for the sliders.
  • Min Value is the minimum value of the slider, the slider output when the handle is set furthest to the left. This is currently set to 0 for all sliders.
  • Max Value is the maximum value of the slider, the slider output when the handle is set furthest to the right. This is set to 20 for the discrete sliders and 100 for the continuous sliders. A bigger difference between the minimum and maximum value makes the slider behave as a continuous slider.
  • Haptic Strength is the feedback strength for when the slider value is updated. For the continuous sliders with 100 steps, it is set to 200 and for the discrete sliders with 20 steps it is set to 500. You can try out different values for different feedback effects.

Components

The components are animated depending on the interaction state of the slider and consists of the components making up the slider.

Focused

  • Background Focus Color is the color of the background when the slider is focused. It is set to make the background white (from a grayish tone by default).
  • Fill Focus Color is the color of the fill amount when the slider is focused. It is set to make the slider become a little brighter.
  • Label Focus Color is the color of the label text when the slider is focused. It is set to make the label text white (from a grayish tone by default)
  • Slider Focus Scale is the scale of the slider when it is focused. It is currently set to increase 3% when looked at but this value can be customized depending on the use case and how big the default slider scale is.
  • Visual Feedback Duration is the time it takes to lerp from the current colors and scale to the focus colors and scale. Current value is set to 200 ms to give the user enough time to perceive the change.
  • Visual Feedback Animation Curve describes how the visual feedback is animated. The current curve has an ease-in, ease-out behavior, meaning it will start slowly, speed up, and then slow down at the end. You can try different curves for different effects.

Handle

The default color of the handle is currently set to be completely transparent and have its scale set to 0. This is because we wish to not show it until the user actually starts interacting with the sliders (i.e., looks at it and touches the touchpad or presses down the trigger). If you want to know more about why we recommend this, read about slider affordances in our design section.

  • Handle On Touch Color is the color of the handle when the user touches the touchpad or presses down the trigger button. It is set to become visible with a light blue color (transparent by default).
  • Handle Animation Duration is the time it takes to animate and show the handle. It is currently set to 100 ms to feel responsive.
  • Handle Animation Curve is how the handle is animated. It is currently set to continuously slow down to create a satisfying effect for the user. You can try different curves for different effects.

Create Your Own Scene

To add gaze UI to a new or existing scene, you need to add 3 prefabs:

  1. TobiiXR Initializer initializes the SDK and the Tobii G2OM, found in TobiiXR > Prefabs.
  2. Tobii XR Player UI contains the camera and scripts required for controller input, found in TobiiXR > Examples > UI_Example > Prefabs.
  3. Trigger Menu or Touchpad Menu contains the gaze menus, found in TobiiXR > Examples > UI_Example > Prefabs.