Social

This section describes the Social example scene.

In the Social example scene, we provide reusable examples of the following use cases:

  • Two dimensional eye movement: that demonstrates how to visualize eye movement on a two dimensional plane.
  • Three dimensional eye movement: that demonstrates how to visulaize eye movement by rotating three dimensional spherical eyeballs.
  • Blinking and winking: on both two and three dimensional eye representations.
  • Facial expressions: derived from eye movements using blendshapes.

We also provides examples of first person and third person avatar usage.

The scene can be found in TobiiXR > Examples > Social_Example > Scenes.

Check out our design section to learn about how to design for the social use case or eye tracking in general.

Compatibility

This particular Unity Example is compatible with the following headsets:

HMD Compatible Instructions
Tobii HTC VIVE Devkit Yes Getting Started
HTC VIVE Pro Eye Yes, follow instructions Getting Started
Pico Neo 2 Eye Yes, follow instructions Getting Started and Special Setup.

Table of Contents


Scene Contents

The scene consists of the four prefabs TobiiXR Initializer, Tobii XR Player Social, 2D Character, 3D Character and the Environment game objects like lights and floor.

Tobii XR Initializer

This prefab simply ensures the SDK and Tobii G2OM is initialized, and should be included in every scene.

Tobii XR Player Social

Drag this prefab into your scene to instantly provide you with a first person avatar head that demonstrates true gaze directional control, winking, blinking and some other facial blend shape driven expressions. This prefab is designed for use in multiplayer scenarios so unless you create a mirror or view from another player’s perspective you will not see it.

2D Character

The 2D Character model demonstrates the simplest implementation of eye tracking on flat two-dimensional eyes, moving only the pupils in local X Y space. The prefab also includes hands to provide more room for expression. There is no camera attached so this prefab can be used in third person.

3D Character

The 3D Character model demonstrates a more sophisticated method that includes true directional control, winking, blinking and some other facial blend shape driven expressions.


2D Character Scripts

Follow Camera Height

Attached to both avatars’ root game object and is used to keep the avatars at eye level so they can be viewed seated or standing.

Handle 2D Eyes

Contains the code that controls the 2D character’s eyes.

Pupil Transforms

The component only acts on two Transforms passed as left and right pupils, moving their local position in the X and Y coordinates. Winking and blinking will modify the y scale of those Transforms reducing them when the eyes are closed.

Eye Behaviour Values

  • Gaze Direction Smooth Time allows for filtering signal noise. The higher the value the smoother the eyes move but the slower they respond, making the avatar appear less realistic and more sleepy.
  • Blink Speed adjusts animation speed when the avatar blinks.

3D Character Scripts

Follow Camera Height

Attached to both avatars’ root game object and is used to keep the avatars at eye level so they can be viewed seated or standing.

Handle 3D Eyes

This component contains the code that controls the direction of the 3D characters spherical eyes.

Eye Transforms

Acts on two eye Transforms. The left and right eye balls are manipulated by rotation about the center of each eye.

Eye Behaviour Values

  • Cross Eye Correction, different avatar models’ eyes sometimes can appear to converge or diverge even when in fact they do not, this parameter allows you to correct for that phenomenon.
  • Gaze Direction Smooth Time allows for filtering signal noise. The higher the value the smoother the eyes move but the slower they respond, making the avatar appear less realistic and more sleepy.

The next three parameters are used to clamp the eyeball rotation amounts. As model designs vary this will allow you to keep the pupils from rotating too far and being obscured by eyelids etc.

  • Vertical Gaze Angle Upper Limit In Degrees
  • Vertical Gaze Angle Lower Limit In Degrees
  • Horizontal Gaze Angle Limit In Degrees

Handle 3D Eyelids

This component demonstrates how eyelids can be animated in a realistic way based on the signals provided by the Tobii eye tracker. It acts on four Transforms that represent the upper and lower eyelid on both the left and right eyes. It assumes that the eyelid transforms passed are centered at the eyeball center.

  • Blink Speed. Blinks and wink animations are triggered by an eye openness flag, as real eyes take time to blink or wink here you can adjust that animation time.

Handle 3D Expressions

It is surprising how many facial expressions can be derived from the eye tracker signals. This component is included to give a simple demonstration of some basic expressions and how to animate them based on Blendshapes.

Blend Shapes

  • Head Blend Shapes takes the MeshRenderer on which the head blend shapes are exposed. This is specific to the model used here but shows a typical system for passing access to a model’s blend shapes.
  • Teeth Blend Shapes also takes a MeshRenderer exposing blend shapes related to the teeth animations.
  • Blend Shapes Animation Curve allows you to adjust the shape of the blend shape animations curve. Facial muscles, teeth etc. have mass and therefore require some time to accelerate and decelerate. By tapering the start and end of the curve far more realistic movements can be achieved.
  • Blend Shape Animation Time Seconds is the time it takes for the blend shape animations to complete.

Other Scripts

Handle Hands

Used by both avatars to handle the placement and visibility of any type of “Hand” prefab attached via the components Left Hand Prefab and Right Hand Prefab parameters.

Rotate With HMD

Attached to both third person avatars’ heads so they follow the users head movements via the HMD’s rotation.


Create Your Own Avatar

To create a scene with your own 3D avatar you will need 2 components:

  1. Drag the TobiiXR Initializer prefab into your scene found in TobiiXR > Prefabs.
  2. Add the component Handle3DEyes to your avatar’s head GameObject then drag the left and right eye Transforms to the left and right eye component parameters in the inspector.

Warning: If the avatar’s eye models were not created with y-axis up and z-axis forward they may point in an unpredictable direction. It is recommended that you fix the model’s eyes in a 3D modelling application to have the y-axis face upwards and z-axis face forwards before exporting. However you can fix it in Unity by adding an extra empty parent GameObject to the center of each eye and then pass those empty GameObjects Transformsto the component.

Blinking and Winking. If your model has eyelids whose center of rotation is at the center of the eyeballs you can also add the Handle3DEyelids component to the avatar’s head GameObject and drag the left, right, upper and lower lid Transforms into the corresponding component parameters in the inspector.

Facial Expressions. In more complex models blinking, winking and facial expressions are all handled using blendshapes. If you dig into the Handle3DExpressions component code a little you will soon see how to connect those blendshapes to gaze signals, opening the door to all sorts of organically driven facial animations.