Module: VisageTrackerUnityPlugin

VisageTrackerUnityPlugin


The VisageTrackerUnityPlugin is a native plugin (see Native Plugins) written in JavaScript for Unity. The plugin exposes face tracking features of visage|SDK for use in Unity WebGL build.

The plugin code is placed in VisageTrackerUnityPlugin.jslib file and can be found in /lib folder.

Note: This version of the plugin is adapted to Unity version 2019.1, while for all older versions of Unity unityInstance in plugin's functions has to be renamed to gameInstance.

 Dependencies

The plugin depends on visageSDK.js file that loads visageSDK.data. Therefore, visageSDK.js has to be included in WebGL build output file - index.html. Additional information on changing the location of the data file can be seen on the following link: VisageTracker Dependencies

 Usage

For the functions to be accessible from C#, a C# wrapper interface is used. The example can be seen in the following function:

	[DllImport("__Internal")]
	public static extern void _openCamera(int width, int height, int mirrored, string onSucessCallbackCamera, string onErrorCallbackCamera);

The usage is demonstrated in VisageTrackerUnityDemo sample, VisageTrackerNativ.HTML5.cs file. All C# scripts are located in Assets/Scripts folder of the project.
In order to use VisageTrackerUnity plugin in Unity projects visageSDK.js must be included in output index.html file (see link).

 Callbacks

Generally, it is common to use callback functions in JavaScript, due to its asynchronous nature.

Within the plugin, some functions are implemented so they expect a callback function name as a parameter. Callback functions are defined in C# code. Examples can be seen in _getCameraInfo function.

Methods

<static> _preloadFile(fileURL)

Stores urls and names of the files that need to be preloaded into the file system. It is expected that the url for each file will be relative to the main index.html file position. It has to be called before using any of the functionalities from visage|SDK. The recommendation for calling this function is within the function Awake() in Unity.
Parameters:
Name Type Description
fileURL string the name and path to the file.

<static> _initializeLicense(license)

Initializes the license. It is recommended to initialize the license in the Awake() function within the unity script.
Parameters:
Name Type Description
license String license url as relative path from the main index.html file to the directory that contains license file(s) (e.g. licenseFilePath = "StreamingAssets/Visage Tracker/471-308-776-250-553-231-598-221-624-198-522.vlc";).

<static> _initTracker(config, numFaces, callback)

Initializes VisageTracker.
To change the configuration file, _initTracker function should be called again. Callback function is called once the tracker is initialized.
Parameter config is string that represent the path to the tracker configuration file.
Example, if the configuration file is located in StreamingAssets/Visage Tracker folder:

	configFilePath = "StreamingAssets/Visage Tracker/Head Tracker.cfg";

Parameter callback is name of function defined in Unity script. An example of a callback function definition and _initTracker() function call:

	//callback function
	public void CallbackInitTracker()
	{
		Debug.Log("TrackerInited");
		trackerInited = true;
	}
	//call of the _initTracker() function:
	VisageTrackerNative._initTracker(config, MAX_FACES, "CallbackInitTracker");

Parameters:
Name Type Description
config string the name and path to the tracker configuration file (.cfg; default configuration file is provided in lib folder; for further details see VisageTracker Configuration Manual).
numFaces Number The maximum number of faces that will be tracked.
callback string the name of the callback function.

<static> _openCamera(camWidth, camHeight, isMirrored, onSuccessCallbackCamera, onErrorCallbackCamera)

Closes camera if it is already opened.
If the camera is initialized successfully onSuccessCallbackCamera callback function is called, otherwise onErrorCallbackCamera callback function is called. Mirrors the frame if isMirrored parameter is set to 1.
Parameters:
Name Type Description
camWidth Number camera frame width.
camHeight Number camera frame height.
isMirrored Number 1 or 0; if set to 1 frames will be mirrored.
onSuccessCallbackCamera String the name of the callback function that is called if camera is successfully initalized.
onErrorCallbackCamera String the name of the callback function that is called if initialization camera fails.

<static> _closeCamera() → {Boolean}

Stops camera streaming.
Returns:
If camera is already closed returns false, otherwise returns true.
Type
Boolean

<static> _grabFrame()

Captures current frame to be used for face tracking and binding on given texture. Needs to be called before _track and _bindTexture functions.

<static> _getTrackerStatus(trackStatus, length)

Returns tracker status.
Parameters:
Name Type Description
trackStatus Array array that will be filled with tracking statuses for each of the tracked faces.
length Number length of trackStatus array, should be equal to the number of faces given to the _initTracker() function(smaller of equal to 10).

<static> _track()

Performs face tracking on the captured frame (see _grabFrame) and returns tracking status. _openCamera and _initTracker functions need to succeed before calling _track.

<static> _bindTexture(texture)

Update the passed texture with the current captured frame image data (see _grabFrame). The function expects the pointer to the created texture to be passed.
Example:

	private Texture2D texture = null;
	texture = new Texture2D(TexWidth, TexHeight, TextureFormat.RGBA32, false);
	VisageTrackerNative._bindTexture(texture.GetNativeTexturePtr());


Parameters:
Name Type Description
texture Number Pointer to an texture.

<static> _getCameraInfo(camFocus, imgWidth, imgHeight)

Assigns values to the given camera info parameters.
Parameters:
Name Type Description
camFocus Number parameter to which focal distance of the camera will be assigned.
imgWidth Number parameter to which width of the frame will be assigned.
imgHeight Number parameter to which height of the frame will be assigned.

<static> _getHeadTranslation(translation, faceIndex)

Translation of the head from the camera. Parameter translatation is 3-dimensional array which will be filled x, y and z head translation coordinates for the particular face given with faceIndex parameter. The coordinate system is such that when looking towards the camera, the direction of x is to the left, y iz up, and z points towards the viewer. The global origin (0,0,0) is placed at the camera. The reference point on the head is in the center between the eyes. This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will fill translation array with -10000 values.
Parameters:
Name Type Description
translation Array 3D array in which the head translation values will be stored.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getHeadRotation(rotation, faceIndex)

Rotation of the head.
Parameter rotation is three-dimensional array which will be filled x, y and z head rotation coordinates for the particular face given with faceIndex parameter.
This is the estimated rotation of the head, in radians. Rotation is expressed with three values determining the rotations around the three axes x, y and z, in radians. This means that the values represent the pitch, yaw and roll of the head, respectively. The zero rotation (values 0, 0, 0) corresponds to the face looking straight ahead along the camera axis. Positive values for pitch correspond to head turning down. Positive values for yaw correspond to head turning right in the input image. Positive values for roll correspond to head rolling to the left in the input image. The values are in radians.
This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will fill rotation array with 0 values.
Parameters:
Name Type Description
rotation Array 3D array in which the head rotation values will be stored.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getFP_START_GROUP_INDEX() → {Number}

Index of the first feature point group.
Returns:
2
Type
Number

<static> _getFP_END_GROUP_INDEX() → {Number}

Index of the last feature point group.
Returns:
15
Type
Number

<static> _getGroupSizes(groupSizeArray, length)

Get the size of all feature point groups.
Parameters:
Name Type Description
groupSizeArray Array array in which the size of each group will be stored.
length Number length of groupSizeArray array.

<static> _getFeaturePoints3DRelative(N, groups, indices, positions3D, defined, detected, quality, faceIndex)

Returns the 3D coordinates relative to the face origin, placed at the center between eyes in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
Returned coordinates are in the local coordinate system of the face, with the origin (0,0,0) placed at the center between the eyes. The x-axis points laterally towards the side of the face, y-xis points up and z-axis points into the face.
Function returns position for N feature points in the N*3-dimensional positions array, an indication of whether the point is defined as 0 and 1 in the N-dimensional defined array and detected as 0 and 1 in the N-dimensional detected array. The quality of each point as value between 0 and 1 is returned in quality array. For each point, its group and index are specifically listed in the groups and indices arrays.
An example of use:

	// an example for points 2.1, 8.3 and 8.4
	const int N;
	int[] groups = new int[N] {2, 8, 8};
	int[] indices = new int[N]{ 1, 3, 4 };
	float[] positions3D = new float[3*N];
	int[] defined = new int[N];
	int[] detected = new int[N];
	float[] quality = new float[N];

	for (int faceIndex = 0; faceIndex < MAX_FACES; faceIndex++)
	{
		VisageTrackerNative._getFeaturePoints3DRelative(N, groups, indices, positions3D, defined, detected, quality, faceIndex);
	}

NOTE: quality information is returned exclusively within _getFeaturePoints2D() function. Therefore, this function will always return -1 for the quality of each point.

Parameters:
Name Type Description
N Number number of feature points.
groups Array N-dimensional array of the groups of the feature points.
indices Array N-dimensional array of the indices of the feature points within the groups.
positions3D Array N*3-dimensional array filled with resulting facial feature points positions.
defined Array N-dimensional array filled for every feature point with a value that indicates whether the point is defined. Value 1 is assigned to defined points, and 0 to undefined points.
detected Array N-dimensional array filled for every feature point with a value that indicates whether the point is detected. Value 1 is assigned to detected points, and 0 to undetected points.
quality Array N-dimensional array filled for every feature point with a value from 0 to 1, where 0 is the worst and 1 is the best quality.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getFeaturePoints3D(N, groups, indices, positions3D, defined, detected, quality, faceIndex)

Returns the global 3D feature point position in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
The coordinate system is such that when looking towards the camera, the direction of x is to the left, y iz up, and z points towards the viewer. The global origin (0,0,0) is placed at the camera.
Function returns position for N feature points in the N*3-dimensional positions array, an indication of whether the point is defined as 0 and 1 in the N-dimensional defined array and detected as 0 and 1 in the N-dimensional detected array. The quality of each point as value between 0 and 1 is returned in quality array. For each point, its group and index are specifically listed in the groups and indices arrays.
An example of use:

	// an example for points 2.1, 8.3 and 8.4
	const int N;
	int[] groups = new int[N] {2, 8, 8};
	int[] indices = new int[N]{ 1, 3, 4 };
	float[] positions3D = new float[3*N];
	int[] defined = new int[N];
	int[] detected = new int[N];
	float[] quality = new float[N];

	for (int faceIndex = 0; faceIndex < MAX_FACES; faceIndex++)
	{
		VisageTrackerNative._getFeaturePoints3D(N, groups, indices, positions3D, defined, detected, quality, faceIndex);
	}

NOTE: quality information is returned exclusively within _getFeaturePoints2D() function. Therefore, this function will always return -1 for the quality of each point.

Parameters:
Name Type Description
N Number number of feature points.
groups Array N-dimensional array of the groups of the feature points.
indices Array N-dimensional array of the indices of the feature points within the groups.
positions3D Array N*3-dimensional array filled with resulting facial feature points positions.
defined Array N-dimensional array filled for every feature point with a value that indicates whether the point is defined. Value 1 is assigned to defined points, and 0 to undefined points.
detected Array N-dimensional array filled for every feature point with a value that indicates whether the point is detected. Value 1 is assigned to detected points, and 0 to undetected points.
quality Array N-dimensional array filled for every feature point with a value from 0 to 1, where 0 is the worst and 1 is the best quality.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getFeaturePoints2D(N, groups, indices, positions2D, defined, detected, quality, faceIndex)

Returns the feature points position in normalized 2D screen coordinates, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
The 2D feature point coordinates are normalised to image size so that the lower left corner of the image has coordinates 0,0 and upper right corner 1,1.
Function returns position for N feature points in the N*2-dimensional positions array, an indication of whether the point is defined as 0 and 1 in the N-dimensional defined array and detected as 0 and 1 in the N-dimensional detected array. The quality of each point as value between 0 and 1 is returned in quality array. For each point, its group and index are specifically listed in the groups and indices arrays.
An example of use:

	// an example for points 2.1, 8.3 and 8.4
	const int N;
	int[] groups = new int[N] {2, 8, 8};
	int[] indices = new int[N]{ 1, 3, 4 };
	float[] positions2D = new float[2*N];
	int[] defined = new int[N];
	int[] detected = new int[N];
	float[] quality = new float[N];

	for (int faceIndex = 0; faceIndex < MAX_FACES; faceIndex++)
	{
		VisageTrackerNative._getFeaturePoints2D(N, groups, indices, positions2D, defined, detected, quality, faceIndex);
	}

Parameters:
Name Type Description
N Number number of feature points.
groups Array N-dimensional array of the groups of the feature points.
indices Array N-dimensional array of the indices of the feature points within the groups.
positions2D Array N*2-dimensional array filled with resulting facial feature points positions.
defined Array N-dimensional array filled for every feature point with a value that indicates whether the point is defined. Value 1 is assigned to defined points, and 0 to undefined points.
detected Array N-dimensional array filled for every feature point with a value that indicates whether the point is detected. Value 1 is assigned to detected points, and 0 to undetected points.
quality Array N-dimensional array filled for every feature point with a value from 0 to 1, where 0 is the worst and 1 is the best quality.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getAllFeaturePoints3DRelative(featurePointArray, length, faceIndex)

Returns for all feature points the position in 3D coordinates relative to the face origin, placed at the center between eyes in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
Returned coordinates are in the local coordinate system of the face, with the origin (0,0,0) placed at the center between the eyes. The x-axis points laterally towards the side of the face, y-xis points up and z-axis points into the face.
For example, first feature point is 2.1, therefore first six values in featurePointArray array will be filled with the following values:

	featurePointArray = [x_(2.1), y_(2.1), z_(2.1), defined_(2.1), detected_(2.1), quality_(2.1), ...]

An example of use in Unity:

	//array which will be filled with values
	public float[] featurePointArray = new float[1000];
	
	//call of the _getAllFeaturePoints3DRelative() function
	VisageTrackerNative._getAllFeaturePoints3DRelative(featurePointArray, featurePointArray.Length, faceIndex);
	
NOTE: quality information is returned exclusively within _getAllFeaturePoints2D() function. Therefore, this function will always return -1 for the quality of each point.
Parameters:
Name Type Description
featurePointArray Array array in which the position in x, y and z coordinates, indication whether the point is defined and detected and feature point quality will be stored.
length Number length of featurePointArray.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getAllFeaturePoints3D(featurePointArray, length, faceIndex)

Returns for all feature points the global 3D position in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
The coordinate system is such that when looking towards the camera, the direction of x is to the left, y iz up, and z points towards the viewer. The global origin (0,0,0) is placed at the camera.
For example, first feature point is 2.1, therefore first six values in featurePointArray array will be filled with the following values:

	featurePointArray = [x_(2.1), y_(2.1), z_(2.1), defined_(2.1), detected_(2.1), quality_(2.1), ...]
An example of use in Unity:

	//array which will be filled with values
	public float[] featurePointArray = new float[1000];
	
	//call of the _getAllFeaturePoints3D() function
	VisageTrackerNative._getAllFeaturePoints3D(featurePointArray, featurePointArray.Length, faceIndex);

NOTE: quality information is returned exclusively within _getAllFeaturePoints2D() function. Therefore, this function will always return -1 for the quality of each point.
Parameters:
Name Type Description
featurePointArray Array array in which the position in x, y and z coordinates, indication whether the point is defined and detected and feature point quality will be stored.
length Number length of featurePointArray.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getAllFeaturePoints2D(featurePointArray, length, faceIndex)

Returns for all feature points the position in normalized 2D screen coordinates, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
The 2D feature point coordinates are normalised to image size so that the lower left corner of the image has coordinates 0,0 and upper right corner 1,1.
For example, first feature point is 2.1, therefore first five values in featurePointArray array will be filled with the following values:

	featurePointArray = [x_(2.1), y_(2.1), defined_(2.1), detected_(2.1), quality_(2.1), ...]
An example of use in Unity:

	//array which will be filled with values
	public float[] featurePointArray = new float[1000];
	
	//call of the _getAllFeaturePoints2D() function
	VisageTrackerNative._getAllFeaturePoints2D(featurePointArray, featurePointArray.Length, faceIndex);

Parameters:
Name Type Description
featurePointArray Array array in which the position in normalized 2D screen coordinates, indication whether the point is defined and detected and feature point quality will be stored.
length Number length of featurePointArray.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getFaceModelVertexCount() → {Number}

Number of vertices in the 3D face model.
This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will be set to 0.
Returns:
If tracker is tracking, returns vertex count, otherwise 0.
Type
Number

<static> _getFaceModelVertices(vertices, faceIndex)

Array of vertex coordinates of the 3D face model.
The coordinates are in the local coordinate system of the face, with the origin (0,0,0) placed at the center between the eyes. The x-axis points laterally towards the side of the face, y-axis points up and z-axis points into the face.
Function returns vertices position in the vertexNumber*3-dimensional vertices array, where vertexNumber is value obtained from _getFaceModelVertexCount() function, for the particular faceIndex.
An example of use:

	//define MaxVertices value that will be larger than the real number of vertices
	public const int MaxVertices = 1024;
	private float[] vertices = new float[MaxVertices * 3];
	
	//call of _getFaceModelVertices() function
	 VisageTrackerNative._getFaceModelVertices(vertices, faceIndex);
	

Parameters:
Name Type Description
vertices Array array in which the values of vertex coordinates will be stored.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getFaceModelVerticesProjected(verticesProjected, faceIndex)

Array of projected (image space) vertex coordinates of the 3D face model.
The 2D coordinates are normalised to image size so that the lower left corner of the image has coordinates 0,0 and upper right corner 1,1.
Function returns vertices position in the vertexNumber*2-dimensional verticesProjected array, where vertexNumber is value obtained from _getFaceModelVertexCount() function, for the particular faceIndex.
An example of use:

	//define MaxVertices value that will be larger than the real number of vertices
	public const int MaxVertices = 1024;
	private float[] verticesProjected = new float[MaxVertices * 2];
	
	//call of _getFaceModelVerticesProjected() function
	 VisageTrackerNative._getFaceModelVerticesProjected(verticesProjected, faceIndex);

Parameters:
Name Type Description
verticesProjected Array array in which the values of projected vertex coordinates will be stored.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getFaceModelTriangleCount() → {Number}

Number of triangles in the 3D face model. This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will be set to 0.
Returns:
If tracker is tracking, returns triangle count, otherwise 0.
Type
Number

<static> _getFaceModelTriangles(triangles, faceIndex)

Triangles array for the 3D face model.
Each triangle is described by three indices into the list of vertices (see _getFaceModelVertices() function).
Function returns triangles coordinates in the triangleNumber*3-dimensional triangles array, where triangleNumber is value obtained from _getFaceModelTriangleCount() function, for the particular faceIndex. An example of use:

	//define MaxTriangles value that will be larger than the real number of triangles
	public const int MaxTriangles = 2048;
	private int[] triangles = new int[MaxTriangles * 3];
	
	//call of _getFaceModelTriangles() function
	VisageTrackerNative._getFaceModelTriangles(triangles, faceIndex);
	
Parameters:
Name Type Description
triangles Array array in which values of triangle coordinates will be stored.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getFaceModelTextureCoords(texCoord, faceIndex)

Texture coordinates for the 3D face model. A pair of u, v coordinates for each vertex.
Function returns texture coordinates in the vertexNumber*2-dimensional texCoord array, where vertexNumber is value obtained from _getFaceModelVertices() function, for the particular faceIndex.
An example of use:

	//define MaxVertices value that will be larger than the real number of vertices
	public const int MaxVertices = 1024;
	private float[] texCoords = new float[MaxVertices * 2];
	
	//call of _getFaceModelTextureCoords() function
	VisageTrackerNative._getFaceModelTextureCoords(texCoord, faceIndex);
	
Parameters:
Name Type Description
texCoord Array array in which values of texture coordinates will be stored.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _setIPD()

Sets the inter pupillary distance.

<static> _getIPD() → {Number}

Returns the current inter pupillary distance (IPD) setting.
Returns:
Current setting of inter pupillary distance (IPD) in meters.
Type
Number

<static> _setTrackerConfigurationFile(trackerConfigFile, au_fitting_disabled, mesh_fitting_disabled)

Sets tracking configuration.
Parameters:
Name Type Description
trackerConfigFile string the name of the tracker configuration file.
au_fitting_disabled boolean disables the use of the 3D model used to estimate action units. If set to true, estimation of action units shall not be performed, and action units related data will not be available (_getActionUnits() etc.). Disabling will result in a small performance gain.
mesh_fitting_disabled boolean disables the use of the fine 3D mesh. If set to true, the 3D mesh shall not be fitted and the related information shall not be available (_getFaceModelVertices() etc.). Disabling will result in a small performance gain.

<static> _getTrackingQuality(faceIndex) → {Number}

Returns tracking quality level.
This variable is set while tracker is running. If tracking status is TRACK_STAT_OFF, the return value will be -1.
Parameters:
Name Type Description
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.
Returns:
Estimated tracking quality level for the current frame. The value is between 0 and 1.
Type
Number

<static> _getFrameRate() → {Number}

Returns the frame rate of the tracker, in frames per second, measured over last 10 frames.
This variable is set while tracker is running. If tracking status is TRACK_STAT_OFF, the return value will be -1.
Returns:
The frame rate of the tracker.
Type
Number

<static> _getTimeStamp() → {number}

Returns timestamp of the current video frame.
This variable is set while tracker is running. If tracking status is TRACK_STAT_OFF, the return value will be -1.
Returns:
Timestamp of the current video frame.
Type
number

<static> _getActionUnitCount() → {Number}

Number of facial action units. This variable is set only while tracker is tracking (TRACK_STAT_OK) and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details), otherwise it will be set to 0. Number of action units that are defined for current face model.
Returns:
Number of action units.
Type
Number

<static> _getGazeQuality(faceIndex) → {Number}

The session level gaze tracking quality. Quality is returned as a value from 0 to 1, where 0 is the worst and 1 is the best quality. The quality is 0 also when the gaze tracking is off or calibrating. This variable is set while tracker is running. If tracking status for given faceIndex is TRACK_STAT_OFF, the return value will be -1.
Parameters:
Name Type Description
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.
Returns:
Value from 0 to 1.
Type
Number

<static> _getScreenSpaceGazeData(index, gaze, quality, inState, faceIndex)

ScreenSpaceGazeData is a container structure used to hold the gaze point location, current state of screen space gaze tracking system and gaze tracking estimation quality parameters for the current frame. This class is used both to store the calibration points during the calibration phase and the estimation results during the estimation phase. ScreenSpaceGazeData will be returned in index, gaze, quality, inState parameters for the given faceIndex.
Parameters:
Name Type Description
index Number index of the video frame.
gaze Array coordinates of screen space gaze normalized to screen width and height. x = 0 corresponds to the left-most point and 1 to the right-most points of the screen. Defaults to 0.5. y = 0 corresponds to the top-most point and 1 to the bottom-most points of the screen. Defaults to 0.5.
quality Number the frame level gaze tracking quality.
inState Number Flag indicating the state of the gaze estimator for the current frame. If inState is 0 the estimator is off and position is default. If inState is 1 the estimator is calibrating and returns calibration data for the current frame. If inState is 2 the estimator is estimating and returns the estimated screen space gaze coordinates. If inState is -1 the estimator is calibrating but the tracking/screen space gaze estimation failed for the current frame (position is default) If inState is -2 the estimator is estimating but the tracking/screen space gaze estimation failed for the current frame (position is default)
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getFaceScale(faceIndex) → {Number}

Scale of facial bounding box. This variable is set while tracker is running. If tracking status for given faceIndex is TRACK_STAT_OFF, the return value will be -1.
Parameters:
Name Type Description
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.
Returns:
Value between 0.0 and 1.0.
Type
Number

<static> _getEyeClosure(closure, faceIndex)

Returns discrete eye closure value.
Function returns eye closure value in the 2-dimensional closure. An example of use:

	
	private int[] closure = new int[2];
	
	//call of _getEyeClosure() function
	 VisageTrackerNative._getEyeClosure(closure, faceIndex);
	
  
Parameters:
Name Type Description
closure Array array in which the eye closure values will be stored.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getIrisRadius(radius, faceIndex)

Returns iris radius values in px for the given faceIndex.
Function returns iris radius value in the 2-dimensional radius. An example of use:

	
	private float[] radius = new float[2];
	
	//call of _getIrisRadius() function
	 VisageTrackerNative._getIrisRadius(radius, faceIndex);
	
  
Parameters:
Name Type Description
radius Array array in which the iris radius values will be stored.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getActionUnitsNames(auIndex) → {String}

Name of facial Action Unit at given index. This variable is set only while tracker is tracking (TRACK_STAT_OK) and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details), otherwise the return value will be an empty string.
Parameters:
Name Type Description
auIndex Number index of specific action unit in the current tracker configuration.
Returns:
Name of the action unit for a given index according to the configuration used
Type
String

<static> _getActionUnits(actionUnits, faceIndex)

List of current values for facial action units, one value for each action unit. This variable is set only while tracker is tracking (TRACK_STAT_OK) and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details). The action units used by the tracker are defined in the 3D face model file.
Function returns actionUnits values in the auCount-dimensional actionUnits array, where auCount is value obtained from _getActionUnitCount() function, for the particular faceIndex. An example of use:

	//define auCount value that will be larger than the real number of action units
	public const int auCount = 1024;
	private float[] actionUnits = new float[AUCOUNT];
	
	//call of _getActionUnits() function
	 VisageTrackerNative._getActionUnits(actionUnits, faceIndex);
	
Parameters:
Name Type Description
actionUnits Array array in which the action unit values will be stored.
faceIndex Number Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getActionUnitsUsed(auIndex) → {Boolean}

Used facial Action Units. This variable is set only while tracker is tracking (TRACK_STAT_OK) and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details).
Parameters:
Name Type Description
auIndex Number index of specific action unit in the current tracker configuration.
Returns:
True if action unit is used, false if action unit is not used.
Type
Boolean

<static> _getGazeDirectionGlobal(gazeDirection, faceIndex)

Global gaze direction, taking into account both head pose and eye rotation. This variable is set only while tracker is tracking (TRACK_STAT_OK). This is the current estimated gaze direction relative to the camera axis. Direction is expressed with three values determining the rotations around the three axes x, y and z, i.e. pitch, yaw and roll. Values (0, 0, 0) correspond to the gaze direction parallel to the camera axis. Positive values for pitch correspond to gaze turning down. Positive values for yaw correspond to gaze turning right in the input image. Positive values for roll correspond to face rolling to the left in the input image, see illustration below.
The values are in radians.
Function returns gaze direction values in the 3-dimensional gazeDirection array. An example of use:

	
	private float[] gazeDirection = new float[3];
	
	//call of _getGazeDirectionGlobal() function
	 VisageTrackerNative._getGazeDirectionGlobal(gazeDirection, faceIndex);
	
Parameters:
Name Type Description
gazeDirection Array 3D array in which the gaze direction values will be stored.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.

<static> _getGazeDirection(gazeDirection, faceIndex)

Gaze direction. This variable is set only while tracker is tracking (TRACK_STAT_OK). This is the current estimated gaze direction relative to the person's head. Direction is expressed with two values x and y, in radians. Values (0, 0) correspond to person looking straight. X is the horizontal rotation with positive values corresponding to person looking to his/her left.
Y is the vertical rotation with positive values corresponding to person looking down. Function returns gaze direction values in the 2-dimensional gazeDirection array. An example of use:

	
	private float[] gazeDirection = new float[2];
	
	//call of _getGazeDirection() function
	 VisageTrackerNative._getGazeDirection(gazeDirection, faceIndex);
	
Parameters:
Name Type Description
gazeDirection Array 2D array in which the gaze direction values will be stored.
faceIndex Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.