The VisageTrackerUnityPlugin is a native plugin (see Native Plugins) written in JavaScript for Unity. The plugin exposes face tracking features of visage|SDK for use in Unity WebGL build.
The plugin code is placed in VisageTrackerUnityPlugin.jslib file and can be found in /lib folder.
Note: This version of the plugin is adapted to Unity version 2019.1, while for all older versions of Unity unityInstance in plugin's functions has to be renamed to gameInstance.
Dependencies
The plugin depends on visageSDK.js file that loads visageSDK.data. Therefore, visageSDK.js has to be included in WebGL build output file - index.html. Additional information on changing the location of the data file can be seen on the following link: VisageTracker DependenciesUsage
For the functions to be accessible from C#, a C# wrapper interface is used. The example can be seen in the following function:
[DllImport("__Internal")]
public static extern void _openCamera(int width, int height, int mirrored, string onSucessCallbackCamera, string onErrorCallbackCamera);
The usage is demonstrated in VisageTrackerUnityDemo sample, VisageTrackerNativ.HTML5.cs file. All C# scripts are located in Assets/Scripts folder of the project.
In order to use VisageTrackerUnity plugin in Unity projects visageSDK.js must be included in output index.html file (see link).
Callbacks
Generally, it is common to use callback functions in JavaScript, due to its asynchronous nature.Within the plugin, some functions are implemented so they expect a callback function name as a parameter. Callback functions are defined in C# code. Examples can be seen in _getCameraInfo function.
Methods
-
<static> _preloadFile(fileURL)
-
Stores urls and names of the files that need to be preloaded into the file system. It is expected that the url for each file will be relative to the main index.html file position. It has to be called before using any of the functionalities from visage|SDK. The recommendation for calling this function is within the function Awake() in Unity.
Parameters:
Name Type Description fileURL
string the name and path to the file. -
<static> _initializeLicense(license)
-
Initializes the license. It is recommended to initialize the license in the Awake() function within the unity script.
Parameters:
Name Type Description license
String license url as relative path from the main index.html file to the directory that contains license file(s) (e.g. licenseFilePath = "StreamingAssets/Visage Tracker/471-308-776-250-553-231-598-221-624-198-522.vlc";). -
<static> _initTracker(config, numFaces, callback)
-
Initializes VisageTracker.
To change the configuration file, _initTracker function should be called again. Callback function is called once the tracker is initialized.
Parameter config is string that represent the path to the tracker configuration file.
Example, if the configuration file is located in StreamingAssets/Visage Tracker folder:
Parameter callback is name of function defined in Unity script. An example of a callback function definition and _initTracker() function call:configFilePath = "StreamingAssets/Visage Tracker/Head Tracker.cfg";
//callback function public void CallbackInitTracker() { Debug.Log("TrackerInited"); trackerInited = true; } //call of the _initTracker() function: VisageTrackerNative._initTracker(config, MAX_FACES, "CallbackInitTracker");
Parameters:
Name Type Description config
string the name and path to the tracker configuration file (.cfg; default configuration file is provided in lib folder; for further details see VisageTracker Configuration Manual). numFaces
Number The maximum number of faces that will be tracked. callback
string the name of the callback function. -
<static> _openCamera(camWidth, camHeight, isMirrored, onSuccessCallbackCamera, onErrorCallbackCamera)
-
Closes camera if it is already opened.
If the camera is initialized successfully onSuccessCallbackCamera callback function is called, otherwise onErrorCallbackCamera callback function is called. Mirrors the frame if isMirrored parameter is set to 1.
Parameters:
Name Type Description camWidth
Number camera frame width. camHeight
Number camera frame height. isMirrored
Number 1 or 0; if set to 1 frames will be mirrored. onSuccessCallbackCamera
String the name of the callback function that is called if camera is successfully initalized. onErrorCallbackCamera
String the name of the callback function that is called if initialization camera fails. -
<static> _closeCamera() → {Boolean}
-
Stops camera streaming.
Returns:
If camera is already closed returns false, otherwise returns true.- Type
- Boolean
-
<static> _grabFrame()
-
Captures current frame to be used for face tracking and binding on given texture. Needs to be called before _track and _bindTexture functions.
-
<static> _getTrackerStatus(trackStatus, length)
-
Returns tracker status.
Parameters:
Name Type Description trackStatus
Array array that will be filled with tracking statuses for each of the tracked faces. length
Number length of trackStatus array, should be equal to the number of faces given to the _initTracker() function(smaller of equal to 10). -
<static> _track()
-
Performs face tracking on the captured frame (see _grabFrame) and returns tracking status. _openCamera and _initTracker functions need to succeed before calling _track.
-
<static> _bindTexture(texture)
-
Update the passed texture with the current captured frame image data (see _grabFrame). The function expects the pointer to the created texture to be passed.
Example:private Texture2D texture = null; texture = new Texture2D(TexWidth, TexHeight, TextureFormat.RGBA32, false); VisageTrackerNative._bindTexture(texture.GetNativeTexturePtr());
Parameters:
Name Type Description texture
Number Pointer to an texture. -
<static> _getCameraInfo(camFocus, imgWidth, imgHeight)
-
Assigns values to the given camera info parameters.
Parameters:
Name Type Description camFocus
Number parameter to which focal distance of the camera will be assigned. imgWidth
Number parameter to which width of the frame will be assigned. imgHeight
Number parameter to which height of the frame will be assigned. -
<static> _getHeadTranslation(translation, faceIndex)
-
Translation of the head from the camera. Parameter translatation is 3-dimensional array which will be filled x, y and z head translation coordinates for the particular face given with faceIndex parameter. The coordinate system is such that when looking towards the camera, the direction of x is to the left, y iz up, and z points towards the viewer. The global origin (0,0,0) is placed at the camera. The reference point on the head is in the center between the eyes. This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will fill translation array with -10000 values.
Parameters:
Name Type Description translation
Array 3D array in which the head translation values will be stored. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getHeadRotation(rotation, faceIndex)
-
Rotation of the head.
Parameter rotation is three-dimensional array which will be filled x, y and z head rotation coordinates for the particular face given with faceIndex parameter.
This is the estimated rotation of the head, in radians. Rotation is expressed with three values determining the rotations around the three axes x, y and z, in radians. This means that the values represent the pitch, yaw and roll of the head, respectively. The zero rotation (values 0, 0, 0) corresponds to the face looking straight ahead along the camera axis. Positive values for pitch correspond to head turning down. Positive values for yaw correspond to head turning right in the input image. Positive values for roll correspond to head rolling to the left in the input image. The values are in radians.
This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will fill rotation array with 0 values.Parameters:
Name Type Description rotation
Array 3D array in which the head rotation values will be stored. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getFP_START_GROUP_INDEX() → {Number}
-
Index of the first feature point group.
Returns:
2
- Type
- Number
-
<static> _getFP_END_GROUP_INDEX() → {Number}
-
Index of the last feature point group.
Returns:
15
- Type
- Number
-
<static> _getGroupSizes(groupSizeArray, length)
-
Get the size of all feature point groups.
Parameters:
Name Type Description groupSizeArray
Array array in which the size of each group will be stored. length
Number length of groupSizeArray array. -
<static> _getFeaturePoints3DRelative(N, groups, indices, positions3D, defined, detected, quality, faceIndex)
-
Returns the 3D coordinates relative to the face origin, placed at the center between eyes in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
Returned coordinates are in the local coordinate system of the face, with the origin (0,0,0) placed at the center between the eyes. The x-axis points laterally towards the side of the face, y-xis points up and z-axis points into the face.
Function returns position for N feature points in the N*3-dimensional positions array, an indication of whether the point is defined as 0 and 1 in the N-dimensional defined array and detected as 0 and 1 in the N-dimensional detected array. The quality of each point as value between 0 and 1 is returned in quality array. For each point, its group and index are specifically listed in the groups and indices arrays.
An example of use:// an example for points 2.1, 8.3 and 8.4 const int N; int[] groups = new int[N] {2, 8, 8}; int[] indices = new int[N]{ 1, 3, 4 }; float[] positions3D = new float[3*N]; int[] defined = new int[N]; int[] detected = new int[N]; float[] quality = new float[N]; for (int faceIndex = 0; faceIndex < MAX_FACES; faceIndex++) { VisageTrackerNative._getFeaturePoints3DRelative(N, groups, indices, positions3D, defined, detected, quality, faceIndex); } NOTE: quality information is returned exclusively within _getFeaturePoints2D() function. Therefore, this function will always return -1 for the quality of each point.
Parameters:
Name Type Description N
Number number of feature points. groups
Array N-dimensional array of the groups of the feature points. indices
Array N-dimensional array of the indices of the feature points within the groups. positions3D
Array N*3-dimensional array filled with resulting facial feature points positions. defined
Array N-dimensional array filled for every feature point with a value that indicates whether the point is defined. Value 1 is assigned to defined points, and 0 to undefined points. detected
Array N-dimensional array filled for every feature point with a value that indicates whether the point is detected. Value 1 is assigned to detected points, and 0 to undetected points. quality
Array N-dimensional array filled for every feature point with a value from 0 to 1, where 0 is the worst and 1 is the best quality. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getFeaturePoints3D(N, groups, indices, positions3D, defined, detected, quality, faceIndex)
-
Returns the global 3D feature point position in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
The coordinate system is such that when looking towards the camera, the direction of x is to the left, y iz up, and z points towards the viewer. The global origin (0,0,0) is placed at the camera.
Function returns position for N feature points in the N*3-dimensional positions array, an indication of whether the point is defined as 0 and 1 in the N-dimensional defined array and detected as 0 and 1 in the N-dimensional detected array. The quality of each point as value between 0 and 1 is returned in quality array. For each point, its group and index are specifically listed in the groups and indices arrays.
An example of use:// an example for points 2.1, 8.3 and 8.4 const int N; int[] groups = new int[N] {2, 8, 8}; int[] indices = new int[N]{ 1, 3, 4 }; float[] positions3D = new float[3*N]; int[] defined = new int[N]; int[] detected = new int[N]; float[] quality = new float[N]; for (int faceIndex = 0; faceIndex < MAX_FACES; faceIndex++) { VisageTrackerNative._getFeaturePoints3D(N, groups, indices, positions3D, defined, detected, quality, faceIndex); } NOTE: quality information is returned exclusively within _getFeaturePoints2D() function. Therefore, this function will always return -1 for the quality of each point.
Parameters:
Name Type Description N
Number number of feature points. groups
Array N-dimensional array of the groups of the feature points. indices
Array N-dimensional array of the indices of the feature points within the groups. positions3D
Array N*3-dimensional array filled with resulting facial feature points positions. defined
Array N-dimensional array filled for every feature point with a value that indicates whether the point is defined. Value 1 is assigned to defined points, and 0 to undefined points. detected
Array N-dimensional array filled for every feature point with a value that indicates whether the point is detected. Value 1 is assigned to detected points, and 0 to undetected points. quality
Array N-dimensional array filled for every feature point with a value from 0 to 1, where 0 is the worst and 1 is the best quality. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getFeaturePoints2D(N, groups, indices, positions2D, defined, detected, quality, faceIndex)
-
Returns the feature points position in normalized 2D screen coordinates, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
The 2D feature point coordinates are normalised to image size so that the lower left corner of the image has coordinates 0,0 and upper right corner 1,1.
Function returns position for N feature points in the N*2-dimensional positions array, an indication of whether the point is defined as 0 and 1 in the N-dimensional defined array and detected as 0 and 1 in the N-dimensional detected array. The quality of each point as value between 0 and 1 is returned in quality array. For each point, its group and index are specifically listed in the groups and indices arrays.
An example of use:// an example for points 2.1, 8.3 and 8.4 const int N; int[] groups = new int[N] {2, 8, 8}; int[] indices = new int[N]{ 1, 3, 4 }; float[] positions2D = new float[2*N]; int[] defined = new int[N]; int[] detected = new int[N]; float[] quality = new float[N]; for (int faceIndex = 0; faceIndex < MAX_FACES; faceIndex++) { VisageTrackerNative._getFeaturePoints2D(N, groups, indices, positions2D, defined, detected, quality, faceIndex); }
Parameters:
Name Type Description N
Number number of feature points. groups
Array N-dimensional array of the groups of the feature points. indices
Array N-dimensional array of the indices of the feature points within the groups. positions2D
Array N*2-dimensional array filled with resulting facial feature points positions. defined
Array N-dimensional array filled for every feature point with a value that indicates whether the point is defined. Value 1 is assigned to defined points, and 0 to undefined points. detected
Array N-dimensional array filled for every feature point with a value that indicates whether the point is detected. Value 1 is assigned to detected points, and 0 to undetected points. quality
Array N-dimensional array filled for every feature point with a value from 0 to 1, where 0 is the worst and 1 is the best quality. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getAllFeaturePoints3DRelative(featurePointArray, length, faceIndex)
-
Returns for all feature points the position in 3D coordinates relative to the face origin, placed at the center between eyes in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
Returned coordinates are in the local coordinate system of the face, with the origin (0,0,0) placed at the center between the eyes. The x-axis points laterally towards the side of the face, y-xis points up and z-axis points into the face.
For example, first feature point is 2.1, therefore first six values in featurePointArray array will be filled with the following values:featurePointArray = [x_(2.1), y_(2.1), z_(2.1), defined_(2.1), detected_(2.1), quality_(2.1), ...]
An example of use in Unity:
NOTE: quality information is returned exclusively within _getAllFeaturePoints2D() function. Therefore, this function will always return -1 for the quality of each point.//array which will be filled with values public float[] featurePointArray = new float[1000]; //call of the _getAllFeaturePoints3DRelative() function VisageTrackerNative._getAllFeaturePoints3DRelative(featurePointArray, featurePointArray.Length, faceIndex);
Parameters:
Name Type Description featurePointArray
Array array in which the position in x, y and z coordinates, indication whether the point is defined and detected and feature point quality will be stored. length
Number length of featurePointArray. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getAllFeaturePoints3D(featurePointArray, length, faceIndex)
-
Returns for all feature points the global 3D position in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
The coordinate system is such that when looking towards the camera, the direction of x is to the left, y iz up, and z points towards the viewer. The global origin (0,0,0) is placed at the camera.
For example, first feature point is 2.1, therefore first six values in featurePointArray array will be filled with the following values:
An example of use in Unity:featurePointArray = [x_(2.1), y_(2.1), z_(2.1), defined_(2.1), detected_(2.1), quality_(2.1), ...]
NOTE: quality information is returned exclusively within _getAllFeaturePoints2D() function. Therefore, this function will always return -1 for the quality of each point.//array which will be filled with values public float[] featurePointArray = new float[1000]; //call of the _getAllFeaturePoints3D() function VisageTrackerNative._getAllFeaturePoints3D(featurePointArray, featurePointArray.Length, faceIndex);
Parameters:
Name Type Description featurePointArray
Array array in which the position in x, y and z coordinates, indication whether the point is defined and detected and feature point quality will be stored. length
Number length of featurePointArray. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getAllFeaturePoints2D(featurePointArray, length, faceIndex)
-
Returns for all feature points the position in normalized 2D screen coordinates, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
The 2D feature point coordinates are normalised to image size so that the lower left corner of the image has coordinates 0,0 and upper right corner 1,1.
For example, first feature point is 2.1, therefore first five values in featurePointArray array will be filled with the following values:
An example of use in Unity:featurePointArray = [x_(2.1), y_(2.1), defined_(2.1), detected_(2.1), quality_(2.1), ...]
//array which will be filled with values public float[] featurePointArray = new float[1000]; //call of the _getAllFeaturePoints2D() function VisageTrackerNative._getAllFeaturePoints2D(featurePointArray, featurePointArray.Length, faceIndex);
Parameters:
Name Type Description featurePointArray
Array array in which the position in normalized 2D screen coordinates, indication whether the point is defined and detected and feature point quality will be stored. length
Number length of featurePointArray. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getFaceModelVertexCount() → {Number}
-
Number of vertices in the 3D face model.
This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will be set to 0.Returns:
If tracker is tracking, returns vertex count, otherwise 0.
- Type
- Number
-
<static> _getFaceModelVertices(vertices, faceIndex)
-
Array of vertex coordinates of the 3D face model.
The coordinates are in the local coordinate system of the face, with the origin (0,0,0) placed at the center between the eyes. The x-axis points laterally towards the side of the face, y-axis points up and z-axis points into the face.
Function returns vertices position in the vertexNumber*3-dimensional vertices array, where vertexNumber is value obtained from _getFaceModelVertexCount() function, for the particular faceIndex.
An example of use://define MaxVertices value that will be larger than the real number of vertices public const int MaxVertices = 1024; private float[] vertices = new float[MaxVertices * 3]; //call of _getFaceModelVertices() function VisageTrackerNative._getFaceModelVertices(vertices, faceIndex);
Parameters:
Name Type Description vertices
Array array in which the values of vertex coordinates will be stored. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getFaceModelVerticesProjected(verticesProjected, faceIndex)
-
Array of projected (image space) vertex coordinates of the 3D face model.
The 2D coordinates are normalised to image size so that the lower left corner of the image has coordinates 0,0 and upper right corner 1,1.
Function returns vertices position in the vertexNumber*2-dimensional verticesProjected array, where vertexNumber is value obtained from _getFaceModelVertexCount() function, for the particular faceIndex.
An example of use://define MaxVertices value that will be larger than the real number of vertices public const int MaxVertices = 1024; private float[] verticesProjected = new float[MaxVertices * 2]; //call of _getFaceModelVerticesProjected() function VisageTrackerNative._getFaceModelVerticesProjected(verticesProjected, faceIndex);
Parameters:
Name Type Description verticesProjected
Array array in which the values of projected vertex coordinates will be stored. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getFaceModelTriangleCount() → {Number}
-
Number of triangles in the 3D face model. This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will be set to 0.
Returns:
If tracker is tracking, returns triangle count, otherwise 0.
- Type
- Number
-
<static> _getFaceModelTriangles(triangles, faceIndex)
-
Triangles array for the 3D face model.
Each triangle is described by three indices into the list of vertices (see _getFaceModelVertices() function).
Function returns triangles coordinates in the triangleNumber*3-dimensional triangles array, where triangleNumber is value obtained from _getFaceModelTriangleCount() function, for the particular faceIndex. An example of use://define MaxTriangles value that will be larger than the real number of triangles public const int MaxTriangles = 2048; private int[] triangles = new int[MaxTriangles * 3]; //call of _getFaceModelTriangles() function VisageTrackerNative._getFaceModelTriangles(triangles, faceIndex);
Parameters:
Name Type Description triangles
Array array in which values of triangle coordinates will be stored. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getFaceModelTextureCoords(texCoord, faceIndex)
-
Texture coordinates for the 3D face model. A pair of u, v coordinates for each vertex.
Function returns texture coordinates in the vertexNumber*2-dimensional texCoord array, where vertexNumber is value obtained from _getFaceModelVertices() function, for the particular faceIndex.
An example of use://define MaxVertices value that will be larger than the real number of vertices public const int MaxVertices = 1024; private float[] texCoords = new float[MaxVertices * 2]; //call of _getFaceModelTextureCoords() function VisageTrackerNative._getFaceModelTextureCoords(texCoord, faceIndex);
Parameters:
Name Type Description texCoord
Array array in which values of texture coordinates will be stored. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _setIPD()
-
Sets the inter pupillary distance.
-
<static> _getIPD() → {Number}
-
Returns the current inter pupillary distance (IPD) setting.
Returns:
Current setting of inter pupillary distance (IPD) in meters.- Type
- Number
-
<static> _setTrackerConfigurationFile(trackerConfigFile, au_fitting_disabled, mesh_fitting_disabled)
-
Sets tracking configuration.
Parameters:
Name Type Description trackerConfigFile
string the name of the tracker configuration file. au_fitting_disabled
boolean disables the use of the 3D model used to estimate action units. If set to true, estimation of action units shall not be performed, and action units related data will not be available (_getActionUnits() etc.). Disabling will result in a small performance gain. mesh_fitting_disabled
boolean disables the use of the fine 3D mesh. If set to true, the 3D mesh shall not be fitted and the related information shall not be available (_getFaceModelVertices() etc.). Disabling will result in a small performance gain. -
<static> _getTrackingQuality(faceIndex) → {Number}
-
Returns tracking quality level.
This variable is set while tracker is running. If tracking status is TRACK_STAT_OFF, the return value will be -1.Parameters:
Name Type Description faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. Returns:
Estimated tracking quality level for the current frame. The value is between 0 and 1.- Type
- Number
-
<static> _getFrameRate() → {Number}
-
Returns the frame rate of the tracker, in frames per second, measured over last 10 frames.
This variable is set while tracker is running. If tracking status is TRACK_STAT_OFF, the return value will be -1.Returns:
The frame rate of the tracker.- Type
- Number
-
<static> _getTimeStamp() → {number}
-
Returns timestamp of the current video frame.
This variable is set while tracker is running. If tracking status is TRACK_STAT_OFF, the return value will be -1.Returns:
Timestamp of the current video frame.- Type
- number
-
<static> _getActionUnitCount() → {Number}
-
Number of facial action units. This variable is set only while tracker is tracking (TRACK_STAT_OK) and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details), otherwise it will be set to 0. Number of action units that are defined for current face model.
Returns:
Number of action units.- Type
- Number
-
<static> _getGazeQuality(faceIndex) → {Number}
-
The session level gaze tracking quality. Quality is returned as a value from 0 to 1, where 0 is the worst and 1 is the best quality. The quality is 0 also when the gaze tracking is off or calibrating. This variable is set while tracker is running. If tracking status for given faceIndex is TRACK_STAT_OFF, the return value will be -1.
Parameters:
Name Type Description faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. Returns:
Value from 0 to 1.- Type
- Number
-
<static> _getScreenSpaceGazeData(index, gaze, quality, inState, faceIndex)
-
ScreenSpaceGazeData is a container structure used to hold the gaze point location, current state of screen space gaze tracking system and gaze tracking estimation quality parameters for the current frame. This class is used both to store the calibration points during the calibration phase and the estimation results during the estimation phase. ScreenSpaceGazeData will be returned in index, gaze, quality, inState parameters for the given faceIndex.
Parameters:
Name Type Description index
Number index of the video frame. gaze
Array coordinates of screen space gaze normalized to screen width and height. x = 0 corresponds to the left-most point and 1 to the right-most points of the screen. Defaults to 0.5. y = 0 corresponds to the top-most point and 1 to the bottom-most points of the screen. Defaults to 0.5. quality
Number the frame level gaze tracking quality. inState
Number Flag indicating the state of the gaze estimator for the current frame. If inState is 0 the estimator is off and position is default. If inState is 1 the estimator is calibrating and returns calibration data for the current frame. If inState is 2 the estimator is estimating and returns the estimated screen space gaze coordinates. If inState is -1 the estimator is calibrating but the tracking/screen space gaze estimation failed for the current frame (position is default) If inState is -2 the estimator is estimating but the tracking/screen space gaze estimation failed for the current frame (position is default) faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getFaceScale(faceIndex) → {Number}
-
Scale of facial bounding box. This variable is set while tracker is running. If tracking status for given faceIndex is TRACK_STAT_OFF, the return value will be -1.
Parameters:
Name Type Description faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. Returns:
Value between 0.0 and 1.0.- Type
- Number
-
<static> _getEyeClosure(closure, faceIndex)
-
Returns discrete eye closure value.
Function returns eye closure value in the 2-dimensional closure. An example of use:private int[] closure = new int[2]; //call of _getEyeClosure() function VisageTrackerNative._getEyeClosure(closure, faceIndex);
Parameters:
Name Type Description closure
Array array in which the eye closure values will be stored. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getIrisRadius(radius, faceIndex)
-
Returns iris radius values in px for the given faceIndex.
Function returns iris radius value in the 2-dimensional radius. An example of use:private float[] radius = new float[2]; //call of _getIrisRadius() function VisageTrackerNative._getIrisRadius(radius, faceIndex);
Parameters:
Name Type Description radius
Array array in which the iris radius values will be stored. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getActionUnitsNames(auIndex) → {String}
-
Name of facial Action Unit at given index. This variable is set only while tracker is tracking (TRACK_STAT_OK) and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details), otherwise the return value will be an empty string.
Parameters:
Name Type Description auIndex
Number index of specific action unit in the current tracker configuration. Returns:
Name of the action unit for a given index according to the configuration used- Type
- String
-
<static> _getActionUnits(actionUnits, faceIndex)
-
List of current values for facial action units, one value for each action unit. This variable is set only while tracker is tracking (TRACK_STAT_OK) and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details). The action units used by the tracker are defined in the 3D face model file.
Function returns actionUnits values in the auCount-dimensional actionUnits array, where auCount is value obtained from _getActionUnitCount() function, for the particular faceIndex. An example of use://define auCount value that will be larger than the real number of action units public const int auCount = 1024; private float[] actionUnits = new float[AUCOUNT]; //call of _getActionUnits() function VisageTrackerNative._getActionUnits(actionUnits, faceIndex);
Parameters:
Name Type Description actionUnits
Array array in which the action unit values will be stored. faceIndex
Number Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getActionUnitsUsed(auIndex) → {Boolean}
-
Used facial Action Units. This variable is set only while tracker is tracking (TRACK_STAT_OK) and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details).
Parameters:
Name Type Description auIndex
Number index of specific action unit in the current tracker configuration. Returns:
True if action unit is used, false if action unit is not used.
- Type
- Boolean
-
<static> _getGazeDirectionGlobal(gazeDirection, faceIndex)
-
Global gaze direction, taking into account both head pose and eye rotation. This variable is set only while tracker is tracking (TRACK_STAT_OK). This is the current estimated gaze direction relative to the camera axis. Direction is expressed with three values determining the rotations around the three axes x, y and z, i.e. pitch, yaw and roll. Values (0, 0, 0) correspond to the gaze direction parallel to the camera axis. Positive values for pitch correspond to gaze turning down. Positive values for yaw correspond to gaze turning right in the input image. Positive values for roll correspond to face rolling to the left in the input image, see illustration below.
The values are in radians.
Function returns gaze direction values in the 3-dimensional gazeDirection array. An example of use:private float[] gazeDirection = new float[3]; //call of _getGazeDirectionGlobal() function VisageTrackerNative._getGazeDirectionGlobal(gazeDirection, faceIndex);
Parameters:
Name Type Description gazeDirection
Array 3D array in which the gaze direction values will be stored. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. -
<static> _getGazeDirection(gazeDirection, faceIndex)
-
Gaze direction. This variable is set only while tracker is tracking (TRACK_STAT_OK). This is the current estimated gaze direction relative to the person's head. Direction is expressed with two values x and y, in radians. Values (0, 0) correspond to person looking straight. X is the horizontal rotation with positive values corresponding to person looking to his/her left.
Y is the vertical rotation with positive values corresponding to person looking down. Function returns gaze direction values in the 2-dimensional gazeDirection array. An example of use:private float[] gazeDirection = new float[2]; //call of _getGazeDirection() function VisageTrackerNative._getGazeDirection(gazeDirection, faceIndex);
Parameters:
Name Type Description gazeDirection
Array 2D array in which the gaze direction values will be stored. faceIndex
Number value between 0 and MAX_FACES-1, used to access the data of a particular tracked face.