This guide explains how to prepare 3D models to be used in augmented reality (AR) applications based on visage|SDK head tracking. As an example, the guide shows how to prepare eyeglasses for Virtual Try-on application. The same principles may be applied to other objects to be used in AR applications.
Real-life size, in meter units.
X positioning: centre of glasses at X=0
Y positioning: visual axes at Y = 0
Z positioning: eyeball front edge at Z = 0 (therefore lenses at approx Z = 0.02 - 0.025 m depending on glasses model)
If occlusions are not handled, glasses would always appear on top of the face like this:
In reality, parts of the glasses are occluded by the head, nose or ears. To achieve the same effect in augmented reality, an occlusion mask is used. This is an object shaped roughly like a human head and placed inside the glasses. The material of this object is called occluder_mat. At runtime, the occlusion mask covers the object behind it but is not itself visible, achieving the desired effect. The next two images show the occlusion mesh highlighted as wireframe (left), and the final effect (right).
Occlusion mask is available: models/occlusion_mask.obj. When imported into the scene, the mask should automatically be correctly sized and positioned, as follows:
Real-life size, in meter units.
X positioning: point between the eye centres at X=0
Y positioning: eye centres at Y = 0
Z positioning: eyeball front edge at Z = 0
The occlusion mask must not be moved or scaled.
Small adjustments to the mask can be made so it better fits the specific glasses model.
Nose can be widened in order to fit the glasses' nose pads as close as possible. Following images illustrate the correct fitting of the nose:
If the occlusion mask is too wide or too narrow, it can be adjusted by moving the sides of the head or adjusting them so that the temples of the glasses fit closely to the mask.
The ears of the occlusion mask should fully cover the back part of the temples.
This section covers issues of file formats and conversions. The two subsections cover the two 3D platforms typically used to build final AR applications using visage|SDK - Three.js and Unity3D.
Models need to be in OBJ format with a corresponding MTL material file. Most 3D authoring tools provide a way to export an object to OBJ and MTL.
Resulting OBJ file must contain both the glasses and the occlusion mask. Occlusion mask mesh should be grouped under occluder_mat name. For example, in Blender this is done by naming the material of the mesh occluder_mat and exporting to OBJ with setting Material Groups checked.
Materials are defined in the MTL file, usually generated from the materials in the authoring tool. After exporting advanced users can do fine tuning by editing the file. For syntax reference see here. Textures used in the MTL file should be placed in the path relative to the MTL folder. Supported texture formats include JPG, TGA and PNG.
Models are typically built in a 3D authoring tool such as 3ds Max or Blender, then imported into Unity. See a tutorial here for a list of supported formats and details regarding the import process. After importing process is complete check that the size and positioning of the model and the occlusion mask are consistent with the instructions in sections Size and Positioning and Importing the occlusion mask.
Unity3D package contains an occlusion shader that needs to be assigned to the occluder_mask object. Occlusion shader is available in the unity scene in: Materials/