Method

This node generates a 2D mesh using the predefined structure of the geometry generated by the Face Tracker node: 68 vertices linked as triangles with texture coordinates. It takes as input a Face Tracker node, which is required for the node to render anything. When a Face Tracker node is linked the geometry is updated automatically each frame as long as the tracker has found and tracked a face.

The Face Tracking Geometry node allows the geometry produced by the face tracker to be rendered in different ways or multiple times easily with low overhead: the face tracking process is only done once using the Face Tracker. The Face Tracking Geometry node can also be used as input to other nodes which use geometry as part of processing, such as Particle Mesh Emitter, Clone to Mesh, Combine Geometry.

For more information on face tracking see the Face Tracker node.

Setup

This node uses the NVIDIA Broadcast Engine AR SDK as well as a dataset which needs to be manually downloaded, compiled and copied into the Notch application location (due to licensing). This node will not work unless you follow these steps to set up your system prior to launching Notch Builder. These steps are taken directly from the official NVIDIA documentation and are repeated here for convenience.

  1. Make sure Notch Builder is not running
  2. Download and install the NVIDIA Broadcast Engine AR SDK
  3. Generate a 3D Morphable Face Model (3DMM)
    1. Create a temporary folder and download all the following files:
    2. Download the ConvertSurreyFaceModel.exe utility from the BROADCAST-AR-SDK project page on Github.
    3. Download the following Surrey Face Model files from the eos project page on Github — OR: download the entire repo by clicking here. You’ll get a few more files than you need, but you’ll avoid download errors. The files you want are in the archive, in the “share” folder. If you want to download just the files you need, these are the ones:
      1. sfm_shape_3448.bin
      2. expression_blendshapes_3448.bin
      3. sfm_3448_edge_topology.json (beware that some browsers corrupt pure text files like JSON on download!)
      4. sfm_model_contours.json (beware that some browsers corrupt pure text files like JSON on download!)
      5. ibug_to_sfm.txt
    4. Open a Command Prompt (Start menu, CMD) and navigate to the folder where the files downloaded in step 1 and 2 can be found.
    5. Copy and paste this strong into your Command Prompt and hit enter to run it: ConvertSurreyFaceModel.exe --shape=sfm_shape_3448.bin --blend_shape=expression_blendshapes_3448.bin --topology=sfm_3448_edge_topology.json --contours=sfm_model_contours.json --ibug=ibug_to_sfm.txt --out=face_model0.nvf
  4. Copy the newly generated face_model0.nvf to the “models” folder inside your Notch installation folder (which is typically C:\Program Files\Notch).

Troubleshooting

Error: “No shape file was specified”

Make sure you’re copy-pasting the entire command from step 3d above exactly as written. There should be no extra spaces or other characters added. When run correctly, there should be no feedback in the Command Prompt after you’ve run the command, you should just have a new file in the folder.

Error: ERR#8 or “Syntax Error at DOCTYPE

Your download of one or more of the files from Github has failed. You can download the entire directory by clicking here. You’ll get a few more files than you need, but you’ll avoid download errors.

Why does this happen? Well, when you are downloading the files from Github as described in step 3b above, make sure you do not have any content blockers or extra security settings in your browser, as they tend to block or sanitize JS-related files like JSON under certain conditions. They need to be downloaded as their RAW source format to work. One way to do this especially for the .json files is to click the “RAW” option on Github for those files, and then go to “Save page as..” in your browser.

Error: The “Face Tracking Geometry” node is marked as red in the node graph

You’re missing the NVIDIA Broadcast Engine AR SDK. Please download and install it according to the setup guide above.

Transform

Name Description
Position X Move its position along the x-axis in local space.
Position Y Move its position along the y-axis in local space.
Position Z Move its position along the z-axis in local space.
Rotation Heading Rotate the object about the y-axis.
Rotation Pitch Rotate the object about the x-axis.
Rotation Bank Rotate the object about the z-axis.
Scale X Scale along the x-axis.
Scale Y Scale along the y-axis.
Scale Z Scale along the z-axis.

Inherit Transform Channels

Toggle which transform channels should be inherited. By default, all are on.

Rendering

Name Description
Show Polygons Show the polygons of the mesh.
Mesh Type Select which mesh type to use.
  • 68 Mark Landmark, Uses 68 landmark points to generate a 2D mesh.
  • NVIDIA Face Tracking Mesh, Uses the NVIDIA Face Tracking 3D Mesh. Does not support CPU face tracking.
Use Mesh Orientation Rotate the mesh to match the rotation of the face from the face tracker. only functions with the NVIDIA Face Tracking Mesh type.
Visible Control whether the node is visible or not to the scene.

Lines

Name Description
Lines Visible Control whether the node is visible or not to the scene.
Lines Alpha Control the alpha value of the lines.
Colour Control the colour value of the lines.
Use Vertex Colours Colour lines based on the vertex colours of the geometry.
Blend Mode Control how the lines blend with the rest of the scene.
Thick Lines Use lines with thickness relative to their distance to the camera.
Hide Back Face Lines Hides the lines around back faces of geometry.
Lock Width Lock the line width to be consistent regardless of distance from the camera. Only functions with Thick Lines enabled.
Thick Line Width Control the thickness of all the lines. Only functions with Thick Lines enabled.
Show Silhouette Lines Draws lines along the edges of the object relative to the camera.
Show Normal Difference Lines Draw a line along the edges of the mesh, depending on the angle difference between their mutual faces.
Show Unshared Lines Draw lines along all edges of the shape.
Show Other Lines Show all the lines for each edge of the Mesh.
Unshared Lines Weight Greater control over the alpha of the unshared lines.
Silhouette Lines Weight Greater control over the alpha of the silhouette lines.
Normal Difference Lines Weight Greater control over the alpha of the normal difference lines.
Other Lines Weight Greater control over the alpha of the other lines.
Line Normal Difference Angle Change the threshold angle between two face normals that will generate a line along their common edge. Only functions with Show Normal Lines enabled.
Line Normal Fade Sharpness How much the drawn normal lines will fade away the closer the edge angle is to the Line Normal Difference Angle.
Line Depth Bias Exaggerate the width of the lines based on the distance to the camera. not functional with Lock Width enabled.
Line Silhouette Fade Sharpness How much the drawn silhouette lines will fade away based on the size of the edge angle to the camera.

Materials

See Materials.

Inputs

Name Description Typical Node Input
Face Tracker Source Input a node with facial tracking data to drive the effect. Face Tracker
Material Override the default material with another material. Materials
Transform Modifier Links all transform properties to the input node. Null
Target Node Always faces the x-axis towards the inputs anchor point. Null
Local Transform Override Override the transformation values of the node, relative to its parent. Null

Outputs

This node outputs the normal transformation and translation values, but it also outputs geometry which can be Deformed or used as a mesh so