Face Tracker

Updated: 4 Mar 2026

Method #

This node searches the supplied image or video for faces. If a face is found, a mesh is generated then tracked along with the image.

This node uses the NVIDIA RTX AI based face tracking solution to generate a face mesh, and track the points between frames. This features requires an NVIDIA RTX enabled graphics card, as well as the NVIDIA AR drivers (available here). Running on the GPU, this method is far faster than the CPU based alternative and can be used with a 3D Face mesh instead of a 2D mesh.

For more information on the NVIDIA Broadcast Integration, see here

Important information for Step 2: NVIDIA 20xx (Turing), 30xx (Ampere) and 40xx (Ada) cards have different AR SDK downloads. Make sure you download the correct one for your graphics card.

Troubleshooting #

Error: The “Face Tracking Geometry” node is marked as red in the node graph #

You’re missing the NVIDIA Broadcast Engine AR SDK. Please download and install it.

General #

When the Face Tracking node finds a face it generates a mesh which may be used as geometry by the Face Tracking Mesh node, or used to deform a matching 3D object’s vertices using the Face Tracking Deformer node. The image output by the Face Tracking node can either pass through the original input; or output a mask from the detected face geometry in either the RGB or Alpha channels. Only one face can be detected at a time, so multi-face tracking is currently not supported.

The Face Tracker also analyses the detected face pose to determine whether the eyes and mouth are open, the positions of several key points such as the eyes, and the overall approximate rotation and scale of the face. These values are available as envelopes that can be extracted from the node via an Extractor and can, for example, be linked to Image 2D nodes in order to attach 2D images to points on the face. The Face Found envelope is set to 0 if no face is found, or 1 if there is.

To improve performance further, there is the option to downsample the input image before face detection - as this typically does not have to be processed at full resolution to find faces. Downsample Steps defines the number of downsample iterations: 1 means half width, half height and so on.

It must be noted that face tracking, while reasonably solid and robust, is not perfect. There is a degree of tolerance but care must be taken with the input feed to ensure that the algorithm is able to perform to its best. In particular:

• Face detection works best in good, balanced, even lighting, with a subject facing straight on to the camera and good image quality. The technique is not intended to work with moving cameras but is tolerant to some degree.
• Excessive facial hair, hats, novelty glasses, low frame rates and fast motion may impact the result.
• The face must be at least 300 pixels high in the source image. Smaller faces will most likely not be detected.
• The face detection algorithm can detect multiple faces in a single image, but at present only one face is tracked. To keep consistency the largest face in the image is chosen each frame.

Parameters

Inputs