Depth Camera / Kinect Source
Method #
This node is used to read colour and depth data from a Kinect Or RealSense camera. This node carries both colour and depth information. Post Effect nodes and Video Processing nodes attached to this node will operate on the colour data only, unless specifically designed to read the depth data - such as the Depth Camera / Kinect Colour Key node. The RGB channel is read from the camera directly, or set to white if the camera does not provide colour data. The alpha channel is set to a mask value based on the depth, using the near and far clip plane parameters. If “Output Depth” is selected the depth data is output in the colour channel as luminance values.
Where multiple Kinect cameras are connected they may be selected via the “Kinect Camera Index” parameter. A pre-recorded stream may be used instead of live Kinect camera data by setting its resource via the Kinect Stream parameter. Multiple Nodes can be used for streaming both Kinect 2 and Kinect 4 sources together on the same PC.
For the output of the node to work, you will need to activate the Kinect camera you want to use by going to Devices > VideoIn/Camera/Kinect Settings and enabling it. Otherwise, it will not be initialized by Notch. This is by design, so Notch does not lock the device from the use by other software when you are not actively using it in Notch
Kinect 4 features #
Stream Setup #
When Kinect 4 sensor type is selected, additional options are enabled: Depth Sensor Mode and Colour Sensor Mode as Kinect 4 supports a number of range/FOV and colour sensor resolutions. Typically Near/narrow range/FOV depth modes have lower noise characteristics and therefore higher quality than Far/Wide range/FOV depth modes.
Colour modes supported on Kinect 4 can go up to 3072p 4k resolution but the higher the resolution the higher real-time cost. We recommend 1080p colour resolution for real-time.
Body Segmentation #
If Kinect 4 sensor type is selected, there is the option to use automatic body masking, set in the Video In options. This enables up to 6 “bodies” or “players” to be detected and utilize the mask for each of these to automatically segment the bodies from the background.
You can find out more about setting up notch for Kinect here.
Parameters
These properties control the core behaviours of the node.
Parameter Details
Preview In Viewport Preview the effect blended with alpha in the viewport.
Preview RGB In Viewport Preview the colour values in the viewport.
Preview Alpha In Viewport Preview the alpha values in the viewport.
Depth Sensor Mode (Kinect 4 only) The depth sensor mode affecting range and FOV
Colour Sensor Mode (Kinect 4 only) The colour sensor mode affecting resolution of Colour sensor output
Near Clip Plane The closest distance that will be rendered. WARNING: Reducing this value too low can reduce the accuracy of aspects of the render.
Far Clip Plane The furthest distance that will be rendered by this camera. WARNING: Increasing this value too high, without increasing the Near Clip can reduce the accuracy of aspects of the renderer. Generally best left untouched.
Depth Clip Hardness How harsh the falloff is for depth data.
Edge Fade Width Refinement width for depth edge mask
Kinect Depth Scale Scaling of the Kinect Z depth
Output Depth Toggle whether only the depth image is output.
Camera Type Choose the camera type: Kinect 1, Kinect 2, Kinect 4, Realsense, Realsense 2 or First Detected to choose the first sensor Notch detects (has specific sensor type ordering)
Camera Index Choose which depth sensor to use as a source.
Kinect Stream Choose an optional, overriding, depth sensor stream to stream from instead of realtime device streaming.
Align Colour And Depth (Kinect 2 only) Aligns/remaps Colour data to Depth data using sensor intrinsics so the UVs match when mapped onto a projected depth mesh.
Colour Image Offset X/Y (Kinect 2 only) Manually aligns the colour image UVs to align with depth data, used to fine tweak when “Align Colour And Depth” is selected.
Apply Body Masking (Kinect 4 only) If “Body Index Masking” is selected in the Kinect 4 Video In settings, this will mask out depth, alpha and colour pixels that are not a part of a detected body.
Body Mask Index Start (Kinect 4 only) If “Body Index Masking” is selected in the Kinect 4 Video In settings, this allows the user to filter out detected bodies per node.
Body Mask Index End (Kinect 4 only) If “Body Index Masking” is selected in the Kinect 4 Video In settings, this allows the user to filter out detected bodies per node.
Hole Fill Iterations How many iterations will be made to fill gaps in the Kinect depth-map.
Parameter Details
Kinect Alpha Mask Disparity Cutoff (Kinect 2 only) The cut off point for the alpha mask of the kinects disparity.
Depth Edge Dilation Passes Number of dilation passes to fill in the edge of the depth mask, useful for Kinect 2 depth refinement due to noise statistic of the sensor.
Visualize Targets Turn on visualization of Colour, Depth and Depth disparity mask within the viewport when node is selected.
Temporal Filter Depth Turn on Spatio-temporal filtering to reduce sensor noise from the resulting depth image and increase quality.
Parameter Details
Stream Start Frame Select the stream start frame.
Stream End Frame Select the stream end frame.
Stream Frame Rate Change the frame rate of the stream.
Inputs
These properties control the core behaviours of the node.
| Parameter | Details |
|---|---|
| Preview In Viewport | Preview the effect blended with alpha in the viewport. |
| Preview RGB In Viewport | Preview the colour values in the viewport. |
| Preview Alpha In Viewport | Preview the alpha values in the viewport. |
| Depth Sensor Mode | (Kinect 4 only) The depth sensor mode affecting range and FOV |
| Colour Sensor Mode | (Kinect 4 only) The colour sensor mode affecting resolution of Colour sensor output |
| Near Clip Plane | The closest distance that will be rendered. WARNING: Reducing this value too low can reduce the accuracy of aspects of the render. |
| Far Clip Plane | The furthest distance that will be rendered by this camera. WARNING: Increasing this value too high, without increasing the Near Clip can reduce the accuracy of aspects of the renderer. Generally best left untouched. |
| Depth Clip Hardness | How harsh the falloff is for depth data. |
| Edge Fade Width | Refinement width for depth edge mask |
| Kinect Depth Scale | Scaling of the Kinect Z depth |
| Output Depth | Toggle whether only the depth image is output. |
| Camera Type | Choose the camera type: Kinect 1, Kinect 2, Kinect 4, Realsense, Realsense 2 or First Detected to choose the first sensor Notch detects (has specific sensor type ordering) |
| Camera Index | Choose which depth sensor to use as a source. |
| Kinect Stream | Choose an optional, overriding, depth sensor stream to stream from instead of realtime device streaming. |
| Align Colour And Depth | (Kinect 2 only) Aligns/remaps Colour data to Depth data using sensor intrinsics so the UVs match when mapped onto a projected depth mesh. |
| Colour Image Offset X/Y | (Kinect 2 only) Manually aligns the colour image UVs to align with depth data, used to fine tweak when “Align Colour And Depth” is selected. |
| Apply Body Masking | (Kinect 4 only) If “Body Index Masking” is selected in the Kinect 4 Video In settings, this will mask out depth, alpha and colour pixels that are not a part of a detected body. |
| Body Mask Index Start | (Kinect 4 only) If “Body Index Masking” is selected in the Kinect 4 Video In settings, this allows the user to filter out detected bodies per node. |
| Body Mask Index End | (Kinect 4 only) If “Body Index Masking” is selected in the Kinect 4 Video In settings, this allows the user to filter out detected bodies per node. |
| Hole Fill Iterations | How many iterations will be made to fill gaps in the Kinect depth-map. |
| Parameter | Details |
|---|---|
| Kinect Alpha Mask Disparity Cutoff | (Kinect 2 only) The cut off point for the alpha mask of the kinects disparity. |
| Depth Edge Dilation Passes | Number of dilation passes to fill in the edge of the depth mask, useful for Kinect 2 depth refinement due to noise statistic of the sensor. |
| Visualize Targets | Turn on visualization of Colour, Depth and Depth disparity mask within the viewport when node is selected. |
| Temporal Filter Depth | Turn on Spatio-temporal filtering to reduce sensor noise from the resulting depth image and increase quality. |
| Parameter | Details |
|---|---|
| Stream Start Frame | Select the stream start frame. |
| Stream End Frame | Select the stream end frame. |
| Stream Frame Rate | Change the frame rate of the stream. |
| Name | Description | Typical Input |
|---|---|---|
| Effect Mask | Mask out areas that Post-FX applied to this node won’t be applied. | Video Loader |
| Alpha Image | Use a seperate video nodes luminance values to overwrite the alpha channel of the image. | Video Loader |