Notch Notch Manual 0.9.23
 Light | Dark
Depth Camera / Kinect Source

Depth Camera / Kinect Source

Read colour and depth data from a Kinect or RealSense camera.

Method #

This node is used to read colour and depth data from a Kinect Or RealSense camera. This node carries both colour and depth information. Post Effect nodes and Video Processing nodes attached to this node will operate on the colour data only, unless specifically designed to read the depth data - such as the Depth Camera / Kinect Colour Key node. The RGB channel is read from the camera directly, or set to white if the camera does not provide colour data. The alpha channel is set to a mask value based on the depth, using the near and far clip plane parameters. If “Output Depth” is selected the depth data is output in the colour channel as luminance values.

Where multiple Kinect cameras are connected they may be selected via the “Kinect Camera Index” parameter. A pre-recorded stream may be used instead of live Kinect camera data by setting its resource via the Kinect Stream parameter. Multiple Nodes can be used for streaming both Kinect 2 and Kinect 4 sources together on the same PC.

For the output of the node to work, you will need to activate the Kinect camera you want to use by going to Devices > VideoIn/Camera/Kinect Settings and enabling it. Otherwise, it will not be initialized by Notch. This is by design, so Notch does not lock the device from the use by other software when you are not actively using it in Notch

Kinect 4 features #

Stream Setup #

When Kinect 4 sensor type is selected, additional options are enabled: Depth Sensor Mode and Colour Sensor Mode as Kinect 4 supports a number of range/FOV and colour sensor resolutions. Typically Near/narrow range/FOV depth modes have lower noise characteristics and therefore higher quality than Far/Wide range/FOV depth modes.

Colour modes supported on Kinect 4 can go up to 3072p 4k resolution but the higher the resolution the higher real-time cost. We recommend 1080p colour resolution for real-time.

Body Segmentation #

If Kinect 4 sensor type is selected, there is the option to use automatic body masking, set in the Video In options. This enables up to 6 “bodies” or “players” to be detected and utilize the mask for each of these to automatically segment the bodies from the background.

You can find out more about setting up notch for Kinect here.

Parameters

These properties control the core behaviours of the node.

ParameterDetails
Preview In ViewportPreview the effect blended with alpha in the viewport.
Preview RGB In ViewportPreview the colour values in the viewport.
Preview Alpha In ViewportPreview the alpha values in the viewport.
Depth Sensor Mode(Kinect 4 only) The depth sensor mode affecting range and FOV
Colour Sensor Mode(Kinect 4 only) The colour sensor mode affecting resolution of Colour sensor output
Near Clip PlaneThe closest distance that will be rendered. WARNING: Reducing this value too low can reduce the accuracy of aspects of the render.
Far Clip PlaneThe furthest distance that will be rendered by this camera. WARNING: Increasing this value too high, without increasing the Near Clip can reduce the accuracy of aspects of the renderer. Generally best left untouched.
Depth Clip HardnessHow harsh the falloff is for depth data.
Edge Fade WidthRefinement width for depth edge mask
Kinect Depth ScaleScaling of the Kinect Z depth
Output DepthToggle whether only the depth image is output.
Camera TypeChoose the camera type: Kinect 1, Kinect 2, Kinect 4, Realsense, Realsense 2 or First Detected to choose the first sensor Notch detects (has specific sensor type ordering)
Camera IndexChoose which depth sensor to use as a source.
Kinect StreamChoose an optional, overriding, depth sensor stream to stream from instead of realtime device streaming.
Align Colour And Depth(Kinect 2 only) Aligns/remaps Colour data to Depth data using sensor intrinsics so the UVs match when mapped onto a projected depth mesh.
Colour Image Offset X/Y(Kinect 2 only) Manually aligns the colour image UVs to align with depth data, used to fine tweak when “Align Colour And Depth” is selected.
Apply Body Masking(Kinect 4 only) If “Body Index Masking” is selected in the Kinect 4 Video In settings, this will mask out depth, alpha and colour pixels that are not a part of a detected body.
Body Mask Index Start(Kinect 4 only) If “Body Index Masking” is selected in the Kinect 4 Video In settings, this allows the user to filter out detected bodies per node.
Body Mask Index End(Kinect 4 only) If “Body Index Masking” is selected in the Kinect 4 Video In settings, this allows the user to filter out detected bodies per node.
Hole Fill IterationsHow many iterations will be made to fill gaps in the Kinect depth-map.
ParameterDetails
Kinect Alpha Mask Disparity Cutoff(Kinect 2 only) The cut off point for the alpha mask of the kinects disparity.
Depth Edge Dilation PassesNumber of dilation passes to fill in the edge of the depth mask, useful for Kinect 2 depth refinement due to noise statistic of the sensor.
Visualize TargetsTurn on visualization of Colour, Depth and Depth disparity mask within the viewport when node is selected.
Temporal Filter DepthTurn on Spatio-temporal filtering to reduce sensor noise from the resulting depth image and increase quality.
ParameterDetails
Stream Start FrameSelect the stream start frame.
Stream End FrameSelect the stream end frame.
Stream Frame RateChange the frame rate of the stream.

Inputs

NameDescriptionTypical Input
Effect MaskMask out areas that Post-FX applied to this node won’t be applied.Video Loader
Alpha ImageUse a seperate video nodes luminance values to overwrite the alpha channel of the image.Video Loader