Method

This node performs a “chroma keying” process on an image, by which a specific flat colour element (typically a shade of green) is removed and replaced with a different element (transparency). This is commonly known as “green screen keying”.

It uses an innovative method for chroma keying that attempts to derive transparency values and resolve spill with a high degree of automation, making it relatively easy to achieve a good result; but the process it uses may appear unfamiliar. It is strongly recommended to follow the process outlined in the “Usage” section below in order to get the best possible results.

The node first generates a “clean plate”: an image that looks like an empty background with no subjects in shot, but with allowance for variation across the plate. Next, the transparency matte is generated; then spill may be suppressed in the RGB channel if necessary.

Chroma Key node
Green screen footage With Chroma Key node


Usage

The workflow for generating the best possible keying result is detailed below. The node is often capable of generating a good result with only minimal adjustment from defaults when the input is a well-lit, well-separated source with a clean background; some problem cases or ideal refinements will take more tuning.

1. Clean Plate Generator

First a good clean plate image must be generated.

The node uses a reference colour as the initial estimate. Using the input image it detects pixels which are “solid” if they are sufficiently different from the reference colour, groups them into continuous areas and removes them from the image. The remaining pixels are considered the “clean plate”. The holes are then filled by estimation. This results in a “clean plate” background image which allows for local variations in lighting and tone. Generating a high quality clean plate is essential for keying to work well. It should contain as much of the original image as possible but not contain any pixels which are not part of the background: any part of the foreground subject, including low-transparency hair pixels, must be avoided.

The first step in clean plate generation is to select a reference colour using the Generation Mode property. For a green screen, “Auto-Generate From Greens” often gives the best result, and similarly “Auto-Generate From Blues” often does best for blue screens. In order to pick a colour, use “Auto Generate Using Colour Picker Colour As Hint” and then pick a colour using the Key Colour Picker property.

Next, ensure that the right set of pixels are in the clean plate. Use “Show Contributing Clean Plate Pixels” to visualise which pixels of the image are considered “clean plate” pixels, then use the “Clean Plate Inclusion Threshold” parameter to adjust which pixels may contribute to the clean plate. Use “Show Generated Clean Plate” to visualise the generated clean plate image. Ensure that no parts of e.g. the hair or body of the subject have leaked in to the clean plate.

Following clean plate generation, the process automatically extracts “good” solid colours from the image. These are colours which are sufficiently far from the key colour that they are considered opaque and will be retained as-is throughout the keying process. These can be visualised using the “Show Generated RGB Palette” option.

The Contributing Clean Plate Pixels View The Generated Clean Plate View

2. Fully Keyed Areas

Use “Show Fully Keyed Areas” to visualise the pixels which are considered completely transparent after the keying process and adjust this using the Difference Threshold parameter. Ensure that only pixels are marked which are completely part of the background as they will be removed completely.

The Fully Keyed Areas View

3. Solid Areas

Use “Show Solid Areas” to visualise the pixels which are considered “solid” – they will be left completely unprocessed, as-is. The Hue Difference Threshold and Saturation Difference Threshold parameters adjust this. Attempt to make as many pixels “solid” as possible – but if a pixel should be semi-transparent or if it needs any spill processing it should not be part of the solid area.

Pixels which are not part of the solid area or fully keyed area will be processed for transparency evaluation, and optional spill suppression. This does not mean a pixel will end up with a semi-transparent value; it may be evaluated and found to be completely solid or transparent. It is important to ensure that if a pixel may need processing – it may be semi-transparent or may have some spill to process – it must not be in the fully keyed or solid areas. However, the more correct the solid and fully keyed areas are, the less unnecessary processing will occur and the better the result will be.

The Solid Areas View

4. Transparencies / Edges

Use “Show Output Matte” to view the resulting transparency/alpha matte. The matte may now be refined. “Harden Transparencies Amount” acts as a contrast adjustment on the matte, with “Harden Transparencies Threshold” a centre point for adjustment. This is used to harden the matte. Transparency Black Clip and Transparency White Clip adjust the in and out points of the mask, clipping the higher or lower transparency values.

“Shrink Transparency Edge” may be used to cut in the edge of the transparency matte by a small amount to remove halos.

The Output Matte View

5. Spill Suppression

For many input sources, the above steps will be sufficient to give a good keyed result. However, in some cases where there is significant chroma spill – i.e. pixels which should be opaque reflecting the chroma colour, and therefore being detected incorrectly – additional processing is required.

The simplest way to remove spill and resolve colour channel errors is to use the “Global Suppression” option. This performs a simple process to reduce the key colour’s contribution to colours, which often gives a plausible result at low cost. If this is insufficient, spill suppression may be used, as follows.

Turning on “Reflection Suppression” adds an additional processing step which attempts to resolve pixel colours which reflect the key colour, by replacing it with a user-defined Reflection Replacement Colour. Use “Show Reflectance” to visualise the reflectance result.

Reflection estimation is optional and takes additional computation. It is difficult to determine computationally whether a pixel contains the chroma colour because it’s semi-transparent, or because it’s reflective; as such the process works best if asked to handle one or the other. For scenes with lots of transparency, disabling reflection estimation will give the best result. For scenes with reflections it works best if objects in the scene are solid. The additional optional step to harden the alpha channel for the interiors of objects (“Harden Interior Area Transparencies”) – making them more opaque away from the edges – can be particularly useful when combined with reflections, as interior reflective pixels may be incorrectly identified as transparent: hardening interiors can reduce this issue.

6. Refining & Smoothing

“Smooth Transparency” may be used to smooth the result at the cost of detail. “Smooth Colour” may be used to apply smoothing to transparent or reflective pixel colours. These smoothing processes use additional computation. “Temporal Smoothing” uses previous frame data to both reduce computation and temporal flicker.

Finally, particularly problematic areas of the image may be removed or kept manually using a garbage matte or a holdout matte as appropriate.

Chroma Key Nodegraph

Attributes

Name Description
Active Turn the effect on or off. Values < 0.1 will turn the effect off, and deferred rendering must be on.
Blend Mode Choose how the image blends with the scene. See "Blend Modes":http://manual.notch.one/0.9.23/en/topic/blend-modes for details.
Blend Amount Changes the opacity of the new effect over the original image.

Clean Plate Generator

Name Description
Generation Mode The method of generating the key colour.
  • Auto generate from Greens – The node looks for strong green colours in the image and uses these to generate a variable key colour.
  • Auto generate from Blues – The node looks for strong blue colours in the image and uses these to generate a variable key colour.
  • Auto-Generate Using Colour Picker Colour As Hint – The node looks for colours similar to the reference Key Colour in the image and uses these to generate a variable key colour.
  • Flat Plate Using Colour Picker Colour – The Key Colour parameter is used as the key colour for the whole image.
Key Colour Picker Choose the colour to be keyed out of the image.
Clean Plate Inclusion Threshold Threshold determining whether a pixel should contribute to the key colour.
Clean Plate Contribution Weighting Threshold determining whether a pixel’s colour should contribute to the key colour, based on saturation.
Show Contributing Clean Plate Pixels Shows the pixels which contribute to the key colour image.
Show Generated Clean Plate Outputs the generated key colour image.
Show Generated RGB Palette Outputs the generated solid colour palette.

Fully Keyed Areas

Name Description
Difference Threshold Threshold determining whether a pixel should be considered as matching the key colour, based on colour distance.
Show Fully Keyed Areas Shows the pixels which are considered to be fully keyed out / transparent.

Solid Areas

Name Description
Hue Difference Threshold Threshold determining whether a pixel should be considered “solid”, based on hue difference from the key colour.
Saturation Difference Threshold Threshold determining whether a pixel should be considered “solid”, based on saturation difference from the key colour.
Harden Interiors Determines whether the interior areas (away from the key/solid boundary) should be increased in contrast.
Harden Interiors Amount The amount by which the interior areas (away from the key/solid boundary) should be increased in contrast.
Show Solid Areas Shows the pixels which are considered fully opaque.

Transparencies / Edges

Name Description
Harden Transparencies Amount The amount by which the transparency should be increased in contrast (hardened).
Harden Transparencies Threshold The centre value used when the transparency is increased in contrast.
Shrink Transparency Edge Determines whether the generated transparency channel should be eroded to remove semi-transparent halos.
Shrink Transparency Edge Passes The number of erosion passes on the transparency channel.
Transparency Black Clip The black clip value for the transparency matte.
Transparency White Clip The white clip value for the transparency matte.
Show Output Matte Shows the generated opacity channel.

Spill Suppression

Name Description
Global Suppression Uses a simple process to remove the key colour’s chroma from pixel colours.
Reflection Suppression Determines whether reflections of the key colour should be detected and replaced with the reflection colour.
Reflection Suppression Intensity Scales the intensity of the reflectance.
Reflection Replacement Colour Controls the colour of the reflection.
Show Reflectance Shows the generated reflectance, for pixels which are considered to be reflecting the key colour.

Smoothing and Refinement

Name Description
Temporal Smoothing Determines whether temporal smoothing should be applied – using data from the previous frame to smooth and improve the results.
Smooth Transparency Determines whether the generated transparency channel should be smoothed to remove noise.
Smooth Transparency Passes The number of smoothing passes on the transparency channel.
Smooth Colour Determines whether the generated colour channel should be smoothed to remove noise.
Smooth Colour Passes The number of smoothing passes on the colour channel.

Inputs

Name Description
Colour Reference Image Allows an image to be provided as a source for the solid colour palette. This may be useful in the case where there is too much spill in the keyed image to provide a good palette.
Key Plate Image Allows an image to be provided as a source for clean plate generator. Note that the rest of the algorithm is still performed as is, with the assumption that the clean plate is more or less flat – it won’t work for arbitrary background subtraction.
Garbage Matte Image Allows an image to be provided as a garbage matte. Areas of the matte that are non-zero will be considered as completely keyed out, and will not contribute to the clean plate generator either.
Holdout Matte Image Allows an image to be provided as a hold-out matte. Areas of the matte that are non-zero will be considered as completely solid.

Outputs

There are no outputs from this node.