Method

This node performs a “chroma keying” process on an image, by which a specific colour element (typically a shade of green) is removed and replaced with a different element (transparency). This is commonly known as “green screen keying”.

Chroma Key node

On a simple level, chroma keying simply looks at each pixel’s colour and compares it to the key colour. Pixels that don’t match the key colour are left alone; pixels that do match the key colour are given an alpha value of 0, making them transparent. This leaves two problem cases: semi-transparent elements (including shadows) in the scene result in pixel colours which do not match the key colour, but contain an amount of it; and reflective elements in the scene may reflect the key colour. In both cases the desired result is to not only remove the key colour but to reconstruct the element’s original colour, so it can be made to look plausible in the new scene after keying: blending with, or reflecting, the colour of the new background. The third problem case comes when the key colour varies too much across the image, typically due to problems with the way the background screen is lit.

This node is capable of handling both transparency and reflections. It uses a method whereby a set of plausible colours that are sufficiently different to the key colour are extracted from the image, and used to estimate values of transparency, reflectance and the original colour of each pixel in the image.

The key colour may be set manually or detected automatically by using a search for strong green or blue tones in the image. The true key colour often varies across the image due to inconsistent lighting/shadowing; the node is capable of handling such variations and builds up an image for the key value.

Green screen footage With Chroma Key node

Usage

The workflow for generating the best possible keying result is detailed below. The node is often capable of generating a good result with only minimal adjustment from defaults when the input is a well-lit, well-separated source with a clean background; some problem cases or ideal refinements will take more tuning.

Clean Plate Generation

The first step is to generate a good clean plate image. The node does this by finding pixels which match the required colour criteria, but are also sufficiently far from pixels the algorithm considers solid. It then fills in the holes that remain using convolution. The aim is to generate a clean plate which allows for local variations in lighting and tone, but also does not include any pixels which should not be there: e.g. low-transparency hair pixels and so on. It is important to ensure that only pixels which should be completely keyed out are included. It’s better to err on the side of caution here, particularly when working with video: if pixels are incorrectly determined as “clean plate” at this stage it will break the rest of the process.
Use “Show Contributing Clean Plate Pixels” to visualise which pixels of the image are considered “clean plate” pixels, and “Show Generated Clean Plate” to visualise the generated clean plate image. The “Clean Plate Generator” properties can be used to adjust which pixels are considered as part of the clean plate, leaving the rest of the space to be filled in by convolution.

The Contributing Clean Plate Pixels View The Generated Clean Plate View

Reference Colours

Following clean plate generation, the process automatically extracts “good” solid colours from the image. These are colours which are sufficiently far from the key colour that they are considered opaque and will be retained as-is throughout the keying process. These can be visualised using the “Show Generated RGB Palette” option, and the “Hue / Saturation Difference” properties may be used to adjust the set.

Transparency Generation

Use “Show Solid Areas” and “Show Fully Keyed Areas” to view the areas of the image which are considered either to match the clean plate, or to be left as-is. The rest of the image will be processed to find a new colour and opacity value. Use the “Hue / Saturation Difference” properties to make the solid area as complete as possible, while erring on the side of caution particularly for video sources. If a pixel should be semi-transparent or it needs some of the key colour removing to return it to its original desired colour, it must not be in the “solid areas” or “fully keyed areas” set.

The Solid Areas View

Use “Show Output Matte” to view the resulting transparency/alpha matte. “Harden Interior Area Transparencies” may be used to increase the contrast of only areas further away from the solid/key boundary. “Shrink Transparency Edge” may be used to cut in the transparency by a small amount, to remove halos. Global adjustments to the matte may be made using the controls in the “Transparencies / Edges” property group.

The Output Matte View

Spill Suppression

For many input sources, the above steps will be sufficient to give a good keyed result. However, in some cases where there is significant chroma spill – i.e. pixels which should be opaque reflecting the chroma colour, and therefore being detected incorrectly – additional processing is required.

The simplest way to remove spill and resolve colour channel errors is to use the “Global Suppression” option. This performs a simple process to reduce the key colour’s contribution to colours, which often gives a plausible result at low cost. If this is insufficient, spill suppression may be used, as follows.

Turning on “Reflection Suppression” adds an additional processing step which attempts to resolve pixel colours which reflect the key colour, by replacing it with a user-defined Reflection Replacement Colour. Use “Show Reflectance” to visualise the reflectance result. It’s important to remember that the previous stages will influence the quality of results in later stages: if the key colour is incorrect or contains a spill from solid colours, or if the solid colour set contains colours it shouldn’t or doesn’t contain pixels which should be valid, the transparency and reflectance stages will be affected detrimentally.

Reflectance vs Transparency

Reflection estimation is optional and takes additional computation. It is difficult to determine computationally whether a pixel contains the chroma colour because it’s semi-transparent, or because it’s reflective; as such the process works best if asked to handle one or the other. For scenes with lots of transparency, disabling reflection estimation will give the best result. For scenes with reflections it works best if objects in the scene are solid. The additional optional step to harden the alpha channel for the interiors of objects (“Harden Interior Area Transparencies”) – making them more opaque away from the edges – can be particularly useful when combined with reflections, as interior reflective pixels may be incorrectly identified as transparent: hardening interiors can reduce this issue.

Refining

“Smooth Transparency” may be used to smooth the result at the cost of detail. “Smooth Colour” may be used to apply smoothing to transparent or reflective pixel colours. These smoothing processes use additional computation. “Temporal Smoothing” uses previous frame data to both reduce computation, optimising the process, and reduce temporal flicker. This should be used where possible.
Finally, particularly problematic areas of the image may be resolved manually using a garbage matte or a holdout matte as appropriate.

Chroma Key Nodegraph

Attributes

Name Description
Active Turn the effect on or off. Values < 0.1 will turn the effect off, and deferred rendering must be on.
Blend Mode Choose how the image blends with the scene. See Blend Modes for details.
Blend Amount Changes the opacity of the new effect over the original image.

Clean Plate Generator

Name Description
Generation Mode The method of generating the key colour.
  • Auto generate from Greens – The node looks for strong green colours in the image and uses these to generate a variable key colour.
  • Auto generate from Blues – The node looks for strong blue colours in the image and uses these to generate a variable key colour.
  • Auto-Generate Using Colour Picker Colour As Hint – The node looks for colours similar to the reference Key Colour in the image and uses these to generate a variable key colour.
  • Flat Plate Using Colour Picker Colour – The Key Colour parameter is used as the key colour for the whole image.
Key Colour Picker Choose the colour to be keyed out of the image.
Clean Plate Inclusion Threshold Threshold determining whether a pixel should contribute to the key colour.
Clean Plate Contribution Weighting Threshold determining whether a pixel’s colour should contribute to the key colour, based on saturation.
Show Generated Clean Plate Outputs the generated key colour image.
Show Contributing Clean Plate Pixels Shows the pixels which contribute to the key colour image.

Fully Keyed Areas

Name Description
Difference Threshold Threshold determining whether a pixel should be considered as matching the key colour, based on colour distance.
Show Fully Keyed Areas Shows the pixels which are considered to be fully keyed out / transparent.

Solid Areas

Name Description
Hue Difference Threshold Threshold determining whether a pixel should be considered “solid”, based on hue difference from the key colour.
Saturation Difference Threshold Threshold determining whether a pixel should be considered “solid”, based on saturation difference from the key colour.
Harden Interiors Determines whether the interior areas (away from the key/solid boundary) should be increased in contrast.
Harden Interiors Amount The amount by which the interior areas (away from the key/solid boundary) should be increased in contrast.
Show Solid Areas Shows the pixels which are considered fully opaque.

Transparencies / Edges

Name Description
Harden Transparencies Amount The amount by which the transparency should be increased in contrast (hardened).
Harden Transparencies Threshold The centre value used when the transparency is increased in contrast.
Shrink Transparency Edge Determines whether the generated transparency channel should be eroded to remove semi-transparent halos.
Shrink Transparency Edge Passes The number of erosion passes on the transparency channel.
Transparency Black Clip The black clip value for the transparency matte.
Transparency White Clip The white clip value for the transparency matte.
Show Output Matte Shows the generated opacity channel.

Spill Suppression

Name Description
Global Suppression Uses a simple process to remove the key colour’s chroma from pixel colours.
Reflection Suppression Determines whether reflections of the key colour should be detected and replaced with the reflection colour.
Reflection Suppression Intensity Scales the intensity of the reflectance.
Reflection Replacement Colour Controls the colour of the reflection.
Show Reflectance Shows the generated reflectance, for pixels which are considered to be reflecting the key colour.

Smoothing and Refinement

Name Description
Temporal Smoothing Determines whether temporal smoothing should be applied – using data from the previous frame to smooth and improve the results.
Smooth Transparency Determines whether the generated transparency channel should be smoothed to remove noise.
Smooth Transparency Passes The number of smoothing passes on the transparency channel.
Smooth Colour Determines whether the generated colour channel should be smoothed to remove noise.
Smooth Colour Passes The number of smoothing passes on the colour channel.

Debug Visualisation

Name Description
Show Generated RGB Palette Outputs the generated solid colour palette.

Inputs

Name Description Typical Node Input
Active Turn the effect on or off. Values < 0.1 will turn the effect off, and deferred rendering must be on. Envelope Modifier
Colour Reference Image Allows an image to be provided as a source for the solid colour palette. This may be useful in the case where there is too much spill in the keyed image to provide a good palette.
Key Plate Image Allows an image to be provided as a source for clean plate generator. Note that the rest of the algorithm is still performed as is, with the assumption that the clean plate is more or less flat – it won’t work for arbitrary background subtraction.
Garbage Matte Image Allows an image to be provided as a garbage matte. Areas of the matte that are non-zero will be considered as completely keyed out, and will not contribute to the clean plate generator either.
Holdout Matte Image Allows an image to be provided as a hold-out matte. Areas of the matte that are non-zero will be considered as completely solid.

Outputs

There are no outputs from this node.