Notch Notch Manual 0.9.23
 Light | Dark
Chroma Key

Chroma Key

Generate a mask for an image using Chroma Keying.

image

Example .dfx

Method #

This node performs an intelligent “chroma keying” process on an image, by which a specific flat colour element (typically a shade of green) is removed and replaced with a different element (transparency). This is commonly known as “green screen keying”.

It uses an innovative method for chroma keying that attempts to derive transparency values and resolve spill with a high degree of automation, making it relatively easy to achieve a good result; but the process it uses may appear unfamiliar. It is strongly recommended to follow the process outlined in the “Usage” section below in order to get the best possible results.

The node first generates a “clean plate”: an image that looks like an empty background with no subjects in shot, but with allowance for variation across the plate. Next, the transparency matte is generated; then spill may be suppressed in the RGB channel if necessary.

chroma-key-example

Usage #

The workflow for generating the best possible keying result is detailed below. The node is often capable of generating a good result with only minimal adjustment from defaults when the input is a well-lit, well-separated source with a clean background; some problem cases or ideal refinements will take more tuning.

1. Clean Plate Generator #

First a good clean plate image must be generated.

The node uses a reference colour as the initial estimate. Using the input image it detects pixels which are “solid” if they are sufficiently different from the reference colour, groups them into continuous areas and removes them from the image. The remaining pixels are considered the “clean plate”. The holes are then filled by estimation. This results in a “clean plate” background image which allows for local variations in lighting and tone. Generating a high quality clean plate is essential for keying to work well. It should contain as much of the original image as possible but not contain any pixels which are not part of the background: any part of the foreground subject, including low-transparency hair pixels, must be avoided.

The first step in clean plate generation is to select a reference colour using the Generation Mode property. For a green screen, “Auto-Generate From Greens” often gives the best result, and similarly “Auto-Generate From Blues” often does best for blue screens. In order to pick a colour, use “Auto Generate Using Colour Picker Colour As Hint” and then pick a colour using the Key Colour Picker property.

Next, ensure that the right set of pixels are in the clean plate. Use “Show Contributing Clean Plate Pixels” to visualise which pixels of the image are considered “clean plate” pixels, then use the “Clean Plate Inclusion Threshold” parameter to adjust which pixels may contribute to the clean plate. Use “Show Generated Clean Plate” to visualise the generated clean plate image. Ensure that no parts of e.g. the hair or body of the subject have leaked in to the clean plate.

Areas of the image which should be removed from the key process and made transparent can be marked by supplying a mask - e.g. using the Curve Mask Generator - to the “Garbage Matte Image” input.

Following clean plate generation, the process automatically extracts “good” solid colours from the image. These are colours which are sufficiently far from the key colour that they are considered opaque and will be retained as-is throughout the keying process. These can be visualised using the “Show Generated RGB Palette” option.

The Contributing Clean Plate Pixels View The Generated Clean Plate View
chromakeykeycontributingpixels chromakeycolourkey

2. Fully Keyed Areas #

Use “Show Fully Keyed Areas” to visualise the pixels which are considered completely transparent after the keying process and adjust this using the Difference Threshold parameter. Ensure that only pixels are marked which are completely part of the background as they will be removed completely.

The Fully Keyed Areas View

3. Solid Areas #

Use “Show Solid Areas” to visualise the pixels which are considered “solid” - they will be left completely unprocessed, as-is. The Hue Difference Threshold and Saturation Difference Threshold parameters adjust this. Attempt to make as many pixels “solid” as possible - but if a pixel should be semi-transparent or if it needs any spill processing it should not be part of the solid area.

Pixels which are not part of the solid area or fully keyed area will be processed for transparency evaluation, and optional spill suppression. This does not mean a pixel will end up with a semi-transparent value; it may be evaluated and found to be completely solid or transparent. It is important to ensure that if a pixel may need processing - it may be semi-transparent or may have some spill to process - it must not be in the fully keyed or solid areas. However, the more correct the solid and fully keyed areas are, the less unnecessary processing will occur and the better the result will be.

The Solid Areas View

4. Transparencies / Edges #

There are two methods by which the transparency/alpha channel may be generated, selected via the “Transparency Output Mode” parameter:

  • Transparency: attempts to generate a full alpha channel, with semi-transparent areas in the source such as hair or glass preserved as semi-transparent values in the alpha channel.
  • Mask: all pixels in the alpha channel are either fully opaque or fully transparent. Both methods use the same source data; setting up the solid and fully keyed areas properly is essential for both methods. The “Mask” method fills in the rest of the pixels, that fall into neither area, using segmentation; the transparency method attempts to reconstruct an alpha value using probabilities. The “Mask” method may be easier to control and obtain clean results in many cases, but at the cost of being unable to handle transparent features such as hair.

Use “Show Output Matte” to view the resulting transparency/alpha matte. The matte may now be refined. “Harden Transparencies Amount” acts as a contrast adjustment on the matte, with “Harden Transparencies Threshold” a centre point for adjustment. This is used to harden the matte. Transparency Black Clip and Transparency White Clip adjust the in and out points of the mask, clipping the higher or lower transparency values.

“Shrink Transparency Edge” may be used to cut in the edge of the transparency matte by a small amount to remove halos.

The “Antialias Transparency” option applies antialiasing post process techniques to the alpha channel after generation to smooth / anti-alias the edges of the matte, at the cost of additional processing time.

The Output Matte View

5. Spill Suppression #

For many input sources, the above steps will be sufficient to give a good keyed result. However, in some cases where there is significant chroma spill - i.e. pixels which should be opaque reflecting the chroma colour, and therefore being detected incorrectly - additional processing is required.

The simplest way to remove spill and resolve colour channel errors is to use the “Global Suppression” option. This performs a simple process to reduce the key colour’s contribution to colours, which often gives a plausible result at low cost. If this is insufficient, spill suppression may be used, as follows.

Turning on “Reflection Suppression” adds an additional processing step which attempts to resolve pixel colours which reflect the key colour, by replacing it with a user-defined Reflection Replacement Colour. Use “Show Reflectance” to visualise the reflectance result.

Initial stages of the process will greatly influence the quality of results in later stages: if the key colour is incorrect or contains a spill from solid colours, or if the solid or fully keyed areas are incorrect, the transparency and reflectance stages will be affected detrimentally. To fix problems go back to those initial stages first.

Reflection estimation is optional and takes additional computation. It is difficult to determine computationally whether a pixel contains the chroma colour because it’s semi-transparent, or because it’s reflective; as such the process works best if asked to handle one or the other. For scenes with lots of transparency, disabling reflection estimation will give the best result. For scenes with reflections it works best if objects in the scene are solid. The additional optional step to harden the alpha channel for the interiors of objects (“Harden Interior Area Transparencies”) - making them more opaque away from the edges - can be particularly useful when combined with reflections, as interior reflective pixels may be incorrectly identified as transparent: hardening interiors can reduce this issue.

6. Refining & Smoothing #

Temporal Noise Reduction applies a temporal median filter to the resulting colour and alpha channels to reduce temporal noise/flicker.

“Smooth Transparency” may be used to smooth the result at the cost of detail. “Smooth Colour” may be used to apply smoothing to transparent or reflective pixel colours. These smoothing processes use additional computation. “Temporal Smoothing” uses previous frame data to both reduce computation and temporal flicker.

Finally, particularly problematic areas of the image may be removed or kept manually using a garbage matte or a holdout matte as appropriate.

If the image suffers from a great deal of “key spill” or reflectance - i.e. large areas of the image are tinted with the key colour that should remain opaque - it may be possible that not enough solid colours can be extracted for the rest of the process to produce a good result. In this case it is possible to input a second reference image using the “Colour Reference Image” input. This image should be a reference that does not contain any tint of the key colour.

Parameters

These properties control the core behaviours of the node.

ParameterDetails
ActiveEnables or disables the effect. Disabling the effect means it will no longer compute, so disabling a node when not in use can improve performance.
Blend ModeChoose how the image blends with the scene. See Blend Modes for details.
Blend AmountChanges the opacity of the new effect over the original image.

These properties identify the colour plate being used.

ParameterDetails
Generation ModeThe method of generating the key colour.
  • Auto generate from Greens : The node looks for strong green colours in the image and uses these to generate a variable key colour.
  • Auto generate from Blues : The node looks for strong blue colours in the image and uses these to generate a variable key colour.
  • Auto-Generate Using Colour Picker Colour As Hint : The node looks for colours similar to the reference key colour in the image and uses these to generate a variable key colour.
  • Flat Plate Using Colour Picker Colour : The key colour parameter is used as the key colour for the whole image.
Key Colour PickerChoose the colour to be keyed out of the image.
Clean Plate Inclusion ThresholdThreshold determining whether a pixel should contribute to the key colour.
Clean Plate Contribution WeightingThreshold determining whether a pixel’s colour should contribute to the key colour, based on saturation.
Show Contributing Clean Plate PixelsShows the pixels which contribute to the key colour image.
Show Generated Clean PlateOutputs the generated key colour image.
Show Generated RGB PaletteOutputs the generated solid colour palette.

These properties find the fully keyed areas of the image, or areas which are to be fully transparent.

ParameterDetails
Difference ThresholdThreshold determining whether a pixel should be considered as matching the key colour, based on colour distance.
Show Fully Keyed AreasShows the pixels which are considered to be fully keyed out / transparent.

These properties control the solid areas of the image, or areas which are to be completely solid.

ParameterDetails
Hue Difference ThresholdThreshold determining whether a pixel should be considered “solid”, based on hue difference from the key colour.
Saturation Difference ThresholdThreshold determining whether a pixel should be considered “solid”, based on saturation difference from the key colour.
Harden InteriorsDetermines whether the interior areas (away from the key/solid boundary) should be increased in contrast.
Harden Interiors AmountThe amount by which the interior areas (away from the key/solid boundary) should be increased in contrast.
Show Solid AreasShows the pixels which are considered fully opaque.

These properties control semi transparent areas of the image, such as thin clothes or hair, where the key transparency needs to vary.

ParameterDetails
Transparency Output ModeSelects the method by which the transparency / alpha channel is generated.
  • Transparency : A full transparency channel is generated with semi-transparent areas reconstructed for features such as glass and hair.
  • Mask : A mask is generated, containing only fully opaque or fully transparent values.
Harden Transparencies AmountThe amount by which the transparency should be increased in contrast (hardened).
Harden Transparencies ThresholdThe centre value used when the transparency is increased in contrast.
Shrink Transparency EdgeDetermines whether the generated transparency channel should be eroded to remove semi-transparent halos.
Shrink Transparency Edge PassesThe number of erosion passes on the transparency channel.
Transparency Black ClipThe black clip value for the transparency matte.
Transparency White ClipThe white clip value for the transparency matte.
Antialias TransparencyApplies antialiasing to the alpha channel as a post process.
Show Output MatteShows the generated opacity channel.

These properties attempt to reduce the effects of the keyed colour on the rest of the image.

ParameterDetails
Global SuppressionUses a simple process to remove the key colour’s chroma from pixel colours.
Reflection SuppressionDetermines whether reflections of the key colour should be detected and replaced with the reflection colour.
Reflection Suppression IntensityScales the intensity of the reflectance.
Reflection Replacement ColourControls the colour of the reflection.
Show ReflectanceShows the generated reflectance, for pixels which are considered to be reflecting the key colour.

These properties attempt to reduce any noise that maybe incurred during the keying process.

ParameterDetails
Temporal SmoothingDetermines whether temporal smoothing should be applied - using data from the previous frame to smooth and improve the results.
Smooth TransparencyDetermines whether the generated transparency channel should be smoothed to remove noise.
Smooth Transparency PassesThe number of smoothing passes on the transparency channel.
Smooth ColourDetermines whether the generated colour channel should be smoothed to remove noise.
Smooth Colour PassesThe number of smoothing passes on the colour channel.
Temporal Noise ReductionApplies a temporal median filter to the colour and alpha channels to reduce temporal noise.

Inputs

NameDescriptionTypical Input
Colour Reference ImageAllows an image to be provided as a source for the solid colour palette. This may be useful in the case where there is too much spill in the keyed image to provide a good palette.Video Loader
Key Plate ImageAllows an image to be provided as a source for clean plate generator. Note that the rest of the algorithm is still performed as is, with the assumption that the clean plate is more or less flat - it won’t work for arbitrary background subtraction.Video Loader
Garbage Matte ImageAllows an image to be provided as a garbage matte. Areas of the matte that are non-zero will be considered as completely keyed out, and will not contribute to the clean plate generator either.Video Loader
Holdout Matte ImageAllows an image to be provided as a hold-out matte. Areas of the matte that are non-zero will be considered as completely solid.Video Loader

Related Videos