Chroma Key

Method #

This node performs an intelligent “chroma keying” process on an image, by which a specific flat colour element (typically a shade of green) is removed and replaced with a different element (transparency). This is commonly known as “green screen keying”.

It uses an innovative method for chroma keying that attempts to derive transparency values and resolve spill with a high degree of automation, making it relatively easy to achieve a good result; but the process it uses may appear unfamiliar. It is strongly recommended to follow the process outlined in the “Usage” section below in order to get the best possible results.

The node first generates a “clean plate”: an image that looks like an empty background with no subjects in shot, but with allowance for variation across the plate. Next, the transparency matte is generated; then spill may be suppressed in the RGB channel if necessary.

Usage #

The workflow for generating the best possible keying result is detailed below. The node is often capable of generating a good result with only minimal adjustment from defaults when the input is a well-lit, well-separated source with a clean background; some problem cases or ideal refinements will take more tuning.

1. Clean Plate Generator #

First a good clean plate image must be generated.

The node uses a reference colour as the initial estimate. Using the input image it detects pixels which are “solid” if they are sufficiently different from the reference colour, groups them into continuous areas and removes them from the image. The remaining pixels are considered the “clean plate”. The holes are then filled by estimation. This results in a “clean plate” background image which allows for local variations in lighting and tone. Generating a high quality clean plate is essential for keying to work well. It should contain as much of the original image as possible but not contain any pixels which are not part of the background: any part of the foreground subject, including low-transparency hair pixels, must be avoided.

The first step in clean plate generation is to select a reference colour using the Generation Mode property. For a green screen, “Auto-Generate From Greens” often gives the best result, and similarly “Auto-Generate From Blues” often does best for blue screens. In order to pick a colour, use “Auto Generate Using Colour Picker Colour As Hint” and then pick a colour using the Key Colour Picker property.

Next, ensure that the right set of pixels are in the clean plate. Use “Show Contributing Clean Plate Pixels” to visualise which pixels of the image are considered “clean plate” pixels, then use the “Clean Plate Inclusion Threshold” parameter to adjust which pixels may contribute to the clean plate. Use “Show Generated Clean Plate” to visualise the generated clean plate image. Ensure that no parts of e.g. the hair or body of the subject have leaked in to the clean plate.

Areas of the image which should be removed from the key process and made transparent can be marked by supplying a mask - e.g. using the Curve Mask Generator - to the “Garbage Matte Image” input.

Following clean plate generation, the process automatically extracts “good” solid colours from the image. These are colours which are sufficiently far from the key colour that they are considered opaque and will be retained as-is throughout the keying process. These can be visualised using the “Show Generated RGB Palette” option.

The Contributing Clean Plate Pixels View	The Generated Clean Plate View

2. Fully Keyed Areas #

Use “Show Fully Keyed Areas” to visualise the pixels which are considered completely transparent after the keying process and adjust this using the Difference Threshold parameter. Ensure that only pixels are marked which are completely part of the background as they will be removed completely.

3. Solid Areas #

Use “Show Solid Areas” to visualise the pixels which are considered “solid” - they will be left completely unprocessed, as-is. The Hue Difference Threshold and Saturation Difference Threshold parameters adjust this. Attempt to make as many pixels “solid” as possible - but if a pixel should be semi-transparent or if it needs any spill processing it should not be part of the solid area.

Pixels which are not part of the solid area or fully keyed area will be processed for transparency evaluation, and optional spill suppression. This does not mean a pixel will end up with a semi-transparent value; it may be evaluated and found to be completely solid or transparent. It is important to ensure that if a pixel may need processing - it may be semi-transparent or may have some spill to process - it must not be in the fully keyed or solid areas. However, the more correct the solid and fully keyed areas are, the less unnecessary processing will occur and the better the result will be.

4. Transparencies / Edges #

There are two methods by which the transparency/alpha channel may be generated, selected via the “Transparency Output Mode” parameter:

• Transparency: attempts to generate a full alpha channel, with semi-transparent areas in the source such as hair or glass preserved as semi-transparent values in the alpha channel.
• Mask: all pixels in the alpha channel are either fully opaque or fully transparent. Both methods use the same source data; setting up the solid and fully keyed areas properly is essential for both methods. The “Mask” method fills in the rest of the pixels, that fall into neither area, using segmentation; the transparency method attempts to reconstruct an alpha value using probabilities. The “Mask” method may be easier to control and obtain clean results in many cases, but at the cost of being unable to handle transparent features such as hair.

Use “Show Output Matte” to view the resulting transparency/alpha matte. The matte may now be refined. “Harden Transparencies Amount” acts as a contrast adjustment on the matte, with “Harden Transparencies Threshold” a centre point for adjustment. This is used to harden the matte. Transparency Black Clip and Transparency White Clip adjust the in and out points of the mask, clipping the higher or lower transparency values.

“Shrink Transparency Edge” may be used to cut in the edge of the transparency matte by a small amount to remove halos.

The “Antialias Transparency” option applies antialiasing post process techniques to the alpha channel after generation to smooth / anti-alias the edges of the matte, at the cost of additional processing time.

5. Spill Suppression #

For many input sources, the above steps will be sufficient to give a good keyed result. However, in some cases where there is significant chroma spill - i.e. pixels which should be opaque reflecting the chroma colour, and therefore being detected incorrectly - additional processing is required.

The simplest way to remove spill and resolve colour channel errors is to use the “Global Suppression” option. This performs a simple process to reduce the key colour’s contribution to colours, which often gives a plausible result at low cost. If this is insufficient, spill suppression may be used, as follows.

Turning on “Reflection Suppression” adds an additional processing step which attempts to resolve pixel colours which reflect the key colour, by replacing it with a user-defined Reflection Replacement Colour. Use “Show Reflectance” to visualise the reflectance result.

Initial stages of the process will greatly influence the quality of results in later stages: if the key colour is incorrect or contains a spill from solid colours, or if the solid or fully keyed areas are incorrect, the transparency and reflectance stages will be affected detrimentally. To fix problems go back to those initial stages first.

Reflection estimation is optional and takes additional computation. It is difficult to determine computationally whether a pixel contains the chroma colour because it’s semi-transparent, or because it’s reflective; as such the process works best if asked to handle one or the other. For scenes with lots of transparency, disabling reflection estimation will give the best result. For scenes with reflections it works best if objects in the scene are solid. The additional optional step to harden the alpha channel for the interiors of objects (“Harden Interior Area Transparencies”) - making them more opaque away from the edges - can be particularly useful when combined with reflections, as interior reflective pixels may be incorrectly identified as transparent: hardening interiors can reduce this issue.

6. Refining & Smoothing #

Temporal Noise Reduction applies a temporal median filter to the resulting colour and alpha channels to reduce temporal noise/flicker.

“Smooth Transparency” may be used to smooth the result at the cost of detail. “Smooth Colour” may be used to apply smoothing to transparent or reflective pixel colours. These smoothing processes use additional computation. “Temporal Smoothing” uses previous frame data to both reduce computation and temporal flicker.

Finally, particularly problematic areas of the image may be removed or kept manually using a garbage matte or a holdout matte as appropriate.

If the image suffers from a great deal of “key spill” or reflectance - i.e. large areas of the image are tinted with the key colour that should remain opaque - it may be possible that not enough solid colours can be extracted for the rest of the process to produce a good result. In this case it is possible to input a second reference image using the “Colour Reference Image” input. This image should be a reference that does not contain any tint of the key colour.

Parameters

Inputs