Chroma KeyGenerate a mask for an image using Chroma Keying.
This node performs an intelligent “chroma keying” process on an image, by which a specific flat colour element (typically a shade of green) is removed and replaced with a different element (transparency). This is commonly known as “green screen keying”.
It uses an innovative method for chroma keying that attempts to derive transparency values and resolve spill with a high degree of automation, making it relatively easy to achieve a good result; but the process it uses may appear unfamiliar. It is strongly recommended to follow the process outlined in the “Usage” section below in order to get the best possible results.
The node first generates a “clean plate”: an image that looks like an empty background with no subjects in shot, but with allowance for variation across the plate. Next, the transparency matte is generated; then spill may be suppressed in the RGB channel if necessary.
The workflow for generating the best possible keying result is detailed below. The node is often capable of generating a good result with only minimal adjustment from defaults when the input is a well-lit, well-separated source with a clean background; some problem cases or ideal refinements will take more tuning.
First a good clean plate image must be generated.
The node uses a reference colour as the initial estimate. Using the input image it detects pixels which are “solid” if they are sufficiently different from the reference colour, groups them into continuous areas and removes them from the image. The remaining pixels are considered the “clean plate”. The holes are then filled by estimation. This results in a “clean plate” background image which allows for local variations in lighting and tone. Generating a high quality clean plate is essential for keying to work well. It should contain as much of the original image as possible but not contain any pixels which are not part of the background: any part of the foreground subject, including low-transparency hair pixels, must be avoided.
The first step in clean plate generation is to select a reference colour using the Generation Mode property. For a green screen, “Auto-Generate From Greens” often gives the best result, and similarly “Auto-Generate From Blues” often does best for blue screens. In order to pick a colour, use “Auto Generate Using Colour Picker Colour As Hint” and then pick a colour using the Key Colour Picker property.
Next, ensure that the right set of pixels are in the clean plate. Use “Show Contributing Clean Plate Pixels” to visualise which pixels of the image are considered “clean plate” pixels, then use the “Clean Plate Inclusion Threshold” parameter to adjust which pixels may contribute to the clean plate. Use “Show Generated Clean Plate” to visualise the generated clean plate image. Ensure that no parts of e.g. the hair or body of the subject have leaked in to the clean plate.
Areas of the image which should be removed from the key process and made transparent can be marked by supplying a mask - e.g. using the Curve Mask Generator - to the “Garbage Matte Image” input.
Following clean plate generation, the process automatically extracts “good” solid colours from the image. These are colours which are sufficiently far from the key colour that they are considered opaque and will be retained as-is throughout the keying process. These can be visualised using the “Show Generated RGB Palette” option.
|The Contributing Clean Plate Pixels View||The Generated Clean Plate View|
Use “Show Fully Keyed Areas” to visualise the pixels which are considered completely transparent after the keying process and adjust this using the Difference Threshold parameter. Ensure that only pixels are marked which are completely part of the background as they will be removed completely.
Use “Show Solid Areas” to visualise the pixels which are considered “solid” - they will be left completely unprocessed, as-is. The Hue Difference Threshold and Saturation Difference Threshold parameters adjust this. Attempt to make as many pixels “solid” as possible - but if a pixel should be semi-transparent or if it needs any spill processing it should not be part of the solid area.
Pixels which are not part of the solid area or fully keyed area will be processed for transparency evaluation, and optional spill suppression. This does not mean a pixel will end up with a semi-transparent value; it may be evaluated and found to be completely solid or transparent. It is important to ensure that if a pixel may need processing - it may be semi-transparent or may have some spill to process - it must not be in the fully keyed or solid areas. However, the more correct the solid and fully keyed areas are, the less unnecessary processing will occur and the better the result will be.
There are two methods by which the transparency/alpha channel may be generated, selected via the “Transparency Output Mode” parameter:
Use “Show Output Matte” to view the resulting transparency/alpha matte. The matte may now be refined. “Harden Transparencies Amount” acts as a contrast adjustment on the matte, with “Harden Transparencies Threshold” a centre point for adjustment. This is used to harden the matte. Transparency Black Clip and Transparency White Clip adjust the in and out points of the mask, clipping the higher or lower transparency values.
“Shrink Transparency Edge” may be used to cut in the edge of the transparency matte by a small amount to remove halos.
The “Antialias Transparency” option applies antialiasing post process techniques to the alpha channel after generation to smooth / anti-alias the edges of the matte, at the cost of additional processing time.
For many input sources, the above steps will be sufficient to give a good keyed result. However, in some cases where there is significant chroma spill - i.e. pixels which should be opaque reflecting the chroma colour, and therefore being detected incorrectly - additional processing is required.
The simplest way to remove spill and resolve colour channel errors is to use the “Global Suppression” option. This performs a simple process to reduce the key colour’s contribution to colours, which often gives a plausible result at low cost. If this is insufficient, spill suppression may be used, as follows.
Turning on “Reflection Suppression” adds an additional processing step which attempts to resolve pixel colours which reflect the key colour, by replacing it with a user-defined Reflection Replacement Colour. Use “Show Reflectance” to visualise the reflectance result.
Initial stages of the process will greatly influence the quality of results in later stages: if the key colour is incorrect or contains a spill from solid colours, or if the solid or fully keyed areas are incorrect, the transparency and reflectance stages will be affected detrimentally. To fix problems go back to those initial stages first.
Reflection estimation is optional and takes additional computation. It is difficult to determine computationally whether a pixel contains the chroma colour because it’s semi-transparent, or because it’s reflective; as such the process works best if asked to handle one or the other. For scenes with lots of transparency, disabling reflection estimation will give the best result. For scenes with reflections it works best if objects in the scene are solid. The additional optional step to harden the alpha channel for the interiors of objects (“Harden Interior Area Transparencies”) - making them more opaque away from the edges - can be particularly useful when combined with reflections, as interior reflective pixels may be incorrectly identified as transparent: hardening interiors can reduce this issue.
Temporal Noise Reduction applies a temporal median filter to the resulting colour and alpha channels to reduce temporal noise/flicker.
“Smooth Transparency” may be used to smooth the result at the cost of detail. “Smooth Colour” may be used to apply smoothing to transparent or reflective pixel colours. These smoothing processes use additional computation. “Temporal Smoothing” uses previous frame data to both reduce computation and temporal flicker.
Finally, particularly problematic areas of the image may be removed or kept manually using a garbage matte or a holdout matte as appropriate.
If the image suffers from a great deal of “key spill” or reflectance - i.e. large areas of the image are tinted with the key colour that should remain opaque - it may be possible that not enough solid colours can be extracted for the rest of the process to produce a good result. In this case it is possible to input a second reference image using the “Colour Reference Image” input. This image should be a reference that does not contain any tint of the key colour.
These properties control the core behaviours of the node.
|Active||Enables or disables the effect. Disabling the effect means it will no longer compute, so disabling a node when not in use can improve performance.|
|Blend Mode||Choose how the image blends with the scene. See Blend Modes for details.|
|Blend Amount||Changes the opacity of the new effect over the original image.|
These properties identify the colour plate being used.
|Generation Mode||The method of generating the key colour.
|Key Colour Picker||Choose the colour to be keyed out of the image.|
|Clean Plate Inclusion Threshold||Threshold determining whether a pixel should contribute to the key colour.|
|Clean Plate Contribution Weighting||Threshold determining whether a pixel’s colour should contribute to the key colour, based on saturation.|
|Show Contributing Clean Plate Pixels||Shows the pixels which contribute to the key colour image.|
|Show Generated Clean Plate||Outputs the generated key colour image.|
|Show Generated RGB Palette||Outputs the generated solid colour palette.|
These properties find the fully keyed areas of the image, or areas which are to be fully transparent.
|Difference Threshold||Threshold determining whether a pixel should be considered as matching the key colour, based on colour distance.|
|Show Fully Keyed Areas||Shows the pixels which are considered to be fully keyed out / transparent.|
These properties control the solid areas of the image, or areas which are to be completely solid.
|Hue Difference Threshold||Threshold determining whether a pixel should be considered “solid”, based on hue difference from the key colour.|
|Saturation Difference Threshold||Threshold determining whether a pixel should be considered “solid”, based on saturation difference from the key colour.|
|Harden Interiors||Determines whether the interior areas (away from the key/solid boundary) should be increased in contrast.|
|Harden Interiors Amount||The amount by which the interior areas (away from the key/solid boundary) should be increased in contrast.|
|Show Solid Areas||Shows the pixels which are considered fully opaque.|
These properties control semi transparent areas of the image, such as thin clothes or hair, where the key transparency needs to vary.
|Transparency Output Mode||Selects the method by which the transparency / alpha channel is generated.
|Harden Transparencies Amount||The amount by which the transparency should be increased in contrast (hardened).|
|Harden Transparencies Threshold||The centre value used when the transparency is increased in contrast.|
|Shrink Transparency Edge||Determines whether the generated transparency channel should be eroded to remove semi-transparent halos.|
|Shrink Transparency Edge Passes||The number of erosion passes on the transparency channel.|
|Transparency Black Clip||The black clip value for the transparency matte.|
|Transparency White Clip||The white clip value for the transparency matte.|
|Antialias Transparency||Applies antialiasing to the alpha channel as a post process.|
|Show Output Matte||Shows the generated opacity channel.|
These properties attempt to reduce the effects of the keyed colour on the rest of the image.
|Global Suppression||Uses a simple process to remove the key colour’s chroma from pixel colours.|
|Reflection Suppression||Determines whether reflections of the key colour should be detected and replaced with the reflection colour.|
|Reflection Suppression Intensity||Scales the intensity of the reflectance.|
|Reflection Replacement Colour||Controls the colour of the reflection.|
|Show Reflectance||Shows the generated reflectance, for pixels which are considered to be reflecting the key colour.|
These properties attempt to reduce any noise that maybe incurred during the keying process.
|Temporal Smoothing||Determines whether temporal smoothing should be applied - using data from the previous frame to smooth and improve the results.|
|Smooth Transparency||Determines whether the generated transparency channel should be smoothed to remove noise.|
|Smooth Transparency Passes||The number of smoothing passes on the transparency channel.|
|Smooth Colour||Determines whether the generated colour channel should be smoothed to remove noise.|
|Smooth Colour Passes||The number of smoothing passes on the colour channel.|
|Temporal Noise Reduction||Applies a temporal median filter to the colour and alpha channels to reduce temporal noise.|
|Colour Reference Image||Allows an image to be provided as a source for the solid colour palette. This may be useful in the case where there is too much spill in the keyed image to provide a good palette.||Video Loader|
|Key Plate Image||Allows an image to be provided as a source for clean plate generator. Note that the rest of the algorithm is still performed as is, with the assumption that the clean plate is more or less flat - it won’t work for arbitrary background subtraction.||Video Loader|
|Garbage Matte Image||Allows an image to be provided as a garbage matte. Areas of the matte that are non-zero will be considered as completely keyed out, and will not contribute to the clean plate generator either.||Video Loader|
|Holdout Matte Image||Allows an image to be provided as a hold-out matte. Areas of the matte that are non-zero will be considered as completely solid.||Video Loader|