Video Inpaint ML
Description
Video Inpaint ML removes areas covered by an input alpha channel and generates a replacement using machine learning. Use it to remove unwanted objects, such as a person, wire, or tracking marker, while filling the background consistently over time.
Create the removal matte upstream, then generate either the Work Range or Current Frame. Generation runs as a background task and saves each result to the node’s assets. Use the Tasks window or Asset Browser to choose the result shown in the node’s output.
Node Group
AI/ML, Image.
Workflow
1 Load the footage, create a Session and save the project.
2 Create the removal matte. Use a Roto node to draw and track shapes around the unwanted object, or create an ML matte using Mask ML and Matte Assist ML. Include motion blur, shadows, and reflections where necessary. You may need to grow the matte with a Matte Repair node to fully cover the object’s edges. Adjust the matte Blur for a smooth transition.
3 Connect the matte to Video Inpaint ML. Add Video Inpaint ML after your matte-generating nodes. The image’s alpha channel defines the removal area, while RGB supplies the footage used to generate the fill.
4 Run Auto DOD if your source has a variable resolution, such as a stabilized plate, since Video Inpaint ML requires a consistent image size throughout the sequence.
5 Set the generation controls. Start with Quality at 12 and increase it as necessary. Change Random Seed to try a different fill, then choose a generation method:
• Current Frame: Creates a single repaired image using neighboring frames as context. Bring the result into the tree as a source, hold it, and track it into place over the original footage.
• Work Range: Creates a sequence of repaired frames over the session’s work range. Set the desired range before generating.
6 Choose the generated result. Monitor progress in the Tasks window. When generation completes, click Use Latest, or click Show Results and choose a result in the Asset Browser. Results are not selected automatically.
Controls
Auto DOD
Auto DOD analyzes the sequence, inserts a DOD (Domain of Definition), and automatically crops the footage to a fixed image size for processing.
Note: Video Inpaint ML requires a consistent image size throughout the sequence. If your source has a variable resolution, such as a stabilized plate, run Auto DOD first.
Frame Range
Select from All, Work Range or Custom.
Start Frame
With a Custom frame range, selects the start frame.
End Frame
With a Custom frame range, selects the end frame.
DOD Padding %
Controls the percentage of padding around the DOD.
Size Consistency %
Controls how much the DOD box is allowed to change size from frame to frame. At low values the box tightly follows the subject and changes size often. At high values the box stays the same size and provides a more stable working area. The default setting of 50% offers a balance between adaptability and stability.
Static Max DOD
Creates a fixed DOD that covers the combined image or matte bounds over the selected frame range, including DOD Padding. The DOD maintains the same size and position on every frame. Use for variable-resolution sources, such as stabilized plates, so Video Inpaint ML receives a consistent image size. Size Consistency is disabled when this option is enabled.
Generate
Work Range
Generates a fill for the session’s current work range. The result is saved as a sequence in the node’s assets. Set the work range before starting the task. Changing the range afterward does not change an existing result.
Current Frame
Generates a fill for the current frame using neighboring frames as context. Only the current frame is saved, and the resulting asset appears at that frame when selected. Use Current Frame to create a single repaired image for a held plate that can be tracked into place and composited over the original footage.
Random Seed
Sets the seed for the noise used to generate the fill. Change it to try a different replacement with the same removal matte.
Quality
Sets the number of denoising steps used to generate the fill. Lower values process faster; higher values allow more steps to develop the result.