While you can do a lot with Premiere Pro and other video editing apps, Adobe After Effects (AE) goes further, adding motion graphics, compositing and other more intensive visual effects work. Here we'll show you how to do rotoscoping with After Effects. The art of roto has been around almost as long as cinema itself. Early silent movies shot on black-and-white film would sometimes have colour painted onto the image — frame by tedious frame; for example, 1925’s Phantom of the Opera. However, it was animator Max Fleischer who patented the practice of taking live action footage and tracing it to create realistic animation. Walt Disney also used this technique in his first feature, Snow White and the Seven Dwarfs.
Over the past century, things haven’t changed much — we just do it on computer now. This involves tracing a vector mask around an object or person in a filmed image so it can be separated out. If the person is moving around, then tracing every frame still becomes necessary. This is different to greenscreen or bluescreen, where the background colour can be keyed out more easily. Roto is used all the time to cut out parts of images, for adding effects to a specific object within a frame, or removing objects from a frame. In fact, rotoscoping and masking, despite their labour-intensive tedium, are crucial skills for anyone getting into video compositing.
Rotoscoping in After Effects
The Mask tool works much like the Pen tool you’d find in an illustration or paint program. You draw a shape around an object (a pen tablet highly recommended) and each node has bezier handles to adjust its curvature and angle. Within AE there’s a RotoBezier mode for masking, which is best to use most of the time since it’s more intuitive than traditional Bezier curves. Here’s a basic approach to getting a good rotoscoped mask on a moving person in a shot.
- Create a new composition (Ctrl-N) for the shot you want to work on. In AE every clip appears on its own track, unlike the Premiere Pro timeline, and a composition might have only one shot.
- Press Ctrl-D to duplicate this clip, so you have the same clip on two tracks. The lower tracks will serve as the background, while the upper one is the one you’ll rotoscope. Start on frame 1. Draw as detailed a mask as you can, following the shape precisely. How accurate you are here will impact on your ‘animation’, as it’s difficult to fix mistakes later. Sometimes if your subject and the background blend together it can be hard to see where the edges are. Use your best guess and stick with it, even if it ends up being a bit wrong. Consistency of movement is more important than pixel-perfect accuracy.
- Press M on the keyboard to reveal the ‘Mask Path’ option on the mask you just created. Click the stopwatch icon to the left to enable keyframe animation.
- Within AE, Ctrl-right arrow/left arrow will move you forward and back one frame at a time. Ctrl-Shift-right arrow/left arrow moves you forward and back 10 frames at a time. It will depend how much movement there is, but often it’s best to jump forward 10 frames and then adjust the mask to fit your subject. Try to move the nodes in a logical way; for example, if there’s a node on the person’s elbow, make sure that node is moved to the same position on the elbow’s new location. This will help create a smooth animation from one keyframe to the next. Use Z-click and Z-Alt-click to zoom and spacebar-drag to move the image around.
- Next, go to frame 5, between the two keyframes you just made and again adjust any nodes that may be off-target. If the movement is erratic, you’ll have to adjust the mask for each and every frame — there’s no way around it. However, if the motion is smooth, fewer keyframes will mean a smoother, less noticeable mask.
