FaceAnim

Although simple to use, the FaceAnim tool is actually quite powerful. It lets you use a 2D video source to produce data that can be used to drive joysticks within the Facial Editor. This document explains how to use it.

FaceAnim Video Tool Overview

The Save, Load, and Print buttons are non-functional, standard MFC UI elements that can be ignored. Several file menu options, however, are required to get started.

Initial Setup

There are several basic principles to FaceAnim. First, there is a standard set of markers required so that the data loaded into the Facial Editor can be used.

Above is an image of where the markers should be located physically on an actor's face. This is the basic marker setup that is compatible within the Facial Editor once compatible joystick names are created for them.

Required Markers for the Actor

• Nose: Reference marker for the set.
• Chin: Marker used for jaw movement. Should be placed above the bottom of the jaw line so that from a 2D, forward looking perspective a clear cross (+) is visible.
• Left_Mouthcorner: Center of cross should not be placed directly at the corner of the mouth. The tip of the horizontal bar of the cross should instead only touch the mouth corner.
• Right_Mouthcorner: Center of cross should not be placed directly at the corner of the mouth. The tip of the horizontal bar of the cross should instead only touch the mouth corner.
• Left_lip_top: Should be placed below each nostril, while the bottom of the cross of the marker should not cover the lip itself.
• Right_lip_top: Should be placed below each nostril, while the bottom of the cross of the marker should not cover the lip itself.
• Left_nose: Marker should be placed near the nose where the strongest level of movement will occur, usually just above the nostril and below the pupil.
• Right_nose: Marker should be placed near the nose where the strongest level of movement will occur, usually just above the nostril and below the pupil.
• Left_cheek: Marker best placed on the higher areas of the cheek bone.
• Right_cheek: Marker best placed on the higher areas of the cheek bone.
• Left_eyebrow_left: Brow markers should be drawn far enough away from the brow line so the full marker avoids the hair of the brow, but close enough to it to get accurate movements.
• Right_eyebrow_left: Same as above.
• Left_eyebrow_right: Same as above.
• Right_eyebrow_right: Same as above.

Capturing the Data

General Rules
There are several rules to remember when capturing data for use with FaceAnim.

• You will need a neutral background to record with. Blue or a mid toned gray works best. Internally, video footage is converted to grayscale so color is not important; only luminance.
• Markers need to contrast with the color of the actor's skin. Black for light skin and white for darker skin tones works best.
• Crosses need to have sharp, 90 degree angles at their centers for best results. Vertical and horizontal cross bars work fine.
• Keep neutral lighting conditions and avoid hotspots on the nose or other shiny areas of the skin. Use face powder if necessary.
• Video resolutions have few limitations. For general production purposes a standard 4:3 aspect with a resolution of 640x480 should be enough.

Preparation
Begin by using a standard eye shadow pen and start applying crosses to the actors face. The markers need to be positioned in their logical regions and where the most movement for that part of the face is located (the nose being the only exception).

For darker skinned actors a white marker is more practical. For Caucasian and light skinned individuals, a black marker is most practical. This is because FaceAnim works on contrast between the markers to the actor's skin. If the software cannot read or "see" the markers, the results will be unsatisfactory.

After all markers are laid out, you are ready to start recording.

Recording
A standard camcorder supporting 640x480 at 30 fps should be enough for standard use.

During recording, make sure the actor starts in the neutral or "rest" pose with all markers at their default positions. The software handles occlusions quite well, but for best results the actor should keep head movement to a minimum during the take. The head can move and will actually be translated into movement when imported into the facial editor, but to keep occlusions to a minimum it's best for them to remain as still as possible.

Marker occlusion can be tolerated up to a second or more depending on how many markers become lost in the overall frame. If the occlusion is kept to a single marker or two, you will generally not run into any serious problems during processing.

Note: A fast codec that produces high quality results is important. HUFFYUV codec is one example that produces good quality results and is fast. File sizes will end up much larger than other codec's such as DIVX but the results will be more accurate. However Huffyuv wouldn't work on a 64bit machine, then you could for example use Cinepak codec.

Processing the Data

After capturing the video, you can process it. You can either load a single video file, or batch process an entire directory.

Keybindings in FaceAnim
L – Toggle display of labels on or off.
E – Switch between Edit Mode and processing mode.
Any other key in Edit Mode – Process frame by frame.

Processing a Single File

• Start FaceAnim and click on the folder icon or navigate to the file menu and click open.
• Find the video file and open it.
• Upon opening the file, FaceAnim will silently process the video and output two files automatically. One is a template text file that will store the capture data when the final processing takes place. The second file is a bitmap image (.bmp) of the first frame of your captured video.
• Navigate to the folder where the videos are stored and check the .bmp image and see if the first frame was captured correctly. Occasionally compression artifacts or other issues will be present on the first frame depending on the codec you use. Just open the video in any video editing software and either re-save the file or delete the first few bad frames, the .txt file and .bmp file. Then reprocess the video.
• Once you have a good .bmp frame to work with, open it up in any image editing software such as Adobe Photoshop.
• Choose the pen tool or another tool which will not dither the color. Make the color of the pen 100% green (R-0, G-255, B-0).
• You need to place a single green pixel at the center of each marker. Be sure that the pen is set to a diameter of 1 and that hardness, and opacity are both set to 100%. This will make sure that you have a single, pure green pixel at the center of the marker which FaceAnim uses for tracking. Save the .bmp file.
• Once the file has been modified with the identified markers, you can reopen the video file. This will first find the .bmp file with green pixel identifiers and then open Edit Mode of FaceAnim. It can now start tracking markers and processing them into data compatible with the Facial Editor.

• At the top-left of the window, there is some debugging information that displays the current frame, total frames, and current error count (lower is better). It also displays whether you are in Edit Mode or not.
• Pressing E will begin processing the video automatically. Watch the video carefully as it tracks the markers. If you notice the red circles drifting away from their correct position, stop processing by pressing the E key. This will activate Edit Mode.
• Press the L key to display the marker names. In Edit Mode, find where the tracked marker has moved to. Once you have found where it has drifted, click on the small green circle that appears when the mouse pointer is over it.
• Now click the correct position on the face where the marker should be located. The software will try and track the marker from this new position and in most cases will have no trouble doing so. Continue to process the video by pressing the E key again.
• Repeat these steps until the software has finished processing the entire video.

Batch Processing

• Click on the file menu and select Batch Process Folder.
• Navigate to the folder which contains the .avi files you wish to process. Press OK.
• On first run through, this will produce the empty container .txt files which data will eventually be stored in, as well as the first frames of each video in .bmp format.
• Label all .bmp files with a single pure green pixel at the center of each marker.
• Once complete, choose Batch Process Folder again and choose the same folder.
• The software will now begin to process each video individually. Similar to processing a single file you should watch the processing and stop it where necessary to correct marker tracking.
• Repeat until all videos are processed.

After completing the above steps, you will now have a .txt file that contains the tracked marker data ready for import directly into the Facial Editor. The character will need a joystick setup with names identical to those in the calibration file. Please refer to the Facial Editor Tutorial and the Facial Editor Reference documentation for how to load video extracted sequences.