Animated Blendshapes Tutorial

Overview

This article covers how to create and implement animated blendshapes from Maya into CRYENGINE.

Tutorial Files

Source Maya ASCII scenes with exported CRYENGINE files:

(Do remember that you still need to have a modified "SkeletonList.xml" where you have added the skeleton CHR of this tutorial. Just in case you want to skip all these steps and want to open the character in CE Character Tool: SkeletonList.zip )

Backup your "SkeletonList.xml" before overwriting it!

Setup Your Content In Maya

Requirements for the blendshape scene in Maya:

All blendshape meshes (do not delete them!) must exist in your Maya scene and should be in the same world space location as the skinned base mesh. So, move your blendshape meshes on top of the skinned base mesh:
In some rare situations you may have setup your Maya as a Z-Up. If that is the case then for this tutorial you will need to reset it to a Y-Up. You must also set your scene to 30FPS and to "centimeters" when it comes to exporting:
You must "smooth bind" at least one joint to the blendshape base mesh!
In the tutorial scene the base head mesh was bound to a some common joints (top spine, neck, head, jaw, etc.).
Keep the maximum influence count to eight! For performance sake keep it to four.
CPU/Software Skinning supports 8 weights per vertex and blendshapes.
GPU Skinning support supports 8 weights per vertex, but no blendshapes.
The top-level bound joint is "Spine04". Don't skin the "root" joint into your character mesh - it is for CRYENGINE export:
Notice the empty "SceneRoot" group and "root" joint created for CRYENGINE orientation:
Make sure the root node of the skeleton hierarchy has "zero" rotations - this means CRYENGINE interpretes it as Zero. Since CRYENGINE 3.8 and in respect to Maya there is a a properly orientated, empty "SceneRoot" group node and a "root" joint as the top-level node of the deforming skeleton. They have been setup with both looking forward with their Z-axes aligned to the World Y-axis and the their Y-axes aligned to the World Z-axis:
For each blendshape mesh you must create a joint in the origin and name it <BLENDSHAPE_MESH_NAME> + "_blendWeightVertex", e.g. "sdk_player_head_brows_raise_blendWeightVertex". The "blendWeightVertex" joints have been parented under the root joint for the skeleton hierarchy. (Use the script below in step 5!)
You must map the output range of the blendShape node weights from 0 to 1 to 0 to 100 of the translateX attribute of these joints. (Add an inbetween multiplyDivide node!)

Rather than manually creating the joints and connections you may want to use the Python script below for the setup:

from maya import cmds

def createBlendControlVertex(node, p=None, connect=True, debug=1):
    '''
    Check outgoing "weight" connections of a blendShape node, create the appropriate blendWeightVertex joints. 
  Optional: Connect and map the "weight" output range [0,1] of the blendShape node to [0,100] to the 
  translateX attribute of these helper joints. 
  Optional: Parent all blendWeightVertex joints under a transform node
    '''
    if debug: print node
    blendNodes=[]
    blendVertexJnts=[]
    try:
        shapes = cmds.blendShape(node, t=1, q=1)
        shapeNode = cmds.listRelatives(node, children=1, shapes=1)
        blendShapeNode = cmds.ls(cmds.listHistory(node),type='blendShape')[0]
        for shape in shapes:
            n = shape.split('mesh__')[-1] + '_blendWeightVertex'
            if n not in blendVertexJnts:
                cmds.select(cl=1)
                jnt = cmds.joint(name=n)
                blendVertexJnts.append(jnt)
                blendNodes.append(jnt)
            if connect:
                try:
                    mult = cmds.shadingNode('multiplyDivide', asUtility=1, name = shape + '_MULT')
                    cmds.connectAttr((blendShapeNode + '.' + shape.split('mesh__')[-1]), mult + '.input1.input1X')
                    cmds.setAttr(mult + '.input2X', 100)
                    cmds.connectAttr(mult + '.output.outputX', (n + '.translateX'))
                except Exception as e:
                    print e
    except:
        print 'createBlendControlVertex>>>', node, 'has no blendshapes!'
    if p:
        if blendNodes:
            cmds.parent(blendNodes, p)
    return blendNodes

Paste the script into a Python script tab of Maya's Script Editor and execute it just once. Then run the script by the command. REMEMBER: If you decide to use another head mesh then don't forget to replace the "sdk_player_head":

createBlendControlVertex( 'sdk_player_head', p=None, connect=True, debug=1 )

You should parent those blendWeightVertex joints to the CRYENIGINE "root" joint. However, you can also run the command (shown below) if the "root" joint has already been created:

createBlendControlVertex( 'sdk_player_head', p='root', connect=True, debug=1 )

A dummy quad/triangle polymesh object must be created and smooth skinned to a node of the skeleton hierarchy - it's best to select and include the top-most deforming influence as the only deforming joint.
Why do you need this dummy? Because this node only includes information about the skeleton hierarchy and thus keeps the model with skin and joint data separated from the skeleton. Hence, when you attach a "skin attachment" in CRYENGINE's Character Tool both model + skinning data are separated from the actual skeleton/joint (s) data. You could also attach the head mesh, then consecutively add more upper body apparel, etc.
Finally, you can add some blendshape animation. If you have animated the blendshape node beforehand, then you will notice the "blendWeightVertex" joints are moving when scrubbing the timeline.

Requirements before exporting to CRYENGINE: (applies to all character exports with ".SKIN" and *.CHR files)

A cryExportNode must be created for the skinned mesh with the blendShape node. Use the "TOOLS" from crytek shelf or create as shown in the screenshots below:
A second cryExportNode must be created for the skinned dummy quad/triangle mesh object:
In the cryExportNode of the skinned mesh with your blendshapes, set the export file type as "*.SKIN". Activate "Eight Weights Per Vertex":
In the cryExportNode of the skinned dummy mesh, set the export file type as "*.CHR".
Add a proper material to the skinned blendshape mesh and perhaps a standard Phong material to the dummy object. Click the "MAT.ED" icon from crytek shelf to add a new material group, then add the shaders you have created. A dx11Shader has been added for the "sdk_player_head" mesh and a standard Phong for all the rest. The dxShader will be used later for the wrinkle maps in the wrinkle map tutorial for CRYENGINE.
Make sure you have saved the Maya scene (if you haven't already) before the next step. Press the "EXPORT" icon in crytek shelf and export both cryExportNodes:
Pay special attention to/from what frame you export your "*.SKIN" and "*.CHR" files! Rewind your Maya timeline to the frame where your bind pose is!
In our case we bound the geometry to the skeleton in Frame 0.
In the "Animation Export" tab add a new animation for export. Type in a name for the animation (that will be displayed in CRYENGINE), plus the start and end frame positions. Then point the Exporter to the root node of the skinned character: this is the "root" joint in our case - you may also want to manually add the export folder (this should be a sub-folder of "CRYENGINE_ROOT_FOLDER>\GameSDK\Animations\...").
Finally, press "Export all Anims".
Note: "Export Selected Anims" will show up if you have an animation selected!
The animation named "default" is the standard animation and will be played automatically in the Sandbox Editor (i.e. when you add this character as "AnimObject" entity).
There are 4 files you have exported: a *.CHR file, a *.SKIN file the intermediate animation *.I_CAF file and perhaps the CRYENGINE *.MTL material file.

Setup Your Content In CRYENGINE

This section explains how to add a new character to CRYENGINE using the Character Tool.

Open the Sandbox Editor from your CRYENGINE folder, this will be located somewhere in: "\Bin64\Editor.exe". Open the Character Tool:
The screenshot below shows the left pane split into two vertical layouts, this is activated using the icon highlighted by the red arrow and provides two viewing filters. You might want to set the lower pane to the filter "Animation" in the course of this tutorial.
First add the new skeleton to the SkeletonList:
At the bottom of the list, browse for the *.CHR file you have exported from Maya and give it a name. Press the save icon to save the SkeletonList.xml. You should notice the asterisk will go away ( "*Skeleton List" becomes "Skeleton List" again ).
Create a new *.CDF file and give the CRYENGINE character a nice name:
Assign your exported skeleton ".CHR" file in the skeleton section. The yellow warn boxes will help you fix what is missing to get a correct CRYENGINE character.
Add a skin attachment to the skeleton *.CHR file.
By default the Character Tool will add a "Joint Attachment", change this to a "Skin Attachment" and browse for the *.CHR file you have exported earlier.
This will be the current configuration of the character. Now assign the material you have exported. We will fix the black shaded material for the head in the last step. This is because we assigned it a dx11Shader in Maya, which CRYENGINE doesn't support.
Activate the "Software Skinning" option for the blendshapes to work!
The Character Tool is missing the location of the animations (and more). A *.CHRPARAMS file will handle this. To create a new *.CHRPARAMS for this character, click on the Skeleton field - this is the area marked in red (screenshot above) and that is loaded with the *.CHR.
Add an "Animation Set Filter" and point to the directory of your exported "default.i_caf" animation file! The *.I_CAF file is an intermediate format before compression. If you wish, you can add a new "Animation Events" after this step to get rid of the yellow warning box - just select the animation folder of the *.I_CAF.
Browse the directory containing the *.I_CAF file(s):
On the left you will find the associated animations with the skeleton *.CHR.
Select the "default" animation from the left pane. You now need to add a new *.ANIMSETTINGS file and save it:
In order for the the Character Tool to update your blendshape animation go to the left pane - switch by double-clicking LMB on any existing *.CDF and then go back to this tutorial's *.CDF that you created. (You might need to exit and restart the Sandbox Editor and then go back to the Character Tool).
Browse the left pane for the saved character (sdk_blendshape_tutorial.cdf ). Select and double-click "default" animation from the bottom left (or press spacebar). You should see both the skeletal animation of the head/neck joint AND the blendshape animation playing as well:

Tip: Check for the red marked areas (screenshot above). The options identified will help you inspect the scene you have created.
You may also want to correct the "sdk_player_head" material. Open the Material Editor from the Sandbox Editor and edit the material to something more useful. The Character Tool window will update the changes made:

Summary

To conclude this tutorial, here is a list of the files you created in Maya and exported to CRYENGINE: (replace the file name if you chose to use different names):

saved your Maya scene file as a Maya ASCII file before export
exported a *.SKIN file: sdk_player_head.skin containing the model with skeleton and skinning data
exported a *.CHR file: sdk_player_head_skel.chr containing the skeleton hierarchy
*.CHR and *.SKIN requires you to export from the bind/dag pose frame
exported a *.I_CAF file: default.i_caf containing the animation data
exported a *.MTL file: sdk_player_head.mtl containing the CE material info

In the CRYENGINE Character Tool you edited/created the following files (either by yourself or automatically by the CRYENGINE Character Tool):

In the default SkeletonList.xml in "<CRYENGINE_ROOT_FOLDER>\GameSDK\Animations" folder you added your new skeleton
created a *.CDF file: sdk_player_head_blendshape_tutorial.cdf which defines the animated CRYENGINE character
created a *.CHRPARAMS file: sdk_player_head_skel.chrparams containing the location of the related animation .i_caf/.caf files
created a *.ANIMSETTINGS file: default.animsettings containing the compression ratio for your animation

As a follow-up to this blendshape tutorial we will continue with a wrinkle map setup which will add more detail: Wrinkle Maps Tutorial

More advanced information regarding blendshape animation inside CRYENGINE can be found below - these are mostly used for debugging purposes.

Engine CVars

General CVars:

ca_vaBlendEnable 1/0 - general CVar to enable/disable blend shapes on CPU. default: 1
ca_vaEnable 1/0 - general CVar to enable/disable all CPU based vertex animation (including linear skinning on CPU). default: 1
ca_vaScaleFactor 1 - a multiplier scalar applied on top of the deltas. default: 1
ca_vaProfile 0/1 - displays information about the current blend vertices. default: 0

Blend Culling CVars:

ca_vaBlendCullingThreshold - the higher the number then the more blend shapes will be culled (based on how many pixels they cover). default: 1.0
ca_vaBlendCullingDebug 0/1 - set to 1 to show the difference between 'with' and 'without' blend culling (by toggling the feature every other frame). default: 0
ca_vaBlendPostSkinning 0/1 - perform vertex animation blends post skinning (instead of pre-skinning). default: 0

Tangent Updates

Non-rigid deformation requires tangent updates in order to obtain the correct shading. Geometric real-time tangent updates are expensive, so in order to minimize CPU cost we use vertex colors to transfer a blue (0,0,255) painted mask inside Maya that marks the most important facial parts. The budget for XBox One was ~2K vertices.

Tangent updates only work with 8 weights skinning and software skinning (Flags=8) in the .cdf file.

To enable/disable/preview the triangles marked by the mask in the ENGINE use the following CVars: