md4 v4 file spec 2006.01.17
     by gongo






     the md4 file format is a skeletal version of id software's md3 mesh model format. as is in the gpl'd quake 3 1.32b source base, the md4 format isn't fully implemented. to further complicate things, up until now, there haven't been any publicly availble build tools for md4, so developing for it has been difficult at best.

     as the quest for skeletal animation in quake 3 began, i set out to learn as much as i could about the other file formats implemented in third-party games developed from the quake 3 engine. of particular interest to me was the wolfenstein series, namely return to castle wolfenstein's mds format and enemy territory's mdm/mdx formats. after many nights spent googling in vain for the mdm build tools, i finally stumbled upon the mds build tools... only to find that the skelout plugin was for 3dsmax r3 only...

     in a last ditch attempt to find a suitable quake 3 based skeletal model format, i googled for sdks for elite force, american mcgee's alice, and heavy metal fakk 2. i found the sdk for heavy metal fakk 2, and noticed that the format is more or less like the mdm/mdx, except it's skb/ska. while this system of having a separate base frame and animation file was intreging to me, i noticed that i'd have to do extensive changes to the md4 code already in place in the quake 3 source, which was something that i wasn't quite prepared to do yet... i have no previous experience with skeletal model formats, hence why i was looking into return to castle wolfenstein's mds format, just for simplicity's sake it would be better to work with just one file first. and once again, the skelout plugin was available for 3dsmax r2 and r3 only...

     frustrated, sleep deprived, and confused, my spirits were down and i was about to abandon the project altogether in favor of skm. i figured that since there's much more info about skm available on quakesrc, and it's still being used and developed, it would probably be the best alternative. as i deleted the fakk 2 tools directory, i noticed a folder named 'source'... i unzipped the sdk again, and took a look at the tools source folder, only to find the full source for skelout and max2skl, the skb/ska build tool. i started digging around in the max2skl source and found that it had some preliminary md4 exporting capabilities... from there, i compiled the skelout plugin with the 3dsmax 6 sdk and instantly had a working skl file exporter.

     as i posted on the forums looking for more info about the mds, mdm/mdx, and skb/ska, someone suggested that i simply complete the md4 format and tailor it to my own needs. inspired, my heart took courage, and i set out to finish the md4 exporter and finalize the md4 implementation in the quake 3 source. the following is the result of roughly 2 weeks worth of labor, working on this during every spare moment i had.

typedef struct {
  int ident
  int version
  char name[64]
  int numFrames
  int numBones
  int numSurfs
  int ofsFrames
  int ofsBones
  int ofsSurfs
  int ofsEnd
} md4Header_t

     ident should match MD4_IDENT ("IDP4") and version should match MD4_VERSION (currently "4") in qfiles.h. the offsets will give you the position of the desired data within the file by in place conversion. for example, (md4BoneName_t *)((byte *)header + header->ofsBones) will give you the first md4BoneName_t in the array of md4BoneName_t numBones in length.



typedef struct {
  float bounds[3][2]
  float localOrigin[3]
  float radius
  md4BoneFrame_t bones[numBones]
} md4Frame_t

     bounds gives the frame's min and max for the bounding box. localOrigin is just bounds min + bounds max divided by 2. radius will give the distance to bounds min or max from localOrigin, used for sphere culling. bones is an array of md4BoneFrame_t numBones in length.



typedef struct {
  float matrix[3][4]
} md4BoneFrame_t

     matrix is the bone's axis and offset.



typedef struct {
  char name[32]
  int parent
  int flags
} md4BoneName_t

     parent gives the index to the parent bone in the main bones list. flags holds any of the bone flags MD4_BONE_FLAG_H (head), MD4_BONE_FLAG_U (upper), MD4_BONE_FLAG_L (lower), or MD4_BONE_FLAG_T (tag). bone flags are still currently under development, but future uses may include animating bone groups by flags or using flagged bones as tags.



typedef struct {
  int ident
  char name[64]
  char shader[64]
  int shaderIndex
  int lodBias
  int minLod
  int ofsHeader
  int numVerts
  int ofsVerts
  int numTris
  int ofsTris
  int numBoneRefs
  int ofsBoneRefs
  int ofsCollapseMap
  int ofsEnd
} md4Surface_t

     shader and shaderIndex give the name and index of this surface's shader in the main shader list. currently only one shader per surface is supported. lodBias is a placeholder for use in the renderer. the calculated lod is put in lodBias so that when the surface is passed to the renderer, the proper level of detail collapse map will be used. lodBias is 0 by default (full detail). minLod is the maximum level of collapsing that can be done before the mesh starts to substantially lose shape, or in other words, the lowest level of detail. ofsCollapseMap gives the position of the collapse map for this surface. it is an array of ints numVerts in length.



typedef struct {
  float vertex[3]
  float normal[3]
  float texCoords[2]
  int numWeights
  md4Weight_t weights[numWeights]
} md4Vertex_t

     vertex is this vertex's position, with normal giving the direction vector of the normal. texCoords gives the vertex position on the texture. weights is a list of md4Weight_t numWeights in length. each vertex can have no more than 8 influences or weights.



typedef struct {
  int boneIndex
  float boneWeight
  float offset[3]
} md4Weight_t

     boneIndex gives the index to the bone in the array of bone references for the surface, not the main bones list. boneWeight is any value between 0.01 and 1.0. the sum of all bone weights for any given vertex must equal 1.0. offset gives the direction vector of the weight's influence.



typedef struct {
  int indexes[3]
} md4Triangle_t

     indexes points to the vertex indices from the main vertex list that make up this triangle. indexes is altered for dynamic level of detail via collapse mapping with triangle->indexes[i] = collapsemap[triangle->indexes[i]] to get the index of the vertex this collapses to.



     boneRefs is an array of ints numBoneRefs in length. the values in the array are indices into the main bone list. boneRefs is used in order to know which bones need to be translated and interpolated to get the final frame position for the current surface in the model.



     collapseMap is an array of ints numVerts in length. the values stored in the array are indices into the vertex list for the current surface. the array lists the indices in collapsing order, such that index numVerts is the first to collapse, and index 0 is the last vertex and never collapses. every collapse removes 2 triangles along the edge uv where u is the collapsing vertex and v is the point u collapses to. this allows scaling of level of detail to a resolution of 1 vertex. ideally, one should not collapse the mesh down past the minLod in order to avoid excessive distortion of the mesh's overall shape. for example, to collapse a mesh down to it's minimum resolution, all vertices with index [minLod, numVerts) should be replaced with collapseMap[index] until the value is within [0, minLod), leaving minLod vertices to be rendered. in the process, some triangles will have all 3 vertices collapse together. these triangles need to be removed from the render list to avoid rendering one-dimensional triangles. generally speaking, texture mapping is retained by using the texCoords of the vertex collapsed to. the result is fairly faithful to the original full resolution texture mapping, with an acceptable amount of error (seams in texture mapping become more obvious). implementation in-game would entail scaling the level of detail from minLod at lowest to numVerts at highest, with the varying degrees of detail more or less spaced evenly between the two points.



  MD4_MAX_VERTS 4096

     bones, frames, and surfaces maximums are per model. triangle and vertex maximums are per surface. note that these limits are similar to those of the md3 format for consistency's sake. i could set the limits higher, but given that the md4 spec technically maxes out at 8192 tris/surf * 32 surfs = 262144 tris, i don't think it's really necessary...