MoP: declarations not decls

2007-12-04T15:10:14Z

declarations not decls

MoP at 09:40, 17 October 2007

2007-10-17T09:40:41Z

New page

Material decls are similar to shader files from Quake3, but they are a lot more powerful. Materials have a lot of power, especially when combined with scripts and GL programs.

== Tables ==

In Quake3, you could do all kinds of neat things with sin waves, saw tooth waves, square waves and other types of waves, but you were pretty much screwed if you wanted any kind of non standard wave form. In Doom 3, you can define arbitrary data lookup tables, then reference them in your materials.
The format of a table definiton is:

table <tablename> { [snap] [clamp] { <data>, <data>, ... } }
Where [snap] is an optional key word which means "jump directly from one value to the other (don't blend between values)" and [clamp] is an optional key word which means "don't wrap around if the index is outside the range of table elements, (return the first value if less or the last value if it's more)". Both keywords are optional, and can be used together.
Now, armed with this knowledge, we can easily construct a square wave lookup table:

table squarewave { snap { 0, 1 } }
The sin table is a bit harder, but lucky for you, it's already defined shaderDemo.mtr

== Materials ==

Every material file has a global section, followed by one or more stages. The global section, as the name implies, sets properties that affect all the stages. In Quake3 the stages were pretty easy to understand because the engine rendered them in order: stage 1, then stage 2, then stage 3, etc.. In Doom 3 the order is not quite as simple because of the way the lighting system works, but for the most part, they are still rendered in order.
Let's look at a simple material (some random wall in the alpha labs):

material textures/alphalabs/a_lfwall21b
{
qer_editorimage textures/alphalabs/a_lfwall21b
bumpmap textures/base_wall/lfwall21_local
diffusemap textures/alphalabs/a_lfwall21b
specularmap textures/alphalabs/a_lfwall21b_s
}
The first line is the name of the material (which in this case also happens to be the name of the diffuse map). Notice the use of the 'material' keyword. This is required for all materials, unlike Doom3.

The global section begins right after the open curly brace. The first key word we see is qer_editorimage, which tells Radiant which image to use in the editor. This is the image used in the texture window, as well as in the 3D view. Next we define the bump map (normal map), the diffuse map (colors), and the specular map (gloss). This is about as basic of a material file you can get, while still using the advanced lighting features. Technically all you need is 'diffusemap', in which case the specular map is black, the bump map is blue, and the editorimage is the diffusemap. Creating an entire map with those types of materials will look roughly like Quake 3.
In most cases you won't need to specify the qer_editorimage unless you specifically need something which looks different to the diffusemap.

This material looks like it doesn't have any stages, but in fact it has 3 stages. The 'bumpmap' 'diffusemap' and 'specularmap' keywords are simply shortcuts for stages as we will see later.

== Shaders in Materials ==

ETQW has the ability to use custom GL vertex and fragment programs when rendering a stage of a material. This is mostly seen in the 'heat haze' effects around fire, in the warping effects on glass, and in the warping effects on various projectiles (like the BFG). For documentation on how to write your own renderprograms programs, see the separate [[Render program documentation]]

There is nothing stopping an artist from writing those, but I would really let a programmer handle writing the renderprogram. The main thing an artist or level designer needs to know is how to use the renderprogram in their materials.
This is the material file for the hornet smoke which uses one of the heathaze effects. Using other renderprograms works in a similar way:

material particles/penta/hornetheathaze {
noshadows
translucent
nonsolid
{
program heatHazeWithVertex
deformScroll 0, 0
deformMagnitude 1
bumpMap textures/particles/smoke/smokenormal.tga
maskAlpha
}
}

There is a single stage in this material. It uses a special program, this means it will not render like any other material but that it will use that program to do some special effect. It also means that certain parameters are surrendly read by the engine.
So the heatHazeWithVertex causes the engine to render the stage as a heathaze surface, it also looks at the values of deformScroll and deformMagnitude to define the specific strength of the heathaze effect (how much warping is going on). The engine automatically knows at which parameters to read when you use a certain program, so while you can set these parameters on any material only programs that actually use them will care about their value. Also note how the texture is specified using the bumpMap [bleh] command, altough you use the bumpmap command here it won't render like any other bumpmapped surface since the "program heatHazeWithVertex" told the engine to do special rendering for this surface.

The list of new "commands" that are available when you use a certain program is not fixed, so every new program may define new variables you can set. Most programs tend to use the default values like "map", "bumpMap", "diffuseMap", ... so most of the time you don't need to know about any special variables to use the program.

== Maths and Logic ==

Materials in Doom 3 can contain mathematical and logical expressions. These expressions are evaluated for each material every frame, and are what cause normally boring surfaces to come alive. This is a replacement for the rather limited shader commands in Quake 3 such as tcMod scroll, rgbGen, etc.
Let's take a look at a material to see these features in use:

material models/weapons/soulcube/soulcube3fx
{
noSelfShadow
translucent
noShadows
{
if ( parm7 > 3 )
blend add
map models/weapons/soulcube/soulcube3fx
rgb scTable[ time * .5 ]
}
}
In Doom 3, certain stages of the material can be selectively turned on and off. The 'if' command in this example means "only draw this material if parm7 is greater than 3." There are a few places where parm7 can get set, one of them is in the editor with the 'shaderParm0' to 'shaderParm11' keys. Another place is in the script file (such as with weapons). Of course, it can also get set in the code.

Notice the use of a lookup table in this material. Here we are using 'time' (which is a floating point number that increases forever) to look up a value in 'scTable' (which was defined using a 'table' decl earlier).

The mathematical operators you can use in a material are %, /, *, -, and +. You can also use the boolean operators <, >, <=, >=, ==, !=, &&, and ||. The meaning of the symbols is the same as in C/C++, Java, PHP, etc. Mathematical expressions should be enclosed in parenthesis (there are cases where they don't have to be, such as when they are used as an index to a lookup table, but it never hurts to have too many. For the operands, you can use a lookup table, any numerical constant, and any of the following variables:

time: Forever increasing floating point value that returns the current time in seconds
parm0-parm11: Parameters that can be set a number of ways (discussed above)
global0-global7: Not used right now
fragmentPrograms: Constant that is 1 if ARB fragment programs are available. This is mostly used for disabling a stage by specifying "if ( fragmentPrograms == 1 )"
sound: The current sound amplitude of the entity using this material. This is used to create light materials that pulse with the sound.

== New backend ==

The new backend slightly changes the way how shaders work, altough internally the system is very different we tried to keep external changes as small as possible. The next few topics are mainly "copy pastable" cases.

=== Alphatested surfaces ===

Unlike doom3 bump/specular/diffuse maps are not treated as separate material stages anymore, so instead of having three stages with one map each you now have one stage with the three maps specified at once. This makes for less stages and a generally more intuitive way of creating shaders.
The biggest change because of this is the alpha testing. While you specified the bump and specular stages like normal stages and specified added the alphatest on the diffuse stage before, now you need one stage with has diffuse+bump+specular maps with alpha testing enabled.
An alpha tested stage now looks like this:

material textures/alphademo {
{
diffusemap textures/decals/fgrill2_d.tga
specularmap textures/decals/fgrill2_s.tga
bumpmap addnormals( textures/decals/fgrill_local.tga , heightmap ( textures/decals/fgrill2_b.tga, 1 ) )

alphaTest 0.5
}
}

=== Clamped textures ===

Clamping was specified in the shader/stage body before, this is not very logical as it is actually an image parameter. Therefore clamping now has to be specified before image names.

material textures/clampdemo {
{
blend add
map clamp textures/decals/blast.tga
}
}

=== Animated texture coordinates ===

Since bump/normal/specular are now part of a single stage( see alphatest information ) the texture matrices can't be specified per stage anymore, therefore there is a new "texturematrix" command you can use to specify the texture matrix of a specific texture.

material textures/matrixdemo {
{
bumpmap textures/base_wall/wire_fence2_local.tga
specularmap textures/base_wall/a_wire_fence2_s.tga
diffusemap textures/base_wall/a_wire_fence2_d.tga
alphaTest 0.5

textureMatrix bumpMatrix {
translate 2, 2
}

textureMatrix specularMatrix {
translate 4, 4
}

textureMatrix diffuseMatrix {
translate 2, 2
}
}

You can still use the old texture matrix system, but this means it will only affect the diffuse map and NOT the specular/normal map. If you are writing a non bumpmapped material you should also use the old way of specifying texture matrices.

== Reference (Beware: copy paste from doom 3 out of date likely!) ==

=== Global Keywords for regular materials ===
{| style="border: 1px solid #AAAAAA;" width="100%"
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | qer_editorimage <map>
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Image to display in the editor
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | description <string>
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Just a simple description for people using this material
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | polygonOffset
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | [float] offset the depth buffer to combat z-fighting
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | noShadows
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Don't cast shadows
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | noSelfShadow
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | This material doesn't cast shadows on the model it's on (but it does on other models)
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | forceShadows
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Allows nodraw surfaces to cast shadows
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | noOverlays
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Overlay / Decal suppression
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | forceOverlays
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Force decal overlays for alpha tested or translucent surfaces
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | translucent
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | The engine thinks this is a translucent material (which means no ambient/zfill pass will be done for this material)
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | forceOpaque
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Opposite forces the engine to think this is a nontranslucent material.
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | twoSided
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Draw the front and back. Implies no-shadows, because the shadow volume would be coplanar with the surface, giving depth fighting</td>
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | backSided
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Draw only the back. This also implies no-shadows</td>
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | mirror
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Use to make mirrors
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | noFog
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Don't fog this surface
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | unsmoothedTangents
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Uses the single largest area triangle for each vertex, instead of smoothing all
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | guisurf <guifile>
guisurf entity[2|3]
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | This surface has a gui on it. Use "somegui.gui" to specify the gui, or entity, entity2, entity3, etc for the level designer to set it in Radiant.
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | sort <type>
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Type is one of: subview, opaque, decal, far, medium, close, almostNearest, nearest, postProcess
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | spectrum <int>
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Spectrums are used for "invisible writing" that can only be illuminated by a light of matching spectrum
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | deform <type>
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Type is one of: sprite, tube, flare, expand, move, turbulent, eyeBall, particle, particle2
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | decalInfo <staySeconds>
<fadeSeconds> [start rgb] [end rgb]
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Used in decal materials to set how long the decal stays, and how it fades out.
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | renderbump <args...>
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | Renderbump command options, without "renderbump" at the start
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | diffusemap <map>
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | shortcut for
{
blend diffusemap
map <map>
}
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | specularmap <map>
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | shortcut for
{
blend specularmap
map <map>
}
|-
| valign="top" style="border-bottom: 1px solid #AAAAAA; background: #F0F0F0;" | bumpmap <map>
| valign="top" style="border-bottom: 1px solid #AAAAAA;" | shortcut for
{
blend bumpmap
map <map>
}
|-
| valign="top" style="background: #F0F0F0;" | DECAL_MACRO
| valign="top" | shortcut for
polygonOffset 1
discrete
sort decal
noShadows
|}

=== Global Keywords for light materials ===
<table border = 1>
<tr>
<td>noShadows</td>
<td>This light doesn't cast shadows</td>
</tr>
<tr>
<td>forceShadows</td>
<td>fog, blend, and ambient lights don't cast shadows by default. This forces them to cast shadows</td>
</tr>
<tr>
<td>noPortalFog</td>
<td>This fog volume won't ever consider a portal fogged out</td>
</tr>
<tr>
<td>fogLight</td>
<td>Option to fill with fog from viewer instead of light from center</td>
</tr>
<tr>
<td>blendLight</td>
<td>Perform simple blending of the projection, instead of interacting with bumps and textures</td>
</tr>
<tr>
<td>ambientLight</td> <td>An ambient light has non-directional bump mapping and no specular</td>
</tr>
<tr>
<td>lightFalloffImage <map> </td><td>specifies the image to use for the third axis of projected light volumes</td>
</tr>
</table>

=== Global Surface Parameters ===
solid may need to override a clearSolid
water used for water
playerclip solid to players
monsterclip solid to monsters
moveableclip solid to moveable entities
ikclip solid to IK
blood used to detect blood decals
trigger used for triggers
aassolid solid for AAS
aasobstacle used to compile an obstacle into AAS that can be enabled/disabled
flashlight_trigger used for triggers that are activated by the flashlight
nonsolid clears the solid flag
nullNormal renderbump will draw as 0x80 0x80 0x80, which won't collect light from any angle
areaportal divides areas
qer_nocarve don't cut brushes in editor
discrete surfaces should not be automatically merged together or clipped to the world, because they represent discrete objects like gui shaders mirrors, or autosprites
noFragment dmap won't cut surface at each bsp boundary
slick affects game physics (not implemented?)
collision collision surface. if a model has no collision surfaces, then all surfaces are considered collision surfaces
noimpact don't make impact explosions or marks
nodamage no falling damage when hitting
ladder player can climb up this surface
nosteps no footstep sounds
metal metal
stone stone
flesh flesh
wood wood
cardboard cardboard
liquid liquid
glass glass
plastic plastic
ricochet behaves like metal but causes a ricochet sound
surftype10 not used
surftype11
surftype12
surftype13
surftype14
surftype15

=== Stage Keywords ===
blend <type>
blend <src>, <dst>

Blend types: Type Src Dst
blend gl_src_alpha gl_one_minus_src_alpha
add gl_one gl_one
filter gl_dst_color gl_zero
modulate gl_dst_color gl_zero
none gl_zero gl_one
bumpmap Normal map
diffusemap Diffuse map
specularmap Specular map

Source blend modes: gl_one Constant 1
gl_zero Constant 0
gl_dst_color The color currently on the screen
gl_one_minus_dst_color One minus the color currently on the screen
gl_src_alpha The alpha channel of the source image
gl_one_minus_src_alpha One minus the alpha channel of the source image
gl_dst_alpha The alpha channel of the screen image
gl_one_minus_dst_alpha One minus the alpha channel of the screen image
gl_src_alpha_saturate Minimum of the source alpha and one minus screen alpha

Destination blend modes: gl_one Constant 1
gl_zero Constant 0
gl_src_color The color of the source image
gl_one_minus_src_color One minus the color of the source image
gl_src_alpha The alpha channel of the source image
gl_one_minus_src_alpha One minus the alpha channel of the source image
gl_dst_alpha The alpha channel of the screen image
gl_one_minus_dst_alpha One minus the alpha channel of the screen image

map <map> The image program to use for this stage
remoteRenderMap <int> <int> Width and Height of the buffer to render a remote image in to (for cameras). The entity this material is applied to has to support remote render views.
mirrorRenderMap <int> <int> Width and Height of the buffer to render a mirror in to. This of course makes this stage a mirror stage, which is different from using the 'mirror' global keyword because that makes the entire material a mirror, rather than just one stage.
videomap [loop] <file> This stage uses a video stream as an image map
soundmap [waveform] This stage uses a sound meter from the sound system as an image map. Specify 'waveform' to get a scope rather than bars.
cubeMap <map> This stage uses a cube map as the image map. Looks for _px, _py, _pz, _nx, _ny, _nz for the positive x, y, z, and negative x, y, z sides
cameraCubeMap <map> This stage uses a cube map in camera space. Looks for _forward, _back, _left, _right, _up, and _down
ignoreAlphaTest Always use DEPTHFUNC_LEQUAL rather than DEPTHFUNC_EQUAL which is normally used for opaque and alpha tested surfaces
nearest Use nearest texture filtering
linear Use linear texture filtering
clamp Same as the global keywords. Use to override a global clamp for a specific stage.
zeroclamp
alphazeroclamp
noclamp Use to set texture repeat for a stage when global clamp is set
uncompressed Do not compress this image in medium quality mode
highquality
forceHighQuality Do not compress this image in low quality mode
nopicmip Ignore the image_downSize cvar
vertexColor Multiply the pixel color by the vertex color
inverseVertexColor Multiply the pixel color by one minus the vertex color
privatePolygonOffset <float> Explict larger (or negative) polygon offset for this stage
texGen <type> Type is one of: normal, reflect, skybox, wobbleSky <exp> <exp> <exp>
scroll <exp>, <exp> Scroll the texture coordinates
translate <exp>, <exp>
scale <exp>, <exp> Just scales without a centering
centerScale <exp>, <exp> Subtracts 0.5, then scales, then adds 0.5
shear <exp>, <exp> Subtracts 0.5, then shears, then adds 0.5
rotate <exp> Subtracts 0.5, then rotates, then adds 0.5
maskRed Don't write to the red channel
maskGreen Don't write to the blue channel
maskBlue Don't write to the green channel
maskAlpha Don't write to the alpha channel
maskColor Shortcut for
maskRedmaskGreenmaskBlue
maskDepth Don't write to the depth buffer
alphaTest <exp> Only write if the alpha value is greater than <exp>
red <exp> Set the red vertex color
green <exp> Set the green vertex color
blue <exp> Set the blue vertex color
alpha <exp> Set the alpha vertex value
rgb <exp> Shortcut for
red <exp>green <exp>blue <exp>
rgba <exp> Shortcut for
red <exp>green <exp>blue <exp>alpha <exp>
color <exp0>, <exp1>, <exp2>, <exp3> Shortcut for
red exp0green exp1blue exp2alpha exp3
colored Shortcut for
color parm0, parm1, parm2, parm3
if <exp> Conditionally disable stages
fragmentProgram <prog> Use an ARB fragment program with this stage
vertexProgram <prog> Use an ARB vertex program with this stage
program <prog> Shortcut for
fragmentProgram <prog>vertexProgram <prog>
vertexParm <index> <exp0>
[,exp1] [,exp2] [,exp3] Values to pass to the vertex program. One expression gets repeated across all 4 values. Two expressions put 0, 1 in z, w. Three expressions put 1 in w.
fragmentMap <index> [options] <map> The image map to use for texture unit <index>
[options] can be cubeMap, cameraCubeMap, nearest, linear, clamp, noclamp, zeroclamp, alphazeroclamp, forceHighQuality, uncompressed, highquality, or nopicmip
megaTexture <mega> This stage uses a mega texture (super secret)

=== Parameters Key ===
<float> Any number
<int> Any number without a fractional part
<string> Any value enclosed in quotes
<index> An integer number that is an index into an array
<map> An image map, which may include image programs (below)
<prog> A vertex / frament program. Written using the GL ARB shader language. These files are stored in the glprogs directory
<exp> An expression that is evaluated every frame.

=== Image Program Functions ===
These can be used anywhere that accepts <map> and can be nested heightmap(<map>, <float>) Turns a grayscale height map into a normal map. <float> varies the bumpiness
addnormals(<map>, <map>) Adds two normal maps together. Result is normalized.
smoothnormals(<map>) Does a box filter on the normal map, and normalizes the result.
add(<map>, <map>) Adds two images without normalizing the result
scale(<map>, <float> [,float] [,float] [,float]) Scales the RGBA by the specified factors. Defaults to 0.
invertAlpha(<map>) Inverts the alpha channel (0 becomes 1, 1 becomes 0)
invertColor(<map>) Inverts the R, G, and B channels
makeIntensity(<map>) Copies the red channel to the G, B, and A channels
makeAlpha(<map>) Sets the alpha channel to an average of the RGB channels. Sets the RGB channels to white.

← Older revision		Revision as of 15:10, 4 December 2007
Line 1:		Line 1:
−	Material ~~decls~~ are similar to shader files from Quake3, but they are a lot more powerful. Materials have a lot of power, especially when combined with scripts and GL programs. Material declarations are stored in ''.mtr'' files.	+	Material declarations are similar to shader files from Quake3, but they are a lot more powerful. Materials have a lot of power, especially when combined with scripts and GL programs. Material declarations are stored in ''.mtr'' files.

	For a simple overview of how to create a new material, [[Adding Custom Textures\|see this tutorial]].		For a simple overview of how to create a new material, [[Adding Custom Textures\|see this tutorial]].

Materials - Revision history

MoP: declarations not decls

MoP: shuffled headings around for better order, made the example less doom3-like, moved reference to new page for filesize

192.168.0.129: Tabulated everything

MoP at 09:40, 17 October 2007