B Tutorial Shader Shader Tutorial by Scion

Users who are viewing this thread

Duh

Duke
Scion said:

shaderscva2.png


Shader Tutorial - Make Your Own
by Scion



Since there isn`t anything about shaders out there and I have experimented a little bit with it myself, I thought I could share my experiences for those who wants to try it out for themself.

Disclaimer:
There might be errors.








Introduction

You might want to read Wikipedia on shaders.

We are going to talk about the DirectX 9 shaders. Of nVIDIA cards, 5xxx FX - cards are not supported. I`m not sure exactly what shader version M&B uses, if it is shader version 2 it is supported by all 6xxx cards, if it is shader version 3, it is supported from 68xx cards.

First, M&B supports only �local� shaders for single materials. It cannot do global shaders like fading, bloom etc. You might achieve the effect of a global shader by mixing a global shader technique into all of your shaders. Hopefully, global shaders are in the future plans of Armagan.

Know that there is two different kind of shaders we operate with: vertex shaders and pixel shaders.
From Wikipedia:
  • Vertex shaders are applied for each vertex and run on a programmable vertex processor. Vertex shaders define a method to compute vector space transformations and other linearizable computations. That is, the vertex shader takes the vertex data and can change it as the programmer desires.
  • Pixel shaders � or "fragment programs" in OpenGL nomenclature � are used to compute properties which, most of the time, are recognized as pixel colors.

There seems to be two different shader languages avaliable for M&B:
  • HLSL (High Level Shader Language): developed by Microsoft for use with the Microsoft Direct3D API. I believe this is DirectX 9 shaders in the M&B configuration window.
  • The one located in myshader.pp. You can refer directly to such a file from a BRF file,but it is only a pixel shader. I`m not sure exactly what this.

So the approach I want to present here is HLSL, to modify mb.fx in your Mount&Blade root directory. This means that the shader you make by following this, won`t be contained within your mod-directory.

You can however extend the mb.fx file without too much effort in an installer script, without messing up the existing shaders. This by simply adding text at the end of the file. It`s not really the �right way� to do things, but as long as it works like it should, who cares?

Now, the main steps in this tutorial are:
  • Making a BRF File
  • Copy an existing shader
  • Edit the copied shader







PART 1 - Getting dirty with a BRF File

First the references. There WAS a video tutorial on this by a guy called n00854180t at http://forums.taleworlds.com/index.php?topic=6039.0
Janus talks about something related at http://forums.taleworlds.com/index.php/topic,6628.0.html
Unfortunately, it seems like the video is lost forever. However, do not weep. We can make this anyway.

First step in this tutorial is to make a shader declaration in a BRF file. What we need is:
  • The name/ID of the shader: to be able to name what shader we want to use in a material.
  • The name of the code (technique) we want. Since we want to start from skratch, it can be called whatever we want.
  • The fallback shader; a crappy shader that will be used if the shader you have assigned is too heavy for the players graphics card.

Since BRF Edit, to my knowledge, don`t support what we are going to do, we need to HEX edit a BRF file. I use PSPad Editor myself, which is a general programming editor.

Important: As mentioned a lot in the upcoming text, it can be smart to have a name length on your shader which is equal to an already existing shader's name length. Renaming �dot3� to �iron� makes this part overwith in notime.


What to do:
  • Make a new file in BRF Edit. (File->New)
  • Import from Mount&Blade/CommonRes/core_shaders.brf : some shader similar to yours. (This due to the �Fallback� shader: if your shader is not supported by the players card. I`ll further assume we selected �dot3�. Preferably, you choose a shader with a name length equal to your new shader`s name length.
  • Save file, add to module.ini
  • Open BRF-file in a HEX Editor. You will see something similar to this:
sc01ts0.jpg



What you now want to change is the name. Preferably you choose a shader with a name length equal to yours, because then you just replace the characters �dot3� with your new shader name. If this is not the case, you need to edit the positions where name lengths is. Here you see 04 and 08, which is the length of dot3 and dot3_ffp. Don`t bother changing dot3_ffp if it reassembles your new one, it`s the fallback shader.

Change name length for the two first, remember HEX numbers go from 0-F (16 digit system) , so a name length of 16 will here be �10�, not �16�. 15 will be �0F�. Read up on HEX numbers if you`re not sure, or alternatively choose a shader with right name length :wink: . I had bad experiences with editing lengths myself. Then, copy parts of your new name and paste after the name to obtain the right length, this is the safe way.

Example - Changing dot3 to my_shader:
  • Replace length: 04 -> 09
  • Copy dot3: paste after 3. Copy 3, paste after 3 again.
  • Overwrite (press insert?) dot3dot33 with my_shader

Both places! First one is for reference from materials, the second is a reference to mb.fx. If you choose the same name for both or not, is not important.
Save the HEX edited file, then open it in BRF Edit. If you get any error there, something is wrong. If you struggle too long to get it right, import a shader with same name length as yours.







PART 2 - Edit mb.fx

Open mb.fx in a suitable editor. The ones I have used have no syntax highlighting, so if you find a good one, yell out. Note that there exist a �shader authoring tool� called NVIDIA FX Composer at
http://developer.nvidia.com/object/fx_composer_home.html
fully supporting HLSL which is used in mb.fx. It even has some very nice examples of shaders. I could not get my M&B shaders to work in this program however, so I won`t go into it. Just note, it could be worth it (instant preview of shaders for example).

This part can only be to get you started and is not going into HLSL itself, you can figure out tons of this yourself by searching for Direct X shaders, �.fx� shader, HLSL etc. On for example Google. We will rather just discuss the issues I find most important for beginners.




2.1  Some Basics


Note the following in the top part of the mb.fx file:

Code:
texture diffuse_texture;             <- Main texture
texture specular_texture;            <- Alpha channel
texture normal_texture;              <- Bump map
texture env_texture;                 <- Enviro
texture shadowmap_texture;           <- ?

These are the different textures assigned by a material in BRF Edit. Setting bump map there will be normal_texture here.

You get values in a pixel shader from such a texture a the tex2D function:
Code:
tex2D(NormalTextureSampler, In.Tex0).rgb
Here we get the RGB value of the normal texture at the current pixel.
There are several �samplers�, in fact one for each of the textures discussed above.

Each shader have a Pixel shader and vertex shader. Prefix = PS is a pixel shader, prefix = VS is a vertex shader.



2.2  First Step � Technique

The technique where you tell what pixel shader and vertex shader that will be used for your shader name made in the BRF file. It is what you refered to from the BRF file.

At the bottom of mb.fx you`ll see the techniques. The dot3 technique looks like this:
Code:
technique dot3
{
	pass P0
	{
               VertexShader = compile vs_2_0 vs_main_dot3_bump(PCF_NONE);
               PixelShader = compile ps_2_0 ps_main_dot3_bump(PCF_NONE);
	}
}

We see that the vertex shader you have refered to from the BRF file is vs_main_dot3_bump, and the pixel shader is ps_main_dot3_bump. Copy the text above and paste it at the end of the file. Change the name of the technique (�dot3�) to your own shader. Note that you don`t have to use the dot3 shader, whatever shader you like can be used. We won`t bother with the specifics of �dot3�. However, this name has to be identical to the name you use for the actual shaders in step 3: this is the main references to these shaders, and the entrypoint from M&B in-game. When we take a closer look at the shader code, we also see that input and output have to correspond with the respective shader. Input for a water shader will not work for a dot3 shader for example, they need different input.

In the beginning of mb.fx you will see
Code:
#define PCF_NONE					0
#define PCF_DEFAULT					1
#define PCF_NVIDIA					2
#define PCF_ATI						3

This refers to the type of graphics card the shader will work for. We see that for the shaders above that we use PCF_NONE. I`m not sure about if this states that it should be the one in the configuration of M&B, but it works perfectly on my NVIDIA graphics card. Might be unfinished business from Ikisoft, maybe that they must write a shader for each card, but just haven`t done it yet. Experts, please fill us in.



2.3  Second Step � PS Input and VS Output

At almost at the beginning of mb.fx, you`ll see this:
Code:
struct PS_INPUT_DOT3_BUMP
{
   float4 Color					: COLOR0;
   float4 SunColor				: COLOR1;
   float2 Tex0					: TEXCOORD0;
   float4 SunLight				: TEXCOORD1;
   float4 ShadowTexCoord		        : TEXCOORD2;
   float2 TexelPos				: TEXCOORD3;
};

struct VS_OUTPUT_DOT3_BUMP
{
   float4 Pos				    : POSITION;
   float4 Color			            : COLOR0;
   float4 SunColor			    : COLOR1;
   float2 Tex0			            : TEXCOORD0;
   float4 SunLight			    : TEXCOORD1;
   float4 ShadowTexCoord		    : TEXCOORD2;
   float2 TexelPos			    : TEXCOORD3;
   float  Fog				    : FOG;
};

You can use existing input and output structures (and thus not copy anything here); but if you want to customize the input/output structures, do the following: copy the lines and paste at the end of the file. Change DOT3_BUMP to whatever you like for your own shader. PS_OUTPUT is a generic structure for all pixel shaders, so you don`t have to bother.

Data is transferred as following:
  • Game data is sent into the header of the vertex shader.
  • The vertex shader builds the vertex shader output structure.  (VS_OUTPUT_...)
  • The pixel shader imports (through PS_INPUT_...) arguments given by the vertex shader output structure, or direct input from the game.

2.4  Third Step � The actual shader programs

Copy these two, which are the actual pixel shader and vertex shader code respectively:
Code:
PS_OUTPUT ps_main_dot3_bump(PS_INPUT_DOT3_BUMP In, uniform const int PcfMode)
{ 
    PS_OUTPUT Output;
    float3 normal = 2.0f * tex2D(NormalTextureSampler, In.Tex0).rgb - 1.0f;
    float3 diffuse = 2.0f * In.Color.rgb - 1.0f;
    float3 sun_color = 2.0f * In.SunColor.rgb - 1.0f;
   
	float4 light_amount = vMaterialColor; 
    if (PcfMode != PCF_NONE)
    {
		float sun_amount = GetSunAmount(PcfMode, In.ShadowTexCoord, In.TexelPos);
		light_amount += ((saturate(dot(sun_color, normal)) * sun_amount));
    }
    light_amount += (saturate(dot(diffuse, normal)));
	Output.RGBColor = tex2D(MeshTextureSampler, In.Tex0) * light_amount;
    Output.RGBColor.a = In.Color.a;
	return Output;
}
Code:
VS_OUTPUT_DOT3_BUMP vs_main_dot3_bump (uniform const int PcfMode, float4 vPosition : POSITION, float3 vNormal : NORMAL, float2 tc : TEXCOORD0, float4 vColor : COLOR0, float4 vLight : COLOR1)
{
   VS_OUTPUT_DOT3_BUMP Out = (VS_OUTPUT_DOT3_BUMP)0;

   Out.Pos = mul(matWorldViewProj, vPosition);
   Out.Tex0 = tc;
   Out.Color = vColor;
   Out.SunColor = vLight;

   float3 P = mul(matWorldView, vPosition); //position in view space

	if (PcfMode != PCF_NONE)
	{
		//shadow mapping variables
		float4 vWorldPos = (float4)mul(matWorld,vPosition);
		
		float3 vWorldN = mul((float3x3)matWorld, vNormal);
		float wNdotSun = max(-0.0001, dot(vWorldN, -vSunDir));
		Out.SunLight = (wNdotSun) * vSunColor;

		float4 ShadowPos = mul(matSunViewProj, vWorldPos);
		Out.ShadowTexCoord = ShadowPos;
		Out.ShadowTexCoord.z /= ShadowPos.w;
		Out.ShadowTexCoord.w = 1.0f;
		Out.TexelPos = Out.ShadowTexCoord * fShadowMapSize;
		//shadow mapping variables end
	}

   //apply fog
   float d = length(P);
   Out.Fog = saturate((fFogEnd - d) / (fFogEnd - fFogStart));
   
   return Out;
}

Change their names, the input and output names to your own.

This is mainly the code you want to edit! Add some light amount, play around. The pixel shader is easiest to get to work for a newbie. Put �Output.RGBColor.r = Output.RGBColor.r*1.1;� to enhance red for example, or play around with the normal map for some interresting effects.

Input arguments for the vertex shader are not contained in a input structure as for the pixel shader. It simply lies in the header of the function:
Code:
VS_OUTPUT_DOT3_BUMP vs_main_dot3_bump (uniform const int PcfMode, float4 vPosition : POSITION, float3 vNormal : NORMAL, float2 tc : TEXCOORD0, float4 vColor : COLOR0, float4 vLight : COLOR1)



2.5  The not moving costume/armor problem

Only the shaders with the suffix �_skin� supports skeletons. Why? Best to illustrate by example. Look at the following code in vs_main_dot3_bump:
Code:
Out.Pos = mul(matWorldViewProj, vPosition);

Replace it with the following code, which you will find in all �skin� shaders:

Code:
float4 vObjectPos = mul(matWorldArray[vBlendIndices.x], vPosition - matBoneOriginArray[vBlendIndices.x]) * vBlendWeights.x
   + mul(matWorldArray[vBlendIndices.y], vPosition - matBoneOriginArray[vBlendIndices.y]) * vBlendWeights.y
   + mul(matWorldArray[vBlendIndices.z], vPosition - matBoneOriginArray[vBlendIndices.z]) * vBlendWeights.z
   + mul(matWorldArray[vBlendIndices.w], vPosition - matBoneOriginArray[vBlendIndices.w]) * vBlendWeights.w;
   
   Out.Pos = mul(matWorldViewProj, vObjectPos);

You need to calculate the position relative to the bones, something the original dot3 �position� does not take into account.

You also have to add some arguments in the header of the vertex shader, in specific you want to know something called blendWeights and blendIndices. Change
Code:
VS_OUTPUT_DOT3_BUMP vs_main_dot3_bump (uniform const int PcfMode, float4 vPosition : POSITION, float3 vNormal : NORMAL, float2 tc : TEXCOORD0, float4 vColor : COLOR0, float4 vLight : COLOR1)
to
Code:
VS_OUTPUT_DOT3_BUMP vs_main_dot3_bump (uniform const int PcfMode, float4 vPosition : POSITION, float3 vNormal : NORMAL, float2 tc : TEXCOORD0, float4 vColor : COLOR0, float4 vLight : COLOR1, float4 vBlendWeights : BLENDWEIGHT, float4 vBlendIndices : BLENDINDICES)



2.6  Some important code segments for pixel shaders

Getting main (diffuse) texture value:
Code:
float4 diffuse_texture_val = tex2D(MeshTextureSampler, In.Tex0);

Getting vertex colored value:
Code:
float3 diffuse = 2.0f * In.Color.rgb - 1.0f;

Getting the normal for a normal map:
Code:
float3 normal = 2.0f * tex2D(NormalTextureSampler, In.Tex0).rgb - 1.0f;

Getting the Sun Color:
Code:
float3 sun_color = 2.0f * In.SunColor.rgb � 1.0f;

Getting light amount:
Code:
float4 light_amount = vMaterialColor; 
float sun_amount = GetSunAmount(PcfMode, In.ShadowTexCoord, In.TexelPos);
light_amount += ((saturate(dot(sun_color, normal)) * sun_amount));
light_amount += (saturate(dot(diffuse, normal)));

Output Color:
Code:
Output.RGBColor
Output.RGBColor.rgb    <-- all channels
Output.RGBColor.r      <-- red channel
Output.RGBColor.g      <-- green channel
Output.RGBColor.b      <-- blue channel

Output.RGBColor.a      <-- alpha channel, not 0 will give transparency



2.7  Post Mortem � Trying your shader in-game

In BRF Edit, use your shader for some material. Run Mount&Blade. If you get into the main menu, your mb.fx file compiled. If you get an error about the effect file, you failed.

Important! You don`t have to restart M&B to see changes; only press Ctrl-F in Edit mode (or was it some other key in that area, I can`t remember). This will recompile mb.fx without restarting the game, and you will see the changes immediately.



2.8  Scions Iron Shader modification

Screenshot: http://img233.imageshack.us/img233/6786/shadersc05st9.jpg
Screenshot (tweaked values): http://img237.imageshack.us/my.php?image=shadersc04ck2.jpg

This was made for chainmails and �hard armor�, where I made for each costume a normal map (see yoshiboy`s texturing tutorial for how to make them) and assigned the normal map as bump texture in the corresponding material in BRF Edit. My shader has it flaws, but I post in case it can be of any use:
Code:
////////////////////////////////////////////////////////////////////////////////

struct VS_OUTPUT_IRON_SHADER
{
   float4 Pos                   : POSITION;
   float4 Color			: COLOR0;
   float4 SunColor		: COLOR1;
   float2 Tex0			: TEXCOORD0;
   float4 SunLight		: TEXCOORD1;
   float4 ShadowTexCoord	: TEXCOORD2;
   float2 TexelPos		: TEXCOORD3;
   float3 worldPos              : TEXCOORD4;
   float3 worldNormal           : TEXCOORD5;
   float  Fog			: FOG;
};

Code:
VS_OUTPUT_IRON_SHADER vs_iron_shader (uniform const int PcfMode, float4 vPosition : POSITION, float3 vNormal : NORMAL, float2 tc : TEXCOORD0, float4 vColor : COLOR0, float4 vLight : COLOR1, float4 vBlendWeights : BLENDWEIGHT, float4 vBlendIndices : BLENDINDICES)
{
   VS_OUTPUT_IRON_SHADER Out = (VS_OUTPUT_IRON_SHADER)0;
   
   float4 vObjectPos = mul(matWorldArray[vBlendIndices.x], vPosition - matBoneOriginArray[vBlendIndices.x]) * vBlendWeights.x
   + mul(matWorldArray[vBlendIndices.y], vPosition - matBoneOriginArray[vBlendIndices.y]) * vBlendWeights.y
   + mul(matWorldArray[vBlendIndices.z], vPosition - matBoneOriginArray[vBlendIndices.z]) * vBlendWeights.z
   + mul(matWorldArray[vBlendIndices.w], vPosition - matBoneOriginArray[vBlendIndices.w]) * vBlendWeights.w;
   float3 vObjectN = normalize(mul((float3x3)matWorldArray[vBlendIndices.x], vNormal) * vBlendWeights.x
   + mul((float3x3)matWorldArray[vBlendIndices.y], vNormal) * vBlendWeights.y
   + mul((float3x3)matWorldArray[vBlendIndices.z], vNormal) * vBlendWeights.z
   + mul((float3x3)matWorldArray[vBlendIndices.w], vNormal) * vBlendWeights.w);
   
   float4 vWorldPos = mul(matWorld,vObjectPos);
   Out.Pos = mul(matViewProj, vWorldPos);
   float3 vWorldN = mul((float3x3)matWorld, vObjectN); //normal in world space
   
   float3 P = mul(matWorldViewProj, vWorldPos); //position in view space
   
   Out.worldPos = vWorldPos;
   Out.worldNormal = vWorldN;

   float4 diffuse_light = vAmbientColor;
   
	//directional lights, compute diffuse color
	diffuse_light += max(0, dot(vWorldN, -vSkyLightDir)) * vSkyLightColor;

	//point lights
	for(int j = 0; j < iLightPointCount; j++)
	{
		int i = iLightIndices[j];
		float3 point_to_light = vLightPosDir[i]-vWorldPos;
		float LD = length(point_to_light);
		float3 L = normalize(point_to_light);
		float wNdotL = dot(vWorldN, L);
		
		float fAtten = 0.1f/LD + 0.9f / (LD * LD);
		//compute diffuse color
		diffuse_light += max(0, wNdotL) * vLightDiffuse[i] * fAtten;
	}
   // ...

   //Out.Pos = mul(matWorldViewProj, vPosition);
   Out.Tex0 = tc;
   Out.Color = (vMaterialColor * vColor * diffuse_light);
   Out.SunColor = vLight;

   P = mul(matWorldView, vPosition); //position in view space

	if (PcfMode != PCF_NONE)
	{
		
		float3 vWorldN = mul((float3x3)matWorld, vNormal);
		float wNdotSun = max(-0.0001, dot(vWorldN, -vSunDir));
		Out.SunLight = (wNdotSun) * vSunColor;

		float4 ShadowPos = mul(matSunViewProj, vWorldPos);
		Out.ShadowTexCoord = ShadowPos;
		Out.ShadowTexCoord.z /= ShadowPos.w;
		Out.ShadowTexCoord.w = 1.0f;
		Out.TexelPos = Out.ShadowTexCoord * fShadowMapSize;
		//shadow mapping variables end
	}

   //apply fog
   float d = length(P);
   Out.Fog = saturate((fFogEnd - d) / (fFogEnd - fFogStart));
   
   return Out;
}

Code:
struct PS_INPUT_IRON_SHADER
{
	float4 Color		: COLOR0;
	float4 SunColor		: COLOR1;
	float2 Tex0		: TEXCOORD0;
	float4 SunLight		: TEXCOORD1;
	float4 ShadowTexCoord	: TEXCOORD2;
	float2 TexelPos		: TEXCOORD3;
    float3 worldPos             : TEXCOORD4;
    float3 worldNormal          : TEXCOORD5;

};

Code:
PS_OUTPUT ps_iron_shader(PS_INPUT_IRON_SHADER In, uniform const int PcfMode)
{
    PS_OUTPUT Output;
    
 // Compute half vector for specular lighting
 //   float3 vHalf = normalize(normalize(-ViewPos) + normalize(g_vLight - ViewPos));
 

	if ((PcfMode != PCF_NONE))
    {
        float3 normal = 2.0f * tex2D(NormalTextureSampler, In.Tex0).rgb - 1.0f;
        float3 diffuse = 2.0f * In.Color.rgb - 1.0f;
        float3 sun_color = 2.0f * In.SunColor.rgb - 1.0f;
        
        float4 light_amount = vMaterialColor; 

		float sun_amount = GetSunAmount(PcfMode, In.ShadowTexCoord, In.TexelPos);
		light_amount += ((saturate(dot(sun_color, normal)) * sun_amount));

        light_amount += (saturate(dot(diffuse, normal)));              
       
		float3 vHalf = normalize(normalize(vCameraPos - In.worldPos) - vSunDir);
		float4 outColor = tex2D(MeshTextureSampler, In.Tex0);
		// Compute normal dot half for specular light
		float4 fSpecular = vSpecularColor * pow( saturate( dot( vHalf, normalize( In.worldNormal) ) ), fMaterialPower) * outColor.a;
		Output.RGBColor = 1.2*pow(outColor,2)*((1.2*In.Color + (In.SunLight + fSpecular) * sun_amount)) *light_amount*light_amount;
		Output.RGBColor += max(light_amount, 0.005f)*outColor*0.18f;
		
		Output.RGBColor.a = 1.0f;

    }
    else
    {
    	Output.RGBColor = tex2D(MeshTextureSampler, In.Tex0) * In.Color;
    	Output.RGBColor.a = 1.0f;
    }
    return Output;
}

Code:
technique daoc_iron_shader
{
	pass P0
	{
      VertexShader = compile vs_2_a vs_iron_shader(PCF_NVIDIA);
      PixelShader = compile ps_2_a ps_iron_shader(PCF_NVIDIA);
	}
}

I mostly messed around with the following two lines, getting significant variety in the armor look by tweaking the formula:

Code:
Output.RGBColor = 1.2*pow(outColor,2)*((1.2*In.Color + (In.SunLight + fSpecular) * sun_amount)) *light_amount*light_amount;
Output.RGBColor += max(light_amount, 0.005f)*outColor*0.18f;







3  Some references

HLSL Introduction
http://docendo.bai.nu/img/kapitel/0735616531.htm

FX Composer
http://developer.nvidia.com/object/fx_composer_home.html
Has some really nice examples.

HLSL Tutorial - Normal mapping (Bump mapping technique)
http://dotnet.org.za/pieterg/archive/2005/07/29/40407.aspx

Microsoft HLSL Workshop � Some tutorials
http://msdn2.microsoft.com/en-us/library/bb173495.aspx



That`s it. I encourage people to share their shaders. :smile:



All credit goes to Scion. Original thread taken from the MBX forum here:
http://mbx.streetofeyes.com/index.php/topic,367.msg4453.html#msg4453

I hope noone minds it, if i move some of the good stuff from there to here as i feel these things deserve more exposure.
 
Very interesting, thanks for posting this. :smile:

Although the linked quote is weird, as it's a linked quote from MBX, so here it links to this thread: http://forums.taleworlds.com/index.php/topic,367.msg3825.html#msg3825
Don't know if that matters, just wanted to say that.
 
Back
Top Bottom