How to create a CgFx like effect system?

Question

Seriouse graphics engine like CryEngine3, Unreal Engine 3 have their customized shader language and effect system. While trying to find some effect system for my small graphics framework, it looks like nvidia CgFx is the only choice (seems Khronos had a project called glFx, but the project page is 404 now).

I have several reasons to make a effect system of my own:

1. I need more control about how and when to pass the shader parameters.
2. In order to reuse shader snippets, I want to create some c++ macro like mechanism. It's also useful to use macro to do some conditional compilation, and that the way CryEngine used to produces various effects.
3. Looks like GLSL don't have such effect system

so I am wondering how to create a effect system? Do I need to write grammar parser from scratch or there's already some code/tools able to do this thing?

PS: I am using OpenGL with both GLSL and CG.

Answer 1

Back in the day when I was using HLSL, I developped a little shader system which allowed me to specify all my parameters through data, so that I could just edit a sort of XML file containing the list of parameters and the shader codes, and after saving, the engine would automatically reload it, rebind all parameters, etc.

It's nothing compared to what's found in the UDK, but pretty convenient, and I guess you're trying to implement something like that ?

I it is, then here are a few stuff to do. First, you need to create a class to abstract shader parameter handling (binding, setting, etc.) Something along these lines :

class IShaderParameter
{
protected:
    IShaderParameter(const std::string & name)
        : m_Uniform(-1)
        , m_Name(name)
    {}
    GLuint m_Uniform;
    std::string m_Name;
public:
    virtual void Set(GLuint program) = 0;
};

Then, for static parameters, you can simply create an overload like this :

template < typename Type >
class StaticParameter
    : public IShaderParameter
{
public:
    StaticParameter(const std::string & name, const Type & value)
        : IShaderParameter(name)
        , m_Value(value)
    {}
    virtual void Set(GLuint program)
    {
        if (m_Uniform == -1)
            m_Uniform = glGetUniformLocation(program, m_Name.c_str());
        this->SetUniform(m_Value);
    }
protected:
    Type m_Value;
    void SetUniform(float value) { glUniform1f(m_Uniform, value); }
    // write all SetUniform specializations that you need here
    // ...
};

And along the same idea, you can create a "dynamic shader parameter" type. For example, if you want to be able to bind a light's parameter to your shader, create a specialized parameter's type. In its constructor, pass the light's id so that it will know how to retrieve the light in the Set method. With a little work, you can have a whole bunch of parameters that you can then automatically bind to an entity of your engine (material parameters, light parameters, etc.)

The last thing to do is create a little custom file format (I used xml) to define your various parameters, and a loader. For example, in my case, it looked like this :

<shader>
    <param type="vec3" name="lightPos">light_0_position</param>
    <param type="vec4" name="diffuse">material_10_diffuse</param>
    <vertexShader>
      ... a CDATA containing your shader code
    </vertexShader>
</shader>

In my engine, "light_0_position" would mean a light parameter, 0 is the light's ID, and position is the parameter to get. Binding between the parameter and the actual value was done during loading so there was not much overhead.

Anyway, I don't if that answer your question, and don't take these sample codes too seriously (HLSL and OpenGL shaders work quite differently, and I'm no OpenGL expert ^^) but hopefully it'll give you a few leads :)

Answer 2

1. Could you elaborate on this? By working with OpenGL directly you have a full control over the parameters being passed to the GPU. What exactly are you missing?

2. (and 3.) GLSL does support re-using the code. You can have a library of shaders providing different functions. In order to use any function you just need to pre-declare it in the client shader (vec4 get_diffuse();) and attach the shader object implementing the function to the shader program before linking.