How to use the C++ interface
Albeit slightly old question, I recently struggled with using the C++ interface, so I decided to post my findings here.
The includer
This is probably the most tricky part. Something that we don't think about when we write C or C++ programs, but GLSL allows the inclusion of other shader files through #include pre-processing directive, and its behaviour is not defined for us.
In the file glslang/Public/ShaderLang.h we have an abstract class defined, summarized:
class Includer {
public:
struct IncludeResult {
// [...]
};
virtual IncludeResult* includeSystem(const char* /*headerName*/,
const char* /*includerName*/,
size_t /*inclusionDepth*/) { return nullptr; }
virtual IncludeResult* includeLocal(const char* /*headerName*/,
const char* /*includerName*/,
size_t /*inclusionDepth*/) { return nullptr; }
virtual void releaseInclude(IncludeResult*) = 0;
virtual ~Includer() {}
};
Its usage is only very vaguely described, but from what I can gather (by reading the code) we should create a class which inherits from Includer, and which implements the methods shown above. In essense, these methods search the local filesystem for a GLSL file matching headerName, reads its contents, and returns its content in a new Includer::IncludeResult struct.
And to make it even more problematic, this header lookup might itself result in a recursive query!
Custom includer
This implementation is roughly copy/pasted from my own code (which unfortunately I cannot open-source atm.). It should be reasonably easy to understand how it works:
// includes
#include <glslang/Public/ShaderLang.h>
#include <map>
#include <vector>
class GlslShaderIncluder : public glslang::TShader::Includer
{
public:
// explicit GlslShaderIncluder(fileio::Directory* shaderdir)
// : mShaderdir(shaderdir) {}
// Note "local" vs. "system" is not an "either/or": "local" is an
// extra thing to do over "system". Both might get called, as per
// the C++ specification.
//
// For the "system" or <>-style includes; search the "system" paths.
virtual IncludeResult* includeSystem(
const char* headerName, const char* includerName, size_t inclusionDepth) override;
// For the "local"-only aspect of a "" include. Should not search in the
// "system" paths, because on returning a failure, the parser will
// call includeSystem() to look in the "system" locations.
virtual IncludeResult* includeLocal(
const char* headerName, const char* includerName, size_t inclusionDepth) override;
virtual void releaseInclude(IncludeResult*) override;
private:
static inline const std::string sEmpty = "";
static inline IncludeResult smFailResult =
IncludeResult(sEmpty, "Header does not exist!", 0, nullptr);
// const fileio::Directory* mShaderdir {nullptr};
std::map<std::string, IncludeResult*> mIncludes;
std::map<std::string, std::vector<char>> mSources;
};
The implementation was, mostly because of time limits, unfinished. System includes are not supported (missing use case), but that should be similar to how local includes are handled. The code also depends on a custom file I/O library, so those parts are commented out (it would also work with using std::ifstream or fopen, the fileio namespaces uses std::filesystem underneath):
IncludeResult* GlslShaderIncluder::includeSystem(
const char* headerName, const char* includerName, size_t inclusionDepth)
{
// TODO: This should be used if a shader file says "#include <source>",
// in which case it includes a "system" file instead of a local file.
log_error("GlslShaderIncluder::includeSystem() is not implemented!");
log_error("includeSystem({}, {}, {})", headerName, includerName, inclusionDepth);
return nullptr;
}
IncludeResult* GlslShaderIncluder::includeLocal(
const char* headerName, const char* includerName, size_t inclusionDepth)
{
log_debug("includeLocal({}, {}, {})", headerName, includerName, inclusionDepth);
// std::string resolvedHeaderName =
// fileio::directory_get_absolute_path(mShaderdir, headerName);
if (auto it = mIncludes.find(resolvedHeaderName); it != mIncludes.end())
{
// `headerName' was already present, so return that, and probably log about it
return it->second;
}
// if (!fileio::file_exists(mShaderdir, headerName))
// {
// log_error("#Included GLSL shader file \"{}\" does not exist!", resolvedHeaderName);
// return &smFailResult;
// }
// mSources[resolvedHeaderName] = {}; // insert an empty vector!
// fileio::File* file = fileio::file_open(
// mShaderdir, headerName, fileio::FileModeFlag::read);
// if (file == nullptr)
// {
// log_error("Failed to open #included GLSL shader file: {}", resolvedHeaderName);
// return &smFailResult;
// }
// if (!fileio::file_read_into_buffer(file, mSources[resolvedHeaderName]))
// {
// log_error("Failed to read #included GLSL shader file: {}", resolvedHeaderName);
// fileio::file_close(file);
// return &smFailResult;
// }
IncludeResult* result = new IncludeResult(
resolvedHeaderName, mSources[resolvedHeaderName].data(),
mSources[resolvedHeaderName].size(), nullptr);
auto [it, b] = mIncludes.emplace(std::make_pair(resolvedHeaderName, result));
if (!b)
{
log_error("Failed to insert IncludeResult into std::map!");
return &smFailResult;
}
return it->second;
}
void GlslShaderIncluder::releaseInclude(IncludeResult* result)
{
log_debug("releaseInclude(result->headerName: {})", result->headerName);
if (auto it = mSources.find(result->headerName); it != mSources.end())
{
mSources.erase(it);
}
if (auto it = mIncludes.find(result->headerName); it != mIncludes.end())
{
// EDIT: I have forgotten to use "delete" here on the IncludeResult, but should probably be done!
mIncludes.erase(it);
}
}
Using the includer
After defining a custom includer, we can procede to inject it into the glslang loader (which is also an object):
std::vector<char> buffer; // contains the shader file
glslang::TShader shader(EShLangVertex); // example, use this code for each separate shader file in the shader program
const char* sources[1] = { buffer.data() };
shader.setStrings(sources, 1);
// Use appropriate Vulkan version
glslang::EShTargetClientVersion targetApiVersion = glslang::EShTargetVulkan_1_3;
shader.setEnvClient(glslang::EShClientVulkan, targetApiVersion);
glslang::EShTargetLanguageVersion spirvVersion = glslang::EShTargetSpv_1_3;
shader.setEnvTarget(glslang::EshTargetSpv, spirvVersion);
shader.setEntryPoint("main"); // We can specify a different entry point
// The resource is an entire discussion in and by itself, here just use default.
TBuiltInResource* resources = GetDefaultResources();
// int defaultVersion = 110, // use 100 for ES environment, overridden by #version in shader
const int defaultVersion = 450;
const bool forwardCompatible = false;
const EShMessages messageFlags = (EShMessages)(EShMsgSpvRules | EShMsgVulkanRules);
EProfile defaultProfile = ENoProfile; // NOTE: Only for desktop, before profiles showed up!
// NOTE: Here a custom file I/O library is used, your implementation may be different.
fileio::Directory* shaderdir = ...
GlslShaderIncluder includer(shaderdir);
std::string preprocessedStr;
if (!shader.preprocess(
resources, defaultVersion, defaultProfile, false, forwardCompatible, messageFlags, &preprocessedStr, includer))
{
log_error("Failed to preprocess shader: {}", shader.getInfoLog());
// FAIL
}
const char* preprocessedSources[1] = { preprocessedStr.c_str() };
shader.setStrings(preprocessedSources, 1);
if (!shader.parse(resources, defaultVersion, defaultProfile, false,
forwardCompatible, messageFlags, includer))
{
vtek_log_error("Failed to parse shader: {}", shader.getInfoLog());
// FAIL
}
glslang::TProgram program;
program.addShader(&shader);
if (!program.link(messageFlags))
{
vtek_log_error("Failed to link shader: {}", program.getInfoLog());
// FAIL
}
// Convert the intermediate generated by glslang to Spir-V
glslang::TIntermediate& intermediateRef = *(program.getIntermediate(lang));
std::vector<uint32_t> spirv;
glslang::SpvOptions options{};
options.validate = true;
// TODO: We can also provide a logger to glslang.
// glslang::spv::SpvBuildLogger logger;
// glslang::GlslangToSpv(intermediateRef, spirv, &logger, &options);
glslang::GlslangToSpv(intermediateRef, spirv, &options);
At the end, we have an array std::vector<uint32_t> spirv which contains the SPIR-V bytecode. This may be used to create a Vulkan shader module. Here, I must note that codeSize should be multiplied with 4 to ensure that we send the correct number of bytes to the GPU:
VkShaderModule module;
VkShaderModuleCreateInfo createInfo{};
createInfo.sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO;
createInfo.codeSize = spirvCode.size() * sizeof(uint32_t);
createInfo.pCode = spirvCode.data();
VkResult result = vkCreateShaderModule(dev, &createInfo, nullptr, &module);
if (result != VK_SUCCESS)
{
log_error("Failed to create {} shader module!", type);
// FAIL
}
Roughly speaking, that's it. This system handles recursive local includes, with some error handling. I hope it helps the next person.
