The bgfx shader compiler, 14 years of not being a compiler

Introduction

bgfx renders through many backends, and each of them wants shaders in its own language: HLSL for Direct3D, Metal Shading Language on Apple platforms, GLSL for OpenGL and ESSL for OpenGL ES, SPIR-V for Vulkan, WGSL for WebGPU, etc. Nobody wants to write and maintain one shader per API. So bgfx has a shader compiler, shaderc, that takes a single source written in bgfx’s GLSL-flavoured dialect and produces a binary blob for whichever backend you ask for.

shaderc is the jankiest part of bgfx. I’ll say that upfront, because the rest of this post explains why it is janky, why it still works after fourteen years, and why I haven’t replaced it. It is not a compiler in the textbook sense. There is no parser, no AST, no IR of its own. It is a C preprocessor, a pile of #defines, textual patching of the source, and then a hand-off to somebody else’s real compiler. The price is that every target reports errors in its own way. The benefit is that adding a new target has never required touching a front-end, because there isn’t one. The language you write is GLSL. For every target that is not OpenGL, the text that actually gets compiled is HLSL.

shaderc was in the initial commit of bgfx, on 3 April 2012, as a single tools/shaderc.cpp of 664 lines. It came with two third-party dependencies, both of which outlived every other decision made that day: fcpp, a C preprocessor with roots in 1984, and glsl-optimizer, an extraction of Mesa’s GLSL compiler.

Android and Native Client got GLSL through glsl-optimizer. Windows and Xbox 360 got HLSL through D3DXCompileShader from the D3DX9 library:

The output container from that first version, a uniform table (name, type, count, register) followed by the shader blob, is still the same shape today, just with more fields.

How does it work?

Take the simplest example in the repo, vs_cubes.sc:

 1$input a_position, a_color0
 2$output v_color0
 3
 4#include "../common/common.sh"
 5
 6void main()
 7{
 8	gl_Position = mul(u_modelViewProj, vec4(a_position, 1.0) );
 9	v_color0 = a_color0;
10}

Next to it sits varying.def.sc, which is the only place where types and semantics of inputs and outputs are declared:

1vec4 v_color0    : COLOR0    = vec4(1.0, 0.0, 0.0, 1.0);
2
3vec3 a_position  : POSITION;
4vec4 a_color0    : COLOR0;

A typical compile looks like this:

shaderc -f vs_cubes.sc --type vertex --platform windows -p s_5_0 -i src -o vs_cubes.bin

Then compiling it takes these steps:

1. Preprocess, first pass. The source goes through the C preprocessor once, with the platform, language and shader type defines set, so that includes are pulled in and any #if around the $input and $output lines is resolved.

2. Parse the $input and $output lines. These are not preprocessor directives. shaderc reads them off the top of the preprocessed text and strips them. They list which of the varyings from varying.def.sc this shader uses. shaderc also hashes the sorted list so that at runtime bgfx can verify a vertex shader’s outputs match a fragment shader’s inputs.

3. Build a preamble. This is where the target language is decided, and it is done by generating text and prepending it to the source. For HLSL-family targets the preamble starts with the type aliases:

 1	preprocessor.writef(
 2		"#define lowp\n"
 3		"#define mediump\n"
 4		"#define highp\n"
 5		"#define ivec2 int2\n"
 6		"#define ivec3 int3\n"
 7		"#define ivec4 int4\n"
 8		"#define uvec2 uint2\n"
 9		"#define uvec3 uint3\n"
10		"#define uvec4 uint4\n"
11		"#define vec2 float2\n"
12		"#define vec3 float3\n"
13		"#define vec4 float4\n"
14		"#define mat2 float2x2\n"
15		"#define mat3 float3x3\n"
16		"#define mat4 float4x4\n"
17		);

And then the actual trick. GLSL has void main() with globals for inputs and outputs. HLSL wants an entry point with typed, semantic-annotated parameters and a return struct. shaderc generates a macro for that from the varyings, roughly for the vertex shader above:

 1struct Output
 2{
 3	vec4 gl_Position : SV_POSITION;
 4#define gl_Position _varying_.gl_Position
 5	vec4 v_color0 : COLOR0;
 6#define v_color0 _varying_.v_color0
 7};
 8#define void_main() \
 9Output main( \
10	vec3 a_position : POSITION \
11	, vec4 a_color0 : COLOR0 \
12	) \
13{ \
14	Output _varying_; \
15	v_color0 = vec4(1.0, 0.0, 0.0, 1.0);
16#define __RETURN__ \
17	} \
18	return _varying_

The vec3 and vec4 in there are fine, because the type aliases above turn them into float3 and float4 on the way through. smooth and flat from varying.def.sc are rewritten to linear and nointerpolation while the struct is being generated.

4. Patch the source. The shader still says void main(). shaderc finds it and overwrites one byte, the space, with an underscore, so it now reads void_main() and expands to the macro above. For vertex shaders it then finds the closing brace of main and inserts __RETURN__; in front of it. That is the entire “translation” of the entry point: one byte and one string insert.

1	*const_cast<char*>(entry.getPtr() + 4) = '_';

The fragment shader side is the same idea, except main stays void and outputs become out parameters. gl_FragColor becomes bgfx_FragData0 : SV_TARGET0, gl_FragCoord becomes a SV_POSITION parameter, gl_FrontFacing is SV_IsFrontFace. Which of these get added to the signature is decided by searching the source text for the identifier.

5. Preprocess, second pass. The preamble plus the patched source go through the preprocessor again, and this time the macros do their work. void_main() expands, vec4 becomes float4, and bgfx_shader.sh, which the shader included through common.sh, supplies the rest. That header is where SAMPLER2D, texture2D, mul, mtxFromRows, vec4_splat and friends are defined per language. For HLSL it wraps texture and sampler pairs into structs so that GLSL-style texture2D(s_tex, uv) works; for GLSL it defines mul(a, b) as a * b. The header, not the compiler, is where most of the language knowledge lives:

1// HLSL
2#define SAMPLER2D(_name, _reg) \
3	uniform SamplerState _name ## Sampler : REGISTER(s, _reg); \
4	uniform Texture2D    _name ## Texture : REGISTER(t, _reg); \
5	static BgfxSampler2D _name = { _name ## Sampler, _name ## Texture }
6
7// GLSL
8#define SAMPLER2D(_name, _reg) uniform sampler2D _name
9#define mul(_a, _b) ( (_a) * (_b) )

For vs_cubes.sc, after that pass, the entry point FXC sees is roughly:

 1struct Output
 2{
 3	float4 gl_Position : SV_POSITION;
 4	float4 v_color0 : COLOR0;
 5};
 6
 7Output main(float3 a_position : POSITION, float4 a_color0 : COLOR0)
 8{
 9	Output _varying_;
10	_varying_.v_color0 = float4(1.0, 0.0, 0.0, 1.0);
11	_varying_.gl_Position = mul(u_modelViewProj, float4(a_position, 1.0) );
12	_varying_.v_color0 = a_color0;
13	return _varying_;
14}

Above that sits the rest of bgfx_shader.sh: built-in uniforms, mul, splat helpers, the lot. --preprocess dumps this text. When a target compiler complains, the code it is complaining about is this, not the .sc you wrote.

6. Hand it to a real compiler. The preprocessed text is now legal GLSL or legal HLSL, and it goes to FXC, DXC, Glslang, or whatever the profile wants. What comes back is the shader binary. Uniform and sampler reflection is taken from the compiler that produced it: D3DReflect after FXC, DXC’s reflection blob after DXIL, Glslang’s live uniforms or SPIRV-Cross resources after anything that went through SPIR-V. shaderc writes that into its own container, ahead of the shader blob, together with the $input and $output hashes from step 2.

Before and after glsl-optimizer

This is what shaderc looked like just before the glsl-optimizer removal in August 2026, with every target it had accumulated:

The left side is the legacy. glsl-optimizer’s front-end was Mesa as of the mid-2010s, and it never implemented GLSL 4.x, so anything above 4.00 (and ESSL above 3.00) had, since 2017, been passed through untouched. In practice that meant a GLSL 4.30 shader got no validation at all at build time. You found out it was broken when the driver rejected it at runtime, on whichever machine happened to run it first. Meanwhile every extension needed on the GLSL side was detected by grepping the source for identifiers and prepending #extension lines. glsl-optimizer would then add its own EXT and ARB suffixes to the functions those extensions provide, which had to be patched back out so that the same shader would load across GLSL and ESSL versions, and the runtime would re-add whatever the driver actually needed.

After removing glsl-optimizer and raising the minimum API versions on OpenGL to 4.3 and OpenGL ES to 3.0, and replacing fcpp with a preprocessor of my own, this is how the compiler looks today:

Every target now has a validated front-end; every target except FXC and DXC produces SPIR-V on the way. The GLSL / HLSL fork is still there: a shader can compile as spirv and fail as 430, because those are not the same program.

Could it be simpler?

If you look carefully at the diagram, you might notice that the GLSL and HLSL patching could be unified, because Glslang accepts HLSL and SPIRV-Cross is already used to output GLSL/ESSL. This would simplify the preprocessor macros and the patching, but having two paths keeps some flexibility for when a new shader language needs to be added, which doesn’t happen often.

Another thing that might not be obvious is that the bgfx shader language could simply be HLSL. Some of the bgfx shaping happens with preprocessor macros, but it’s feasible to do that transformation inside Glslang traversers, with plain HLSL as the front-end. It might look like the obvious choice to switch the default bgfx shader language to HLSL.

The problem is that this would tie shaderc to Glslang even further, and I don’t control where Glslang goes. This is not hypothetical. In April 2026 Glslang proposed deprecating and removing its HLSL front-end, and the README now reads:

The HLSL front-end is deprecated as of April 2026 and will be removed at the next major version of glslang. […] Projects that require continued HLSL support should maintain a fork of glslang at the tag corresponding to the deprecation announcement.

Had the bgfx shader language been HLSL, every existing shader would now need to move. Because the language is GLSL and HLSL is only an intermediate, the exposure is limited to the spirv, metal and wgsl paths, and those can be moved to another HLSL compiler, or to the GLSL front-end, without touching a single .sc file.

It cuts the other way too. The WGSL path exists only because I carry a stripped-down copy of Tint for one translation, SPIR-V to WGSL. SPIRV-Cross already does SPIR-V to GLSL, ESSL and MSL for me, so in January 2026 I asked whether it could do WGSL as well and let me drop Tint. The maintainer’s answer:

I don’t have bandwidth to implement and maintain yet another backend. There is also no motivation to effectively duplicate the effort that Tint already does.

Fair enough on bandwidth. But SPIRV-Cross is not one person’s side project, it is a Khronos project, and WGSL is a W3C standard implemented in every major browser. If the “Cross” in SPIRV-Cross is understood as “to every shading language”, this is Khronos declining to cross to one of them, and pointing at Google’s compiler instead. That is the whole reason shaderc looks the way it does. Every project in the pipeline solves its own problem, none of them shares bgfx’s vision, and nobody is obliged to. The Frankenstein is not a design choice. It is what you get when you assemble a compiler from parts whose owners don’t answer to you, and the seams between them are yours to maintain.

glsl-optimizer was a great idea!

glsl-optimizer was built for Unity 3.0, where it made mobile shaders faster on drivers whose own compilers did little or no optimization. bgfx picked it up for the same reason: in 2012 the GLSL targets were Android and Native Client, and the shader compilers behind those were not something you wanted to trust with unoptimized code.

It is simple. The library is Mesa’s GLSL front-end, Mesa’s IR, Mesa’s optimization passes, and one file, ir_print_glsl_visitor.cpp, that walks the IR and prints GLSL. Metal support, when it came, was a second printer, ir_print_metal_visitor.cpp. The public API is a dozen C functions in one header, and the ones you need are initialize, optimize, get status, get log, get output, cleanup. The initial shaderc.cpp used it in 30 lines.

The code is clear. Mesa’s GLSL IR is a plain tree of ir_instruction nodes with a visitor.

It outputs source. This is the property I want to underline, because it is the one that mattered most and the one that is rarest. Most shader compilers produce an IR that a driver consumes: DXBC, DXIL, SPIR-V. Or there is no optimizer, because text is always treated as source, and there is no split between text as source and text as IR. glsl-optimizer showed, in 2010, that the sane way to deal with text as an interchange format is not to ship the shader as the author wrote it, but to run it through a real compiler with real optimization passes and print the result: the minimum code required for the shader, with every dead uniform and unreachable branch already gone. I made this same argument in the WebGPU post when defending WGSL as textual IR; glsl-optimizer is where I got this “aha” moment about how shader code/text should be treated.

Why it had to go. Its author stopped working on it in mid-2016 when Unity moved to a proper (proprietary) shader compiler. Mesa has since moved its optimization work to NIR, so the GLSL IR that glsl-optimizer is built on is no longer maintained upstream either. It’s a dead project.

What makes shaderc janky?

In a 2014 survey of cross platform shaders, bgfx was the example of the first, and least sophisticated, approach on the list:

Write some helper macros to abstract away HLSL & GLSL differences, and make everyone aware of all the differences.

Twelve years later that is still what shaderc is, with real compilers bolted on behind the macros. A commenter on Hacker News put it more bluntly, in a thread about shader compilation for SDL_GPU:

BGFX uses a different approach. You basically write your shader in a GLSL-like language but it’s all just (either very clever or very horrible) macro expansions that handles all the platform differences. With that you get a very performant backend for OpenGL, WebGL, Vulkan, Metal, Direct3D 11 and 12 and Playstation. It’s shader compiler program basically does minimal source level transformations before handing it over to the platforms’ own shader compiler if available.

I’ll take “either”.

Error messages depend on the target. A shader that fails on s_5_0 gives you an FXC error. The same shader on spirv gives you a Glslang error, on metal a Glslang error followed possibly by a SPIRV-Cross assertion, on wgsl any of those plus a Tint diagnostic, and on 430 a Glslang error in the GLSL dialect. They report different line numbers, because each one is looking at a different preamble prepended to your code. shaderc tries to help by printing the patched source around the reported line, but you still have to know which tool is talking.

Features are detected by substring. Whether gl_FragDepth appears in the source decides whether SV_DEPTH goes into the signature.

The entry point is a macro. For a vertex shader, everything about main is done by rewriting one byte and inserting __RETURN__ before the last brace. An early return; inside a vertex shader’s main is a compile error, because main now returns Output. The docs say attributes and varyings can only be accessed from main(), and now you know why: outside main there is no _varying_ to expand to.

The dialect is real. SAMPLER2D(s_tex, 0) instead of uniform sampler2D, vec4_splat(x) instead of vec4(x), mul(a, b) instead of a * b, mtxFromRows instead of a matrix constructor, etc. Each of those exists because the two sides disagree and the preprocessor cannot resolve the disagreement without a hint from you.

How do others do it?

The alternatives I have seen in other open source libraries and engines do not do much better. Most take a similar Frankenstein approach: a shader compiler assembled from parts of other shader compilers, written by different teams with different goals. Here is how the ones I know of get from what you write to what the GPU driver accepts, sorted into four groups by how much of the job they actually do themselves.

Janky. Text goes in, gets transformed, and is handed to whichever compiler the target uses. Each of those compilers is free to accept or reject the result on its own terms, so the same source can pass on one API and fail on another. That is the jank, and this is the shaderc family:

Text translators. One parser of their own, one set of emitters of their own, and nothing you would call an optimizer in between. They print what you wrote in the target’s language and let the driver’s compiler do the rest. All three exist because a browser needed to compile somebody else’s shading language safely:

A real front-end. One compiler parses and validates the source, so an error is the same error no matter which backend you asked for, and one source really does target every backend. What happens after that is somebody else’s back-end: usually SPIR-V as the interchange format and SPIRV-Cross or Tint printing whatever the target wants. Some of these have a parser of their own in front (Godot, O3DE, Stride), but it only produces text for the next tool in line:

Actual compilers. Text in, optimization in the middle, and out comes whatever the API accepts, with no other compiler in between. Where the API accepts source, as OpenGL, Metal and WebGPU do, that source counts as the API’s IR. By that definition the list is short:

Slang qualifies everywhere except Direct3D, because nobody in these tables emits DXIL on their own. It is LLVM 3.7 bitcode that until 2025 had to be signed by Microsoft’s dxil.dll before a driver would accept it, and the one open source emitter I know of is Mesa’s nir_to_dxil, which is how Godot got there. glsl-optimizer is in the table because, for the one API it served, it was exactly this: parse, optimize, print what the driver accepts. The text translators above would qualify too if they optimized anything, and that is the situation glsl-optimizer was written to fix in 2010. The browsers have chosen to live with it.

Only the last group comes close to having no seam, and the one Slang keeps is Direct3D’s doing, not Slang’s. Everything above it hands text to a compiler someone else maintains. The real front-ends hide that better than shaderc does, at the cost of Glslang and SPIRV-Cross being load-bearing, which is exactly the exposure the HLSL front-end deprecation just demonstrated. None of them removes it, and the ones that look cleaner usually support fewer targets.

Conclusion

And yet it works! Fourteen years, eight target languages, every one of them added without a front-end rewrite. The structure that made this possible is the same one that makes the error messages inconsistent: shaderc owns the front of the pipeline, delegates the back, and the seam between them is text. Nearly everyone else in the tables above has that seam in the same place.

The one real newcomer in those tables is Slang. It is an actual compiler: a front-end, an IR, modules, generics, and code generation for HLSL, GLSL, SPIR-V, MSL, WGSL, and even Slang round-trip from one source. That is the set bgfx needs. It came out of NVIDIA research and has been hosted by Khronos since late 2024. The press release claims that “Valve compiled the entire production Source 2 HLSL codebase with Slang while modifying only 10 lines of code.” It is the first thing in this space that solves the problem instead of routing around it. It did not exist in 2012, which is why shaderc is a preprocessor and a hand-off. If this were started today, Slang would be the approach it would take.