18

I am trying to create a framework that works with METAL Api (iOS). I am pretty new to this platform and I would like to know how to build the framework to work with .metal files (I am building a static lib, not dynamic). Should they be a part of the .a file, or as a resource files in the framework bundle? Or is there an other way to do that? Thanks.

Update: For those who tackle this - I ended up following warrenm's 1's suggested option - converted the .metal file into a string and calling newLibraryWithSource:options:error:. Although it is not the best in performance it allowed me to ship only one framework file, without additional resources to import. That could be useful to whoever creating framework that uses Metal, ARKit, etc with shader files.

DocForNoc
  • 346
  • 2
  • 13
  • Glad you found an approach that worked for you. For posterity, I've commented below and shared a proof-of-concept project that allows you to maintain the benefit of shader pre-compilation while still only shipping one file. If I were doing this in production, it's almost certainly the approach I would take. – warrenm Oct 15 '17 at 20:55

3 Answers3

23

There are many ways to provide Metal shaders with a static library, all with different tradeoffs. I'll try to enumerate them here.

1) Transform your .metal files into static strings that are baked into your static library.

This is probably the worst option. The idea is that you preprocess your Metal shader code into strings which are included as string literals in your static library. You would then use the newLibraryWithSource:options:error: API (or its asynchronous sibling) to turn the source into an MTLLibrary and retrieve the functions. This requires you to devise a process for doing the .metal-to-string conversion, and you lose the benefit of shader pre-compilation, making the resulting application slower.

2) Ship .metal files alongside your static library and require library users to add them to their app target

All things considered, this is a decent option, though it places more of a burden on your users and exposes your Metal shader source (if that's a concern). Code in your static library can use the "default library" (newDefaultLibrary), since the code will be compiled automatically by Xcode into the app's default.metallib, which is embedded in the app bundle as a resource.

3) Ship a .metallib file alongside your static library

This is a good middle ground between ease-of-use, performance, and security (since it doesn't expose your shader source, only its IR). Basically, you can create a "Metal Library" target in your project, into which you put your shader code. This will produce a .metallib file, which you can ship along with your static library and have your user embed as a resource in their app target. Your static library can load the .metallib at runtime with the newLibraryWithData:error: or newLibraryWithURL:error: API. Since your shaders will be pre-compiled, creating libraries will be faster, and you'll keep the benefit of compile-time diagnostics.

warrenm
  • 31,094
  • 6
  • 92
  • 116
  • Thanks for your detailed answer. Just to make things clear - if I don't want to ship another file to my users - the 1'st solution is only one fits? – DocForNoc Oct 15 '17 at 08:56
  • I suppose you could embed the pre-compiled library by taking the bytes from a metallib file and writing them as a literal array of bytes in your static library source. That would satisfy the one-file requirement while also giving the benefits of precompilation. – warrenm Oct 15 '17 at 17:32
  • 1
    I just tested this last approach and it does work. Here's a proof-of-concept project that does it all: build a `.metallib`, bake it into a header file, and create a `MTLLibrary` at runtime from which pipelines can be made: https://www.dropbox.com/s/8w30r1gyutj9twc/EmbeddedKernelSample.zip?dl=0. This is by no means production-ready, but it should illustrate the approach well enough. – warrenm Oct 15 '17 at 20:48
  • Great! I will give this one a try. – DocForNoc Oct 16 '17 at 09:29
  • I added the metallib by drag&dropping it to Copy Files phase in Build Phases, and selecting Resources as Destination. That works, but it hard-codes the source to something like ../../../Library/Developer/Xcode/DerivedData/VidWorkspace-gnnwdwbnewfpadcksukplsporkda/Build/Products/Debug-iphoneos/MyMetalLib.metallib which doesn't survive if I want to reuse the project somewhere else... How do you tell Xcode to point at the right location every time? – endavid Mar 18 '18 at 16:09
  • To make metallib, we can also use metal commandline tool, `xcrun -sdk iphoneos metal -c MyLibrary.metal -o MyLibrary.air xcrun -sdk iphoneos metallib MyLibrary.air -o MyLibrary.metallib ` – JohnHanr Jan 11 '19 at 07:23
  • Hello @warrenm! Could you please elaborate what kind of performance hit my app would experience if I choose an option number 1? I understand that compilation of the library from the string source might take a while but I do not understand why it would have an impact on my kernels performance – Eugene Alexeev Aug 10 '21 at 09:01
  • It shouldn't affect the performance of the compiled kernel; I was strictly referring to the overhead of the Metal compiler. – warrenm Aug 10 '21 at 18:39
21

As someone looking to include metal shader functions in a SceneKit / ARKit related framework, the available answers led me in the wrong direction. There is a much simpler solution that uses makeDefaultLibrary(bundle: Bundle) (iOS 10+) to access the functions included in a framework's .metal dependencies. Adding here for people in a similar position.

TL;DR, Access a Framework's MTLLibrary like this:

        //Get the framework bundle by using `Bundle(for: type(of: self))` from inside any framework class.
        //Then use the bundle to define an MTLLibrary.
        let frameworkBundle = Bundle(for: type(of: self))
        let device = MTLCreateSystemDefaultDevice()
        do {
            let bundleLib = try device?.makeDefaultLibrary(bundle: frameworkBundle)
            print(bundleLib.functionNames) //we can access our framework's metal functions! No build tricks/workarounds.
        } catch {
            print("Couldn't locate default library for bundle: \(frameworkBundle)")
            print( error )
        }

Xcode creates a default library of shader functions at build time by compiling .metal dependencies. This is true of both framework targets and app targets, so the real question is, how do I access my framework’s default library?

It’s possible to access a framework’s default library using the using the makeDefaultLibrary(bundle: Bundle) method on MTLDevice. The sample code above shows more detail.

For Scenekit/ARKit with SCNProgram

The bundle library can be set as an SCNProgram’s library property, and then fragment and shader functions can be defined just as if the .metal file was included in the main project:

        //The SCNProgram that will use our framework's metal functions
        var program = SCNProgram()

        //Use the framework's bundle to define an MTLLibrary.
        let frameworkBundle = Bundle(for: type(of: self))
        let device = MTLCreateSystemDefaultDevice()
        do {
            let bundleLib = try device?.makeDefaultLibrary(bundle: frameworkBundle)

            //set the SCNProgram's library, and define functions as usual
            program.library = bundleLib
            program.fragmentFunctionName = "yourCustomFrameworkFragmentFunction"
            program.vertexFunctionName = "yourCustomFrameworkVertexFunction"
        } catch {
            print("Couldn't locate default library for bundle: \(frameworkBundle)")
            print( error )
        }
Jim Martin
  • 211
  • 2
  • 3
0

The approach suggested by the questioner could not possibly work (hence, the lack of sample code). A Metal shader (.metal) is just a collection of functions, it does not a MTLLibrary (.metallib) make. Here is working code that compiles a Metal shader from a character (const char *) array (not the same as NSString); it is followed by instructions for converting a .metal file to a .metallib file prior to runtime.

Compiling a Metal Shader during Runtime

The following sample could also be used to provide users with a Shader Editor, and can allow you to update just the shader portion of your app without requiring the user to update the entire app:

NSError* error = NULL;
const char* vshSource =
"using namespace metal;\n"
"typedef struct {\n"
"    packed_float2 position;\n"
"    packed_float2 texcoord;\n"
"} Vertex;\n"

"typedef struct {\n"
"    float3x3 matrix;\n"
"    float3 offset;\n"
"} ColorConversion;\n"

"typedef struct {\n"
"    float4 position [[position]];\n"
"    float2 texcoord;\n"
"} Varyings;\n"

"vertex Varyings vertexPassthrough(\n"
"device Vertex* verticies [[ buffer(0) ]],\n"
"unsigned int vid [[ vertex_id ]]\n"
") {\n"
"   Varyings out;\n"
"   device Vertex& v = verticies[vid];\n"
"    out.position = float4(float2(v.position), 0.0, 1.0);\n"
"    out.texcoord = v.texcoord;\n"
"    return out;\n"
"}\n";

const char* fshSource =
"using namespace metal;\n"
"typedef struct {\n"
    "packed_float2 position;\n"
    "packed_float2 texcoord;\n"
"} Vertex;\n"

"typedef struct {\n"
    "float3x3 matrix;\n"
    "float3 offset;\n"
"} ColorConversion;\n"

"typedef struct {\n"
    "float4 position [[position]];\n"
    "float2 texcoord;\n"
"} Varyings;\n"

"fragment half4 fragmentColorConversion(\n"
                                       "Varyings in [[ stage_in ]],\n"
                                       "texture2d<float, access::sample> textureBGRA [[ texture(0) ]],\n"
                                       "constant ColorConversion &colorConversion [[ buffer(0) ]]\n"
                                       ") {\n"
    "constexpr sampler s(address::clamp_to_edge, filter::linear);\n"
    "return half4(half3(textureBGRA.sample(s, in.texcoord).rgb), 1.0);\n"
"}\n";

id <MTLFunction> vertexProgram;
id <MTLLibrary> vertexLibrary = [_device newLibraryWithSource:[NSString stringWithUTF8String:vshSource] options:NULL error:&error];
if (NULL != vertexLibrary)
{
    vertexProgram = [vertexLibrary newFunctionWithName:@"vertexPassthrough"];
} else {
    NSLog(@"Error compiling vertex program: %@", error.description);
}

id <MTLFunction> fragmentProgram;
id <MTLLibrary> fragmentLibrary = [_device newLibraryWithSource:[NSString stringWithUTF8String:fshSource] options:NULL error:&error];
if (NULL != fragmentLibrary)
{
    fragmentProgram = [fragmentLibrary newFunctionWithName:@"fragmentColorConversion"];
}  else {
    NSLog(@"Error compiling fragment program: %@", error.description);
}

The following are excerpts from an Apple Developer Documentation publication; although the information is relatively rudimentary, use it as a basis for a common framework shared by you and your audience when communicating about its subject matter.

Creating Libraries During the App Build Process

The accepted answer is flat-out wrong for the same reasons; and, it's claims about performance trade-offs are questionable. Here are the only definitive statements that can be made about compiling Metal shaders and creating Metal libraries, followed by actual code:

Functions and Libraries

This chapter describes how to create a MTLFunction object as a reference to a Metal shader or compute function and how to organize and access functions with a MTLLibrary object.

MTLFunction Represents a Shader or Compute Function

A MTLFunction object represents a single function that is written in the Metal shading language and executed on the GPU as part of a graphics or compute pipeline. For details on the Metal shading language, see the Metal Shading Language Guide.

To pass data or state between the Metal runtime and a graphics or compute function written in the Metal shading language, you assign an argument index for textures, buffers, and samplers. The argument index identifies which texture, buffer, or sampler is being referenced by both the Metal runtime and Metal shading code.

For a rendering pass, you specify a MTLFunction object for use as a vertex or fragment shader in a MTLRenderPipelineDescriptor object, as detailed in Creating a Render Pipeline State. For a compute pass, you specify a MTLFunction object when creating a MTLComputePipelineState object for a target device, as described in Specify a Compute State and Resources for a Compute Command Encoder.

A Library Is a Repository of Functions

A MTLLibrary object represents a repository of one or more MTLFunction objects. A single MTLFunction object represents one Metal function that has been written with the shading language. In the Metal shading language source code, any function that uses a Metal function qualifier (vertex, fragment, or kernel) can be represented by a MTLFunction object in a library. A Metal function without one of these function qualifiers cannot be directly represented by a MTLFunction object, although it can called by another function within the shader.

The MTLFunction objects in a library can be created from either of these sources:

  • Metal shading language code that was compiled into a binary library format during the app build process.
  • A text string containing Metal shading language source code that is compiled by the app at runtime.

Compiling shader language source files and building a library (.metallib file) during the app build process achieves better app performance than compiling shader source code at runtime. You can build a library within Xcode or by using command line utilities.

Using Xcode to Build a Library

Any shader source files that are in your project are automatically used to generate the default library, which you can access from Metal framework code with the newDefaultLibrary method of MTLDevice.

Using Command Line Utilities to Build a Library

Figure 8-1 shows the command line utilities that form the compiler toolchain for Metal shader source code. When you include .metal files in your project, Xcode invokes these tools to build a library file that you can access in your app at run time.

To compile shader source into a library without using Xcode:

  1. Use the metal tool to compile each .metal file into a single .air file, which stores an intermediate representation (IR) of shader language code.
  2. Optionally, use the metal-ar tool to archive several .air files together into a single .metalar file. (metal-ar is similar to the Unix ar.)
  3. Use the metallib tool to build a Metal .metallib library file from IR .air files or from archive .metalar files.

Example: Building a Library File with Command Line Utilities

xcrun -sdk macosx metal MyLibrary.metal -o MyLibrary.air
    xcrun -sdk macosx metallib MyLibrary.air -o MyLibrary.metallib

To access the resulting library in framework code, call the newLibraryWithFile:error: method:

NSError *libraryError = NULL;
NSString *libraryFile = [[NSBundle mainBundle] pathForResource:@"MyLibrary" ofType:@"metallib"];
id <MTLLibrary> myLibrary = [_device newLibraryWithFile:libraryFile error:&libraryError];
if (!myLibrary) {
    NSLog(@"Library error: %@", libraryError);
}
James Bush
  • 1,485
  • 14
  • 19
  • Hi James, I was the one asked the question. Your answer is very informative, thanks for sharing. One thing I do have to say - converting the shader files to string indeed allowed me to build a static lib. code: `id defaultLibrary = [_device newLibraryWithSource: options:[MTLCompileOptions new] error:&errors]; id capturedImageVertexFunction = [defaultLibrary newFunctionWithName:];` – DocForNoc May 01 '18 at 12:33
  • I updated my answer to demonstrate exactly how to compile a Metal shader during runtime. – James Bush May 02 '18 at 17:14