How can I make webpack embed my *.wasm for use in a web worker?

Question

I have some rust code that compiles to web assembly using wasm-pack and wasm-bindgen. I want to call into this code from a web worklet/worker. The entire app should eventually be just one single *.js file, with everything else inlined.

This is what I imagine my build process to look like:

1. Use wasm-pack to compile the rust code to *.wasm and *.js bindings (this step works just fine)
2. Use webpack to build a self-contained *.js file that I can load as a worklet/worker. The *.wasm must be included in this file. (this step fails)
3. Use webpack again to build my final app/package, inlining the worklet/worker file from step 2. (this step works just fine)

My problem is in step 2: I can't make webpack inline the *.wasm into the worklet/worker file. I tried this in my webpack config:

entry: {
    worker: {
        import: './src/worker.ts',
        filename: '../lib/worker.js',
    }
},

// ...

module: {
    rules: [
    
        // ...

        {
            test: /\.wasm$/,
            // 1st option: type: 'webassembly/sync',
            // 2nd option: type: 'asset/inline',
        },

        // ...

    ],
},

No matter what I do, webpack always emits two files, one worker.js with my worklet/worker script itself, and another one, vendor_my_package_name_wasm_js.js that contains just the *.wasm and its bindings. Obviously, when loading the worker.js as a web worker, it fails - the second file can't be loaded from the worker scope.

My goal is to include everything in worker.js and NOT have a separate file emitted. But how do I do that?

Edit: Documenting steps towards a solution:

Webpack native wasm-loading doesn't seem to allow inlining the wasm file. We can try to use a regular raw-loader:

// in module.rules
{
    test: /\.wasm$/,
    loader: 'raw-loader',
},

This results in the following error:

ERROR in ./node_modules/my-module/my-wasm-file.wasm
Module parse failed: magic header not detected
File was processed with these loaders:
 * ../../node_modules/raw-loader/dist/cjs.js
You may need an additional loader to handle the result of these loaders.

This happens because there's still an implicit default rule that kicks in. We can disable it by overwriting the default rules to only consider json and js files:

// in webpack.config.js
    module: {
        defaultRules: [
            {
                type: 'javascript/auto',
                resolve: {},
            },
            {
                test: /\.json$/i,
                type: 'json',
            },
        ],
        rules: [
            // ...
            {
                test: /\.wasm$/,
                loader: 'raw-loader',
            },
        ],
    },

Now we finally have our worker bundled into a single *.js file! However, when loading it, we end up in this error:

Uncaught ReferenceError: document is not defined

pointing to this piece of webpack-generated code:

/* webpack/runtime/jsonp chunk loading */
/******/    (() => {
/******/        __webpack_require__.b = document.baseURI || self.location.href; // <<< error here
/******/        
/******/        // object to store loaded and loading chunks
/******/        // undefined = chunk not loaded, null = chunk preloaded/prefetched
/******/        // [resolve, reject, Promise] = chunk loading, 0 = chunk loaded
/******/        var installedChunks = {
/******/            "myModuleName": 0
/******/        };
/******/        
/******/        // no chunk on demand loading
/******/        
/******/        // no prefetching
/******/        
/******/        // no preloaded
/******/        
/******/        // no HMR
/******/        
/******/        // no HMR manifest
/******/        
/******/        // no on chunks loaded
/******/        
/******/        // no jsonp function
/******/    })();

For some reason webpack tries to support loading stuff dynamically (?). We can isolate the problem to this piece of code that was generated by wasm-pack as part of the javascript bindings when using the --target=web CLI argument:

async function init(input) {
    if (typeof input === 'undefined') {
        input = new URL('my_wasm_file.wasm', import.meta.url);
    }
    const imports = {};
    // ...

Apparently the possibility of having to generate a URL makes webpack rely on document which is not available when loading the worker script in the worker scope. Uncommenting the new URL() part makes the document reference disappear from the webpack output.

Not sure where to go from here. Write my own wasm-loader? I worked on that for a while, base64 encoding the wasm file and inlining it as a string - but then I have to dramatically change the consumer code to manually load the wasm asynchronously. This means that I can't use the wasm-bindgen bindings anymore as they rely on either the URL part shown above (when using --target=web) or the bundling logic of webpack 5 (when using --target=bundler) which I can't get supported from my own simple wasm-loader attempts. Essentially that means that I have to provide my own JS bindings, which is inconvenient.

There must be a better way - right?

Answer 1

The solution

1. Build the wasm itself: cargo build --target=wasm32/unknown/unknown
2. Build the JS-bindings: wasm-bindgen --out-dir=dist --target=web --omit-default-module-path my-wasm-package.wasm.
3. Consume the wasm in your worklet script like this:

import init, { /* other stuff */ } from 'my-wasm-package';
import wasmData from 'my-wasm-package/my-wasm-package.wasm';

const wasmPromise = init(wasmData);

4. Use this webpack config to build the worklet script into one single *.js file:

// ...

module.exports = {

    // ...

    module: {
        rules: [
            {
                test: /\.ts$/,
                loader: 'ts-loader',
                options: {
                    configFile: 'tsconfig.json',
                },
            },
            {
                test: /\.wasm$/,
                type: "asset/inline",
            },
        ],
    },
};

Why it works

The core problem I observed was that the JS bindings generated by wasm-bindgen contain the URL keyword - which makes webpack initialise in such a way that it expects the document object to be defined. That object is undefined in the worklet scope, so initializing webpack would crash, even if we never entered the code section that contains the URL.

If we don't use wasm-pack to build the wasm and the bindings in one go, we can pass additional arguments to wasm-bindgen - mainly the --omit-default-module-path argument which removes the section with the URL from the bindings. Now webpack won't reference document when it initializes and we can use the bindings without modification.

From here it's simple: We bundle the wasm as a b64 encoded asset and pass it to the init() function that comes with the JS bindings.

Answer 2

I could get accepted solution to be part of my build process, so I did this:

https://github.com/SonOfLilit/vscode-web-wasm-rust

It involved writing a webpack WASM Loader plugin (and patching webpack to make that possible, PR submitted) to use this loader:

/******/    /* webpack/runtime/wasm loading */
/******/    (() => {
/******/        __webpack_require__.v = (exports, wasmModuleId, wasmModuleHash, importsObj) => {
/******/            var vscode = require('vscode');
/******/            var wasmPath = __webpack_require__.p + wasmModuleHash + '.module.wasm';
/******/            var req = vscode.workspace.fs.readFile(vscode.Uri.file(wasmPath));;
/******/            return req
/******/                .then((bytes) => (WebAssembly.instantiate(bytes, importsObj)))
/******/                .then((res) => (Object.assign(exports, res.instance.exports)));
/******/        };
/******/    })();

as well as setting __webpack_public_path__ in activate before trying to import the WASM:

export function activate(context: vscode.ExtensionContext) {
  __webpack_public_path__ =
    context.extensionUri.toString().replace("file:///", "") + "/dist/web/";

  let disposable = vscode.commands.registerCommand(
    "vscode-rust-web.helloWorld",
    () => {
      require("rust-wasm").then((rust: any) => {
        vscode.window.showInformationMessage(rust.greet());
      });
    }
  );

  context.subscriptions.push(disposable);
}

Answer 3

The accepted answer is great for a situation that matches the context of the question exactly (you want to build your own Rust code into a WASM module and then use it in a worker). I faced the same exact error messages in a slightly different context, but I thought my findings may be of help here nonetheless.

My context was: I have a WASM dependency that has been built by someone else so it is less convenient to change the flags used to build the JS bindings (such as wasm-bindgen's --omit-default-module-path). In my case WASM was not even built from Rust, but its JS bindings had the same URL markers as described in the question.

DISCLAIMER: I'm neither a WASM nor a webpack expert, so I'm not aware of the full implications of the below approach. In my context it didn't cause unintended side-effects, but I'm not confident that's a sure thing.

The solution was to specify a baseUri property for an entry in webpack's config:

entry: {
    worker: {
        import: './src/worker.ts',
        filename: '../lib/worker.js',
        baseUri: 'my:worker.ts',
    }
},

As of this writing the property doesn't seem to be present in the official docs, but has been contributed.

This changes the webpack-generated code to

/******/    /* webpack/runtime/jsonp chunk loading */
/******/    (() => {
/******/        __webpack_require__.b = "my:worker.ts";
/******/        // ... rest remains the same as in the question's description ...
/******/    })();

This removes references to document and avoids the error at runtime.