2

I'm writing a browser game in Rust. The basic flow is in WASM memory we have an array of pixels. Every frame we change some pixels based on user input, then we draw that image to a <canvas>. I'm doing this by creating an ImageBitmap from an ImageData from a UInt8ClampedArray. Here's (a simplified version of) my code right now:

src/lib.rs:

#[wasm_bindgen]
pub struct Board {
  data: Vec<u32>,
}

#[wasm_bindgen]
pub struct WasmSlice {
  pub adr: usize,
  pub len: usize,
}

#[wasm_bindgen]
impl Board {
  pub fn get_image_slice(&self) -> WasmSlice {
    WasmSlice {
      adr: self.data.as_ptr() as u32,
      len: self.data.len() * 4, // sizeof u32 / sizeof u8 = 4
    }
  }
}
index.js:

import * as wasm from ...;

const board = wasm.Board.new();
const ctx = document.getElementById("canvas").getContext("2d");

loop();

function loop() {
  board.do_stuff();
  draw();
  requestAnimationFrame(loop);
}

function draw() {
  const { adr, len } = board.get_image_slice();
  createImageBitmap(new ImageData(
    new Uint8ClampedArray(wasm.memory.buffer).subarray(adr, adr + len),
    1200,
    1200
  ))
    .then(bitmap => {
      ctx.drawImage(bitmap, 0, 0, 600, 600);
    })
    .catch(console.error);
}

First off, you'll notice I'm doing some manual pointer work. I'm doing this for two reasons: first, I couldn't find a good way to pass a slice from Rust to JS with wasm_bindgen, and I didn't like the idea of creating the ImageBitmap / ImageData in Rust with web_sys. Second, I needed to transmute my Vec<u32> into a Vec<u8>. Doing manual pointer stuff solved both these problems.

Now I could have drawn the ImageData directly to the canvas, but using an ImageBitmap allows me to scale the image, which I am using to get an antialiasing effect. It is also much slower, as right now half of my program's time is spent in createImageBitmap. I assume this takes so long because it's copying the image onto the GPU. When I tried just using ctx.putImageData() my program performed much better. I have also done some experiments confirming that new ImageData() and new UInt8ClampedArray() are not making copies, and are just holding references to the wasm memory buffer. My experiments also suggest that when I use ctx.putImageData() I'm still making a copy.

So am I right that the image has to be copied at least once per frame to get from wasm memory to my screen? How can I be sure that I'm not doing excessive copies?

The only alternative I can see to copying the whole image would be updating specifically the pixels that I change each frame. If I tried for example:

let img = ctx.getImageData();
// Do stuff to img.
ctx.putImageData(img);

would that code copy the canvas's image buffer out and back in, or is img a direct reference to the screen buffer so there's no copying?

If I skip using an ImageBitmap so that I don't have to copy to the GPU, then whether I'm diffing specific pixels or redrawing the whole image, I'll have to do my own antialiasing, right? In this case I could antialias pixels just when they're updated, which should be faster than using the GPU but would be more manual.

A. Kriegman
  • 510
  • 1
  • 4
  • 18
  • 1
    Not time to compose a proper answer but worth a read: https://github.com/whatwg/html/issues/5173 Also, instead of `ctx.drawImage(await createImageBitmap(ImageData))` you may want to try `ctx.putImageData(ImageData);ctx.drawImage(ctx.canvas)` I'd expect it to be a bit faster, though I'm not sure. – Kaiido Dec 22 '21 at 06:12
  • Thank you, I'm getting much better performance now with that approach. Redrawing from canvas back into itself didn't work, but with an offscreen canvas it worked fine. ```const ctx = document.getElementById("board").getContext("2d"); const ctxOff = document.createElement("canvas").getContext("2d"); function draw() { const { x: ptr, y: len } = board.get_image_slice(); ctxOff.putImageData(new ImageData( new Uint8ClampedArray(memory.buffer).subarray(ptr, ptr + len), 1200, 1200 ), 0, 0); ctx.drawImage(offscreen, 0, 0, 600, 600); } ``` – A. Kriegman Dec 22 '21 at 14:54

0 Answers0