Going parallel with reusable resources

In the previous section, we went over the basics of creating compute shaders and managing GPU memory. In this section, we will write a compute shader that runs more than one thread in parallel and look at what that changes.

So far, most of our shaders have looked like this:

import tgpu from 'typegpu';

const root = await tgpu.init();

const program = root.createGuardedComputePipeline(() => {
  'use gpu';
  // Run some code.
});

program.dispatchThreads();

The createGuardedComputePipeline() method is a simple way to create a function that runs on the GPU. Calling dispatchThreads() without arguments dispatches one logical GPU thread. That is useful for the smallest examples, but most compute workloads are interesting because they run many copies of the same program in parallel.

From one thread to many

To run more than one thread, add a positional argument to the shader function and pass a thread count to dispatchThreads(). In a one-dimensional dispatch, that argument is the thread index.

Let’s create a program that increments an entire array of values instead of incrementing a single value:

import tgpu, { d } from 'typegpu';

const root = await tgpu.init();

const valuesMutable = root.createMutable(d.arrayOf(d.u32, 16));

const program = root.createGuardedComputePipeline((x) => {
  'use gpu';
  valuesMutable.$[x]++;
});

export async function execute() {
  const threadCount = Math.floor(Math.random() * 16) + 1;
  program.dispatchThreads(threadCount);

  return {
    threadCount,
    values: await valuesMutable.read(),
  };
}

ValuesNot run

00

10

20

30

40

50

60

70

80

90

100

110

120

130

140

150

Run

Each time you run the example, it chooses a random thread count between 1 and 16. If it dispatches 6 threads, thread 0 increments array element 0, thread 1 increments array element 1, and so on, up to thread 5. The preview highlights the part of the array touched by the latest dispatch.

Dispatch size and bounds

The dispatchThreads() method controls how many logical GPU threads will run the shader function. If we dispatch 16 threads for a 16-element array, we increment the whole array. If we dispatch 4 threads, only the first 4 elements are incremented because x takes values from 0 to the passed count minus 1.

We still have to be careful not to read or write outside the bounds of the array. WebGPU prevents those accesses from reaching memory outside the binding, but that does not make them useful. The shader may still continue with meaningless data, or a write you expected may not happen.

Tip

If the dispatch size might be larger than the buffer length, guard the array access inside the shader:

const program = root.createGuardedComputePipeline((x) => {
  'use gpu';
  if (x >= valuesMutable.$.length) {
    return;
  }

  valuesMutable.$[x]++;
});

A useful mental model

For the increment example above, you can think of the shader as something close to this JavaScript loop:

const values = Array.from({ length: 16 }, () => 0);

for (let x = 0; x < values.length; x++) {
  values[x]++;
}

This is a useful analogy for understanding how the thread index maps to data. But it is only an analogy. The shader invocations run in parallel, and their execution order is not guaranteed.

Caution

When one thread reads data that another thread may be writing, the loop analogy breaks down:

const program = root.createGuardedComputePipeline((x) => {
  'use gpu';
  if (x >= valuesMutable.$.length || x === 0) {
    return;
  }

  const sum = valuesMutable.$[x - 1] + valuesMutable.$[x];
  valuesMutable.$[x] = sum;
});

The result is nondeterministic because each thread may read valuesMutable.$[x - 1] before or after the neighboring thread writes to it. When shader invocations need to coordinate, the algorithm has to account for that explicitly.

Multi-dimensional compute shaders

When using createGuardedComputePipeline(), we can pass a function that takes up to three arguments: x, y, and z. These are the thread indices along the x, y, and z axes, respectively.

On a high level, a 2D compute shader can often be reduced to a 1D compute shader by flattening the coordinates into one index. The same is true for 3D work. That translation is sometimes useful, but it can make the shader harder to read because the first thing each thread has to do is recover the coordinates it actually cares about.

Note

There is also a practical reason to keep work split across dimensions. WebGPU limits how much work can be dispatched along a single axis. The limit is high, but large workloads can still run into it.

For grid-shaped workloads, spreading the work across x and y lets the dispatch shape match the data while staying within those per-axis limits.

Let’s take a look at a simple 2D compute shader that increments a randomly sized rectangle inside a grid:

import tgpu, { d } from 'typegpu';

const root = await tgpu.init();

const WIDTH = 8;
const HEIGHT = 6;

const valuesMutable = root.createMutable(
  d.arrayOf(d.arrayOf(d.u32, HEIGHT), WIDTH),
);

const program = root.createGuardedComputePipeline((x, y) => {
  'use gpu';
  valuesMutable.$[x][y]++;
});

export async function execute() {
  const dispatchWidth = Math.floor(Math.random() * WIDTH) + 1;
  const dispatchHeight = Math.floor(Math.random() * HEIGHT) + 1;
  program.dispatchThreads(dispatchWidth, dispatchHeight);

  return {
    dispatchWidth,
    dispatchHeight,
    values: await valuesMutable.read(),
  };
}

GridNot run

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

Run

This time, the shader callback receives two indices. The x argument selects the outer array, and the y argument selects the inner array, so valuesMutable.$[x][y] points at one cell in the grid.

The dispatch uses the same shape, only now we provide two counts instead of one:

const dispatchWidth = Math.floor(Math.random() * WIDTH) + 1;
const dispatchHeight = Math.floor(Math.random() * HEIGHT) + 1;

program.dispatchThreads(dispatchWidth, dispatchHeight);

This asks TypeGPU to run dispatchWidth * dispatchHeight logical threads, but each thread still receives its own coordinate pair. For a 4 x 2 dispatch, x ranges from 0 to 3, while y ranges from 0 to 1. The preview highlights the rectangle touched by the latest dispatch.

This is the main reason to use a 2D dispatch for grid-shaped data: the shader can stay in the same coordinate system as the problem. The same pattern extends to 3D by adding a third callback argument and passing a third number to dispatchThreads().

Swapping resources

So far, we have used createMutable() to create buffers that shader code can read and write directly. This is very convenient for simple shaders and requires no additional setup. We refer to this path as Fixed Resources: the shader references one particular GPU resource and TypeGPU binds it for us.

Sometimes, though, we want to swap which buffer is used by the shader for a particular dispatch. Later, the same idea will apply to other resource types too. For that, TypeGPU offers Laid Out Resources.

Laid out resources split the setup into two pieces:

• A bind group layout describes the shape of the resources the shader expects.
• A bind group provides the concrete GPU resources that satisfy that layout.

The layout is created first, without providing any actual buffers. For buffers, each layout entry uses the same schema we would use to create the buffer. If the entry is a storage buffer, it can also include an access mode:

import tgpu, { d } from 'typegpu';

const bindGroupLayout = tgpu.bindGroupLayout({
  counter: { storage: d.u32, access: 'mutable' },
  values: { storage: d.arrayOf(d.u32, 16) },
});

Each key in the bind group layout maps to a single resource. For the buffer resources we have explored so far, the equivalent layout entries look like this:

Fixed Resources shorthand	Laid Out Resources entry
root.createUniform(d.u32)	{ uniform: d.u32 }
root.createReadonly(d.u32)	{ storage: d.u32 }
root.createMutable(d.u32)	{ storage: d.u32, access: 'mutable' }

Storage entries are readonly by default, so { storage: d.u32 } and { storage: d.u32, access: 'readonly' } describe the same shader access.

To provide the resources, create a bind group with root.createBindGroup(layout, entries):

const counterBuffer = root.createBuffer(d.u32).$usage('storage');
const valuesBuffer = root.createBuffer(d.arrayOf(d.u32, 16)).$usage('storage');

const bindGroup = root.createBindGroup(bindGroupLayout, {
  counter: counterBuffer,
  values: valuesBuffer,
});

Note

Because bind group entries are typed, providing a resource with the wrong schema or usage is a TypeScript error.

const wrongBuffer = root.createBuffer(d.f32).$usage('storage');
const invalidBindGroup = root.createBindGroup(bindGroupLayout, {
  counter: wrongBuffer,
Error ts(2322)  ― Type 'TgpuBuffer<F32> & StorageFlag' is not assignable to type 'GPUBuffer | (TgpuBuffer<U32 | Atomic<U32> | DecoratedLocation<U32>> & StorageFlag)'.
  Type 'TgpuBuffer<F32> & StorageFlag' is not assignable to type 'TgpuBuffer<U32 | Atomic<U32> | DecoratedLocation<U32>> & StorageFlag'.
    Type 'TgpuBuffer<F32> & StorageFlag' is not assignable to type 'TgpuBuffer<U32 | Atomic<U32> | DecoratedLocation<U32>>'.
      Types of property 'dataType' are incompatible.
        Type 'F32' is not assignable to type 'U32 | Atomic<U32> | DecoratedLocation<U32>'.
          Property '[$memIdent]' is missing in type 'F32' but required in type 'U32'.
  values: valuesBuffer,
});

Here is that resource swap in a small particle simulation. The shader stays the same, while the selected bind group decides which particle buffer is advanced:

import tgpu, { d, std } from 'typegpu';

const root = await tgpu.init();

const PARTICLE_COUNT = 96;

const Particle = d.struct({
  position: d.vec2f,
  velocity: d.vec2f,
});
const ParticleArray = d.arrayOf(Particle, PARTICLE_COUNT);

const particleLayout = tgpu.bindGroupLayout({
  particles: { storage: ParticleArray, access: 'mutable' },
});

declare const initialStates: { position: d.v2f; velocity: d.v2f }[][]

const particleBuffers = initialStates.map((state) =>
  root.createBuffer(ParticleArray, state).$usage('storage'),
);

const bindGroups = particleBuffers.map((particles) =>
  root.createBindGroup(particleLayout, { particles }),
);

const simulate = root.createGuardedComputePipeline((i) => {
  'use gpu';
  const particle = particleLayout.$.particles[i];
  particleLayout.$.particles[i].position = std.fract(
    particle.position + particle.velocity,
  );
});

declare function render(selected: number): void;

export function run(selected: number) {
  const bindGroup = bindGroups[selected]!;

  simulate.with(bindGroup).dispatchThreads(PARTICLE_COUNT);
  render(selected);
}

Orbit0 steps

Buffer ABuffer BBuffer C

Run

The visible snippet hides the rendering code behind render(...) because render pipelines and SDF drawing have not been introduced yet. The full example below includes that part.

Show full example code

import { sdBox2d } from '@typegpu/sdf';
import tgpu, { common, d, std } from 'typegpu';
import { createExampleRoot } from './runnable/index.ts';

const PARTICLE_COUNT = 96;
const PARTICLE_HALF_SIZE = 0.0065;

const Particle = d.struct({
  position: d.vec2f,
  velocity: d.vec2f,
});
const ParticleArray = d.arrayOf(Particle, PARTICLE_COUNT);

export type ParticleValue = {
  position: d.v2f;
  velocity: d.v2f;
};

export type BufferIndex = 0 | 1 | 2;

export const BUFFER_COLORS = [
  d.vec3f(0.4, 0.91, 0.98),
  d.vec3f(0.98, 0.66, 0.83),
  d.vec3f(0.75, 0.95, 0.39),
] as const;

function createOrbitState(): ParticleValue[] {
  return Array.from({ length: PARTICLE_COUNT }, (_, i) => {
    const t = (i / PARTICLE_COUNT) * Math.PI * 2;
    const radius = 0.28 + Math.sin(i * 0.7) * 0.045;
    const tangent = t + Math.PI / 2;

    return {
      position: d.vec2f(0.5 + Math.cos(t) * radius, 0.5 + Math.sin(t) * radius),
      velocity: d.vec2f(Math.cos(tangent) * 0.0065, Math.sin(tangent) * 0.0065),
    };
  });
}

function createDriftState(): ParticleValue[] {
  const columns = 12;
  const rows = PARTICLE_COUNT / columns;

  return Array.from({ length: PARTICLE_COUNT }, (_, i) => {
    const x = i % columns;
    const y = Math.floor(i / columns);
    const offset = Math.sin(i * 1.37) * 0.012;

    return {
      position: d.vec2f((x + 0.5) / columns, (y + 0.5) / rows + offset),
      velocity: d.vec2f(0.009, 0.004),
    };
  });
}

function createBurstState(): ParticleValue[] {
  return Array.from({ length: PARTICLE_COUNT }, (_, i) => {
    const t = (i / PARTICLE_COUNT) * Math.PI * 2;
    const ring = (i % 12) / 12;
    const radius = 0.03 + ring * 0.12;
    const speed = 0.004 + ring * 0.007;

    return {
      position: d.vec2f(0.5 + Math.cos(t) * radius, 0.5 + Math.sin(t) * radius),
      velocity: d.vec2f(Math.cos(t) * speed, Math.sin(t) * speed),
    };
  });
}

const INITIAL_STATES = [createOrbitState(), createDriftState(), createBurstState()] as const;

export async function createBindGroupProgram(canvas: HTMLCanvasElement) {
  const root = await createExampleRoot();
  const context = root.configureContext({ canvas, alphaMode: 'premultiplied' });

  const computeLayout = tgpu.bindGroupLayout({
    particles: { storage: ParticleArray, access: 'mutable' },
  });
  const displayLayout = tgpu.bindGroupLayout({
    particles: { storage: ParticleArray },
  });

  const particleBuffers = INITIAL_STATES.map((state) =>
    root.createBuffer(ParticleArray, state).$usage('storage'),
  );
  const computeBindGroups = particleBuffers.map((particles) =>
    root.createBindGroup(computeLayout, { particles }),
  );
  const displayBindGroups = particleBuffers.map((particles) =>
    root.createBindGroup(displayLayout, { particles }),
  );

  const particleColor = root.createUniform(d.vec3f, BUFFER_COLORS[0]);

  const simulate = root.createGuardedComputePipeline((i) => {
    'use gpu';
    const particle = computeLayout.$.particles[i];
    computeLayout.$.particles[i].position = std.fract(particle.position + particle.velocity);
  });

  const render = root.createRenderPipeline({
    vertex: common.fullScreenTriangle,
    fragment: ({ uv }) => {
      'use gpu';
      const background = d.vec3f(0.09, 0.08, 0.15);
      const gridColor = d.vec3f(0.19, 0.18, 0.29);
      const gridUv = std.fract(uv * 8);
      const verticalGrid = std.min(gridUv.x, 1 - gridUv.x);
      const horizontalGrid = std.min(gridUv.y, 1 - gridUv.y);
      const gridDistance = std.min(verticalGrid, horizontalGrid);
      const gridMask = 1 - std.smoothstep(0.003, 0.007, gridDistance);

      let particleDistance = d.f32(1);
      for (const particle of displayLayout.$.particles) {
        const offset = std.abs(uv - particle.position);
        const wrappedOffset = std.min(offset, 1 - offset);
        particleDistance = std.min(
          particleDistance,
          sdBox2d(wrappedOffset, d.vec2f(PARTICLE_HALF_SIZE)),
        );
      }

      const particleMask = 1 - std.smoothstep(0, std.fwidth(particleDistance), particleDistance);
      const withGrid = std.mix(background, gridColor, gridMask);
      const withParticles = std.mix(withGrid, particleColor.$, particleMask);

      return d.vec4f(withParticles, 1);
    },
  });

  function resizeCanvasToDisplaySize() {
    const pixelRatio = Math.min(window.devicePixelRatio || 1, 2);
    const width = Math.min(1024, Math.max(1, Math.round(canvas.clientWidth * pixelRatio)));
    const height = Math.min(1024, Math.max(1, Math.round(canvas.clientHeight * pixelRatio)));

    if (canvas.width !== width || canvas.height !== height) {
      canvas.width = width;
      canvas.height = height;
    }
  }

  function draw(bufferIndex: BufferIndex) {
    const displayBindGroup = displayBindGroups[bufferIndex];
    if (!displayBindGroup) {
      return;
    }
    resizeCanvasToDisplaySize();
    particleColor.write(BUFFER_COLORS[bufferIndex]);
    render.with(displayBindGroup).withColorAttachment({ view: context }).draw(3);
  }

  return {
    computeBindGroups,
    draw,
    particleCount: PARTICLE_COUNT,
    root,
    simulate,
  };
}

Now the same simulation step can be written in either style. The Fixed Resources version is bound to one particular set of resources, while the Laid Out Resources version keeps the pipeline reusable and supplies the resources when dispatching.

Laid Out Resources

import tgpu, { d } from 'typegpu';

const root = await tgpu.init();

const Particle = d.struct({
  position: d.vec3f,
  velocity: d.vec3f,
});

const deltaTimeBuffer = root.createBuffer(d.f32).$usage('uniform');
const particlesBuffer = root.createBuffer(d.arrayOf(Particle, 512)).$usage('storage');

const simulationLayout = tgpu.bindGroupLayout({
  deltaTime: { uniform: d.f32 },
  particles: { storage: d.arrayOf(Particle), access: 'mutable' },
});

const bindGroup = root.createBindGroup(simulationLayout, {
  deltaTime: deltaTimeBuffer,
  particles: particlesBuffer,
});

const simulate = root.createGuardedComputePipeline((x) => {
  'use gpu';
  const particle = simulationLayout.$.particles[x];
  simulationLayout.$.particles[x].position =
    particle.position + particle.velocity * simulationLayout.$.deltaTime;
});

simulate.with(bindGroup).dispatchThreads(512);

Fixed Resources

import tgpu, { d } from 'typegpu';

const root = await tgpu.init();

const Particle = d.struct({
  position: d.vec3f,
  velocity: d.vec3f,
});

const deltaTimeUniform = root.createUniform(d.f32);
const particlesMutable = root.createMutable(d.arrayOf(Particle, 512));

const simulate = root.createGuardedComputePipeline((x) => {
  'use gpu';
  const particle = particlesMutable.$[x];
  particlesMutable.$[x].position =
    particle.position + particle.velocity * deltaTimeUniform.$;
});

simulate.dispatchThreads(512);

The tradeoff is a little more setup, but the shader is no longer bound to one concrete resource. It can be separated cleanly from runtime resource setup and reused with different bind groups.

Tip

Layouts are the only way to swap resources for a shader after pipeline creation, but they are not the only way to create modular shaders. We will cover slots and accessors in later sections. They provide a similar way of defining resource-independent shaders, but each change requires recompiling the shader, which can degrade performance.

Since bind groups are immutable by nature, you need to create a new bind group each time you want to use a new resource. If you have a limited set of combinations, you can cache and reuse them.

Summary

You now know how to dispatch many GPU threads, map them to 1D and 2D data, and keep shader accesses in bounds. You have also seen when fixed resources are enough and when bind groups are useful for reusing the same shader with different buffers.