Streaming

Building a whole world up front makes load time grow with its size, and memory with it. A chunked world builds only what is near the camera and frees what falls behind, so what is resident stays bounded while the world it is cut from need not be. This page covers the chunk loop, placing things so that chunk order does not matter, and streaming terrain refined to 50 or 100 voxels per metre.

Chunks around a focus pointCopy Anchor Link

makeChunkedWorld splits the world into columns (a square footprint over the full height) and calls your generate(ctx) for each one it builds. You tell it where to keep the world with focus(x, z, radius, keep?), and give it a time budget each frame with step(ms).

import { makeChunkedWorld, scatterRegion } from "@voxolith/engine";
import { generateTerrain } from "@voxolith/gen-terrain";

const SIZE = { x: 2560, y: 192, z: 2560 };
const terrain = generateTerrain({ width: SIZE.x, depth: SIZE.z, height: SIZE.y }, seed);
const REACH = 40; // half the widest tree, so trees rooted next door are drawn too

const world = makeChunkedWorld({
  target: renderer,
  size: SIZE,
  chunk: 64,
  seed,
  generate(ctx) {
    ctx.edit({ ...ctx.box, y1: terrain.maxY() }, (cells, ox, oy, oz) =>
      terrain.fillBrick(cells, ox, oy, oz, terrainBase));
    scatterRegion({ cell: 24, seed, salt: 1 },
      ctx.box.x0 - REACH, ctx.box.z0 - REACH, ctx.box.x1 + REACH, ctx.box.z1 + REACH, (p) => {
        if (p.x < 0 || p.z < 0 || p.x >= SIZE.x || p.z >= SIZE.z) return;
        if (terrain.waterAt(p.x, p.z) || p.rng() < 0.6) return;
        const v = oaks.pick(p.rng);
        ctx.blit(v.entity.model, { x: p.x, y: terrain.heightAt(p.x, p.z) + 1, z: p.z }, oakBase, v.orientation);
      });
  },
});

// Every frame, from the render callback:
const [x, , z] = orbit.target();
world.focus(x, z, 900); // queues what is missing, nearest first; frees what is past `keep`
if (world.pending) world.step(4); // at most 4 ms of building this frame

• focus is cheap to call every frame: it only diffs the resident set. Chunks inside radius are queued nearest-first, so the view fills from the middle. Chunks beyond keep (default radius * 1.5) are cleared; the gap stops a camera on the boundary rebuilding the same chunk.

• step(budgetMs) builds queued chunks until the budget runs out and returns how many are left, so you can show progress. pending and resident report the queue and the live set.

• ctx.edit and ctx.blit are clamped to the chunk, and ctx.rng is seeded from the chunk coordinates.

• When focus moves, chunks still queued are re-ranked from the new point, so the queue stays nearest first to where the camera is now.

To build the middle of the view before showing the page, wait for the ground around the camera rather than the whole radius. world.whenReady(x, z, radius) resolves once every chunk whose centre lies within radius is built (pendingWithin and readyAround give the same as data); the world builds only in step, so keep stepping meanwhile, with a larger budget behind the loading screen, yielding to the browser between steps. Chunk centres are what is tested, so add half a chunk's diagonal to cover a disc:

world.focus(x, z, 2400);
const near = 1800 + 256 * Math.SQRT1_2; // every 256-voxel chunk reaching into 18 m
while (!world.readyAround(x, z, near)) {
  world.step(40); // behind the loading screen a frame can take more
  await new Promise((r) => requestAnimationFrame(() => setTimeout(r, 0))); // let the frame paint
}
loadingScreen.ready();

The valley and nightwood examples wait this way for 18 m and 8 m of ground (their ?near=), and let the rest stream in after the first picture.

Placement must not depend on orderCopy Anchor Link

Chunks are built in whatever order the camera reaches them, so generate must be a pure function of the chunk: no shared counters, no global rejection sampling, no Math.random. Two tools make that easy:

• scatterRegion(opts, x0, z0, x1, z1, visit) is a jittered grid. Each cell has one candidate whose position and rng come from a hash of its coordinates, so any chunk can work out its own candidates and its neighbours'. Draw every decision about a candidate (whether it exists, which species, which variant) from p.rng. Use a different salt per layer, so trees and grass do not land on the same points.
• ctx.blit clips to the chunk. Offer every entity that could reach into the chunk, including ones rooted next door, by widening the scatter rectangle by the largest entity's reach. Each chunk then writes only its own voxels and a tree over a boundary comes out whole whichever side is built first.

Decide anything global (where the village goes, which pads are levelled, where the paths run) once, up front and deterministically, and have chunks only read it. The world example's layout.ts does this for a settlement.

Streaming fine terrainCopy Anchor Link

At 20, 50 or 100 voxels per metre the ground no longer fits in memory at once, so it has to stream. refineTerrain(terrain, k, { seed }) from @voxolith/gen-terrain is the same terrain k times finer, answered a brick at a time. The coarse heightfield stays the source of truth: levelled pads stay flat, the river stays put. The fine one adds a smooth surface, grass blades and pebbles, and only a skin of ground below the surface.

Edit one box per 8 x 8 brick column, sized by columnSpan (columnBoxes from the engine makes them), and send a chunk's boxes in one editMany. Each column is then computed once for its whole stack of bricks, about 40 ms per 256² chunk.

import { columnBoxes, makeChunkedWorld } from "@voxolith/engine";
import { refineTerrain } from "@voxolith/gen-terrain";

const K = 10;
const ground = refineTerrain(terrain, K, { seed });

const world = makeChunkedWorld({
  target: renderer,
  size: { x: terrain.width * K, y: 192 * K, z: terrain.depth * K },
  chunk: 256,
  seed,
  generate(ctx) {
    const boxes = columnBoxes(ctx.box, (ox, oz) => ground.columnSpan(ox, oz));
    renderer.editMany(boxes, (cells, ox, oy, oz) => ground.fillBrick(cells, ox, oy, oz, terrainBase));
  },
});

That is still 50 ms or so a chunk on the thread that draws. The fill is a pure function of the terrain, so it can move to workers: makeChunkedWorld({ fill, boxes }) with a makeChunkFillPool sends each chunk's boxes to a few workers that rebuilt the terrain from fineTerrainInit, and the main thread only copies the bricks back in, about 10 ms a chunk at 100 voxels per metre (measured). The wiring, the worker file and what to watch are in ground on workers.

Fine models are sparse, so the valley example draws every house, tree and rock as a static instance placed once for the whole map, and streams only the ground.

CostsCopy Anchor Link

• The GPU cost of a fine valley grows fast with the scale: the valley example measures about 19 MB at 20 voxels per metre (a phone's default), 126 MB at 50 (a desktop's) and 560 MB at 100. It defaults to three model variants per species (?variants=3) for that reason; ten match the coarse page exactly but need over a gigabyte at 100.
• A soft fog at the streaming radius hides the edge. The valley example adds a light haze whatever the weather and keeps the zoom limit well inside it.
• Frame time spent in step is frame time not spent drawing. A budget of 4 to 6 ms per frame while moving keeps the view responsive. With fill, step applies at least one chunk that came back whatever the budget, and a chunk's copy is about 10 ms at 100 voxels per metre, so a frame that applies one takes that long.

See alsoCopy Anchor Link

• Engine: placement and worlds
• Generators: terrain
• Worlds and storage for edit, editMany and clear
• API: makeChunkedWorld, ChunkContext, scatterRegion, refineTerrain, FineTerrain, columnBoxes, makeChunkFillPool