Scaling out

One server runs one production pipeline at a time. To run more at once, add servers. They coordinate through the shared database and never talk to each other.

A server runs one production pipeline at a time. The nodes within that pipeline all run at once and stream to each other, so this is one pipeline's full parallelism, not one busy core. Beside it, each server runs one Test run (the builder's 50-row preview) in a small memory slice of its own, so building never waits for a pipeline to finish.

More servers run more different pipelines at once. One pipeline still runs once at a time in each environment, whatever the number of servers: two runs of it would read the same new files and rows twice. While one is going, a Run of the same pipeline is refused and names it, a schedule that comes due records a failed run saying why, a file trigger waits for its next check, a Flow step waits its turn, and the Run API answers 429 so the caller retries.

To run more pipelines at once, add servers:

1. Put the same files on the new server: the same compose.yaml and the same version as your existing one (copy both files across, or paste them from the install page if every server is being moved to the current release).
2. In its .env, set COMPOSE_PROFILES=core (or worker). Put the connection string for the one central Postgres all servers share into secrets/db.key (see Configure). Every profile reads the same file, so this step is identical on every server.
3. Copy the same secrets/master.key file across, and use the same LICENSE_KEY. On both servers the file belongs to uid 10001, so read it with sudo, and on the new one put it in with sudo tee and hand it over as Configure shows. Put it in place before the first start on the new server: Trinium refuses to start against a database it did not encrypt rather than generating a second key over your data. A worker never creates a key or changes the database at all; without the file it refuses to start and names it.
4. If you serve HTTPS, each server needs its own tls/cert.pem and tls/key.pem, or put them all behind one load balancer and leave tls/ empty.

They coordinate entirely through the shared database, so any number of servers can share one workload safely and none needs to know the others exist. Work is claimed by whichever server is free.

On Pro, each server you add is a licensed server: contact us to add it to your license before you start it. Enterprise covers any number of servers.

Every server lists itself in Settings → Servers the first time it starts, named after the last number of its address (worker-046); rename it there if you like. It keeps that name through restarts and upgrades, because its identity is a file in its own data volume, so never copy the filedata volume from one server to another, or the two will claim the same identity. When you retire a server, remove it from that list.

Choosing which server runs what

By default any free server takes the next run. To decide it yourself, open Settings → Servers and press Routing on a server:

• Tags are labels you make up, such as eu, fast or big-memory. A pipeline whose Run on (in its Edit dialog on the Pipelines page) names a tag runs only on servers carrying it: its Staging, Production and Test runs alike.
• Also run pipelines that name no server tag: turn it off to keep a server for its tagged pipelines only: a fast lane for a 15-minute sync that must never wait behind a two-hour load.
• Runs: Staging and Production, Staging only, or Production only, to keep production on its own servers without changing any pipeline. Every Test run is a Staging run.

Trinium refuses a change that would leave untagged pipelines, or Test runs, with no server, and a run that no server could take is refused by name instead of waiting for ever.

More than one server requires shared file storage. The filedata volume is local to each server, so a file uploaded on one is invisible to another's engine. Before adding a second server, switch Settings → File storage to S3 or Azure Blob (Google Cloud Storage is offered for backups, not for the file store). This is a requirement, not an optimization.

Local backups stay on the server that took them. Every server that serves the web runs the backup schedule, whichever gets there first takes the backup, and a Local backup is written to that server's own backups/ folder. History names the server that took each one, and on any other server it says Unable to locate the backup. So download or restore it through that server, or use a cloud destination, which every server reaches.