Deploy workers / Fleet management

Manage a fleet

Use this reference after connecting your workers. For your first controller and worker, follow Deploy workers. Each worker keeps its own identity, data directory, and concurrency limit.

Pair a child over Tailscale SSH

If your Linux child supports Tailscale SSH, Horde can install, enroll, and start it directly. On a child with Horde installed, stop its unconfigured daemon before first-time pairing:

horde stop
horde network setup --worker

Alternatively, prepare Tailscale SSH yourself. Use a non-root login and allow that SSH connection and outbound child connections to the controller on TCP port 7443 in your tailnet policy.

On a parent with Horde networking configured:

horde network peers
horde network add alice@worker
horde runtime list

Replace alice@worker with the child’s login and hostname. Pairing installs Horde if absent and waits for its authenticated connection. Configure the child’s model providers separately. For devices without SSH access, use the invitation walkthrough.

Inspect and update workers

Run these commands on the controller. Use an ID from runtime list and replace VERSION with a published Horde release:

horde runtime list
horde runtime inspect RUNTIME_ID
horde runtime update RUNTIME_ID --version VERSION --request-id update-worker-VERSION
horde runtime restart RUNTIME_ID --request-id restart-worker-1
horde call management_events '{"after":0}'

Reuse a request ID only to retry the same operation. Wait for an update to report succeeded before updating the next worker. Failed or uncertain updates pause further fleet updates; inspect the result before resuming them.

Managed binary updates verify signatures and checksums, drain active work, preserve state, and restart. If the replacement cannot start and the service manager terminated its update helper, restore the previous launcher and restart the service. See update and recovery details.

Package-manager and source installations use their owning installer. Docker and Kubernetes updates use signed release images. AX workers require a new runtime with a new pinned image; see AX deployment.

Start workers at boot

For workers you enroll through Tailscale SSH, select service installation during initial pairing from the controller:

horde network setup --service
horde network add alice@worker --service

The first command enables controller boot startup. The second enables it on the selected SSH-paired worker and requires passwordless sudo there. Do not use SSH pairing to replace a worker already enrolled through an invitation. Without these flags, pairing starts the daemon without installing a boot service. Configure each paired worker’s executors and authentication separately.

Containers and managed sandboxes use their deployment’s restart policy or a sandbox supervisor. Preserve one data directory per worker across restarts.

Enroll workers without SSH

For workers started by your deployment tools, use automatic fleet enrollment. A private fleet invitation lets each worker generate its own identity and connect outbound. Keep its recovery credential available and preserve its data directory; do not clone an enrolled worker’s directory to create another worker.

Certificate lifetimes

Controllers generated by Tailscale setup have one-year certificates. A fleet worker’s recovery invitation must still be valid, and the worker must not have been revoked. See enrollment recovery and revocation before removing credentials.

Project access

For project creation, repository registration, and provider accounts, start with Work in a project.

Enrollment identifies a worker. Grant each project the workers it should use:

horde project runtime-grant PROJECT RUNTIME_ID

A manually configured receiver needs the same immutable project ID and matching execution grants. See project authorization across hosts for both sides of that setup, and project capacity and credentials for scheduling and account behavior.