Architecture overview
zone9 is three things that talk to each other over narrow, explicit interfaces: a control plane, one or more regions, and a set of appliances that are just virtual machines.
┌─ Users ───────────────────────────────────────────────────────────────────┐│ Browser console · REST API client · MCP-speaking agent │└───────────────────────────────┬───────────────────────────────────────────┘ │ OIDC session or bearer token┌───────────────────────────────┴───────────────────────────────────────────┐│ CONTROL PLANE (Kubernetes, or anywhere else) ││ ││ Next.js console ──► Hono API ──► PostgreSQL ││ │ ▲ ││ │ │ jobs, intent, allocations ││ └────────────┘ ││ ││ Holds NO hypervisor credentials in agent mode. │└───────────────────────────────┬───────────────────────────────────────────┘ │ HTTPS, opened BY the region, outbound only┌───────────────────────────────┴───────────────────────────────────────────┐│ REGION (your datacentre) ││ ││ zone9-agent (Go) ──► Proxmox VE REST API ││ │ edge router API ││ │ ││ └── holds the credentials. They never leave this box. ││ ││ ┌── Appliances: ordinary VMs, locked, no SSH ────────────────────────┐ ││ │ zone9-lb (HAProxy) · gateway (nftables, Tailscale) · │ ││ │ zone9-s3 (Garage) · Kubernetes nodes (Talos Linux) │ ││ │ Each PULLS its own configuration from the API. │ ││ └────────────────────────────────────────────────────────────────────┘ │└───────────────────────────────────────────────────────────────────────────┘What each plane is responsible for
Section titled “What each plane is responsible for”The control plane decides. It authenticates, authorises, validates, and — critically — allocates. Every private address, every subnet range, every management index is handed out by transactional SQL in one place. Nothing downstream invents an identifier or an address. This is what makes concurrent provisioning safe, and it is also a security property: a compromised agent cannot quietly assign itself another tenant’s address range, because the panel would reject the result.
It never blocks on infrastructure. An API call writes intent and a job row, then returns.
The region executes. The agent claims jobs for its own region, talks to Proxmox over its REST API, and reports back. It holds the credentials. It connects outbound, so the region needs no inbound firewall hole and no VPN into your datacentre.
Appliances converge. A load balancer, a gateway or a storage node fetches its own configuration, applies it, and reports which version it applied. Nobody pushes to them.
Which failures each plane survives
Section titled “Which failures each plane survives”This table is the practical reason for the split. Read it as a promise you can test.
| Failure | What keeps working |
|---|---|
| Control plane down | Everything already provisioned. VMs run, traffic flows, load balancers keep their last configuration, clusters are unaffected. You cannot create or change anything. |
| Agent VM down | Everything already provisioned, plus appliances (they talk to the API directly, not through the agent). Jobs queue up and run when it returns. |
| Network between region and control plane cut | Same as above. The region is autonomous by design. |
| Proxmox node down | Standard Proxmox behaviour. zone9 adds nothing and takes nothing away. |
| zone9 deleted entirely | Your VMs, disks, networks and firewall rules stay exactly as they are. Every one of them is inspectable and repairable with qm, pvesh and the PVE web UI. |
The last row is a design goal, not an accident: see What zone9 is.
The two directions of data
Section titled “The two directions of data”Understanding these two flows explains most of the panel’s behaviour.
Intent flows down. You express what you want. The panel stores it, versions it, and turns it into jobs or into configuration that appliances fetch.
Truth flows up. The panel reads Proxmox to learn what actually exists. Appliances report the configuration version they actually applied. Nothing is assumed to have worked because a call returned 200.
The gap between the two is what the panel displays as applying, applied, degraded. See Intent, truth and reconciliation.
Where the boundaries are enforced
Section titled “Where the boundaries are enforced”- TypeScript and Go never share code. The API (TypeScript) and the executor (Go) talk only through the OpenAPI contract and Postgres tables. If business logic starts accumulating in the HTTP layer, that is a design violation with a defined remedy: move it to the executor or express it in the database contract.
- Proxmox is touched only over REST. No SSH to hypervisor nodes, no writing under
/etc/pve, no shelling out toqmorpvesh. This is absolute, and it has cost features — where a feature required a wider privilege, the feature was redesigned. - The panel does not manage infrastructure. Switches, VLAN trunks, hypervisor node configuration and the edge router’s own policy are provisioned once, out of band. A control plane that reconfigures the network is a control plane whose bug takes the datacentre with it.
- Control plane and regions — the two region modes
- Intent, truth and reconciliation — the central idea
- The job model — idempotence, leases, resumption
- Appliances and the pull model — how VMs get configured
- Technology stack — every component and what it actually runs