2. Network and edge

Two networks to build: an SDN fabric for tenant private networks, and a shared public segment carrying routed public addresses.

Decide the address plan first

Write these down before creating anything. Renumbering an EVPN fabric later is a project, not a change.

Decision	Guidance
Tenant address pool	A large private range that overlaps nothing in your datacentre. One /20 per private network, one /24 per subnet
Public block	Routed to your edge’s WAN address. Confirm with your upstream that the whole block is routed, not bridged
Underlay	Addresses the hypervisors use to carry VXLAN between themselves

VLAN or EVPN

	VLAN mode	EVPN mode
Segment	A tagged VLAN per subnet	A VXLAN VNet per subnet
Isolation	VLAN separation	A VRF per private network
Physical switch	Must trunk every VLAN you use	Carries only the underlay
Scale	Bounded by the VLAN identifier space	Effectively unbounded
Complexity	Lower	Higher — needs a working underlay and routing daemon

EVPN is the better model for a multi-tenant platform and is what production uses. VLAN mode is a legitimate choice for a small installation with a switch you already control.

EVPN fabric

Create one controller for the cluster, peering the nodes:

pvesh create /cluster/sdn/controllers --controller z9evpn --type evpn \
  --asn 65000 --peers <node1-underlay>,<node2-underlay>,<node3-underlay>

Then, per private network, one zone and its VNets:

pvesh create /cluster/sdn/zones --zone z9v<N> --type evpn --controller z9evpn \
  --vrf-vxlan <vrf-vni> --mtu 1450

pvesh create /cluster/sdn/vnets --vnet z9v<N>a0 --zone z9v<N> --tag <l2-vni>
pvesh create /cluster/sdn/vnets/z9v<N>a0/subnets \
  --subnet <10.x.y.0/24> --type subnet --gateway <10.x.y.1>

MTU 1450 is not optional. The VXLAN header costs 50 bytes. Leaving the default produces a network where small packets work and large ones vanish — the hardest class of problem to diagnose.

Verify the underlay can actually carry it, between two nodes:

ping -M do -s 8972 -c3 <other-node-underlay>

Danger

Do not set --isolate-ports 1 on a VNet. Proxmox applies port isolation to the entire VNet, which isolates platform components along with guests and kills the management path.

The symptom is deeply misleading: the bridge FDB learns both MAC addresses, ip -br link show master lists both taps, and tcpdump is completely silent.

Clearing the configuration afterwards is not enough — the flag is written when the tap joins the bridge, so running machines keep it. Clear it live: ip link set <tap> type bridge_slave isolated off

Shared public segment

Public addresses are routed to virtual machines directly, with no NAT, so every public interface shares one layer-2 segment on a VLAN:

pvesh create /cluster/sdn/zones --type vlan --zone z9vlan --bridge vmbr0
pvesh create /cluster/sdn/vnets --vnet z9pub --zone z9vlan --tag <vlan-id>

No SDN subnet is defined here. The gateway for that block lives on your edge router; Proxmox is not doing address management for public addresses.

Applying

pvesh set /cluster/sdn

Applying SDN configuration is supposed to be hitless. Run a continuous ping to an existing production machine through the apply and measure it rather than trusting it.

pvesh get /cluster/sdn/zones --output-format json
systemctl status frr --no-pager | head -3

The edge

Your edge router owns the public block and the default route for tenant traffic. What it must do:

1. Route the public block toward the region, without translating it. A public address is configured on the virtual machine itself — that is the product’s model.
2. Carry the public segment to the hypervisors on the VLAN you chose.
3. Apply north-south policy you want at the edge, above per-machine firewalls.

zone9 programs only what it owns through the edge adapter — address objects and the policy entries tied to them. Your own rules are left alone.

Two operating modes are available for the adapter, and starting in the safe one is recommended: an analysis mode that only reads and refuses writes, and an apply mode.

Verify before moving on

• A test machine on a tenant VNet reaches another machine on the same VNet.
• A machine with a public address is reachable from the internet, not just from your office.
• The two are correct at the same time — a public address on a machine that also has a private interface must not break either.