Guides

BGP

Per-VRF BGP with an embedded GoBGP speaker (active-active).

SiHA embeds a GoBGP speaker that runs inside the appliance process (pilotd). It announces prefixes to upstream fabric switches, learns routes from them, and programs the best paths into VPP’s FIB — one entry per VRF. Multiple equal-cost next-hops are installed as ECMP, which is the basis for the active-active load-balancer demo.

Two resource types drive the feature:

  • BGPConfig — one per VRF, holds ASN, router-id, and the maintenance drain switch.
  • BGPPeer — one per neighbor, names the remote and controls prefix filters.

A third type, BGPLearnedRoute, is read-only: pilotd writes one entry per best-path learned and installed.

The per-VRF model

Every BGPConfig resource id is the VRF name. The reserved id default is the default VRF (VPP table 0) and must leave vrf empty; any other id must set vrf to match. Each VRF presents its own localAsn on the wire, so one appliance can run distinct ASNs toward different tenant fabrics from a single GoBGP process.

The router-id is global in v1 (set on the default config). All VRFs share it.

BGPConfig

type: BGPConfigs.bgp.siha
metadata:
  namespace: bgp
  id: default            # "default" → default VRF, table 0
spec:
  localAsn: 65010
  routerId: 192.168.100.1    # required on id=default; IPv4 only
  # announceNextHop overrides the next-hop attribute on outbound UPDATE messages
  # (useful when the appliance is behind a NAT or re-announcing from a VRF).
  announceNextHop: ""        # omit to use the local session address
  # listenAddresses makes GoBGP bind TCP/179 on these addresses so remote peers
  # can dial in (e.g. Cilium agents, a fleet of k8s nodes). Each address must
  # be on a linux-cp-mirrored Interface. Empty = dial-out only.
  listenAddresses: []
  # dynamicNeighbors accepts inbound sessions from any address in a CIDR
  # without requiring a BGPPeer resource per host. Requires listenAddresses.
  dynamicNeighbors: []
  # drained withdraws every announced prefix while keeping sessions alive and
  # continuing to learn routes — the maintenance drain switch. Flip to true
  # before draining traffic from this node; flip back to restore.
  drained: false

For a tenant VRF the id must equal the vrf field:

type: BGPConfigs.bgp.siha
metadata:
  namespace: bgp
  id: tenant-a
spec:
  vrf: tenant-a          # must match id; no routerId required here
  localAsn: 65020        # this VRF presents AS 65020 on the wire
  drained: false

BGPPeer

type: BGPPeers.bgp.siha
metadata:
  namespace: bgp
  id: spine1
spec:
  neighborAddress: 192.168.100.254   # remote BGP speaker; must be IPv4 (v1)
  peerAsn: 65000
  localAddress: 192.168.100.1        # the local address GoBGP dials from
                                     # (must be on a linux-cp-mirrored Interface)
  holdTimeSec: 0                     # 0 = controller default (90s); min 3
  # announce: prefixes this appliance originates and sends to the peer.
  announce:
    - 10.100.0.0/24
    - 10.100.1.0/24
  # acceptPrefixes: import filter — only routes in these CIDRs are installed in
  # the FIB. Empty accepts everything the peer advertises.
  acceptPrefixes:
    - 0.0.0.0/0
  # exportPrefixes: export filter — only routes in these CIDRs are advertised
  # to this peer (includes re-advertised routes learned from other peers).
  # When set it must cover every prefix in announce.
  exportPrefixes: []
  # maxPrefixes: tears down the session if the peer sends more than this many
  # prefixes. 0 = unlimited.
  maxPrefixes: 0
  # md5Password: TCP MD5 session authentication (optional).
  md5Password: ""
  vrf: ""                # empty = default VRF; set to bind to a tenant VRF

Binding a peer to a tenant VRF:

type: BGPPeers.bgp.siha
metadata:
  namespace: bgp
  id: bird2-tenant-a
spec:
  neighborAddress: 10.30.0.1
  peerAsn: 65030
  localAddress: 10.30.0.2
  announce:
    - 10.100.2.0/24
  vrf: tenant-a          # learned routes go into VPP table for tenant-a

Reach (linux-cp)

VPP owns the data NICs; the node kernel has no route to data subnets. The BGP TCP session must therefore run on a linux-cp mirror address — declare lcpHostIf on the Interface that carries the local address. VPP punts inbound TCP/179 traffic to the lcp tap, where GoBGP’s socket receives it; outbound SYNs leave via the same tap and re-enter VPP through the mirrored path. The same constraint applies to listenAddresses on BGPConfig.

A peer whose localAddress is not on an lcp-enabled interface will fail to establish (the kernel has no route to originate the TCP SYN).

Active-active ECMP

When multiple peers advertise the same prefix, GoBGP selects the best path and any equal-cost siblings. All are programmed into the VRF’s VPP FIB table as a multipath entry. VPP hashes flows across next-hops, giving per-flow load-balancing without any additional configuration.

This is the model behind the active-active LB demo: two appliance nodes, each a BGP speaker, announce the same VIP prefix; the fabric ECMP-hashes inbound flows across them.

Draining for maintenance

Set drained: true on a BGPConfig to withdraw all announced prefixes while keeping sessions established and continuing to install learned routes. The appliance stops attracting new flows but keeps forwarding existing ones through its own VPP FIB. Flip back to false to re-announce.

S="sihactl --sihaconfig ~/.siha/config --addr <node-ip>:6443"

$S get bgpconfig default -o yaml         # inspect current drain state
$S edit bgpconfig default                # set drained: true / false

BGPConfigStatus.drained mirrors the live drain state for observability.

Dynamic neighbors

For large fleets (Cilium agents, k8s nodes) where adding one BGPPeer per host is impractical, dynamicNeighbors lets any address within a CIDR establish a session without a dedicated resource:

spec:
  localAsn: 65010
  routerId: 192.168.100.1
  listenAddresses: ["192.168.100.1"]   # GoBGP must bind for peers to dial in
  dynamicNeighbors:
    - prefix: 10.0.0.0/24
      peerAsn: 65000
      acceptPrefixes: ["10.0.0.0/8"]
      exportPrefixes: ["10.100.0.0/16"]

Dynamic sessions are ephemeral: their status appears under BGPPeerStatus with an id prefixed dyn- (or dyn-<vrf>- for non-default VRFs). They do not survive a pilotd restart unless the peer re-dials.

Operating

S="sihactl --sihaconfig ~/.siha/config --addr <node-ip>:6443"

# BGP global state (one row per VRF).
$S get bgpconfig

# Peer list: vrf, neighbor, asn, announced/accepted/exported prefixes, session
# state, last error.
$S get bgppeer

# Full session detail for one peer.
$S get bgppeer spine1 -o yaml

# Learned routes in the FIB (prefix, next-hops, peer, vrf, conflict).
$S get bgplearnedroute

# Watch route convergence live.
$S watch bgplearnedroute

The state column in sihactl get bgppeer reflects the GoBGP session FSM: ESTABLISHED means the session is up and prefixes are flowing; ACTIVE means GoBGP is dialing but has not yet connected; other values (IDLE, OPENSENT, …) indicate transient states or held-down sessions.

BGPLearnedRoute.conflict, when non-empty, explains why a learned route was not installed in VPP — for example, a static Route resource already owns the prefix (static wins over BGP). A non-empty conflict is not an error; it is the expected outcome whenever a locally-configured route covers a BGP-learned one.

BGPConfigStatus (read-only, same id as the config) reports running, establishedSessions, totalSessions, and lastError. Inspect it with:

$S get bgpconfigstatus default -o yaml

v1 limitations

  • Non-overlapping neighbor IPs across VRFs. GoBGP v1 keys sessions by IP address globally. Two BGPPeer resources (in different VRFs) with the same neighborAddress collide. This is rejected at apply time. Per-netns isolation (allowing overlapping transport IPs per VRF) is v2 and is gated on a live spike.
  • Single router-id. The OPEN message uses the global router-id declared on the default BGPConfig. Per-VRF router-ids are v2.
  • IPv4 unicast only. All addresses, prefixes, and next-hops must be IPv4. IPv6 is v2; VPNv4 / inter-VRF route leaking is v3.
  • VRF announce is experimental. Originating prefixes into a non-default VRF (announce on a VRF-bound BGPPeer) is code-complete but has no shipped fixture that exercises it end-to-end. Treat it as experimental until a live env validates it.