Existing reverse proxies hold backend addresses in a config file and reload when something changes. That model works naturally when backend IPs stay stable.

A container orchestration environment is different. Kubernetes Pods rotate IPs, and autoscaling adds and removes instances on the order of minutes. Rewriting the config file and reloading on every change makes propagation slow and breaks connections during the reload.

Envoy was designed with that environment in mind. It receives configuration over an API, applies changes without a hot reload, and exposes L7 traffic in fine detail.

The Static Reverse Proxy

A static reverse proxy hits four limits in a cloud-native environment.

  1. Manual reload — config changes require a SIGHUP-style reload. High change frequency raises operational cost, and the reload itself can briefly interrupt connections.
  2. Static upstreams — backend addresses live in the config file. Representing a dynamic Kubernetes Service requires an external tool to rewrite the file.
  3. Weak L7 observability — per-request metrics, distributed tracing, and detailed access logs are not present by default. Extra modules or external tools fill the gap.
  4. Limited L7 routing expressiveness — host- and path-based routing is straightforward, but header- or metadata-based routing and policies like retry, timeout, and circuit breaking are hard to express flexibly.

xDS API

Envoy takes its configuration over a family of gRPC (HTTP/2-based RPC framework) APIs called xDS (Discovery Service). They deliver listeners, clusters, routes, endpoints, and secrets dynamically.

Each kind splits into its own API.

  • LDS (Listener Discovery Service) — which ports Envoy listens on and which protocols they speak
  • CDS (Cluster Discovery Service) — the logical backend clusters
  • EDS (Endpoint Discovery Service) — the actual endpoints (IP/port) for each cluster
  • RDS (Route Discovery Service) — host/path → cluster routing rules
  • SDS (Secret Discovery Service) — secrets like TLS certificates
sequenceDiagram
    participant CP as Control Plane
    participant Envoy as Envoy (data plane)
    participant Backend
    Envoy->>CP: gRPC stream subscribe (LDS/CDS/RDS/EDS)
    CP-->>Envoy: Listener / Cluster / Route / Endpoint
    Note over Envoy: Applied without a hot reload
    Backend->>CP: Pod added / removed
    CP-->>Envoy: EDS push (endpoint change)

The control plane pushes changes to Envoy, and Envoy applies them without a hot reload. One mechanism resolves both manual reload and static upstreams at once.

The control plane is not tied to Envoy itself. Istio’s Istiod, AWS App Mesh, Contour, or a custom control plane built with go-control-plane all qualify, as long as they speak xDS.

Filter Chain

Request processing in Envoy is expressed as a filter chain. When a listener accepts a connection, the network filter chain runs; for HTTP requests, an HTTP filter chain follows.

flowchart LR
    Conn[Incoming connection] --> L[Listener]
    L --> NF[Network Filter Chain]
    NF --> HCM[HTTP Connection Manager]
    HCM --> HF[HTTP Filter Chain]
    HF --> R[Router]
    R --> C[Upstream Cluster]

The HTTP filter chain holds routing, authentication, rate limiting, retry, timeout, circuit breaking, and header transformation — each policy as a filter. Composing policies this way is more expressive than nginx’s directive-based config. Routing complexity gets absorbed into the filter chain.

Filters can be C++ built-ins or WebAssembly (Wasm) modules loaded dynamically. Custom policies can be injected without rebuilding Envoy.

Observability

Envoy includes stats, distributed tracing, and access logs in its default behavior.

Stats are exposed in a Prometheus-compatible format. Request counts, latency, and response code distributions get collected per cluster, route, and listener. Tracing integrates with OpenTelemetry, Zipkin, and Jaeger. Access logs ship as structured formats (JSON, etc.) to stdout or a file.

None of the three require extra modules — configuration alone turns them on. Observability is part of the default behavior.

Control Plane and Configuration Complexity

For Envoy to behave dynamically, a control plane has to exist. It can run as a standalone reverse proxy, but xDS only delivers its value alongside a control plane. Operationally the control plane becomes its own system to maintain.

Configuration is also more involved than nginx. Listeners, clusters, routes, and endpoints are separate objects, and the order of filters in a chain affects behavior. For purely static traffic, Envoy is heavier than needed.

Service Mesh Data Plane

Envoy is widely adopted as the data plane of a service mesh. Istio, Consul Connect, Kuma, and AWS App Mesh use Envoy as the data plane and build their own control planes on top.

flowchart TB
    subgraph CP[Control Plane]
        Istiod[Istiod / Consul / App Mesh]
    end
    subgraph DP[Data Plane]
        E1[Envoy sidecar]
        E2[Envoy sidecar]
        E3[Envoy sidecar]
    end
    Istiod -.xDS.-> E1
    Istiod -.xDS.-> E2
    Istiod -.xDS.-> E3
    E1 <--> E2
    E2 <--> E3

An Envoy sidecar runs alongside each Pod, and the control plane manages the configuration for every sidecar. Policies like mTLS (mutual TLS), retry, and traffic shifting apply without changes to application code.

The same Envoy can also run as a standalone ingress or API gateway. Contour, Emissary-ingress, and Gloo are examples.

Closing

A static reverse proxy works well when backends stay stable and routing stays simple. Once backends rotate frequently and L7 policies grow detailed in a cloud-native setup, the static model no longer suffices.

Envoy implements that dynamic model through xDS and filter chains. The choice between models tracks how dynamic the traffic actually is.

References