Network layers and load balancers

Key points:

Use L4 for non-HTTP protocols, fixed IPs and raw speed; use L7 to route, retry, rate-limit or authenticate by request.
L4 balances connections, not requests - put an L7 proxy in front of HTTP/2 and gRPC.
Where TLS terminates is a security decision: whatever terminates it can read the traffic.
Trust X-Forwarded-For and CF-Connecting-IP only from proxies you control.
Know the AWS limits: 50 listeners per NLB, 60 security group rules by default, idle timeouts of 60 s (ALB) and 350 s (NLB TCP).

The layers that matter

Of the OSI model’s seven layers, four matter in day-to-day infrastructure work:

Layer	Name	What it deals with	Examples
L2	Data link	MAC addresses on one network segment	ARP, switches
L3	Network	IP addresses and routing between networks	IP, routers, security groups (partly)
L4	Transport	Connections and ports	TCP, UDP
L7	Application	The actual request	HTTP, gRPC, DNS

TLS sits between L4 and L7, encrypting application data over TCP. “L4” or “L7” load balancer means how much of the traffic it can see.

L4 vs L7

Use L4 when: you need non-HTTP protocols, fixed IPs or raw speed. Use L7 when: you need to route, retry, rate-limit or authenticate by request.

	L4 load balancer	L7 load balancer
Sees	IPs and ports only	Full requests: host, path, headers, cookies
Balances	Connections	Requests
Can route by	Port	Host, path, header, method
TLS	Passes it through (or terminates it, without reading requests)	Terminates it to read requests
Can retry, rate limit, authenticate	No	Yes
Speed and protocols	Very fast; any TCP or UDP protocol	More work per request; HTTP-family protocols
Examples	AWS NLB, Azure Load Balancer, kube-proxy	AWS ALB, Envoy, nginx, HAProxy (HTTP mode), Cloudflare

A request through a typical stack

flowchart LR
    U[User] -->|"TLS leg 1"| CF["Cloudflare<br/>L7: WAF, cache"]
    CF -->|"TLS leg 2"| NLB["AWS NLB<br/>L4: forwards TCP"]
    NLB --> EG["Envoy Gateway<br/>L7: terminates TLS,<br/>routes, rate-limits"]
    EG --> POD[Pod]

Cloudflare terminates the user’s TLS and opens its own connection - two TLS legs.
The NLB never decrypts; it forwards TCP to Envoy.
Envoy terminates TLS, applies policies and picks a pod.

Where TLS terminates

Treat this as a security decision: anything that terminates TLS can read the traffic.

Mode	How	Trade-off
Termination	Decrypts and forwards plain traffic	Simplest; the internal hop is unencrypted
Re-encryption	Decrypts, inspects, opens new TLS to the backend	Encrypted end to end, but the balancer still sees everything
Passthrough	Routes on the SNI (hostname sent in plain text at the start of the TLS handshake) without decrypting	Only the backend holds the key

Passthrough is used in migrating 50+ TCP services to Envoy Gateway.

Keeping the client’s IP

Every proxy opens a new connection, so the backend sees the proxy’s IP, not the user’s.

At L7, the proxy adds X-Forwarded-For or Cloudflare’s CF-Connecting-IP. Trust them only from proxies you control - anyone can send them.
At L4, the balancer prepends a PROXY protocol header to the connection. Both sides must agree, or every connection fails.
In Kubernetes, set externalTrafficPolicy: Local on the Service to avoid the extra node hop that hides the client IP.

Gotchas

L4 balances connections, not requests. HTTP/2 and gRPC keep one connection open, so all requests hit one backend. Put an L7 proxy in front.
NLB listener limits. An AWS NLB allows 50 listeners, so fifty-one TCP services means sharding across load balancers. See running 50+ TCP services on EKS.
Security group rule quotas fail silently. The AWS Load Balancer Controller adds a rule per port; past the default 60 per group, updates fail with no error in the logs. Watch the rule count as you add ports.
Idle timeouts kill long-lived connections. An ALB closes idle connections after 60 seconds by default, an NLB after 350 seconds for TCP. Use TCP keepalives or raise the timeout.
ALB IP addresses change. ALB IPs are not fixed. Never put them in a firewall rule or DNS A record; an NLB can have fixed Elastic IPs.
Every hop has its own header size limit. Large auth cookies can pass Cloudflare and Envoy and fail at nginx behind them - the smallest limit wins. Check limits on every hop.
Health checks at the wrong layer. An L4 check passes while the port accepts connections, even if every request returns 500. Check an HTTP endpoint when you can.
Cross-zone traffic costs money. Cross-zone balancing evens load but can add data transfer charges; it is off by default on NLBs, on for ALBs. Decide per load balancer.
DNS caching delays cutovers. Clients keep the old address for hours on a high TTL. Lower the TTL days before moving a load balancer.