Load balancers, proxies & CDNs

One server can only do so much, and it will break one day. Real sites run several copies of their app and put a load balancer in front: it spreads the visitors out and quietly stops sending them to a copy that's broken. Further out, a CDN keeps copies of the site's files all over the world, close to the visitors. This lesson puts both in front of the club's website.

Spreading the load

                  ┌─▶ web1 (127.0.0.1:8081)
visitors ─▶ HAProxy :80 ─▶ web2 (127.0.0.1:8082)
                  └─▶ web3 (127.0.0.1:8083)

Visitors only ever talk to the load balancer. It picks a server for each request (or connection), using one of a few algorithms:

Health checks

A load balancer keeps knocking on every server (every 2 seconds by default in HAProxy). When a server stops answering, it's marked DOWN and gets no more visitors until it's healthy again. That's how you can restart or update one server at a time without anyone noticing. Forget the check keyword and HAProxy never notices a dead server: some visitors get errors.

Layer 4 vs layer 7

• Layer 4 (TCP) balancers just pass connections through. They're fast and work for anything (databases, mail…), but can't look inside.
• Layer 7 (HTTP) balancers read each request: they can send /api/ to one group of servers and /images/ to another, add headers, and handle HTTPS themselves (TLS termination: the certificate lives on the balancer).

Because the app now only sees connections from the balancer, it loses the visitor's real address. Layer 7 balancers add it to a header, X-Forwarded-For (HAProxy: option forwardfor), so logs and rate limits still work.

HAProxy on each family

HAProxy's config is the same everywhere. Installing and starting it is where the families differ:

A minimal config has a frontend (where visitors arrive) and a backend (the servers):

frontend club
    bind *:80
    default_backend club_apps

backend club_apps
    balance roundrobin
    server web1 127.0.0.1:8081 check
    server web2 127.0.0.1:8082 check

CDNs: copies close to the visitors

A CDN (content delivery network, like Cloudflare, Fastly or CloudFront) is a huge network of caching reverse proxies in cities all over the world. DNS sends each visitor to a nearby edge server. If the edge has a fresh copy of the file, it answers straight away (HIT). If not, it fetches it from your server, the origin, keeps a copy, and answers (MISS).

• Cache-Control: public, max-age=86400 (sent by your origin) says "anyone may keep this for a day".
• Age: 120 says the cached copy is two minutes old.
• Changed a file? Either purge it from the CDN, or give it a new name (logo.v2.png, ?v=2). That second trick is called cache busting.
• Pages that are different for every user (anything with a login) must not be cached: Cache-Control: private or no-store.

Try it: put a load balancer in front 🎛️

The club's web app now runs three copies on this server (web1, web2, web3, ports 8081–8083), but nothing spreads the visitors over them. A colleague left a starter config in ~/club-lb.cfg. Set up HAProxy, prove it balances, then break a server on purpose.

Quick check