A frontend, an API and a blog usually live behind one domain; give each its own port or subdomain and you deal with CORS, a TLS certificate per endpoint, and every service doing its own rate limiting, logging and gzip. This post builds an Nginx reverse proxy (1.29.8, in Docker Compose) that routes by path to three services, and runs every request to show the real results. Afterwards you’ll avoid the four usual mistakes: a location that matches too much, a slash in proxy_pass that rewrites the path, spoofed X-Forwarded-* headers, and defaults that don’t suit production.
Worth reading first: Advanced REST API design (the section on URL conventions behind an API gateway).
1. Reverse proxy vs forward proxy, load balancer and API gateway
Key idea: a reverse proxy sits on the server side and takes requests on behalf of the services inside. Load balancers and API gateways are reverse proxies specialised in one more job: spreading load, or API-aware logic.
- A forward proxy sits on the client side (a corporate proxy, a VPN); the destination sees the proxy’s IP.
- A reverse proxy sits on the server side; clients don’t know how many services are behind it.
- A load balancer: a reverse proxy spreading load across copies of the same service. It can work at layer 4 (TCP, doesn’t read HTTP) or layer 7 (reads HTTP).
- An API gateway: a layer-7 reverse proxy plus API-aware logic: token validation, per-client rate limits, quotas, request transformation, response aggregation, API docs. Think Kong, APISIX, Spring Cloud Gateway, AWS API Gateway.
Nginx can play all three server-side roles; here it routes by path.
- Checking a JWT correctly (pinning the algorithm,
exp,iss,aud), whether at the gateway or in the service: Authentication: session cookies and JWT. - Nginx’s per-IP
limit_reqis only one layer of defence against application-layer DDoS; combining it with per-user or per-API-key limits: Web attacks and the OWASP Top 10.
2. What we’re building
Key idea: one Nginx in front of three services: /api/ goes to the backend, /blog/ and /content/ to the blog, everything else to the frontend.
The course exercise draws this layout; redrawn:
flowchart LR
C[Client] --> P[Nginx :80]
P -- "/ (e.g. /home)" --> FE[Frontend :8000]
P -- "/api/... (e.g. /api/orders)" --> BE[Backend :8080]
P -- "/blog/... and /content/..." --> BL[Blog :9000]
| Path the client calls | Goes to | Path the backend receives |
|---|---|---|
/, /home |
frontend | unchanged |
/api/orders/145 |
backend | unchanged, /api/orders/145 |
/blog/posts/1 |
blog | prefix stripped: /posts/1 |
/content/hot |
blog | unchanged, /content/hot |
The blog gets two treatments on purpose (prefix kept, prefix stripped) so you can see how proxy_pass handles paths.
3. The Docker Compose lab
Key idea: the three services behind the proxy are fake Nginx containers that return JSON with the path and headers they received. One look at the response tells you what the proxy did.
gateway.conf (the main proxy):
upstream backend {
server be:8080;
keepalive 32;
}
server {
listen 80;
server_name _;
client_max_body_size 10m;
# Common headers for every proxied location
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Request-Id $request_id;
proxy_connect_timeout 2s;
proxy_read_timeout 30s;
location = /healthz {
access_log off;
return 200 "ok\n";
}
# /api/... -> backend, path kept as is (no URI part in proxy_pass)
location /api/ {
proxy_pass http://backend;
}
# /blog/... -> blog, prefix stripped (URI part "/" in proxy_pass)
location /blog/ {
proxy_pass http://blog:9000/;
}
# /content/... -> blog, path kept
location /content/ {
proxy_pass http://blog:9000;
}
# everything else -> frontend
location / {
proxy_pass http://fe:8000;
}
}
The rest of the lab: echo.conf, compose.yaml, commands and a Docker Desktop note
echo.conf (for fe, be and blog):
server {
listen 8000;
listen 8080;
listen 9000;
default_type application/json;
location / {
return 200 '{"service":"$hostname","port":"$server_port","uri":"$request_uri","host":"$http_host","xff":"$http_x_forwarded_for","real_ip":"$http_x_real_ip","proto":"$http_x_forwarded_proto"}\n';
}
}
compose.yaml:
services:
gateway:
image: nginx:1.29-alpine
ports: ["8088:80"]
volumes: ["./gateway.conf:/etc/nginx/conf.d/default.conf:ro"]
depends_on: [fe, be, blog]
fe:
image: nginx:1.29-alpine
hostname: fe
volumes: ["./echo.conf:/etc/nginx/conf.d/default.conf:ro"]
be:
image: nginx:1.29-alpine
hostname: be
volumes: ["./echo.conf:/etc/nginx/conf.d/default.conf:ro"]
blog:
image: nginx:1.29-alpine
hostname: blog
volumes: ["./echo.conf:/etc/nginx/conf.d/default.conf:ro"]
Start it and test the config:
docker compose up -d
docker compose exec gateway nginx -t
# nginx: configuration file /etc/nginx/nginx.conf test is successful
curl -s -H 'X-Forwarded-For: 6.6.6.6' http://localhost:8088/api/orders/145
On Docker Desktop for Mac, the original exercise’s network_mode: host doesn’t behave as it does on Linux (the “host” is Docker’s VM, not your Mac). Using Compose’s default network and calling services by name, as above, works everywhere.
What actually came back (trimmed):
| Request | Service | uri the backend got |
Note |
|---|---|---|---|
/ |
fe | / |
|
/home |
fe | /home |
|
/api/orders/145 |
be | /api/orders/145 |
prefix kept |
/apiv2 |
fe | /apiv2 |
doesn’t match /api/, falls through to / |
/blog/posts/1 |
blog | /posts/1 |
prefix stripped |
/blog |
(proxy) | 301 → /blog/ |
see section 5 |
/content/hot |
blog | /content/hot |
|
/healthz |
(proxy) | ok |
answered by the proxy |
The headers the backend saw for /api/orders/145: host: localhost, xff: 6.6.6.6, 172.22.0.1, real_ip: 172.22.0.1, proto: http. The xff line shows the proxy appends the address it sees to whatever the client sent; it doesn’t replace it (section 6).
4. Which location wins
Key idea: the longest matching prefix wins, and file order doesn’t matter. But an exact = match stops at once, and a regex (checked in file order) beats the prefix unless the prefix has ^~.
Nginx first normalises the URI (decodes %XX, resolves . and .., merges // into /), then picks:
- Check every prefix location and remember the one with the longest matching prefix.
- If a
location = /pathmatches exactly, stop. - If the longest prefix has
^~, stop and skip regexes. - Check regex locations (
~case-sensitive,~*case-insensitive) in file order; the first match wins. - If no regex matches, use the prefix remembered in step 1.
| Declaration | Meaning |
|---|---|
location = /healthz |
exactly /healthz |
location ^~ /static/ |
prefix that beats regexes |
location ~* \.(png|jpg)$ |
regex, checked in order |
location /api/ |
ordinary prefix |
location / |
catch-all |
5. proxy_pass with or without a URI
Key idea: a proxy_pass without a URI (just http://host:port) sends the path exactly as the client sent it. With a URI (even just /), the part of the path that matched the location is replaced by that URI.
This is the most common mistake. Real results from a test server:
| location | proxy_pass | Request | Backend got |
|---|---|---|---|
/api/ |
http://backend |
/api/orders |
/api/orders |
/blog/ |
http://blog:9000/ |
/blog/posts/1 |
/posts/1 |
/v1/ |
http://be:8080/internal/ |
/v1/users |
/internal/users |
/api |
http://be:8080/ |
/api/orders |
//orders |
/api |
http://be:8080/ |
/apiv2 |
/v2 |
~ ^/img/(.*)$ |
http://fe:8000/static/$1 |
/img/a.png |
/static/a.png |
The two /api rows are the classic bug: /api is replaced by /, so the remaining /orders becomes //orders, and /apiv2 becomes /v2 (a different API altogether).
The request /blog (no trailing slash) gets a 301 to /blog/, and the Location loses port 8088. The fix is in the block below.
Automatic redirects: why /blog gets a 301 without the port, absolute_redirect and proxy_redirect
The exact result: 301 Location: http://localhost/blog/. Nginx does this on its own: when a prefix location ends in / and is handled by proxy_pass, a request equal to that string minus the slash is redirected to the version with the slash. The port goes missing because Nginx builds absolute URLs from the server name and the port it listens on (80), and knows nothing about the port mapping outside.
6. X-Forwarded headers and whom to trust
Key idea: through a proxy, the backend loses three facts: the client IP, the domain the client called (Host), and HTTP vs HTTPS. Nginx carries them in headers, but clients can send headers themselves, so only trust the parts your own proxies added.
| Header | Value Nginx sets | Used for |
|---|---|---|
Host |
$host |
the original domain (virtual hosts, building URLs) |
X-Real-IP |
$remote_addr |
the IP that connected directly to the proxy |
X-Forwarded-For |
$proxy_add_x_forwarded_for |
IPs along each hop (each proxy on the way): previous value + , + $remote_addr |
X-Forwarded-Proto |
$scheme |
https when TLS ends at the proxy |
X-Request-Id |
$request_id |
a random id that links proxy logs to backend logs |
- If you set nothing, Nginx sends
Host: $proxy_host(the name inproxy_pass) and noX-Forwarded-*; tested: the backend gothost: be:8080and an emptyxff. - The newer standard is
Forwarded(RFC 7239:Forwarded: for=203.0.113.7;proto=https;host=shop.vn), butX-Forwarded-*is far more common.
How far to trust X-Forwarded-For
The lab sent X-Forwarded-For: 6.6.6.6 and the backend got 6.6.6.6, 172.22.0.1. If the backend takes the first entry as the client IP (very common code), anyone can fake their IP: dodge rate limits, fake their location, pollute the logs.
The right way: walk from right to left, skip the IPs that belong to your own proxies, and the first IP that isn’t yours is the client. Nginx’s realip module does exactly this.
The realip module, tested: config and results
set_real_ip_from 172.16.0.0/12; # only trust proxies in this range
real_ip_header X-Forwarded-For;
real_ip_recursive on; # skip every trusted IP from the right
| Connection from | X-Forwarded-For received |
$remote_addr after realip |
|---|---|---|
| 127.0.0.1 (untrusted) | 6.6.6.6 |
127.0.0.1 (header ignored) |
| 172.22.0.2 (trusted) | 1.2.3.4, 6.6.6.6 |
6.6.6.6 (1.2.3.4 ignored, it’s left of the first untrusted IP) |
On the Spring Boot side
The backend has to read these headers for request.getRemoteAddr(), request.getScheme(), ServletUriComponentsBuilder and redirects to come out right:
# Tomcat RemoteIpValve reads X-Forwarded-For / -Proto / -Host / -Port
server.forward-headers-strategy=native
# Only trust your real proxy/LB IPs (a regex); the default is the private ranges
server.tomcat.remoteip.internal-proxies=10\\.0\\.\\d{1,3}\\.\\d{1,3}
- The value
frameworkuses Spring’sForwardedHeaderFilterinstead of Tomcat’s valve, which suits servers other than Tomcat. - With nothing configured (the default is
noneunless Spring Boot detects a cloud platform), an app behind TLS termination (TLS decrypted at the proxy) generateshttp://links and wrong OAuth2 redirect URIs, andSecurecookies aren’t set.
7. Production settings
Key idea: Nginx’s defaults suit static websites, not necessarily APIs. Six things to revisit: timeouts, keepalive, buffering, retries, body size and upstream DNS. Logs should carry $request_id and $upstream_response_time (details in the WebSocket block below).
| Setting | Default | Problem | What to do |
|---|---|---|---|
| Timeouts | connect, read and send all 60s | read is measured between two reads, not over the whole response: a backend trickling one byte every 50 seconds keeps the connection forever | connect 1–2s, read per the API’s SLA, slow endpoints in their own location |
| Upstream keepalive | none | a new TCP connection per request: tens of thousands of TIME_WAIT sockets, ports run out |
keepalive N; + proxy_http_version 1.1; + proxy_set_header Connection ""; |
| Buffering | proxy_buffering on |
breaks Server-Sent Events and streaming: the client gets big chunks | proxy_buffering off; or the backend sends X-Accel-Buffering: no |
| Retries | proxy_next_upstream error timeout; |
since 1.9.13, no retry of POST, PATCH, LOCK once sent, unless you add non_idempotent |
keep the default, cap tries and time |
| Body | client_max_body_size 1m |
a 3 MB upload gets 413 from Nginx, backend logs are empty |
set it per location |
| Upstream DNS | resolved once, when the config is read | 502 after a container is recreated with a new IP | see the details block below |
Each default in detail: timeouts, keepalive, buffering, upstream DNS
- Timeouts: the three are
proxy_connect_timeout,proxy_read_timeoutandproxy_send_timeout. Keep connect short because inside a data centre a connection takes a few milliseconds. Intentionally slow endpoints (exports, reports) are better moved to async jobs and Idempotency-Key. - Keepalive: the
TIME_WAITpile-up shows at a few thousand requests per second. Before 1.29.7 Nginx spoke HTTP/1.0 to upstreams by default (connection closed after each request); since 1.29.7 the default is 1.1, but writing it out still helps whoever reads the config and whatever machine runs an older build. WhyTIME_WAITsits on the side that closes first and only exhausts ports on the calling side is in Networking for backend engineers. - Buffering: Nginx reads the backend’s response quickly into buffers (spilling to temp files if needed) and feeds it to the client at the client’s pace, freeing the backend early. Great for normal APIs; turn it off only on streaming locations.
- Body: the upload gets
413 Request Entity Too Large; give uploads a dedicated location with a larger limit.client_max_body_size(default1m) must be at least Spring’sspring.servlet.multipart.max-file-size/max-request-size, or Nginx answers413before the request reaches Spring. Spring’s limits and storing files safely are in file upload in Spring MVC. - DNS: open-source Nginx resolves host names in
proxy_pass/upstreamwhen it reads the config, and won’t start if a name doesn’t resolve (host not found in upstream).
WebSocket, the shared-header include trap, and logs for observability
WebSocket needs the upgrade headers forwarded and a read timeout longer than the ping interval:
location /ws/ {
proxy_pass http://backend;
# this location has its own proxy_set_header, so repeat the common ones
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 1h;
}
You can’t include the common header file here if it contains Connection "": both Connection lines would be sent. I also tried including a file with proxy_http_version into a location that sets proxy_http_version itself: Nginx refused to start with "proxy_http_version" directive is duplicate. Headers don’t get a duplicate error like that, which is all the more reason to watch them.
Observability. Log JSON with $request_id, $upstream_addr, $upstream_status, $request_time (total time) and $upstream_response_time (backend time). The gap between the last two tells you whether the slowness is on the network to the client or in the backend. Pass X-Request-Id (or better, traceparent) to the backend to link it with application logs; see Distributed tracing with OpenTelemetry.
Exercises
Exercise 1: Route by service name, as in the course’s URL exercise
Task (paraphrased): a gateway sits in front of user-service and order-service. Public URLs look like /user-service/v1/users/34 and /order-service/v1/orders/145; each service only knows /v1/.... Write the Nginx config.
Solution to Exercise 1: a gateway that strips the prefix per service
snippets/proxy-headers.conf, shared:
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
The gateway config:
upstream user_service { server user-service:8080; keepalive 16; }
upstream order_service { server order-service:8080; keepalive 16; }
server {
listen 80;
absolute_redirect off;
proxy_http_version 1.1;
include /etc/nginx/snippets/proxy-headers.conf;
location /user-service/ {
proxy_pass http://user_service/; # strip: /user-service/v1/users/34 -> /v1/users/34
proxy_set_header X-Forwarded-Prefix /user-service;
include /etc/nginx/snippets/proxy-headers.conf; # because this location sets its own header
}
location /order-service/ {
proxy_pass http://order_service/;
proxy_set_header X-Forwarded-Prefix /order-service;
include /etc/nginx/snippets/proxy-headers.conf;
}
location / { return 404; }
}
Test run (nginx -t passes, then real calls): /user-service/v1/users/34 reaches user-service as /v1/users/34 with X-Forwarded-Prefix: /user-service; /order-service/v1/orders/145 reaches order-service as /v1/orders/145; /other gets 404.
Why it works:
- Both the location and the
proxy_passURI end in/, so the prefix is stripped cleanly. - Each location sets its own header (
X-Forwarded-Prefix), so the common headers have to beincluded again there (the inheritance rule from section 6). location / { return 404; }rejects any path that belongs to no service instead of quietly routing it somewhere.- A senior would also question whether public URLs should expose service names at all (see the trade-off in Advanced REST API design).
Exercise 2: http:// links behind TLS
Task: TLS terminates at Nginx; Spring Boot behind it speaks HTTP. After logging in, users are redirected to http://shop.vn/home and the browser shows a warning. Find the cause and fix it.
Solution to Exercise 2: X-Forwarded-Proto and forward-headers-strategy
The cause: Spring builds the redirect URL from request.getScheme(), which is http because the connection from Nginx is plain HTTP.
- Fix both ends: Nginx sends
proxy_set_header X-Forwarded-Proto $scheme;(andHost $host); Spring Boot enablesserver.forward-headers-strategy=nativewithinternal-proxiesmatching Nginx’s IP. - To verify, call an endpoint that returns
request.getScheme()andgetServerName()through the proxy; you should seehttpsandshop.vn. - Add HSTS at Nginx (
add_header Strict-Transport-Security "max-age=31536000" always;) so browsers never tryhttp://again.
Exercise 3: Read the config, predict the result
Given:
location /static/ { root /var/www; }
location ~* \.(png|jpg)$ { root /var/cdn; }
location /api { proxy_pass http://be:8080/; }
Where do GET /static/logo.png and GET /api/v2/products go?
Solution to Exercise 3: regex beats prefix, and the trailing-slash bug
I ran it, and it behaves exactly as below.
/static/logo.pngmatches the prefix/static/, but the regex\.png$matches too and the regex wins, so the file is read from/var/cdn/static/logo.png. Fix it withlocation ^~ /static/./api/v2/productsmatcheslocation /api,/apiis replaced by/, and the backend receives//v2/products. Fix it withlocation /api/(keepproxy_pass http://be:8080/to strip the prefix, or drop its trailing slash to keep it).
Remember
- Can you state the order in which Nginx picks a
location, and why a.pngregex can steal requests fromlocation /api/? - Can you predict the path the backend gets for any location /
proxy_passpair, including//ordersand/apiv2becoming/v2? - Can you explain why the first
X-Forwarded-Forentry can’t be trusted, and configure it correctly in both Nginx and Spring Boot? - Do you know the
proxy_set_headerinheritance trap and howincludeavoids it? - Can you list the defaults to change for production: timeouts, keepalive, buffering, retries, body size, upstream DNS?
Interview questions
Junior
How is a reverse proxy different from a forward proxy?
A forward proxy sits on the client side and goes out on the client’s behalf; a reverse proxy sits on the server side and takes requests in on behalf of the services.
With a forward proxy (a corporate proxy), the destination only sees the proxy. With a reverse proxy, clients only see one address. Reverse proxies terminate TLS, route by path or host, balance load, compress, cache, rate-limit and hide the internal layout.
Why does a backend behind a proxy need X-Forwarded-For and X-Forwarded-Proto?
Because the connection to the backend comes from the proxy: the source IP is the proxy’s, and the scheme is HTTP if TLS ended at the proxy.
Without these headers, logs and rate limits see only the proxy’s IP, and links and redirects the app generates use the wrong http://. X-Forwarded-For carries the chain of IPs, X-Forwarded-Proto the scheme the client used.
Mid
What’s the difference between location /api/ { proxy_pass http://be:8080; } and proxy_pass http://be:8080/;?
Without a URI the path is unchanged (/api/orders); with the URI / the part matching the location is replaced, so the backend gets /orders.
They have to go together: if the location ends in a slash, so must the URI, or you get //orders, or /apiv2 turning into /v2.
In what order does Nginx choose a location?
It remembers the longest matching prefix, stops on an exact = match or if that prefix has ^~, then the first regex matching in file order wins; if none matches, it uses the remembered prefix.
The consequence: a regex catching .png can steal requests from location /api/.
What does upstream keepalive need, and why bother?
keepalive N in the upstream, proxy_http_version 1.1 and proxy_set_header Connection "", so connections are reused instead of opened per request.
Connection "" stops Nginx from sending Connection: close. You skip the TCP (and TLS, if any) handshake on every request, which cuts latency and avoids tens of thousands of TIME_WAIT sockets exhausting ports under load.
You put SSE behind Nginx and clients receive events in big bursts. Why?
Because proxy_buffering is on by default, so Nginx collects the response in buffers before sending it.
Turn it off for that location (proxy_buffering off;) or have the backend send X-Accel-Buffering: no. Also raise proxy_read_timeout above the heartbeat interval so the connection isn’t cut after 60 seconds.
Senior
How do you get the real client IP through several layers of proxies?
Read X-Forwarded-For from right to left, skip the IPs that belong to your proxies; the first untrusted one is the client.
- Never take the first entry, because the client controls it. The outermost layer should overwrite the header with
$remote_addr. - Nginx:
set_real_ip_fromwithreal_ip_recursive on. Spring Boot:forward-headers-strategy=nativeandinternal-proxiesset to the load balancer’s range. - Behind an L4 load balancer there are no HTTP headers, so use the PROXY protocol.
Should the proxy retry requests when a backend fails?
Connection errors before the request was sent are safe to retry; timeouts after sending are only safe for idempotent requests.
- Nginx doesn’t retry POST/PATCH by default because the backend may already have processed them.
- Even for GET, cap the number of tries and the total retry time: when the whole cluster is overloaded, retries multiply the load and spread the outage.
- Retries should live in one layer with backoff, not in the client, the gateway and the service all at once.
After redeploying a backend in Docker, Nginx returns 502 until it’s reloaded. Explain and fix.
Open-source Nginx resolves upstream host names once at start or reload and keeps that IP; the new container has a different IP, so connections to the old one fail.
Fix: use a name with a stable IP (a Kubernetes Service), or a resolver plus a variable in proxy_pass so it re-resolves per TTL, or the resolve parameter in upstream (since 1.27.3). If you keep the static approach, add a reload to the deploy process.
When would you use an API gateway instead of Nginx?
When you need API-aware logic that’s awkward in static Nginx config; if all you need is one entry point with TLS, path routing and per-IP rate limits, Nginx is simpler.
API-aware logic means JWT or API-key auth for partners, per-client quotas, header-based canaries, frequent route changes without reloads, many teams editing routes. Nginx has fewer moving parts. Either way, keep business logic out of the gateway.
Useful phrases for interviews
- “The reverse proxy is our single entry point: it terminates TLS and routes by path prefix.”
- “If the location ends with a slash, the proxy_pass URI should too; otherwise the path gets mangled.”
- “We strip the prefix at the proxy and pass X-Forwarded-Prefix so the service can still build correct links.”
- “We only trust forwarded headers from our own load balancer’s address range.”
- “The client IP is the first untrusted address when you read X-Forwarded-For from the right.”
- “We turned on upstream keepalive, which removed thousands of TIME_WAIT sockets.”
- “Retries at the proxy are limited to idempotent requests, and they’re capped so they can’t amplify an outage.”
- “The gap between request time and upstream response time tells us whether the backend or the network is slow.”
References
- nginx: ngx_http_proxy_module (
proxy_pass,proxy_set_header, timeouts, buffering,proxy_next_upstream) - nginx: ngx_http_core_module,
location - nginx: ngx_http_realip_module
- nginx: ngx_http_upstream_module (
keepalive,resolve) - RFC 7239: Forwarded HTTP Extension
Notes based on the Advanced Backend course (Ronin Engineer), rewritten in my own words.