Scaling a web app to 100k users is a database discipline problem, not a distributed systems one. Fix N+1 queries, indexes, pooling, and queues first.
A client once walked me through an architecture diagram for an app with 200 daily active users. Message queue, three microservices, a read replica, a Redis cluster, and a Kubernetes setup that needed a full-time person just to stay green. The whole thing served fewer requests in a day than my personal blog does, and they were proud of it. I spent the meeting quietly wondering who was going to be awake at 2am when the queue backed up.
Here is what almost nobody tells you when you're starting out: getting to 100k users is, in the overwhelming majority of cases, not a distributed-systems problem. It's a "stop doing dumb things in your database" problem. The complexity people adopt to prepare for scale is usually the exact thing that stops them from ever reaching it, because you cannot move fast when you cannot hold your own system in your head. This post is about the boring, cheap, reason-about-able path to six figures of users — the practical web app scaling strategy I actually reach for in production — and how to know when you have earned the right to get fancy.
Premature scaling isn't just wasted money. That's the part people underestimate. The real cost is cognitive.
Every layer you add is a layer you have to reason about when something breaks at 2am. A stale cache. A queue that's silently backed up. A service that's up but returning garbage because a config drifted three deploys ago. Each of these is a debugging session that simply wouldn't exist if the data lived in one Postgres instance you could open a shell into and query directly.
I've watched teams burn their entire runway building for a scale they never reached. Beautiful event-sourced, CQRS, multi-service platform. What they didn't have was users, because every feature took three weeks to ship across four repositories. A leaner competitor shipped the same feature in an afternoon on a monolith and a single database, and took the market while the "scalable" team was still writing integration tests for the message contracts between their services.
The premature-scaling tax has three parts:
The last one is the killer. An architecture you can't reason about is an architecture that will surprise you, and production surprises are the expensive kind. I would rather run a "boring" monolith I fully understand at 90% of theoretical efficiency than a clever distributed system I understand at 60%, because the 30% I lose on the clever one gets spent tenfold on incident calls. Simplicity is not a phase you grow out of; it's a competitive advantage you protect.
The most expensive engineering decisions I've seen were made on a hunch. "The database is slow, let's add caching." "We're getting a lot of traffic, let's add more servers." Nobody measured anything. They pattern-matched a conference talk to a gut feeling and started building.
Before you touch architecture, get three numbers:
You can get most of this for close to free. Put timing around your handlers, ship the numbers to any APM or distributed-tracing tool, and watch a real load pattern for a week. On a recent project we were convinced our API was CPU-bound and about to provision beefier instances. The trace showed 80% of request time was a single N+1 query pattern fetching related rows in a loop. The "scaling problem" was one JOIN we weren't doing. Fixed it in twenty minutes, and the CPU panic evaporated. We had been one meeting away from paying more money to make an unindexed loop run slightly faster.
The rule I hold to: you are not allowed to buy a solution until you can point at the number the solution moves. If you can't name the metric, you're not scaling, you're decorating.
If you're going to hit a performance wall before 100k users, it's the database. Not the language, not the framework, not the number of app servers. The database.
The good news is that the first wall is almost never "we've outgrown a single Postgres box." Modern single-node PostgreSQL on a mid-tier managed instance will comfortably handle tens of thousands of active users if — and this is the whole game — you're not abusing it. The wall you hit is almost always self-inflicted, and that's actually great news, because self-inflicted walls are the cheap ones to knock down.
EXPLAIN ANALYZE is your friend and it's free.SELECT * on wide tables,** dragging back columns you never render and blob fields you never look at, inflating memory and killing any chance of an index-only scan.Fix these before you even think about replicas or sharding. Here's the N+1 pattern and the fix, which is the highest-leverage change most apps can make:
-- The N+1 disaster: 1 query for posts, then 1 per post for its authorSELECT id, title, author_id FROM posts WHERE feed_id = 42 LIMIT 20;-- ...then, in a loop, 20 more times:SELECT name, avatar_url FROM users WHERE id = $1;-- The fix: one query, one round tripSELECT p.id, p.title, u.name AS author_name, u.avatar_urlFROM posts pJOIN users u ON u.id = p.author_idWHERE p.feed_id = 42ORDER BY p.created_at DESCLIMIT 20;
And when a query is slow, read the query plan before you touch anything. EXPLAIN ANALYZE tells you exactly what Postgres did, not what you hoped it did:
EXPLAIN ANALYZESELECT p.id, p.titleFROM posts pWHERE p.feed_id = 42ORDER BY p.created_at DESCLIMIT 20;-- Seq Scan on posts (cost=... rows=1000000) means you're scanning the whole table.-- Add: CREATE INDEX idx_posts_feed_created ON posts (feed_id, created_at DESC);-- Then it's an Index Scan, and the cliff disappears.
A quick word on indexing, because it's where the leverage is. A composite index on (feed_id, created_at DESC) doesn't just filter by feed_id fast — it also serves the ORDER BY without a separate sort step, so Postgres can walk the index and stop at 20 rows. Order your index columns to match your query's equality-then-range shape. And don't over-index: every index you add is write amplification on every insert and update, so index the queries you actually run hot, not every column someone might filter on someday.
The first legitimate database-scaling move, long before sharding, is a read replica for read-heavy workloads. Point your reporting and heavy read paths at the replica, keep writes on the primary. It's a small operational change and it buys you a lot of headroom. The one trap: replication lag. If a user writes to the primary and immediately reads from a lagging replica, they'll see stale data — so route "read your own write" paths back to the primary and reserve the replica for data that can tolerate a second or two of staleness.
The second move is connection pooling. Postgres connections are expensive — each one is a backend process with its own memory — so a serverless or high-concurrency app can exhaust them fast. Put PgBouncer (or your managed provider's pooler) in transaction mode in front of the database and you'll often "fix" a scaling problem without changing a single query. I've seen an app that fell over at a few hundred concurrent users run clean at several thousand purely by adding a pooler. Nothing else changed. That one still feels like cheating, and it's the first thing I check when someone tells me their database is "maxed out."
Sharding, multi-region writes, and the exotic stuff? That's a problem for a much later you, and you'll know when you're there because you'll have the metrics to prove it, not a hunch and a whiteboard.
Caching is the first optimization everyone reaches for and the one that quietly costs the most. Not in money, in correctness. There are, as the old joke goes, only two hard problems in computer science, and cache invalidation is at least one and a half of them.
The moment you cache, you have two copies of the truth, and the entire history of computer science says keeping two copies in sync is hard. Every cache is a bet that stale data is acceptable for some window. Sometimes that bet is fine. Sometimes it means a user updates their profile, sees the old version, and files a bug you can't reproduce because your cache expired by the time you went looking.
A few hard-won caching rules:
// A safe read-through cache: scoped key, short TTL, DB is source of truthFuture<UserStats> getUserStats(String userId) async { final key = 'stats:v1:$userId'; // scoped per user, versioned final cached = await cache.get(key); if (cached != null) return UserStats.fromJson(cached); final stats = await db.computeUserStats(userId); // the expensive part await cache.set(key, stats.toJson(), ttl: Duration(seconds: 60)); return stats;}Note the version prefix (v1). When your data shape changes, bump it and every old key ages out on its own. No manual purge, no half-migrated cache serving broken JSON to whoever gets the cached copy. Small trick, saves real pain during deploys.
One more failure mode worth naming: the cache stampede (also called the thundering herd). When a hot key expires, every request that was being served from it hits the database in the same instant, and a cold cache after a deploy or a Redis restart can bury an otherwise-healthy database under a synchronized spike. Mitigations range from jittered TTLs to a single-flight lock that lets one request recompute while the others wait. But the real fix is upstream: your database should be fast enough that the cache is an optimization, not a crutch. If your app falls over the instant the cache is cold, you don't have a cache — you have a load-bearing single point of failure that also happens to be your least reliable component. Fix the database first. Cache second.
If there's one architectural move worth making early, it's this one: get slow work out of the request path.
When a user hits submit, they should wait for exactly the work required to give them a correct answer, and not a millisecond more. Sending the welcome email, generating the thumbnail, syncing to the analytics warehouse, calling that flaky third-party API — none of that belongs in the request. Push it onto a queue and return.
This is the first thing I'd call actual architecture, and it earns its complexity because it fixes real, felt problems:
You do not need Kafka for this. For your first 100k users, a database-backed job table or a managed queue (SQS, Cloud Tasks, or a Redis-backed worker like Sidekiq or BullMQ) is plenty, and you can actually reason about it because you can SELECT * FROM jobs WHERE status = 'failed' and see the whole world:
// Enqueue in the request path — fast, returns immediatelyasync function handleSignup(req, res) { const user = await db.users.create(req.body); // the necessary part await jobs.enqueue('send_welcome_email', { userId: user.id }); // the rest, later res.status(201).json({ id: user.id });}// A worker drains the queue out of bandasync function processJob(job) { switch (job.type) { case 'send_welcome_email': await email.sendWelcome(job.data.userId); break; // ...more job types }}Two rules keep queues sane. First, make jobs idempotent. Assume every job can run twice, because eventually one will — whether from a retry, a crash mid-processing, or a redelivery. Most queues give you at-least-once delivery, not exactly-once, so key your side effects such that a double-run is harmless (a welcome email that checks "did we already send this?" before sending, a payment keyed by an idempotency token). Second, give failed jobs somewhere to die. A dead-letter queue plus an alert beats jobs silently retrying forever and hammering a downstream service at 3am. Pair it with exponential backoff so retries don't turn into a self-inflicted denial-of-service. The absence of a dead-letter path is how a queue quietly turns from "resilience" into "invisible outage that also DDoSes your payment provider."
Here's the property that makes horizontal scaling boring, which is exactly what you want it to be: if any app server can handle any request, then adding capacity is just adding servers. No thought required. That's the goal, and it's the whole reason a stateless architecture is worth designing for from the start.
The enemy is state living in a single process:
When your app servers are stateless, your scaling story becomes almost dumb in a good way: a load balancer, N identical boxes behind it, autoscale on CPU or request concurrency. You can kill any box and lose nothing. You can deploy by rolling boxes one at a time with zero downtime. You can double capacity by changing a number in a config file. This is the payoff for keeping state where it belongs — in your database and object storage — instead of smeared across your compute.
I treat statelessness as a design constraint from the first commit, not a migration I do later. It costs almost nothing early and it's genuinely painful to retrofit once three features quietly depend on local memory and nobody remembers which ones.
At some point you want evidence your system holds under load. Good instinct. But most load-testing effort is wasted because people test for a scale they're nowhere near.
If you have 10k users, load-test for 100k. Do not test for 10 million. The bottlenecks at 10 million are completely different, you can't predict them from here, and the fixes for them are exactly the premature complexity this whole post is about avoiding. Test to your next 10x, fix what breaks, and move on. When you get there, test the next 10x. Scaling is a staircase you climb one step at a time, not an elevator you take to the top floor before anyone's moved into the building.
Load-test against something that resembles production: production-shaped data volumes, realistic query mixes, and a database seeded with real row counts. A test against an empty database tells you nothing, because the query that's instant on 100 rows is the exact one that kills you on a million — the sequential scan only hurts once the table is big. A basic run with a tool like k6:
import http from 'k6/http';import { check, sleep } from 'k6';export const options = { stages: [ { duration: '2m', target: 500 }, // ramp to 500 virtual users { duration: '5m', target: 500 }, // hold — this is where problems show { duration: '2m', target: 0 }, // ramp down ], thresholds: { http_req_duration: ['p(95)<400'], // fail the test if p95 blows past 400ms http_req_failed: ['rate<0.01'], // fail if error rate exceeds 1% },};export default function () { const res = http.get('https://staging.example.com/api/feed'); check(res, { 'status is 200': (r) => r.status === 200 }); sleep(1);}The point of the run isn't a vanity number. It's to find the thing that breaks first. Sometimes it's the database connection pool. Sometimes it's an unindexed query that was fine in dev. Sometimes it's a third-party rate limit you forgot existed until 500 virtual users hit it at once. Watch the whole stack while the test runs — the load-test client's error rate on one screen, the database's CPU, connections, and slow-query log on another. Whatever breaks first, that's your actual next bottleneck, the real one, not the imaginary one, and now you can fix it with evidence instead of vibes.
So when do you actually reach for the distributed-systems toolbox — the microservices, the sharding, the streaming brokers? When the metrics leave you no choice. Not before. Concrete signals I trust:
Notice what every one of these has in common: it's a specific, measured pain, not a prophylactic guess. You add the complexity to solve a problem you can point at, not to feel prepared for one you're imagining.
There's a decision test I apply to every proposed piece of sophistication: can we still reason about the system after we add this? If adding a component means nobody on the team can hold the failure modes in their head anymore, that component had better be putting out a fire that's actively burning. "We might need it someday" is not a fire. It's an anxiety, and you don't buy infrastructure to soothe an anxiety.
EXPLAIN ANALYZE; don't pattern-match a conference talk to a hunch.