Event-Driven Architecture Without the Kafka Cargo Cult

Event driven architecture without Kafka: use the transactional outbox pattern with a Postgres or Firestore table, idempotency keys, and a worker to decouple your backend.

A founder I know in Dubai told me last year, with some pride, that his team had "gone event-driven." I asked what that meant on the ground. It meant Kafka, three brokers, a schema registry, a Debezium connector, and a dead-letter topic nobody had ever opened. The product was a booking app doing maybe 400 orders a day. They had wired up a nervous system for an elephant and bolted it to a hamster.

Events are not the problem. I lean on them constantly. The problem is that for most teams under a few hundred requests per second, the message broker is a liability you're paying for out of superstition. A database table and a dull background worker give you the same decoupling for a tenth of the operational cost. This post is about where the line between "you actually need a broker" and "you're cargo-culting one" really sits — and how to build a durable, resilient, event-driven backend out of infrastructure you already own.

What "event-driven architecture" actually buys you, stripped of the jargon

Strip away the conference talks and event-driven architecture is one idea: the thing that produces work should not have to wait for, or even know about, the thing that consumes it.

When a customer pays, the checkout code should not synchronously send the receipt email, update the loyalty points, ping the warehouse, and notify analytics. That's four ways for checkout to get slow, and four ways for checkout to fail because something downstream is having a bad day. Instead, checkout records a fact — "order 8842 was paid" — and moves on. Somebody else deals with the consequences, on their own schedule, retrying if they fall over.

That's the whole prize. You buy three things:

• Decoupling — the producer doesn't import the consumer's code or wait on its latency. Your checkout's p99 stops being hostage to an email provider's p99.
• Resilience — if the email service is down, the order still completes; the email retries later instead of failing the whole request.
• Absorbing spikes — a burst of 5,000 events gets worked through steadily instead of knocking a downstream service over. The queue is a shock absorber.

Notice what's not on that list: Kafka. None of these three benefits require a distributed commit log. They require a durable place to put a fact, and a process that picks facts up and acts on them. A database is a durable place to put a fact. You already have one, you already back it up, and you already know how to operate it.

Commands vs events vs jobs, and why teams conflate them

Half the bad architecture I see comes from mixing up three things that look similar and behave nothing alike. Getting this vocabulary straight is the single highest-leverage thing you can do before you pick any technology.

• A command is an instruction: "send this email." It is addressed to exactly one handler, it expects to be carried out, and it can be refused. Imperative, present tense.
• An event is a fact that already happened: "order was paid." It is past tense, it is broadcast, and it is not addressed to anyone in particular. Zero, one, or ten consumers may care. The producer does not know and should not care.
• A job is a unit of deferred work you scheduled for yourself: "generate the monthly invoice PDF." Often it's just a command you're choosing to run later.

The reason this matters: the number of consumers decides your architecture, not the volume of messages. If exactly one piece of code will ever react to "order paid," you don't need a broadcast bus — you need a queue with one worker, and a plain table does that fine. People reach for Kafka's fan-out because they've mentally labeled everything an "event," when most of what they're shuffling around is really just commands with a single handler.

Be honest about which of the three you have. Most CRUD apps are almost entirely commands and jobs. Genuine multi-consumer events — where you truly don't know who's listening and new listeners appear without you editing the producer — are rarer than the vocabulary suggests. When someone says "we're event-driven," the useful follow-up is: how many independent consumers react to your most important event, and do you control all of them? The answer usually collapses the whole architecture down to a queue.

The transactional outbox pattern: your database is already a queue

Here's the move that makes the simple version safe, and it's the one people skip. It also happens to be the most important pattern in this entire post.

The naive approach is to write your order to the database, then publish an event. Two systems, two writes, no shared transaction. This is the dual-write problem, and its failure mode is nasty: you commit the order, then your process dies before publishing. The order exists; the "order paid" event never fires. No receipt, no fulfillment, and nothing in your logs points at it. Or the reverse — you publish, then the DB write rolls back, and now there's a receipt for an order that doesn't exist. Every message broker on earth has this problem the moment it lives outside your database transaction, Kafka included.

The transactional outbox kills it. You write the business change and the event to the same database, in the same transaction. Either both land or neither does. A separate worker later reads the outbox table and does the delivery. One atomic write, no distributed transaction, no two-phase commit.

-- Same transaction as the business writeBEGIN;UPDATE ordersSET status = 'paid', paid_at = now()WHERE id = 8842;INSERT INTO outbox (id, topic, payload, created_at)VALUES (  gen_random_uuid(),  'order.paid',  '{"orderId": 8842, "amount": 149.00, "currency": "AED"}',  now());COMMIT;

Now a worker drains the outbox on a loop:

-- Claim a batch without two workers grabbing the same rowsSELECT id, topic, payloadFROM outboxWHERE processed_at IS NULLORDER BY created_atLIMIT 100FOR UPDATE SKIP LOCKED;

FOR UPDATE SKIP LOCKED is the quiet hero here. It lets you run several workers in parallel and each one grabs a different batch instead of fighting over the same rows. Without it, your workers either serialize behind a lock or double-process. With it, horizontal scaling of the drain step is free — spin up a second worker and it just claims different rows. Process each row, then stamp processed_at. If the worker crashes mid-batch, the uncommitted rows simply lose their lock and become visible again on the next tick — nothing is lost, because you never marked them done.

That's it. That's a durable, at-least-once event pipeline built entirely out of a table your ORM already manages. No new infrastructure, no new thing to page you at 3am, and it inherits your existing backups and point-in-time recovery for free.

The outbox on Firestore

The shape is nearly identical on Firestore. You write the order and an outbox document in a single WriteBatch, and a scheduled function (or a Firestore trigger, if you're careful about retries) drains it. Because a WriteBatch is atomic, you get the same all-or-nothing guarantee as the SQL transaction.

final batch = firestore.batch();batch.update(orderRef, {'status': 'paid', 'paidAt': FieldValue.serverTimestamp()});batch.set(outboxRef, {  'topic': 'order.paid',  'payload': {'orderId': 8842, 'amount': 149.00, 'currency': 'AED'},  'processedAt': null,  'createdAt': FieldValue.serverTimestamp(),});await batch.commit(); // atomic: both writes land or neither does

One Firestore-specific caveat worth stating plainly: Firestore triggers are themselves at-least-once and can fire more than once for a single write, so whether you drain with a scheduled function or a trigger, the idempotency work in the next section is not optional. A trigger that "usually fires once" is a trigger that will eventually fire twice in production, at the worst possible moment.

Idempotency is the whole game

Once you accept at-least-once delivery — and every real queue, including Kafka, is at-least-once — you have signed up for the same event arriving twice. A worker processes "order.paid," sends the email, then crashes before marking the row done. On restart, it sees the row again and sends a second email. The customer now has two receipts and a bad opinion of you.

Idempotency means processing the same event twice has the same effect as processing it once. This is not optional and it is not a nice-to-have. It is the thing that makes at-least-once delivery survivable, and it's identical whether you're on Kafka or a Postgres table. Anyone selling you a broker as the solution to duplicates is selling you nothing — exactly-once delivery across a network boundary is a mirage, so you still have to do this work yourself no matter what you buy.

The core pattern is a claimed-work table keyed by a natural idempotency key, with the uniqueness guarantee enforced by the database rather than your application code:

-- One receipt per order, enforced by the databaseINSERT INTO sent_receipts (order_id, sent_at)VALUES (8842, now())ON CONFLICT (order_id) DO NOTHING;

If the insert affected zero rows, someone already sent it — skip. The unique constraint on order_id is your source of truth, not application logic you have to remember to write. Push the guarantee down into the database where races go to die. This matters because the alternative — a "have I seen this?" SELECT followed by a SEND followed by an INSERT — has a window between the read and the write where a second worker can slip through and double-send. The constraint closes that window at the storage layer.

Three rules I hold to:

• Derive the key from the event, not from the clock or a random. order.paid:8842 is stable across retries; the same event always produces the same key. A uuid() generated inside the handler is different every run and defeats the entire mechanism.
• Make the check and the effect atomic where you can. A separate "have I seen this?" read followed by a write is a race. A unique-constraint insert (or an upsert) is not.
• Prefer operations that are naturally idempotent. SET status = 'paid' is safe to run ten times — the tenth write produces the same state as the first. balance = balance + 10 is not; run it twice and you've given away money. Naturally idempotent operations need no guard at all, so lean on them.

Get idempotency right and duplicate delivery stops being scary — it becomes a shrug. Get it wrong and no broker on earth will save you.

When a Firestore or Postgres table beats a broker

Here's my honest default. Reach for the table-plus-worker when:

• You have one or a small, known set of consumers. No mystery listeners you don't control.
• Your throughput is hundreds of events per second or less, not tens of thousands. A single Postgres instance handles far more outbox churn than most startups will ever produce, and you can shard or partition long before it becomes a real ceiling.
• Ordering needs are per-entity, not global. "Process events for order 8842 in order" is easy with a table (order by created_at, key your workers by entity). "Global total ordering across all events" is a genuinely hard problem — and you almost certainly don't actually have it.
• Your consumers live in the same system as your producer, or close to it. You're not integrating forty teams across network boundaries.
• You value not running a broker more than you value squeezing out the last drop of throughput.

The operational math is what sells it. A Kafka setup means brokers, a coordinator, retention tuning, partition planning, consumer-group rebalancing, monitoring for all of it, and someone who actually understands it when it breaks at 3am. The outbox means one more table and a cron-driven worker that any backend engineer can read top to bottom. On a recent project we ran the entire async backend — receipts, fulfillment webhooks, push notifications — on a single outbox table for over a year. It cost effectively nothing and it never woke me up.

There's also a cost angle I care about a lot, especially keeping a startup lean. A managed Kafka cluster starts around real monthly money before you've shipped a single feature. An outbox table on infrastructure you're already paying for adds zero to the bill and zero to the on-call surface area. When you're trying to stay cheap and fast, "zero new services" is a feature, not a compromise.

The signals that mean you've genuinely outgrown the simple version

I'm not anti-broker. I'm anti-broker-by-default. There's a real line, and when you cross it, a table stops being the right tool and a proper streaming platform starts earning its keep. Watch for these:

• Fan-out to many independent, unknown consumers. When five teams want to react to "order.paid" and the set changes every quarter, hand-maintaining consumer logic in one worker becomes the bottleneck. A real pub/sub bus (Kafka, Pub/Sub, NATS) earns its keep here because consumers subscribe without you editing the producer.
• Sustained high throughput. When your outbox is churning tens of thousands of rows a second and your database is feeling the write and vacuum pressure, a log built for exactly that workload will serve you better.
• Replay as a first-class need. "Rebuild this projection from the last 90 days of events" is what an event log is for. An outbox you prune weekly can't do it; a retained, immutable log can.
• Cross-organization or cross-datacenter delivery with independent scaling and back-pressure. Different teams, different deploy cadences, real network boundaries between producer and consumer.
• Stream processing — windowed aggregations, joins over streams, continuous queries. That's a genuinely different workload, not something you bolt onto a table.

If you hit one of these for real — not hypothetically, not "we might someday" — then reach for the broker with a clear conscience. The point was never to avoid Kafka forever. It's to not pay for it before it's buying you anything.

A worked example: order confirmation without a single new service

Let me tie it together with the exact case that started this post. A customer pays. We need to send a receipt, notify fulfillment, and record the sale in analytics. Three consumers, no new infrastructure.

Step 1 — checkout writes the fact, atomically. When payment confirms, the same transaction updates the order and inserts one order.paid row into the outbox. Checkout returns immediately. It does not send an email. It does not call fulfillment. It knows nothing about them and never blocks on them.

Step 2 — a worker drains the outbox. A process runs every few seconds (a cron job, a scheduled Cloud Function, a long-running loop — pick your poison), claims a batch with SKIP LOCKED, and dispatches each event to its handlers.

Future<void> drainOutbox() async {  final batch = await claimUnprocessed(limit: 100);  for (final event in batch) {    switch (event.topic) {      case 'order.paid':        await sendReceipt(event);       // idempotent: ON CONFLICT DO NOTHING        await notifyFulfillment(event); // idempotent: keyed by orderId        await recordSale(event);        // idempotent: upsert by orderId        break;    }    await markProcessed(event.id);  }}

Step 3 — every handler is idempotent. sendReceipt inserts into sent_receipts with ON CONFLICT DO NOTHING and only sends if it won the insert. notifyFulfillment posts to a webhook with orderId as the dedupe key. recordSale upserts on orderId. Run the whole batch twice and the outcome is identical — that's the property that makes at-least-once delivery safe.

Step 4 — failures take care of themselves. If notifyFulfillment throws because the warehouse API is down, we don't mark the row processed. Next tick, it comes back around. The receipt was already sent, so its idempotency guard skips it, and only the failed step retries. After N attempts, park the row in a failed state and alert — that's your dead-letter queue, and it's a WHERE processed_at IS NULL AND attempts >= N clause, not a Kafka topic and a consumer group.

Count what we added: one table, one worker loop, three unique constraints. No broker, no schema registry, no consumer groups, no new entry in the on-call runbook. And it has every property the fancy version promised — checkout is decoupled, the pipeline is resilient, duplicates are harmless, and a spike just means the worker takes a few extra minutes to catch up.

When this app grows to where five external teams want that order.paid event on their own schedule, I'll move the delivery step onto a real bus and keep the outbox as the source of truth feeding it. That migration is easy precisely because I didn't couple my business logic to a broker on day one — the business code still just writes a fact to a table, and only the drain step changes.

Key takeaways

• "Event-driven" buys decoupling, resilience, and spike absorption — and none of those three require a message broker. They require a durable place to put a fact and a worker to act on it. Your database is already both.
• Separate commands (one handler), events (broadcast facts), and jobs (deferred work). Most CRUD apps are commands and jobs wearing an event costume. The number of independent consumers, not the message volume, decides your architecture.
• Use the transactional outbox pattern to beat the dual-write problem: write the business change and the event in the same transaction, then drain the table with a worker using FOR UPDATE SKIP LOCKED. Your database is already a durable, at-least-once queue.
• Idempotency is non-negotiable under at-least-once delivery, and no broker does it for you. Key work off the event, enforce uniqueness with a database constraint, and prefer naturally idempotent operations so races die at the storage layer.
• Default to a table plus a worker. Reach for Kafka only when you have real fan-out to unknown consumers, sustained high throughput, event replay, or stream processing — not because a conference slide said "modern architecture."