Order Processing with Kafka Events — System Design Walkthrou

The problem

You are designing the backend for an e-commerce platform that processes customer orders. Each order triggers several downstream actions—inventory updates, payment processing, and shipment scheduling—all of which must be reliably coordinated. The team wants to use Kafka to decouple these systems and ensure that no order event is lost, even under high load.

You are the backend engineer responsible for designing the event-driven order processing system.

If the system is poorly designed, orders may be dropped or processed multiple times, leading to inventory errors, payment issues, and unhappy customers.

Must hold true

Order events must never be lost, even if a downstream service is temporarily unavailable.
Order processing must scale to handle peak loads of up to 10,000 orders per minute.
Downstream services should be decoupled so they can be updated independently.
Order status updates must be queryable by customer support within 5 seconds of order creation.

Functional Requirements

What the system must DO.

Requirement	Priority
The system must publish an event for every new order.	Must have
Inventory must be updated in response to order events.	Must have
Payment processing must be triggered by order events.	Must have
Shipment scheduling must be initiated after successful payment.	Must have
Customer support must be able to query the latest order status within 5 seconds of order creation.	Must have

Non-Functional Requirements

How well it must do it — latency, availability, scale. Measurable, not vibes.

Requirement	Priority
No order events may be lost, even if a downstream service is down.	Must have
The system must handle up to 10,000 new orders per minute at peak.	Must have
Downstream services must be decoupled to allow independent updates.	Nice to have

What you learn by asking

In an interview these facts are NOT volunteered — the candidate earns them with clarifying questions. These are the answers behind this design.

If you ask about…	The answer
Peak order volume	The system must handle up to 10,000 new orders per minute at peak times.
Order event size	Each order event is approximately 2 KB in size.
Order status query frequency	Customer support queries order status for about 5% of orders, typically within 5 minutes of order creation.
Durability requirement	Order events must not be lost even if a downstream consumer is offline for up to 1 hour.
Expected daily order volume	On average, the platform receives 100,000 orders per day.

Capacity Estimation

Rough arithmetic turns the requirements into engineering decisions — it tells you which parts need scaling machinery and which are trivially easy. Only the order of magnitude matters.

Quantity	Answer	How it is derived
Peak Kafka write throughput	~0.33 MB/s	10,000 orders/min × 2 KB = 20,000 KB/min = 20 MB/min. 20 MB/min ÷ 60 ≈ 0.33 MB/s at peak.
Daily storage required for order events	~0.2 GB	100,000 orders/day × 2 KB = 200,000 KB/day = 200 MB/day = 0.2 GB/day.
Peak order status query QPS	~0.83 QPS	10,000 orders/min × 5% = 500 queries/min. 500 ÷ 60 ≈ 8.3 QPS peak, but since queries are spread over 5 minutes, 500/5/60 ≈ 1.67 QPS average, but peak is about 8.3 QPS.

The architecture, one version at a time

Nobody designs the final diagram in one stroke. Strong candidates start with the simplest thing that works, then let the numbers break it: each version fixes exactly one bottleneck with exactly one new component.

At each break you get a chance to call the fix yourself before it is revealed. You can stop reading after any version and still hold a complete, working system in your head — that is the point.

v1 — all services directly called synchronously

Start with the simplest flow: order service calls inventory, payment, and shipment services directly, then updates the status database.

This version is the most straightforward but tightly couples all services. If any downstream service is slow or fails, the whole order creation is delayed or fails.

There is no event log, so failures can cause order loss, and services cannot be updated independently.

v1 — all services directly called synchronously — the picture

Where v1 breaks

If payment or inventory is down, orders are lost or delayed; scaling is limited by synchronous calls.

Quick check

Which single component would you add to fix that?

v2 — downstream service failures cause order loss

The fix: Introduce Kafka as a durable event log between order service and downstream services, decoupling them and ensuring no order event is lost.

Kafka sits between the order service and all downstream services. The order service publishes events to Kafka, and inventory, payment, and shipment services consume them at their own pace.

This removes direct dependencies and allows for retries and recovery if a service is offline.

The finished design

The full reference design, with the reason behind every connection. When you practice, you will rebuild this from a palette that includes decoys.

Why each connection exists

Client AppOrder ServiceClient initiates order creation.
Order ServiceKafka ClusterOrder events are sent to Kafka for durability and decoupling.
Kafka ClusterInventory ServiceInventory service listens for new order events.
Kafka ClusterPayment ServicePayment service processes payments for new orders.
Payment ServiceShipment ServiceShipment only starts after payment is confirmed.
Order ServiceOrder Status DBOrder status is recorded immediately for support queries.
Inventory ServiceOrder Status DB(optional)Status may be updated after inventory adjustment.
Payment ServiceOrder Status DB(optional)Status is updated after payment.
Shipment ServiceOrder Status DB(optional)Status is updated after shipment.

The design in one glance

Each version fixed exactly one problem. If you can retell this table from memory, you can derive the whole architecture on a whiteboard.

Version	What broke	The fix
v1 — all services directly called synchronously	— the starting point	Start with the simplest flow: order service calls inventory, payment, and shipment services directly, then updates the status database.
v2 — downstream service failures cause order loss	If payment or inventory is down, orders are lost or delayed; scaling is limited by synchronous calls.	Introduce Kafka as a durable event log between order service and downstream services, decoupling them and ensuring no order event is lost.

API Design

Every operation traces back to a requirement — nothing extra, nothing missing.

Endpoint	Purpose
POST /orders	Create a new order and publish an event.
GET /orders/{orderId}/status	Query the current status of an order.

Data Model

Entity	Fields
OrderEvent	orderId: string (Unique order identifier), customerId: string (ID of the customer), items: array (List of items in the order), totalAmount: number (Total order value), timestamp: string (Order creation time), status: string (Current order status)

Defending the Design

Interviewers close by stress-testing the design. Strong answers follow one shape: which component → what breaks there → the fix and its cost. Here are this problem's probes and the points a strong answer hits:

How does your design ensure no order events are lost if a downstream service is offline for an hour? Kafka persists events until acknowledged by consumers. Consumer offsets allow resuming after downtime. Kafka retention policy is set longer than max expected downtime. No event is deleted until all consumers have processed it.

What could make the order status database a bottleneck, and how would you address it? High write or read load could overwhelm the DB. Slow queries delay support responses. Can shard or replicate DB for scale. Can use indexing or denormalization for faster lookups.

If order volume grows 10x, what changes would you make to the Kafka cluster? Increase number of partitions for parallelism. Add more brokers to handle throughput. Tune producer/consumer configuration for higher volume. Monitor and adjust retention and disk capacity.

How would you ensure that inventory and payment are each processed exactly once per order? Use idempotent consumers to avoid duplicate processing. Leverage Kafka's transactional APIs if available. Design downstream services to detect and ignore duplicates. Tradeoff: complexity vs. consistency guarantees.

Tip — Key takeaway

Event-driven architectures using Kafka allow you to decouple services, scale order processing, and ensure durability even under failures. The core principle is to use Kafka as a durable, central event log, letting each downstream system process events at its own pace without risking data loss.

Common trap — Common mistakes on this problem

Relying on synchronous calls between services, which destroys decoupling and increases failure risk. · Assuming Kafka alone provides exactly-once semantics without idempotent consumers or transactional logic. · Using a cache or search index for order status queries when a simple database suffices at low QPS.

You've seen the whole design — now build it yourself

The practice run walks the same steps, but YOU do the work: gather the requirements, run the numbers, wire the architecture — with an AI interviewer and per-step feedback.