The problem
Your team is launching a public API for a SaaS product, expected to serve thousands of clients globally. The API must be highly available, resilient to failures, and able to scale with user growth. AWS is the mandated cloud provider, and downtime or slow responses will directly impact paying customers.
You are the lead backend engineer tasked with designing the API's backend architecture for reliability and scalability.
If the design is wrong, customers will experience outages or degraded performance, leading to lost revenue and reputation damage.
Must hold true
- Must use AWS-managed services where possible.
- API must maintain 99.99% availability.
- All data must be stored in a durable, consistent store.
- System must handle regional failover.
Functional Requirements
What the system must DO.
| Requirement | Priority |
|---|---|
| The system must receive and respond to HTTP API requests from clients. | Must have |
| The API must allow clients to write and update data records. | Must have |
| The API must allow clients to read data records efficiently. | Must have |
| All data must be stored in a durable, strongly consistent database. | Must have |
| The system should support automatic failover to another AWS region in case of regional failure. | Nice to have |
Non-Functional Requirements
How well it must do it — latency, availability, scale. Measurable, not vibes.
| Requirement | Priority |
|---|---|
| The API must achieve at least 99.99% availability. | Must have |
| The system must scale to handle peak loads of 10,000 requests per second globally. | Must have |
| API responses should have a median latency below 150 ms. | Nice to have |
What you learn by asking
In an interview these facts are NOT volunteered — the candidate earns them with clarifying questions. These are the answers behind this design.
| If you ask about… | The answer |
|---|---|
| Peak API traffic | Peak load is 10,000 requests per second globally. |
| Data size per request | Each API request reads or writes about 2 KB of data. |
| Data retention and growth | Data is retained indefinitely and grows at 100 GB per month. |
| Availability target | The API must achieve 99.99% availability. |
| AWS region usage | Primary region is us-east-1; failover region is eu-west-1. |
| Budget for managed services | Budget allows for using AWS managed services (RDS, DynamoDB, ALB, Route 53, etc.). |
Capacity Estimation
Rough arithmetic turns the requirements into engineering decisions — it tells you which parts need scaling machinery and which are trivially easy. Only the order of magnitude matters.
| Quantity | Answer | How it is derived |
|---|---|---|
| Monthly API requests | ~25.9B requests/month | 10,000 requests/sec × 60 × 60 × 24 × 30 = 25,920,000,000 requests/month. |
| Annual data storage growth | ~1200 GB/year | 100 GB/month × 12 = 1,200 GB/year. |
| Peak bandwidth needed | ~20 MB/sec | 10,000 requests/sec × 2 KB = 20,000 KB/sec = 20 MB/sec. |
Quick check
Without scrolling back: roughly what did we work out for monthly api requests?
The architecture, built up
Nobody designs the final diagram in one stroke, so this design is built up one component at a time rather than shown finished. Start with the simplest thing that works, then add each piece and see where it attaches.
Each stage lets the numbers break the design, then fixes exactly one bottleneck with exactly one new component — and at each break you get a chance to call the fix yourself before it is revealed. You can stop reading after any stage and still hold a complete, working system in your head — that is the point.
The simplest thing that works
Start with a client, a single API server, and a relational database.
The most basic API design has clients connecting directly to a single API server, which reads and writes data to a relational database. This works for development or very low traffic, but it has clear limitations: any server or database outage means downtime, and it cannot scale to thousands of requests per second.
The design so far
Where it breaks
If the API server crashes or is overloaded, all clients lose access.
Quick check
Which single component would you add to fix that?
One server is a single point of failure
The fix: Add an AWS Application Load Balancer (ALB) to distribute traffic across multiple API servers.
To avoid downtime from a single server failure, we introduce an ALB, which can route requests to multiple API servers. This allows us to scale horizontally and handle more traffic, as well as survive individual server failures.
The design so far
Where it breaks
A full AWS region outage makes the API unreachable, even with Multi-AZ.
Quick check
Which single component would you add to fix that?
No DNS-based failover for regional outages
The fix: Add AWS Route 53 for DNS-based routing and health checks to enable regional failover.
Even Multi-AZ cannot protect against an entire AWS region outage. By introducing Route 53, we can configure DNS health checks and direct clients to a secondary region if the primary is down. This enables true regional failover.
The finished design
The full reference design, with the reason behind every connection. When you practice, you will rebuild this from a palette that includes decoys.
Why each connection exists
- ClientAWS Route 53Route 53 provides DNS resolution and health checks for high availability.
- AWS Route 53AWS Application Load BalancerRoute 53 directs clients to healthy load balancers.
- AWS Application Load BalancerAPI Server (EC2/ECS)ALB distributes traffic across multiple API servers for scalability and redundancy.
- API Server (EC2/ECS)AWS RDS (Multi-AZ)API servers interact with RDS for persistent, consistent storage.
The design in one glance
Each stage fixed exactly one problem. If you can retell this table from memory, you can derive the whole architecture on a whiteboard.
| The design so far | What broke | The fix |
|---|---|---|
| The simplest thing that works | — the starting point | Start with a client, a single API server, and a relational database. |
| One server is a single point of failure | If the API server crashes or is overloaded, all clients lose access. | Add an AWS Application Load Balancer (ALB) to distribute traffic across multiple API servers. |
| No DNS-based failover for regional outages | A full AWS region outage makes the API unreachable, even with Multi-AZ. | Add AWS Route 53 for DNS-based routing and health checks to enable regional failover. |
Good to know — What this design leaves out — on purpose
Every real system also needs monitoring and metrics, logging and tracing, alerting and on-call, CI/CD, auth hardening and abuse limits, and a cost model. None of them appear above, and that is a choice: they sit beside EVERY system in much the same shape, so drawing them here would add the same five boxes to every problem in this library while crowding out the part that is actually specific to this one — the path a request takes and where it breaks. This is also why a metrics or monitoring block is a wrong answer in the practice canvas: it is never the thing that makes this design work. In a real interview, name these in one sentence once the data path is settled — "I would put metrics on the cache hit rate and alert when it drops" — and move on. Reaching for them before the core flow is drawn reads as avoiding the question.
API Design
Every operation traces back to a requirement — nothing extra, nothing missing.
| Endpoint | Purpose |
|---|---|
| POST /records | Create a new data record. |
| PUT /records/{id} | Update an existing data record. |
| GET /records/{id} | Retrieve a data record by ID. |
Quick check
Which one would you call to create a new data record?
Data Model
| Entity | Fields |
|---|---|
| Record | id: string (Unique identifier), data: string (Payload data, up to 2 KB), created_at: timestamp (Creation time), updated_at: timestamp (Last update time) |
Defending the Design
Interviewers close by stress-testing the design. Strong answers follow one shape: which component → what breaks there → the fix and its cost. Here are this problem's probes and the points a strong answer hits:
What are the main bottlenecks if your RDS instance cannot handle the write load, and how can you address them? Write throughput is limited by RDS instance size and IOPS.. Scaling up (vertical scaling) or sharding data across multiple RDS instances can help.. Consider using write-optimized replicas or partitioning if write load continues to grow..
How does your design handle AWS region failure, and what changes are needed for full regional failover? Route 53 can route traffic to a secondary region if the primary is down.. RDS Multi-AZ covers AZ failures but not region-wide outages, so cross-region replication is needed.. API servers and ALB must be deployed in both regions, with data replication and DNS failover configured..
How does the Application Load Balancer contribute to scalability, and what are its limits? ALB distributes traffic across many API servers, allowing horizontal scaling.. ALB can scale automatically but has soft limits that can be raised with AWS support.. If traffic exceeds ALB capacity, requests may be throttled or dropped..
Why is RDS chosen over DynamoDB for this API, and what are the tradeoffs? RDS provides strong consistency and relational features needed for this API.. DynamoDB offers higher scalability but eventual consistency and different data modeling.. RDS is better for transactional workloads; DynamoDB is better for massive scale with simpler access patterns..
Tip — Key takeaway
Designing a highly available API on AWS requires careful use of managed services like Route 53, ALB, and RDS Multi-AZ to meet strict availability and scalability targets. Each component addresses a specific failure or scaling risk, and the architecture must evolve as numbers force new constraints. Understanding AWS service roles and limits is crucial to building resilient, scalable systems.
Common trap — Common mistakes on this problem
Relying on a single API server or database instance, which creates single points of failure. · Using DynamoDB or SQS without a clear need, leading to unnecessary complexity or cost. · Neglecting DNS-based failover, which leaves the system vulnerable to regional outages.
You've seen the whole design — now build it yourself
The practice run walks the same steps, but YOU do the work: gather the requirements, run the numbers, wire the architecture — with an AI interviewer and per-step feedback.