Highly Available API on AWS — System Design Walkthrough

The problem

You are designing a public-facing REST API for a SaaS product serving customers worldwide. The company expects rapid growth and requires the API to be highly available, resilient to failures, and scalable to handle peak loads.

You are a cloud solutions architect tasked with designing the API's backend infrastructure using AWS services.

If the system fails, customers experience downtime, data loss, or inconsistent results, impacting business reputation and revenue.

Must hold true

Must use AWS-managed services where possible
Downtime must not exceed 5 minutes per year (99.999% availability)
API response time must be under 300ms at the 95th percentile
Data must survive the failure of any single AWS region

Functional Requirements

What the system must DO.

Requirement	Priority
The API must handle up to 10,000 requests per second during peak hours.	Must have
The API must ensure strong consistency for critical write operations.	Must have
The system must automatically fail over to another AWS region if one region becomes unavailable.	Must have
The API servers must be stateless to allow easy scaling and recovery.	Must have
The API must authenticate all requests using OAuth 2.0 tokens.	Nice to have

Non-Functional Requirements

How well it must do it — latency, availability, scale. Measurable, not vibes.

Requirement	Priority
The system must achieve at least 99.999% availability.	Must have
The API must respond within 300ms for 95% of requests.	Must have
Use AWS-managed services where possible to reduce operational overhead.	Nice to have

What you learn by asking

In an interview these facts are NOT volunteered — the candidate earns them with clarifying questions. These are the answers behind this design.

If you ask about…	The answer
Peak traffic	The system must handle 10,000 requests per second at peak.
Data size	Each API request reads or writes approximately 1 KB of data.
Data growth	Data is expected to grow by 500 GB per year.
Uptime SLA	The system must achieve at least 99.999% availability (downtime < 5 minutes/year).
Latency target	API responses must be under 300ms for 95% of requests.
Global reach	Users are distributed worldwide, with 60% in North America, 25% in Europe, and 15% in Asia-Pacific.

Capacity Estimation

Rough arithmetic turns the requirements into engineering decisions — it tells you which parts need scaling machinery and which are trivially easy. Only the order of magnitude matters.

Quantity	Answer	How it is derived
Peak API bandwidth	~10 MB/s	10,000 requests/sec × 1 KB/request = 10,000 KB/sec = 10 MB/sec.
Annual storage growth	~500 GB/year	Given directly as 500 GB/year.
Total API requests per day	~864 million requests	10,000 requests/sec × 60 × 60 × 24 = 864,000,000 requests/day = 864 million requests/day.

Quick check

Without scrolling back: roughly what did we work out for peak api bandwidth?

The architecture, built up

Nobody designs the final diagram in one stroke, so this design is built up one component at a time rather than shown finished. Start with the simplest thing that works, then add each piece and see where it attaches.

Each stage lets the numbers break the design, then fixes exactly one bottleneck with exactly one new component — and at each break you get a chance to call the fix yourself before it is revealed. You can stop reading after any stage and still hold a complete, working system in your head — that is the point.

The simplest thing that works

Start with a client, a single API server, and a DynamoDB table to store items.

At first, the simplest approach is to have clients send requests directly to a single API server, which reads and writes data to a DynamoDB table. This setup works for low traffic and a single region, but does not scale or provide high availability.

The design so far

Where it breaks

If the API server fails, all requests fail and the system is unavailable.

Quick check

Which single component would you add to fix that?

No single point of failure at the compute layer

The fix: Add an AWS Application Load Balancer (ALB) to distribute traffic across multiple stateless API servers.

To avoid a single point of failure, introduce an ALB that routes requests to a pool of API servers. This allows for horizontal scaling and resilience if one server goes down.

The design so far

Where it breaks

API must authenticate requests and allow servers to scale statelessly.

Quick check

Which single component would you add to fix that?

Authentication and statelessness

The fix: Add Amazon Cognito to manage OAuth 2.0 tokens and user authentication.

Authentication is offloaded to Cognito, so API servers remain stateless and can be replaced or scaled freely. This is critical for horizontal scaling and secure access control.

The design so far

Where it breaks

Users worldwide experience high latency; 95th percentile response time exceeds 300ms for distant users.

Quick check

Which single component would you add to fix that?

Global reach and low latency

The fix: Add Amazon CloudFront CDN to cache API responses and route requests to the nearest edge location.

CloudFront reduces latency by caching responses and routing user requests to the closest AWS edge location. This improves response times for global users and helps meet latency SLAs.

The design so far

Where it breaks

If an AWS region fails, data and service availability are lost.

Quick check

Which single component would you add to fix that?

Regional failover and data durability

The fix: Add DynamoDB Global Table to replicate data across multiple AWS regions for automatic failover and durability.

DynamoDB Global Tables ensure data is replicated to multiple regions, so if one region fails, traffic can be routed to another region with up-to-date data. This is essential for achieving 99.999% availability and data durability.

The finished design

The full reference design, with the reason behind every connection. When you practice, you will rebuild this from a palette that includes decoys.

Why each connection exists

ClientAmazon CloudFront CDNClients connect to the nearest CloudFront edge for low-latency access.
Amazon CloudFront CDNAWS Application Load BalancerCloudFront forwards API requests to the application load balancer.
AWS Application Load BalancerAPI Server (EC2/Container)The ALB distributes requests to healthy API servers.
API Server (EC2/Container)Amazon CognitoAPI server validates OAuth tokens with Cognito.
API Server (EC2/Container)Amazon DynamoDBAPI server performs strongly consistent reads/writes to DynamoDB.
Amazon DynamoDBDynamoDB Global TableDynamoDB Global Table replicates data across regions for high availability.

The design in one glance

Each stage fixed exactly one problem. If you can retell this table from memory, you can derive the whole architecture on a whiteboard.

The design so far	What broke	The fix
The simplest thing that works	— the starting point	Start with a client, a single API server, and a DynamoDB table to store items.
No single point of failure at the compute layer	If the API server fails, all requests fail and the system is unavailable.	Add an AWS Application Load Balancer (ALB) to distribute traffic across multiple stateless API servers.
Authentication and statelessness	API must authenticate requests and allow servers to scale statelessly.	Add Amazon Cognito to manage OAuth 2.0 tokens and user authentication.
Global reach and low latency	Users worldwide experience high latency; 95th percentile response time exceeds 300ms for distant users.	Add Amazon CloudFront CDN to cache API responses and route requests to the nearest edge location.
Regional failover and data durability	If an AWS region fails, data and service availability are lost.	Add DynamoDB Global Table to replicate data across multiple AWS regions for automatic failover and durability.

Good to know — What this design leaves out — on purpose

Every real system also needs monitoring and metrics, logging and tracing, alerting and on-call, CI/CD, auth hardening and abuse limits, and a cost model. None of them appear above, and that is a choice: they sit beside EVERY system in much the same shape, so drawing them here would add the same five boxes to every problem in this library while crowding out the part that is actually specific to this one — the path a request takes and where it breaks. This is also why a metrics or monitoring block is a wrong answer in the practice canvas: it is never the thing that makes this design work. In a real interview, name these in one sentence once the data path is settled — "I would put metrics on the cache hit rate and alert when it drops" — and move on. Reaching for them before the core flow is drawn reads as avoiding the question.

API Design

Every operation traces back to a requirement — nothing extra, nothing missing.

Endpoint	Purpose
POST /items	Create a new item in the system.
GET /items/{id}	Retrieve an item by its ID.
PUT /items/{id}	Update an existing item.
DELETE /items/{id}	Remove an item from the system.

Quick check

Which one would you call to create a new item in the system?

Data Model

Entity	Fields
Item	id: string (Globally unique identifier), name: string (Item name), createdAt: datetime (Timestamp of creation), updatedAt: datetime (Timestamp of last update), ownerId: string (User who owns the item)

Defending the Design

Interviewers close by stress-testing the design. Strong answers follow one shape: which component → what breaks there → the fix and its cost. Here are this problem's probes and the points a strong answer hits:

What are potential bottlenecks at the AWS Application Load Balancer, and how can they be mitigated? ALB has throughput and connection limits per region; can be mitigated by scaling out, using multiple ALBs, or distributing traffic across regions.. Monitor ALB metrics for surge traffic and pre-scale if predictable.. Consider using AWS Global Accelerator for cross-region load balancing if needed..

How does your design ensure data durability and availability if an entire AWS region fails? DynamoDB Global Tables replicate data across multiple regions, so another region can serve traffic if one fails.. CloudFront can reroute requests to healthy regions.. Failover requires DNS or routing updates to direct clients to the backup region..

How does DynamoDB handle scaling for both read and write throughput at peak load? DynamoDB supports automatic scaling based on traffic patterns.. Provisioned or on-demand capacity can be set to handle 10,000 QPS.. Strong consistency may limit read scalability, so partition keys should be chosen to avoid hot partitions..

Why might you avoid adding an ElastiCache layer in this design, and what trade-offs would it introduce? ElastiCache adds complexity and cost; not needed unless read patterns or latency targets justify it.. Cache invalidation and consistency become challenges with strong consistency requirements.. DynamoDB is fast and scalable enough for the current workload..

Tip — Key takeaway

Designing a highly available API on AWS requires careful attention to statelessness, global data replication, and managed services to meet strict availability and latency SLAs. Each component must be justified by concrete traffic and durability needs, not by habit. Automatic failover and global replication are critical for surviving region failures without data loss.

Common trap — Common mistakes on this problem

Relying on a single AWS region, which risks total outage during a regional failure. · Using a relational database (RDS) instead of DynamoDB, which complicates global replication and scaling. · Adding unnecessary components like queues or caches, increasing complexity without solving a real problem.

You've seen the whole design — now build it yourself

The practice run walks the same steps, but YOU do the work: gather the requirements, run the numbers, wire the architecture — with an AI interviewer and per-step feedback.