What is system design? — System design strategy

Beyond "making it work"

Every developer can build an application that works on their laptop. You type bin/rails server, open localhost:3000, and everything runs. The database responds in milliseconds. There's one user: you. Nothing breaks.

System design is the discipline of making software work when your laptop is no longer enough. When a thousand users hit your app at the same time. When your database has 50 million rows. When a server catches fire at 3 AM and your customers expect the site to stay up. When a deploy has to happen without dropping a single request.

Writing better code won't get you there. System design is about decisions on structure: how components are organized, how they communicate, how they fail, and how they scale. These decisions are invisible while your app is small. Once it grows, they matter more than anything else.

The blueprint analogy

Think of building a house. A carpenter can build a shed without blueprints. Grab some wood, nail it together, put a roof on, and it works. Try building a 40-story skyscraper the same way and it collapses.

The materials are the same. What differs is the planning. A skyscraper needs an architect who decides: How deep should the foundation go? Where do the load-bearing walls go? How does water reach the 40th floor? What happens during an earthquake?

System design is software architecture. You are the architect deciding: How does the database handle 10,000 queries per second? Where does caching go? What happens when the payment service is down? How does the system survive a data center failure?

The carpenter (developer) and the architect (system designer) need different skills. This course teaches you to think like the architect.

Functional vs non-functional requirements

Every software system has two kinds of requirements. Confusing them is the first mistake junior engineers make in system design.

Functional requirements describe what the system does:

Users can sign up and log in
Students can enroll in courses
The system tracks lesson completion and awards XP

These are features. They answer the question "What can a user do?" Most developers spend their whole career here, and Rails makes it easy: a model, a controller, a view, done.

Non-functional requirements describe how well the system does it:

The page loads in under 200ms (performance)
The site stays up 99.9% of the time (availability)
User data is never lost, even during server failures (durability)
The system handles 10,000 concurrent users (scalability)
Passwords are stored with bcrypt, not plaintext (security)

These are qualities. They answer "How fast? How reliable? How big?" Nobody writes user stories for them, yet they decide whether your system survives production.

Functional requirements are negotiable. Non-functional requirements are not. You can launch without a "forgot password" feature. You cannot launch a system that loses data or falls over under load.

In a Rails app, your functional requirements live in controllers and services. Your non-functional requirements live in database indexes, caching strategies, background jobs, deployment configuration and infrastructure decisions. System design is mostly about the second group.

The trade-off mindset

The biggest mental shift in system design is this: there are no right answers, only trade-offs.

Every architectural decision gives up something to gain something else. A system designer's job is to understand what you're trading and decide whether the trade is worth it. A perfect solution doesn't exist.

Three trade-offs govern every system.

Consistency vs availability

When a network partition happens (and it will), you must choose. Does the system show possibly stale data (availability), or does it refuse to respond until it can guarantee fresh data (consistency)?

A banking application chooses consistency, because showing a wrong balance is worse than a brief outage. A social media feed chooses availability: a post from 2 seconds ago is fine if it keeps the site up.

Rails applications usually start with strong consistency: PostgreSQL with a single primary database. As they grow, they add read replicas and caching layers, and sooner or later they face this trade-off head on.

Latency vs throughput

Latency is how fast one request completes. Throughput is how many requests the system handles per second. Optimizing for one often hurts the other.

A Puma worker that processes requests one after another has low latency, because each request gets the CPU's full attention. Its throughput is low for the same reason. More Puma threads raise throughput, but they can raise per-request latency too, through context switching and competition for database connections.

This application's real config/puma.rb sets only the thread count explicitly:

# config/puma.rb
threads_count = ENV.fetch("RAILS_MAX_THREADS", 3)
threads threads_count, threads_count

The file has no workers line. Puma's own generated comment explains why: "You can control the number of workers using ENV['WEB_CONCURRENCY']... The default is already 1." This application's config/deploy.yml sets WEB_CONCURRENCY: 2 and RAILS_MAX_THREADS: 3 as deployment environment variables, and Puma reads the worker count from WEB_CONCURRENCY without needing a line for it.

WEB_CONCURRENCY=2 with RAILS_MAX_THREADS=3 gives you 6 concurrent requests. Raise it to 4 workers and 10 threads and you get 40, but now you need 40 database connections and more RAM. That is the trade-off.

Simplicity vs flexibility

A Rails monolith is simple: one codebase, one deploy, one database. It is also less flexible. You can't scale the search feature separately from the checkout feature.

Microservices are flexible, because each service scales on its own. They are also complex. You need service discovery, distributed tracing, network failure handling, and a whole platform team.

The Rails community's position is clear: start with a monolith. Basecamp, Shopify and GitHub all started as Rails monoliths. Some evolved and some stayed. The monolith is not a temporary hack. It's a legitimate architecture that many billion-dollar companies run in production.

The senior engineer's skill is knowing when the monolith's trade-offs stop working, and not a moment before.

The evolution of a Rails application

Rails applications grow along a predictable path. Knowing the path tells you which system design problems you'll face, and when.

Stage 1: single server

Rails, PostgreSQL and your assets all live on one machine, and Kamal deploys to that single host. This handles more than most people think: a well-tuned Rails app on a decent server can serve thousands of requests per minute.

Stage 2: separate database

PostgreSQL moves to its own server, or to a managed service like RDS. The database and the app server compete for memory and CPU, and separating them gives each room to breathe. Your Kamal deploy now targets the app server only, and the database is managed separately.

Stage 3: background jobs

Slow operations move out of the request cycle. Sending emails, processing uploads and awarding XP happen in Solid Queue workers, on the same server or a separate one. The user gets a fast response while the work happens in the background.

Stage 4: caching

Database queries that rarely change get cached: fragment caching, Russian Doll caching, Solid Cache. You trade memory for speed and accept slightly stale data.

Stage 5: read replicas

One PostgreSQL primary takes the writes, and one or more replicas take the reads. Rails 6+ has built-in support for multiple databases. Most read traffic goes to the replicas, which takes pressure off the primary.

Stage 6: load balancing

Several app servers sit behind a load balancer, which Kamal Proxy handles natively. Your app must be stateless, so session data can't live in server memory. Use database-backed sessions or cookies.

Stage 7: beyond the monolith

Service extraction, event-driven architecture, CQRS. Most Rails applications never need this stage. The ones that do serve millions of users with complex domain logic that has outgrown a single bounded context.

You don't skip stages, and you don't jump to stage 7 because Netflix did. Each stage solves a specific problem the previous stage created.

System design is decision-making

In the end, system design is about decisions more than technology. Every box on an architecture diagram stands for a decision someone made:

"We chose PostgreSQL over MongoDB because our data is relational."
"We chose Solid Queue over Sidekiq because we don't want to manage Redis."
"We chose a monolith over microservices because we have a team of four."

The technology changes. PostgreSQL today, something else tomorrow. The skill is the reasoning behind each decision: the trade-offs you weighed, the constraints you found, the growth you expected.

This course teaches that skill. In the next six lessons you'll learn to estimate requirements, design architectures, choose data storage, plan for scale, handle failure, and explain design decisions clearly.

Summary

Concept	What to remember
System design	Making software work beyond a single laptop: structure, communication, failure, scale
Functional requirements	What the system does (features); easy to add, negotiable
Non-functional requirements	How well it does it (performance, availability, durability); hard to retrofit, not negotiable
Trade-offs	No perfect answers; every decision gains something and loses something
Consistency vs availability	Banks choose consistency; social feeds choose availability
Latency vs throughput	Fast single requests vs many concurrent requests; Puma threads show this directly
Simplicity vs flexibility	Monolith vs microservices; start simple, split only when forced
Evolution stages	Single server → separate DB → background jobs → caching → replicas → load balancing → extraction