The two dials: workers and threads

Free preview 4 min to read

Open config/puma.rb in this repository. Its comments already explain the model, and the file is short enough to read in full:

threads_count = ENV.fetch("RAILS_MAX_THREADS", 3)
threads threads_count, threads_count

port ENV.fetch("PORT", 3000)

plugin :tmp_restart
plugin :solid_queue if ENV["SOLID_QUEUE_IN_PUMA"]

pidfile ENV["PIDFILE"] if ENV["PIDFILE"]

Note what is not there: no workers line. As the file's own comments say, you control the number of workers with WEB_CONCURRENCY, and the default is already 1. With nothing set, Puma runs a single process serving three threads; the deploy environment sets the real numbers.

That is a correct configuration for a small deployment, and one environment variable away from a very different one.

Two dimensions of concurrency

flowchart TB
  M["Puma master process"] --> W1["Worker 1 — a full Ruby process"]
  M --> W2["Worker 2 — a full Ruby process"]
  W1 --> T1["thread"]
  W1 --> T2["thread"]
  W1 --> T3["thread"]
  W2 --> T4["thread"]
  W2 --> T5["thread"]
  W2 --> T6["thread"]

Workers are processes. Each is a separate Ruby VM with its own memory, its own copy of your application, and its own connection pool. Cluster mode, meaning more than one worker, is how a Ruby application uses more than one CPU core.

Threads live inside a worker. They share that process's memory, its object space and its connection pool. Threads are cheap; a worker is expensive.

Total concurrency is the product: workers × threads. Two workers of three threads each handle six requests at once, which is not the same as six workers of one thread, and the difference is what the rest of this course is about.

Why both exist, and the GVL

The file's comments state this part directly, and it is the most important fact for sizing a Ruby application:

due to CRuby's Global VM Lock (GVL) it has diminishing returns and will degrade the response time (latency) of the application

In CRuby, one thread at a time executes Ruby code within a process. Threads still help, because a thread waiting on I/O (a database query, an HTTP call, reading a file) releases the lock, so another thread runs Ruby while the first waits.

Two consequences decide everything downstream:

  • Threads add throughput in proportion to the time your app spends waiting, and nothing more. An application that spends half its time in the database gets a lot from threads. One that spends its time rendering and computing gets very little.
  • Only workers add CPU parallelism. Sixteen threads in one process cannot use two cores. Two workers can.

The comment's warning about latency is the other half. Adding threads means more requests contending for one lock, so each individual request finishes later even as the process handles more of them per second. Throughput and latency are traded against each other, and which you want is a product decision more than a technical one.

Threads and the pool

One line in config/puma.rb reaches outside itself:

Any libraries that use a connection pool or another resource pool should be configured to provide at least as many connections as the number of threads. This includes Active Record's pool parameter in database.yml.

Every thread in a worker may be running a query at the same time, so a pool smaller than the thread count means threads queueing for connections. That appears as latency with no slow query behind it. Lesson 3 is entirely about that relationship, and about the arithmetic of the same repository's database.yml.

What this course covers

Scaling in this context is choosing values for two numbers, then repeating the process across machines.

  1. The two dials (this lesson).
  2. Sizing workers and threads.
  3. The connection pool, and other pools.
  4. Preload, copy-on-write, and memory.
  5. Restarts at the process level.
  6. Horizontal: beyond one machine.
  7. Putting it together: a configuration for this application.

Key takeaways

  • With nothing set, Puma runs one worker with three threads; WEB_CONCURRENCY and RAILS_MAX_THREADS change the two numbers.
  • Workers are processes with their own memory and pool; threads share a worker's memory and pool.
  • The GVL means one thread runs Ruby at a time per process, so threads add throughput only in proportion to time spent waiting on I/O.
  • Only more workers use more CPU cores; more threads trade latency for throughput.
  • A connection pool must be at least as large as the thread count, or threads queue for connections.
Lesson 1 of 7 · free

Keep going with lesson 2

Sizing workers and threads

Picks workers from the minimum of cores, memory and connections, and threads from the I/O share of requests.

A free account also gives you the quizzes, your progress and notes, and the discussion.

Type to start searching...