Splitting tables, not rows

Free preview 3 min to read

Vertical sharding moves whole tables to another database. Horizontal sharding splits one table's rows, and vertical sharding splits the set of tables.

The two axes

Horizontal Vertical
Splits Rows of one table Which tables live where
Each database has The same schema, different rows A different schema, all its rows
Grows with More rows than one machine holds More load than one machine handles
Reversible Barely: rows must be moved live Often: move the tables back

The last row is the one that makes vertical sharding worth learning first. A horizontal shard key is a decision measured in quarters, but moving a table to another database and back is a migration and a deploy.

flowchart TB
  H["horizontal"] --> H1["users 1-1000 here, 1001-2000 there"]
  H1 --> H2["same tables, different rows"]
  V["vertical"] --> V1["users and courses here, events there"]
  V1 --> V2["different tables, all their rows"]
  V2 --> V3["reversible: it is a migration, not a live row move"]

Why it's usually tried first

The seams are already in your schema.

A schema isn't a uniform mesh. Some tables are densely connected to each other and barely connected to the rest. Those boundaries with few connections are candidate split points, and you don't have to invent them. Lesson 3 measures them in this application.

Horizontal sharding, by contrast, requires inventing a key and then living with it.

What it buys

It gives independent capacity to a workload that behaves differently from the rest.

The usual candidates can be recognized by their shape. Some are write-heavy tables that everything else only reads, such as events, logs and audit trails. Some grow without bound, such as analytics. Others have a completely different access pattern, such as a queue polled ten times a second.

Moving those off the main database gives them their own I/O, their own vacuum schedule and their own backup policy. It also stops their growth from crowding the tables that serve your users.

What it doesn't buy

It doesn't help a single table that's too big. If one table dominates, moving its neighbors away leaves it just as big and just as busy. That's a horizontal problem, and vertical sharding can't touch it.

It also doesn't multiply write capacity for the same table. Each table still lives on one machine. You've only chosen which machine.

The constraint that comes with it

Once two tables live in different databases, you can't join them and you can't wrap them in one transaction.

That's the same constraint as horizontal sharding, arriving along a different axis. It's why the seam matters so much: a good seam is one that no query wanted to cross anyway.

Key takeaways

  • Horizontal sharding splits rows, and vertical sharding splits which tables live where.
  • Vertical is usually reversible: a migration and a deploy, not months of moving live rows.
  • The seams already exist in your schema, as regions where tables have few connections to each other.
  • It gives a workload that behaves differently its own I/O, vacuum and backup policy.
  • It can't help one oversized table, and once tables split you lose joins and transactions across the seam.
Lesson 1 of 7 · free

Keep going with lesson 2

You are already doing it

Shows how Rails 8 already splits this application into primary, queue, cache and cable roles, and why the cache moved.

A free account also gives you the quizzes, your progress and notes, and the discussion.

Type to start searching...