Where the time actually goes

This course is about a well-known trick: run the test suite against an in-memory SQLite database instead of a real one, and watch it get faster. The trick is real. Whether it helps you is a question with a measurable answer, and measuring first is the entire discipline.

So here is the measurement, taken from this repository.

The suite

$ bundle exec rspec --profile 8
Finished in 1 minute 11.31 seconds (files took 2.86 seconds to load)
1206 examples, 1 failure

Two runs landed at 70.3 and 71.3 seconds, with about 2.9 seconds of load each. (The one failure is a time-zone spec that depends on the local machine's tzdata, not on anything in this course.) So: 1,206 examples, roughly 70 seconds of running, and about 3 seconds of boot before the first example.

That averages about 58 milliseconds per example. Before reaching for any optimization, the question is what those milliseconds are made of.

The database round trip

adapter: PostgreSQL
SELECT 1 x2000, three runs: 0.062, 0.043, 0.061ms each
savepoint txn x200: 10.2ms, 0.051ms each

About 0.05 milliseconds per query. That is a local PostgreSQL over a Unix domain socket, and it is the number the whole in-memory argument rests on.

An in-memory SQLite database removes the socket, so it removes some fraction of 0.05ms per query. Suppose the suite issues fifty queries per example: a generous estimate for a suite with this many request specs. That is 1,206 × 50 × 0.05ms, or about 3 seconds out of 70. And the in-memory database does not remove all of it; it removes the transport, not the parse, the plan, or the Ruby-side result mapping.

flowchart TB
  T["~70s suite"] --> B["~3s boot: Rails, gems, Zeitwerk"]
  T --> E["~67s examples"]
  E --> S["~27s: 21 sandbox examples waiting out time limits"]
  E --> D["~3s database transport, generous estimate"]
  E --> R["the rest: Ruby, rendering, ActiveRecord, factories"]
  D --> W["what an in-memory database could touch"]

Where the slow examples are

Top 8 slowest examples (29.6 seconds, 41.5% of total time):
  10.01s  Execution::Sandbox stops code that blocks forever
   4.45s  Execution::Sandbox keeps what a fast submission printed while other threads hold the GVL
   3.02s  Execution::Sandbox cleanup after the grade kills a double fork
   3.02s  Execution::Sandbox cleanup after the grade kills a detached fork
   3.02s  Execution::Sandbox cleanup after the grade kills a detached spawn
   2.06s  Execution::GradeSubmissionJob stops code that never terminates instead of hanging the queue
   2.02s  Execution::Sandbox stops code that burns CPU forever
   2.00s  Enrollment::AwardBadgesService.call handles 'The Completionist' badge (max course completion)

Eight examples are 41.5% of the suite, and reading them tells you what each one is spending its time on. Seven of them exercise the grading sandbox. And most run code built to misbehave and wait for a time limit to stop it: the wait is the test. The badge example builds enough state to complete every course. Further down the group list, the authentication specs run BCrypt: deliberately slow, by design, and no database in any form makes it faster.

None of these is slow because of the socket to PostgreSQL.

The rule

Profile before optimizing is advice everyone repeats and few follow, because a suite has no obvious profiler and the intuition feels good enough. RSpec ships one:

$ bundle exec rspec --profile 8

That flag, and the two timings printed at the end of every run, are enough to answer the question this course asks. They cost nothing and they take one run.

The honest conclusion for this repository. An in-memory database would buy a few seconds of a seventy-second suite, and would cost the guarantees of the next four lessons. That is the answer here. It is not the answer everywhere, and lesson 6 is about the projects where it flips.

Key takeaways

• Measured: 1,206 examples, about 70 seconds, about 3 seconds of boot, roughly 58ms per example on average.
• Measured: about 0.05ms per PostgreSQL round trip over a local socket, 0.051ms for a savepoint transaction.
• A generous estimate puts total database transport near 3 seconds of the 70.
• The slowest examples wait out sandbox time limits, build large fixtures or run BCrypt: never the socket.
• --profile and the two lines RSpec already prints answer the question in one run.