Two philosophies, one job: Redis or the database

Free preview 3 min to read

A queue needs somewhere to live

A background job is a promise to do something later: send an email, fetch an image, recalculate a stat. Between the moment your code says "do this later" and the moment a worker runs it, that promise has to sit somewhere durable enough to survive a deploy, a crash or a restart. Two production-proven answers dominate the Ruby ecosystem, and they disagree about where that somewhere should be.

Sidekiq, the gem most Rails developers picture when someone says "background jobs", keeps it in Redis, an in-memory key-value store your app talks to over the network. Solid Queue, the Rails 8 default, keeps it in the relational database (PostgreSQL, in this app) that your models already read and write on every request. Same job, two places to keep the promise while it waits.

This app runs Solid Queue only. grep -i sidekiq Gemfile Gemfile.lock returns nothing, and a grep for redis across the Gemfile, the lockfile and every YAML file in config/ comes back empty too. There's no Redis connection anywhere in this codebase to point Sidekiq at, even if the gem were added tomorrow. The earlier course "Solid Queue: background jobs" already traced a job through this app's Solid Queue tables, its config/queue.yml and its queue connection. None of that is repeated here.

Instead, this course builds a picture of Sidekiq's architecture (its API, its Redis dependency, its retries and its Web UI) and holds each piece up against what the earlier course showed about this app's Solid Queue setup. Claims about Sidekiq come from Sidekiq's own README, wiki and source, checked before they're stated. Claims about this app come from its code, re-checked for this course. Neither gem is "the right one" in the abstract. They're two answers to the same problem, and by the end you'll know which tradeoffs each one makes, beyond its reputation.

What this course covers

Lesson 2 compares how a job class looks in each system, and why this app's jobs would survive an adapter swap unchanged. Lesson 3 covers where queued work lives, a Redis structure or a PostgreSQL row, and what that costs to operate. Lesson 4 compares Sidekiq's automatic retries with retry_on, which this app declares on some jobs and not others. Lesson 5 covers how each system runs the work: threads in one process, or dispatcher and worker processes. Lesson 6 compares Sidekiq's Web UI with this app, which has no jobs dashboard. Lesson 7 puts the evidence side by side, including Rails' own account of why Solid Queue exists and Sidekiq's published benchmarks, for an actual decision.

Lesson 1 of 7 · free

Keep going with lesson 2

Writing the job: Sidekiq::Job vs this app's ApplicationJob

Compares Sidekiq's native perform_async API with this app's ActiveJob classes and what an adapter swap would require.

A free account also gives you the quizzes, your progress and notes, and the discussion.

Type to start searching...