Running Brakeman on this repository — Security audits: Brakeman

The previous course ended with an audit and a list of things a person has to look for. Brakeman is the part of that list a program can do: it reads your Rails source, follows data from user input to dangerous method calls, and reports what it finds.

It has already run against this repository. The report below is what bin/brakeman --no-pager produced the day this course was written, before the fixes in lessons 2, 3 and 7 shipped. Run the same command against this repository today and it reports zero warnings. The six below are kept because triaging a real report is the point, and this one is real.

bin/brakeman --no-pager

== Brakeman Report ==

Rails Version: 8.1.3.1
Brakeman Version: 8.0.4
Checks Run: BasicAuth, ..., SQL, ..., ValidationRegex, ... (79 checks)

== Overview ==

Controllers: 46
Models: 28
Templates: 82
Errors: 0
Security Warnings: 6

== Warning Types ==

Format Validation: 3
SQL Injection: 3

Six warnings, in about a second, across a hundred and fifty files. Lessons 2 and 3 work through them one by one. By the end of this course you will know which ones were real, and you can confirm for yourself that the same command reports zero today.

That is the whole shape of using this tool: it produces a list, and the value comes from what you do with each entry.

What it is

Brakeman is a static analyzer: it reads code without running it. That is what makes it fast and what determines everything it can and cannot do.

Because it does not run the code	Consequence
It needs no database, no server, no test data	It runs in a second, on every pull request
It sees every branch, including ones tests never reach	It finds bugs a passing test suite hides
It cannot know runtime values	It reports what might be user input, not what is

That last row is the source of every false positive you will ever see from it, and the reason the tool reports a confidence level instead of a verdict.

flowchart LR
  S["Your source"] --> P["Parse to a syntax tree"]
  P --> T["Track data flow: params, cookies, model attributes"]
  T --> C["79 checks: does untrusted data reach a dangerous call?"]
  C --> W["Warnings, with a confidence level"]
  W --> H["A person decides: real, or not?"]

How it decides something is dangerous

Brakeman follows two things and looks for them meeting.

Sources of untrusted data: params, cookies, request values, and model attributes that were set from any of those. It propagates through assignments and method calls, which is why it can flag a value three methods away from the controller that received it.

Dangerous sinks: SQL string fragments, render inline:, send, eval, system, redirect_to, raw and html_safe, constantize, File.read. Each sink has a check that knows what safe use looks like for it.

A warning is a path from a source to a sink that Brakeman could not prove safe. The phrasing matters ("could not prove safe", not "is unsafe"), and it is the correct way to read the entire report.

Confidence

Level	What it usually means
High	The pattern is unambiguous: a known-bad construct
Medium	User input probably reaches this, and the tool cannot be sure
Weak	A pattern worth a look, often fine

This repository's six original warnings were three High and three Medium, and that split was itself informative: the High ones are a definite pattern in a validation, and the Medium ones are a query object doing something Brakeman cannot follow. Different confidences, different kinds of work.

A high-confidence warning is not automatically a real vulnerability, and a weak one is not automatically noise. Confidence describes how sure the tool is about the pattern, not how exploitable the result is. That distinction is lesson 4's subject.

What it cannot find

This needs saying early, because a tool that runs in CI invites the belief that it is the security review:

Missing authorization. The whole of A01 from the previous course. A missing check is code that was not written, and there is nothing to analyze.
Business logic flaws. A discount that can be applied twice is a bug in your rules, and Brakeman has no idea what your rules are.
Vulnerable dependencies. That is bundle audit; Brakeman analyzes your code.
Anything at runtime. Configuration in production, an open port, a leaked key.

Brakeman checks for one class of mistake, untrusted data reaching a dangerous call, and it is good at that class. Treating it as coverage of everything is how a team gets a green CI badge and an unaudited application.

What this course covers

Running Brakeman on this repository (this lesson).
The three format-validation warnings.
The three SQL-injection warnings.
Confidence, and triage.
Living with it: CI, baselines and ignore files.
Beyond Brakeman.
Putting it together: a security workflow.

Key takeaways