What is mutation testing?
Mutation testing measures whether a test suite can detect small changes in program behavior. It changes one part of the code, runs selected tests, and then restores the original code.
Coverage and mutation testing answer different questions:
| Signal | Question it answers |
|---|---|
| Code coverage | Did a test execute this code? |
| Mutation testing | Would a test fail if this behavior changed? |
A test can run a line without checking its result. Mutation testing checks whether the tests that reach the line also detect a change to its behavior.
A small example
Section titled “A small example”Suppose production code includes this boundary:
return orderTotal >= freeShippingThreshold;A mutation changes >= to > and runs the selected tests. Tests for totals above and below the threshold may still pass. A test for an order exactly at the threshold would catch the change.
The mutation did not predict a specific bug. It showed that the current tests do not distinguish between the two boundary conditions.
The mutation loop
Section titled “The mutation loop”- Plan a mutation. Choose a small change, such as reversing a condition, changing a return value, or moving a boundary.
- Run selected tests. Run the tests chosen for that code against the mutation.
- Classify the result. If a test fails, the mutation is killed. If the selected tests all pass, it survives.
- Review survivors. Decide whether a test should protect the changed behavior.
Each mutation is evaluated independently. The original source remains the baseline for the next attempt.
What a survivor means
Section titled “What a survivor means”A surviving mutation is a prompt to investigate, not proof that the software is broken. It can point to:
- a missing assertion;
- an untested boundary, error path, or return value;
- behavior that the team does not need to test; or
- an equivalent mutant, where the source changes but the program behaves the same way.
Review the changed behavior in context. If it matters, add the smallest test that proves the expected result and rerun the affected file or Git diff. If it does not matter, do not add a test only to kill the mutation.
How Radforge uses mutation testing
Section titled “How Radforge uses mutation testing”Radforge finds mutation candidates in supported source files, selects tests for each candidate, and reports survivors as findings. The CLI builds the plan from your repository. Hosted runs use that plan and a snapshot of Git-tracked files without changing your working tree.
Mutation testing complements coverage and code review. It does not prove that software is correct or secure. A high mutation score is not useful if the added tests do not protect behavior that matters.