Tests that bite
Mutation testing and other ways to catch tests that pass no matter what.
Agents will write tests that pass whatever the code does. Rule 26 says every test has to prove it's worth keeping, and the proof is a break it catches. Mutation testers make those breaks on purpose. Snapshot and approval tools pin down output a person has looked at once, and the health tools go after the flaky and the slow.
Show tools for
59 tools
Mutation testing
Ours
flinch
No pull request merges while code the Contract covers can break unnoticed.
Start here
Bats-core
Test framework for Bash scripts and Unix programs.
Shell scripts that agents write get a harness with TAP and JUnit output.
Start here
cargo-mutants
Breaks Rust code in small ways and reports which breaks the test suite missed.
Shows which agent-written Rust tests pass whatever the code does.
The maintainer calls it a semi-actively maintained spare-time project.
Start here
Infection
PHP mutation testing for PHPUnit and Pest, with minimum-score thresholds.
The clear PHP choice, and its MSI threshold makes a merge gate easy.
Start here
Mull
LLVM-based mutation testing for C and C++.
Native-code suites get the same escaped-mutant check other stacks have.
Start here
mutaml
Mutation testing for OCaml through a ppx rewriter.
Fills the OCaml gap in mutation tooling.
Start here
Muter
Mutation testing for Swift and Xcode projects.
One of the few ways to check whether agent-written iOS tests would notice a break.
Start here
Mutineer
Mutation testing for Ruby on Prism, with Minitest and RSpec support and no runtime dependencies.
Scores only the lines a pull request changed, and its kill matrix names the tests that never catch anything.
Runs in-process on Ruby 3.4 or later. Apps on older Ruby can run their suite through --test-command.
Start here
mutmut
Python mutation testing with incremental runs and a terminal UI.
Easy to run on pytest suites that agents grow quickly.
Start here
PIT (pitest)
Bytecode mutation testing for the JVM with Maven, Gradle and Ant plugins.
The default check for whether agent-written JUnit tests catch real breaks.
Start here
Stryker.NET
Mutation testing for C# projects, run through dotnet test.
Shows which C# tests pass whatever the code does, a common trait of tests written just to go green.
Start here
Stryker4s
Mutation testing for Scala through sbt and Maven.
Brings a mutation score to Scala suites that agents extend.
Start here
StrykerJS
Mutation testing for JavaScript and TypeScript with runners for Jest, Vitest, Mocha and Karma.
Gives agent-written JS tests a mutation score to gate on in CI.
-
Arcmutate
Commercial PIT extensions for Kotlin and Spring, plus pull request analysis.
Mutation runs only on the changed lines of a PR, which keeps merges quick when agents push often.
-
Cosmic Ray
Python mutation testing that can spread work across many machines.
Suits large Python suites where one machine takes too long.
-
Dextool
libclang tool suite with a mutation testing plugin for C and C++.
An option beside Mull for embedded and safety-critical C code.
-
go-mutesting
Go mutation testing maintained by Avito as a fork of the original zimmski tool.
A long-standing Go option with a different engine from flinch.
The original zimmski/go-mutesting is stale, and this fork carries the work.
-
gremlins
Go mutation testing for small modules and microservices, with CI-friendly output.
Thresholds let you fail a build on a weak mutation score.
Still 0.x, so flags may change between releases.
-
Major
Research mutation framework that hooks into the Java compiler.
Mostly used in studies. It gives a mutation score that is independent of PIT.
Distributed as a zip download, and no public repo or license was found.
-
Mutahunter
Asks an LLM to write realistic mutants for any language, then runs your tests.
LLM mutants can reach bugs that operator-swap mutation misses.
-
mutant
Ruby mutation testing that works with RSpec and Minitest.
The mature Ruby option for proving agent-written specs fail when behavior changes.
Free for open source. Commercial use costs $30 a month or $250 a year per developer.
-
ooze
Go mutation testing that runs as an ordinary go test.
Slots into the existing test command, so there is no new tooling to learn.
-
PIT Descartes
PIT engine that replaces whole method bodies, giving fewer and faster mutants.
A cheaper pass for large Java suites where full runs take too long.
-
Stryker Dashboard
Hosts mutation reports and score badges for any tool that emits the shared report schema.
Publishes mutation scores per branch, which backs rule 29.
-
universalmutator
Regex-based mutation generator that works on nearly any source language.
A fallback when no mutation tool exists for your stack.
Approval and snapshot tests
Start here
insta
Snapshot testing for Rust with inline snapshots and a review CLI.
Snapshots pin current behavior, and a reviewer has to read each diff because an agent can accept a bad one.
Start here
Mneme
Snapshot-style auto assertions for ExUnit that write the expected value into the test.
Brings an inline approval workflow to Elixir tests.
Start here
tasty-golden
Golden test support for the Tasty framework in Haskell.
The usual way to pin output in Haskell tests.
-
Approval Tests (ApprovalTests.Java)
Approval testing for Java that compares output against a stored approved file.
Pins what legacy code does today, so a change an agent makes by accident shows up as a diff.
-
Approvals.NodeJS
Approval testing for Node with adapters for the common test runners.
Approved files sit in plain view, which suits reviewers who find snapshot files too hidden.
-
ApprovalTests.cpp
Approval testing for C++ that works with the major C++ test frameworks.
Gives C++ code with no tests a characterization suite before an agent starts changing it.
-
ApprovalTests.Python
Approval testing for Python with scrubbers and a pytest-friendly API.
Freeze the output of code an agent is about to refactor, then see exactly what moved.
-
expect-test (Rust)
Inline expected-output assertions for Rust that update themselves when you set an environment variable.
Expected values live in the source file, so reviewers see them in the diff.
-
go-snaps
Jest-style snapshot testing for Go.
Gives Go tests a stored-output check with an update flag.
-
goldie
Golden file testing for Go with an update flag.
Suits CLI and template output where one file per case reads best.
-
inline-snapshot
Python snapshots written into the test source, with commands to fix or create them.
Expected values land in the diff, so a changed snapshot gets seen in code review.
-
Jest snapshots
Built-in snapshot assertions in Jest, stored as files or inline.
An agent can accept a snapshot update that nobody reads, so these need a review rule.
-
ppx_expect
Inline expect tests for OCaml that promote new output into the source file.
Jane Street's way to keep expected output visible in review.
-
Smoke
Runs commands and compares stdout, stderr and exit code to expected files.
Golden-master tests for any CLI, whatever language it is written in.
-
spatie/phpunit-snapshot-assertions
Snapshot assertions for PHPUnit.
Pins output in PHP tests with files a reviewer can read.
-
syrupy
pytest snapshot plugin with pluggable serializers and file-per-snapshot output.
Pins structured output in pytest with no hand-written expected values.
-
Verify
Snapshot testing for .NET that stores verified files next to the tests.
Diffs of agent changes become plain files a reviewer can open.
Test suite health
Start here
diff-cover
Reports coverage on only the lines a diff changed, from any XML coverage report.
Lets you require coverage on new lines with no demand on the whole repo.
Start here
Trunk Flaky Tests
Detects flaky tests from CI results and quarantines them so they stop blocking merges.
When green merges itself, quarantine keeps flakes from draining the meaning of green.
-
BuildPulse
Finds and quarantines flaky tests from CI runs, with an MCP server for AI assistants.
An agent can read flake data through the MCP server before it touches a test.
-
cargo-llvm-cov
Rust coverage using LLVM, with region and branch coverage options.
Goes past line coverage for Rust.
-
cargo-nextest
Faster Rust test runner that retries failures and marks tests as flaky.
Separates flaky from broken so agents stop chasing noise.
-
CloudBees Smart Tests
Machine-learning test selection and flaky detection, formerly sold as Launchable.
Cuts test time on big suites while keeping the tests most likely to fail.
Formerly Launchable, which CloudBees acquired in August 2024.
-
Codecov
Hosted coverage with patch coverage checks on each pull request.
Patch coverage is the diff-level gate for rule 29.
-
Codecov Test Analytics
Tracks test run time and failure rates from JUnit XML and flags tests known to be flaky on main.
Tells a PR author whether a red test is a known flake.
-
Datadog Test Optimization
CI test monitoring with flaky test management, early flake detection and test impact analysis.
Retries new tests on first appearance, so agent-added tests get a flake check at once.
-
Develocity Predictive Test Selection
Learns from build history to run only the tests relevant to a change.
Keeps suites fast as agents add tests, and flaky or recently failed tests always run.
-
gotestsum
Go test runner wrapper that reruns failed tests and prints a failure summary.
Its rerun-fails mode records which tests flake.
-
JNose
Detects 21 test smells in JUnit tests and collects JaCoCo coverage.
Flags weak patterns such as empty tests and missing assertions that agents tend to produce.
-
Knapsack Pro
Splits test suites across CI nodes by recorded run time.
Speed matters when agents push many small changes.
-
pytest-randomly
Shuffles test order and reseeds random generators on every run.
Exposes hidden order dependence in tests that were written one at a time.
-
pytest-testmon
Runs only the tests affected by your code changes, tracked per test and method.
Keeps the local loop quick as agent-written suites grow.
-
test-prof
Profilers and factory tools that find slow and wasteful tests in RSpec and Minitest.
Agents add tests fast, and this shows which ones cost the most time.
-
undercover
Warns when changed methods lack test coverage, using git diff and a coverage report.
Fails a PR whose new code has no tests, which is rule 29 at the diff level.
Nothing in this section is built for the stacks you picked, and nothing here works with any language. Pick another stack, or .
Gone quiet 6
These tools are archived or shut down, or nobody has released or committed to them in two years. Some still work fine. Read the note before you adopt one.
-
cupaloy
Go snapshot testing library with custom differs.
A smaller option than go-snaps, though it has been quiet since 2023.
Dormant since May 2023.
-
Exavier
Open source mutation testing library for Elixir.
The permissively licensed Elixir option next to Muzak.
No activity since 2020.
-
mutagen
Earlier Rust mutation tester built on compiler plugins.
Listed so Rust users know to pick cargo-mutants.
Dormant since 2023. Use cargo-mutants.
-
mutatest
Small Python mutation tester that changes operators and runs your tests.
Quiet since 2023, so mutmut is the safer pick.
Dormant since 2023.
-
Muzak
Mutation testing for Elixir run through mix.
The best-known Elixir option, though its last release came in 2022.
Last release was December 2022, and the CC-BY-NC-ND license limits commercial use.
-
pytest-flakefinder
Runs each test many times to surface flaky ones.
A quick flake check for a new agent-written test.
Dormant since 2022.
Missing a tool that belongs here? Suggest it.
Every tool here was checked against its GitHub repository or its own site on 6 October 2026. Stars and dates come from GitHub on that day.