The Contract
Specs that run, so agents get judged against something written down.
Rule 13 says to write the Contract. Rule 20 says to make it executable, and rule 29 says to publish how much of it the code meets. These tools turn a written spec into something a machine can run against the code. The Cucumber family has done this for twenty years, and a newer crop of tools hands the spec straight to a coding agent.
Show tools for
65 tools
Specs that run
Start here
Behat
Gherkin BDD framework for PHP with Symfony and Laravel extensions.
Feature files pin PHP behavior, and the agent has to keep them green.
Start here
Behave
Gherkin feature files with Python step definitions.
CI runs a plain-language contract on every agent change in a Python project.
Start here
Cucumber
Plain-language Given/When/Then scenarios in Gherkin that run as tests against the app.
A Gherkin contract reads cleanly to people and runs on every change, so agent output gets judged against it.
Start here
Cucumber-JVM
Cucumber for Java and other JVM languages, with JUnit Platform integration.
JVM teams run Gherkin contracts in the same build as their unit tests.
Start here
cucumber-rs
A native Rust implementation of Cucumber with async step functions.
Feature files stay executable beside cargo test in Rust projects.
Start here
Godog
The official Cucumber implementation for Go, run through go test.
Go services state their contract in Gherkin and fail the build when an agent drifts from it.
Start here
Pact
Consumer-driven contract tests: consumers record expectations and providers verify them.
When different agents change each side of a service boundary, the contracts get enforced mechanically.
Start here
Quick
Describe/it style behavior specs for Swift and Objective-C, paired with Nimble matchers.
Behavior-style specs give iOS agents a shape to satisfy.
Start here
Reqnroll
Open-source Gherkin BDD framework for .NET, the community successor to SpecFlow.
Executable Given/When/Then contracts that an agent has to satisfy before a change merges.
Start here
Schemathesis
Reads an OpenAPI or GraphQL schema and generates requests that find crashes and spec violations.
The API schema becomes an executable Contract, and the tool tests a server written in any language.
-
byexample
Runs code examples in Markdown and text files as regression tests.
Docs in many languages turn into tests, so prose cannot claim behavior the code lost.
GPL-3.0 licensed, which matters if you embed it.
-
Cabbage
Compiles Gherkin feature files into ExUnit tests for Elixir.
Elixir teams run feature-file contracts through the test runner they already use.
Releases have slowed, with the last activity in March 2025.
-
Concordion
Specification by example in Markdown or HTML, with instrumented prose that runs as tests.
The spec is an ordinary document with checks embedded, so prose and proof share one file.
Updates have slowed.
-
Cucumber-CPP
Lets Cucumber run step definitions written in C++ over a wire protocol.
Native code gets Gherkin-level acceptance checks without a rewrite in another language.
Needs a Ruby Cucumber runner.
-
Elixir ExUnit doctests
Runs iex-style examples in @doc strings as tests.
Module documentation doubles as a checked specification.
-
FitNesse
A wiki where acceptance tests are tables that run against the system.
Non-programmers write expected values in wiki tables and the machine checks them.
-
Gauge
Markdown specifications that run as tests, with plugins for many languages.
Specs live as plain Markdown, which agents and reviewers can both read and edit.
-
Hurl
HTTP requests and assertions written in plain text files and run from a single binary.
An API contract in a readable text file that CI runs and reports on.
-
Karate
Gherkin-style API, mock, performance and UI tests in one framework.
API behavior is specified once and reused for functional and load checks.
-
Prysk
A maintained fork of cram for testing command-line tools via shell transcripts.
Keeps CLI transcript tests alive now that cram has stalled.
-
pytest-bdd
Gherkin scenarios run inside pytest, sharing fixtures with the rest of the suite.
Feature-file contracts sit beside ordinary pytest tests, so one runner and one coverage report cover both.
-
Python doctest
Runs interactive examples embedded in docstrings and text files and checks their output.
Doc examples fail the build when the code changes, which keeps agent-written docs honest.
-
Robot Framework
Keyword-driven acceptance testing with tabular and Gherkin-style test files.
Keyword libraries turn acceptance criteria into checks that run against web, API and embedded targets.
-
Rust doc tests
Code blocks in doc comments compile and run as part of cargo test.
Every documented example is checked on each change, so API docs cannot drift unnoticed.
-
Specmatic
Turns OpenAPI, AsyncAPI, GraphQL and gRPC specs into executable contract tests and stubs.
The API spec is the contract, so agent code is tested against it and dependencies are stubbed from it.
-
Spock
Groovy test framework whose given/when/then blocks read as specifications.
Data-driven where-blocks give agents a compact table of expected behavior to meet.
-
testscript (go-internal)
Shell-like test scripts with txtar archives, extracted from the Go toolchain's own tests.
CLI behavior becomes a plain script with fixtures inline, easy for agents to read and extend.
-
Varar
Turns plain Markdown examples into tests so documentation fails when behavior drifts.
Docs written for agents double as the contract, with one conformance corpus across eight languages.
Spec-driven agent tools
Start here
GitHub Spec Kit
CLI and templates that move agents from spec to plan to tasks to code.
The spec files and task files sit in version control and give the agent a written contract to build from.
-
Agent OS
Extracts coding standards from a codebase and injects them into agent specs.
Standards become written inputs that shape every spec an agent works from.
-
Augment Cosmos
Cloud agent platform with living specs and shared memory for team-scale delivery.
Keeps specs updated as agents change code, which suits teams spread across many repos.
-
BMAD-METHOD
Role-based agent workflow from idea through PRD, architecture and stories to code.
Heavy up-front documents give agents detailed intent and cost the team some ceremony.
-
cc-sdd
Spec-driven harness that turns approved specs into long-running implementation runs with independent review.
An independent review step and auto-debug check each spec-driven run, across eight different agents.
-
Claude Task Master
Parses a PRD into dependency-ordered tasks that agents work through, via CLI and MCP.
A written requirements doc becomes a task graph an agent can walk in order.
Licensed MIT with the Commons Clause, which restricts resale and hosting.
-
CodeMySpec
Enforces a behavioral spec as a gate, then verifies the running app.
A failing behavioral spec blocks the merge, which makes the spec a gate.
The site now describes it as AI harnesses that help one person build, sell and support software.
-
GSD (Get Shit Done)
Context-engineering and spec-driven workflow for Claude Code and other agents.
Work splits into fresh-context phases that follow written plans.
The original repo was archived on 2026-06-26 and the project continues as GSD Core.
-
Kiro
AWS agentic IDE and CLI that turns prompts into requirements, design and tasks, with property-based checks.
Specs in EARS notation and property-based tests give a checkable contract inside one product.
-
OpenSpec
Lightweight spec framework: propose a change, write spec deltas, apply, then archive.
Delta-style specs fit existing codebases, where the contract grows one change at a time.
-
Pimzino claude-code-spec-workflow
Claude Code commands for requirements, design, tasks and implementation, plus a bug-fix flow.
A small example of the spec-first loop built only on Claude Code commands.
Updates have slowed since September 2025.
-
Tessl
Platform for agent skills, policy checks and review loops, started as a spec-centric framework.
A skills registry plus governance tools lets a team enforce shared standards on agent output.
API contracts
Start here
buf
Lints Protobuf files and detects breaking changes against an earlier version.
A breaking proto change fails the check, so an agent cannot quietly ship an incompatible schema.
The Buf Schema Registry is a paid product.
Start here
GraphQL Inspector
Diffs GraphQL schemas, finds breaking changes and validates documents against a schema.
Fails the build when a schema edit would break queries that clients already send.
Start here
oasdiff
Compares two OpenAPI specs and lists changes that would break existing clients.
One command fails the pull request when an agent changes the public API in an incompatible way.
-
Ajv
Validates JSON against JSON Schema in tests and at runtime.
Schemas checked in CI catch a payload shape change before an agent's edit reaches production.
-
Apollo GraphOS schema checks
Checks proposed GraphQL schema changes against real client traffic before they ship.
Real usage data tells you whether a change breaks a client, which an agent has no way to guess.
Custom, proposal and contract checks need the Enterprise plan.
-
AsyncAPI tooling
Spec, CLI, validator and code generators for event-driven APIs described in AsyncAPI.
Gives message-based services a written contract that tests and generators can both use.
-
Microcks
Mocks and contract-tests APIs from OpenAPI, AsyncAPI, gRPC and GraphQL specs.
One spec feeds consumer mocks and provider tests, so both sides of a change get checked.
-
openapi-diff
Compares two OpenAPI specs and reports incompatible changes.
A JVM-friendly option for the same breaking-change gate that oasdiff provides.
-
PactFlow
A hosted Pact broker with can-i-deploy checks and bi-directional contract testing against OpenAPI.
Shows each service's contract status in one place, so a deploy gate can block an incompatible release.
Owned by SmartBear, with a free starter plan.
-
Postman CLI / Newman
Runs Postman collections and tests from the command line in CI.
Teams with collections already built get an API regression gate without writing new tests.
Newman releases have slowed in favor of Postman CLI.
-
Prism
Mocks an API from its OpenAPI file and validates requests and responses against it.
Frontend agents can build against the spec before any backend exists.
-
Spectral
Lints OpenAPI, AsyncAPI and JSON documents against custom rulesets.
House API style becomes rules that fail the build, so agent output stays inside the standard.
-
Spring Cloud Contract
Generates provider tests and consumer stubs from contracts written in Groovy, YAML or Kotlin.
Contracts generate the provider tests, so an agent changing either Spring service faces the same check.
The Spring team moved the project out in July 2026. It continues as Stubborn Contract under Stubborn.sh.
-
Vacuum
A fast OpenAPI linter that reads Spectral rulesets and adds its own rules.
Large specs lint quickly in CI, and existing Spectral rulesets carry over.
-
WireMock
Mocks HTTP APIs with stubs and verifies the requests it receives.
Pins third-party API behavior so agent-written code is tested against a fixed response.
Traceability and contract coverage
-
Allure TestOps
Test management platform that maps requirements to automated tests and reports.
Publishes coverage per requirement from the test runs a team already has.
-
Doorstop
Requirements as YAML files in git, linked by ID, with checks for broken links.
Requirements live beside the code and diff cleanly, so an agent can edit them in a pull request.
-
Jama Connect
Commercial requirements management with live traceability and impact analysis.
Regulated teams need audited links from requirement to test, whoever wrote the code.
-
OpenFastTrace
Traces requirements from Markdown or RST specs to code and tests, and reports gaps.
The build fails when a requirement has no covering test or code, which gives you contract coverage you can publish.
-
Reqflow
Checks coverage between requirement documents and design or test documents.
A light coverage check for document-based requirement chains.
-
sphinx-needs
Sphinx extension that defines requirements, specs and tests as linked objects with filters and charts.
Builds traceability tables and coverage charts straight from the docs in the repo.
-
StrictDoc
Requirements and docs in a plain-text format, with a web editor and source-code traceability.
Links requirements to source files and tests, and exports reports for audits.
-
Tracey
Spec coverage tool that links annotated rules in markdown specs to code references.
Reports which spec rules have an implementation and a test.
-
ubCode
Language server and indexer from useblocks for needs-based documentation.
Checks across large requirement sets run fast and come with editor support.
It builds on Sphinx-Needs, which remains the open source base.
-
Xray (Jira)
Jira test management that links requirements to tests, with Cucumber import and coverage reports.
Shows requirement coverage inside the Jira tracker where the specs already live.
Now sold as Xray by Sembi, formerly Xpand IT.
Nothing in this section is built for the stacks you picked, and nothing here works with any language. Pick another stack, or .
Gone quiet 6
These tools are archived or shut down, or nobody has released or committed to them in two years. Some still work fine. Read the note before you adopt one.
-
cram
Tests for command-line programs written as shell transcripts in text files.
A transcript is the plainest contract for a CLI, and a diff shows drift at once.
Dormant since 2021. Prysk is the maintained fork.
-
Dredd
Checks a running API against its API Blueprint or OpenAPI description.
Older pipelines still run it, and Schemathesis or Specmatic cover the same job with active maintenance.
The owner archived the repo on 2024-11-08.
-
Optic
Diffed OpenAPI specs in pull requests and flagged breaking changes.
Existing users should plan a move to a maintained diff tool such as oasdiff.
Archived on 2026-01-12 after Optic Labs joined Atlassian.
-
SpecFlow
The original Gherkin BDD framework for .NET, now superseded by Reqnroll.
Many .NET codebases still carry SpecFlow feature files that can move to Reqnroll with few changes.
Tricentis ended SpecFlow on 2024-12-31. Reqnroll is the successor.
-
Step CI
Tests REST, GraphQL, gRPC and other APIs from YAML workflows.
Declarative API checks run in CI without a test framework.
Little activity since August 2024.
-
Testing Steps
Acceptance tests written as ordered lists of steps with expectations.
A lighter spec format that reads like manual test steps and still runs.
Missing a tool that belongs here? Suggest it.
Every tool here was checked against its GitHub repository or its own site on 6 October 2026. Stars and dates come from GitHub on that day.