Catalog My learning Blog
Full-Stack Developer Course: React, .NET and SQL Server · Phase 15 · Lesson 148 · 8 min read

Contract, Mutation and Property-Based Testing

On this page
  1. Learning Objective
  2. Concept Explanation
  3. Worked Example: A Consumer Contract, a Mutation Configuration, Property Tests and Schema Fuzzing
  4. Real-World Examples
  5. Review Questions & Answers

Learning Objective

Use consumer-driven contract tests to keep the React client and the .NET API compatible, measure the strength of a test suite with mutation testing, find edge cases with property-based tests, fuzz an API from its OpenAPI document, and apply these techniques to authorization and security behavior.

Concept Explanation

The test pyramid of Day 71 gives the shape of a suite, and this lesson adds techniques that test the tests and explore inputs you did not think of. The first addresses the boundary between teams. When the React client and the API change independently, end-to-end tests are slow and fragile at catching drift, and mocks in the client tests can quietly diverge from the real API. Consumer-driven contract testing solves this. The consumer's tests describe the requests it makes and the responses it needs, in a contract file, using a tool such as Pact, which began at REA Group in Australia. The provider then replays each contract against the real API in its own pipeline. If a change to the API would break a field the client relies on, the provider's build fails before anything is deployed, while fields no consumer uses can be changed freely. It complements the OpenAPI checks of Day 25 by encoding what consumers actually need.

Coverage percentages measure which lines ran, not whether the tests would notice a bug. Mutation testing measures that directly: the tool makes small changes to your code, called mutants, such as changing a greater-than to greater-or-equal, removing a condition, or flipping a boolean, and runs your tests against each one. A mutant that the tests fail on is killed, and one that passes is a survivor, which points to behavior that no test checks. The mutation score is the percentage killed. Stryker provides this for .NET and for JavaScript and TypeScript. It is slow, so run it on the code that matters most, such as pricing, permission logic, and validation, and on changed files in pull requests. It is especially telling for authorization tests: if you can remove a permission check and every test still passes, the suite does not protect that rule.

Property-based testing flips how tests are written. Instead of choosing examples, you state a property that must hold for all inputs, and the tool generates hundreds of random inputs, then shrinks a failing one to a minimal example. Typical properties are round trips, such as decoding an encoded cursor returns the original values; invariants, such as splitting an amount into shares always sums to the total; and idempotence, such as applying an operation twice gives the same result as once. QuickCheck, created by Koen Claessen and John Hughes in 2000, introduced the idea, and FsCheck for .NET and fast-check for TypeScript bring it to this stack. These tests find boundary cases such as empty strings, extreme numbers, and Unicode that examples miss.

Finally, fuzzing throws malformed and unexpected input at the system. Tools such as Schemathesis read your OpenAPI document and generate requests that violate the schema in many ways, checking that the API never returns a server error, never leaks stack traces, and conforms to its documented responses, which finds validation gaps and crashes cheaply. For parsers and security-critical code, coverage-guided fuzzers go deeper. Apply these techniques to security behavior directly. Generate tests for every endpoint asserting that anonymous requests are rejected, that a user cannot access another tenant's resource, and that oversized or malformed input produces a 4xx and not a 5xx. Also manage flaky tests as bugs, quarantining and fixing them, since a suite that people distrust stops protecting anything.

TechniqueQuestion it answersTool examples
Contract testingWill this change break a consumer?Pact, with provider verification in CI
Mutation testingWould my tests notice a bug here?Stryker.NET, StrykerJS
Property-based testingDoes the rule hold for all inputs?FsCheck, fast-check
API fuzzingDoes bad input ever crash or leak?Schemathesis, coverage-guided fuzzers
Approval or snapshot testsDid the output change unexpectedly?Verify
Authorization matrix testsCan anyone do what they should not?Table-driven tests generated from endpoints

Worked Example: A Consumer Contract, a Mutation Configuration, Property Tests and Schema Fuzzing

The consumer test defines the contract. The remaining blocks show the mutation setup, properties in both languages, and a fuzz run.

JavaScript
// Consumer side (React app): the contract states what the client needs. Only these fields matter to it.
import { PactV3, MatchersV3 } from '@pact-foundation/pact';

const provider = new PactV3({ consumer: 'taskboard-web', provider: 'tasks-api' });

it('loads the tasks of a project', async () => {
  provider
    .given('project 1 has tasks')
    .uponReceiving('a request for the tasks of project 1')
    .withRequest({ method: 'GET', path: '/api/projects/1/tasks', headers: { Accept: 'application/json' } })
    .willRespondWith({
      status: 200,
      headers: { 'Content-Type': 'application/json' },
      body: MatchersV3.eachLike({ id: MatchersV3.integer(1), title: MatchersV3.string('Write docs'), state: MatchersV3.regex(/todo|inProgress|done/, 'todo') }),
    });

  await provider.executeTest(async (mock) => {
    const tasks = await loadTasks(mock.url, 1);          // the real client code, against a mock generated from the contract
    expect(tasks[0].title).toBeTruthy();
  });                                                    // writes pacts/taskboard-web-tasks-api.json
});
// Provider side (.NET, in the API pipeline): PactNet replays every contract against the real API with test data set up by "provider states".
// A rename of "title" to "name" fails that build, naming the consumer that depends on the old field.
Code
// stryker-config.json (Stryker.NET): mutate the rules that matter, and fail the build below a threshold.
{
  "stryker-config": {
    "solution": "TaskBoard.sln",
    "mutate": ["**/Domain/**/*.cs", "**/Authorization/**/*.cs", "!**/Migrations/**"],
    "thresholds": { "high": 90, "low": 75, "break": 70 },
    "reporters": ["html", "progress"],
    "since": { "enabled": true, "target": "main" }
  }
}
Java
// A survivor found by mutation testing: changing ">=" to ">" in the rule below did not fail any test.
public bool CanRedeem(Coupon c, DateTimeOffset now) => c.Uses < c.MaxUses && now <= c.ExpiresAt;
// The missing tests are the boundaries. Adding them kills the mutants:
[Fact] public void Cannot_redeem_when_uses_equal_max()  => Assert.False(rules.CanRedeem(Coupon(uses: 5, max: 5), Now));
[Fact] public void Can_redeem_at_the_exact_expiry()     => Assert.True(rules.CanRedeem(Coupon(uses: 0, max: 5, expires: Now), Now));

// Property-based test with FsCheck: shares of an amount always add up to the amount, whatever the inputs.
[Property]
public bool Split_never_loses_or_creates_cents(PositiveInt totalCents, PositiveInt parts)
{
    var shares = Money.Split(totalCents.Get, Math.Min(parts.Get, 1000));
    return shares.Sum() == totalCents.Get && shares.Max() - shares.Min() <= 1;      // fair to within one cent
}
JavaScript
// Property-based test with fast-check: a cursor survives a round trip, and garbage never crashes the decoder.
import fc from 'fast-check';

test('cursor round trip', () => {
  fc.assert(fc.property(fc.integer({ min: 1, max: 2 ** 31 - 1 }), fc.date({ noInvalidDate: true }), (id, at) => {
    expect(decodeCursor(encodeCursor({ id, at }))).toEqual({ id, at });
  }));
});

test('decoder rejects arbitrary input safely', () => {
  fc.assert(fc.property(fc.string(), (junk) => {
    const result = decodeCursor(junk);                     // must return a value or a controlled error, never throw an unexpected exception
    expect(result === null || typeof result === 'object').toBe(true);
  }));
});
Code
# Fuzz the running API from its OpenAPI document. Fails on 5xx responses, schema violations, and stack traces in bodies.
schemathesis run http://localhost:5080/openapi/v1.json \
  --checks not_a_server_error,response_schema_conformance,content_type_conformance \
  --header "Authorization: Bearer $TEST_TOKEN" --max-examples 200 --hypothesis-seed 42

# Security behavior as generated tests: for every operation, an anonymous call must be 401, never 200 or 500.
schemathesis run http://localhost:5080/openapi/v1.json --checks ignored_auth

A mutation score is a tool for finding gaps, not a target to game. Use the survivors as a prompt to ask what behavior the tests should be protecting, and write that test.

Contract tests replay what each consumer needs against the real API. Mutation testing changes the code in small ways and shows which changes no test notices. Property-based tests and OpenAPI fuzzing explore inputs no one thought of, including missing authentication and malformed data.
Contract tests replay what each consumer needs against the real API. Mutation testing changes the code in small ways and shows which changes no test notices. Property-based tests and OpenAPI fuzzing explore inputs no one thought of, including missing authentication and malformed data.

Real-World Examples

  • Pact was created at REA Group, an Australian property portal, around 2013 to solve the problem of testing many services that were developed by different teams, and it became a widely used open source tool for consumer-driven contracts. The approach is particularly valued where front ends and back ends are deployed independently, since it tells the provider precisely who depends on which fields.
  • QuickCheck was introduced in a 2000 paper by Koen Claessen and John Hughes, and its commercial successor was used to find subtle bugs in industrial systems, including race conditions in automotive software. It showed that properties plus generated inputs find classes of bugs that hand-written examples routinely miss, and the idea is now available in most languages.
Key TakeawaysAdvanced testing techniques find bugs that ordinary unit and integration tests miss in a React and .NET system. Use consumer-driven contracts to prevent breaking changes between React and the API, mutation testing to find behavior your tests do not check, especially permissions and business rules, and property-based tests to explore edge cases with round-trip and invariant properties. Fuzz the API from its OpenAPI document, generate authorization and malformed-input tests for every endpoint, and treat flaky tests as defects.

Review Questions & Answers

It records exactly which requests and fields each consumer actually uses, so the provider can verify that a change breaks no real consumer while still being free to change unused parts. An OpenAPI diff flags all differences whether or not anyone depends on them.

The tool changed the code in a small way and all the tests still passed, so no test checks that behavior. It points to a missing test, often at a boundary condition or a removed permission check.

It generates many varied inputs, including extreme and unusual ones, and shrinks failures to a minimal case. Stating a rule that must always hold covers cases the author did not think to write.

It sends many malformed and unexpected requests and checks that the API rejects them cleanly, never crashes or leaks internals, and never accepts unauthenticated calls. It finds validation gaps and missing authentication cheaply and repeatably.