Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Reproducibility

Every scenario run must be reproducible from its inputs. This repository treats nondeterminism as a defect, not a feature.

What makes a run reproducible

A run is fully determined by:

  • scenario id + version — the definition is code or a validated declarative document (the semantic validator runs before execution),
  • fixture catalog — embedded, deterministic, synthetic,
  • environment configuration — the target Environment,
  • seed — every outcome records the seed it ran with; scenarios never consult system randomness silently,
  • clock — the mock harness injects a fixed clock, so phase timings and outcome stamps are byte-deterministic,
  • the invariant registry — attached invariants are named and counted on the outcome.

Where randomness is allowed

Only in the fuzz targets, and only explicitly: each fuzz run takes a seed, records it on every finding, and replays the same iterations for the same seed. A fuzz finding is reducible to a deterministic regression scenario (CT-REG-*) via the reduction module, satisfying spec §26’s “finding → minimal deterministic regression test” pipeline.

Enforcement

  • report and run outputs are byte-stable: parallel execution returns outcomes re-sorted into registry order, so --parallel reports match serial reports exactly (spec §71).
  • The reporting crate asserts its JSON/JUnit/Markdown renderers are byte-identical across iterations.
  • The concurrency bench asserts parallel outcomes equal serial outcomes.
  • Test vectors are deterministic by construction and validated for coherence.

Replaying a failure

A failing run is captured as a classified ScenarioOutcome (never a panic), carrying the seed, environment, observations, and failure category. Re-run with the same selection and seed to reproduce; file a regression issue referencing the seed and scenario id (see the issue templates).