Reproducibility
Every scenario run must be reproducible from its inputs. This repository treats nondeterminism as a defect, not a feature.
What makes a run reproducible
A run is fully determined by:
- scenario id + version — the definition is code or a validated declarative document (the semantic validator runs before execution),
- fixture catalog — embedded, deterministic, synthetic,
- environment configuration — the target
Environment, - seed — every outcome records the seed it ran with; scenarios never consult system randomness silently,
- clock — the mock harness injects a fixed clock, so phase timings and outcome stamps are byte-deterministic,
- the invariant registry — attached invariants are named and counted on the outcome.
Where randomness is allowed
Only in the fuzz targets, and only explicitly: each fuzz run takes a seed,
records it on every finding, and replays the same iterations for the same
seed. A fuzz finding is reducible to a deterministic regression scenario
(CT-REG-*) via the reduction module, satisfying spec §26’s
“finding → minimal deterministic regression test” pipeline.
Enforcement
reportandrunoutputs are byte-stable: parallel execution returns outcomes re-sorted into registry order, so--parallelreports match serial reports exactly (spec §71).- The reporting crate asserts its JSON/JUnit/Markdown renderers are byte-identical across iterations.
- The concurrency bench asserts parallel outcomes equal serial outcomes.
- Test vectors are deterministic by construction and validated for coherence.
Replaying a failure
A failing run is captured as a classified ScenarioOutcome (never a
panic), carrying the seed, environment, observations, and failure
category. Re-run with the same selection and seed to reproduce; file a
regression issue referencing the seed and scenario id (see the issue
templates).