What a production-grade RL environment spec looks like

Most of the roughly 99 tracked environments publish no verifier calibration and no frozen corpus. The ones labs pay for look different. A reference environment contract we examined runs 572 lines and fixes state machine, tool schemas, reward, curriculum, and splits before any simulator code exists, backed by 79 checksum-frozen case assets, down to a 12,840-row inventory subledger and an oracle script per case. A spec of this kind is not a running system. It’s the contract that must exist before a running system is worth building. The contrast with the market is direct: Surge’s CoreCraft argues for chaotic, large-scale enterprise worlds, but chaos that can’t replay byte-identically can’t calibrate a verifier.

Key Takeaways

• The reference contract fixes six things before code: state machine, tool schemas, reward, curriculum, splits, and anti-gaming requirements, plus 79 SHA-256-frozen case assets.
• Determinism isn’t pedantry. RL’s reproducibility crisis is documented back to 2017, and a verifier can only be calibrated against episodes that replay byte-identically.
• The two market philosophies, frozen contracts and chaotic enterprise worlds, will converge on frozen chaos: messy worlds that are deterministic from a seed.

What does a production-grade spec contain?

The reference contract fixes six things before any simulator code is written:

• the episode state machine
• tool schemas
• the reward function
• curriculum
• split policy
• anti-gaming requirements

Every episode follows a state machine from initialization through investigation to submission, with terminal conditions for safety failures, integrity violations, and budget exhaustion. That architecture matches what the strongest published agent benchmarks converged on: τ-bench computes reward by comparing the final database state against the annotated goal state, which is exactly the fix-the-state-machine-first, grade-the-end-state discipline the contract encodes.

The contract also freezes 79 case assets across its reference builds: public performer assets, private evaluator assets, and an oracle script per case. The assets range from schematics and sensor traces to a 12,840-row inventory subledger, policies, private truth files, deterministic tool responses, and authored verifier request/expected-result pairs. Every file has a SHA-256 digest. The corpus is frozen before a single episode is served.

Why does the spec obsess over determinism and splits?

Because RL has been burned before, publicly. Henderson et al. (2017) showed that seeds, implementation details, and uncontrolled nondeterminism swamp reported deep-RL results. And the generalization work behind CoinRun and Procgen demonstrated that agents overfit surprisingly large sets of training levels, which is why the thing you version and freeze is the generator and the split policy, not a pile of instances.

The spec’s answer is structural: splits are fixed in the contract, test families are held out by construction, and every episode replays byte-identically from a seed and version. If the environment can’t do that, the verifier has no stable ground truth to be calibrated against.

What is the market counterpoint?

Surge’s EnterpriseBench CoreCraft argues for chaotic, large-scale enterprise worlds: synthetic organizations with 2,500-plus entities and 23 MCP tools. The thesis is that real enterprise work is messy, and environments should reflect that mess.

Chaos is valuable. But chaos that can’t replay byte-identically can’t calibrate a verifier. If two runs of the same episode produce different states, the verifier can’t distinguish a correct agent from a lucky one. The two philosophies will converge on frozen chaos: environments that are complex and messy but deterministic from a seed, so every episode replays exactly and the verifier’s ground truth is stable.

What does the honesty test look like?

A spec that can’t name what does not exist yet is marketing, not engineering. The reference contract labels every field with its provenance: observed, inferred, author-proposed, expert-validated, or measured in a pilot. Readiness gates are named explicitly, so a reader can tell a validated capability from a planned one. The spec says what is not done, not just what is.

This is not paranoia about hypothetical failure. The Agentic Benchmark Checklist audit found that task-validity and outcome-validity flaws in popular agentic benchmarks distort measured performance by up to 100% relative (Zhu et al., 2025). Named approval decisions and hard dollar ceilings per validation run, with automatic stops, are part of the contract. The acceptance criteria specify deterministic seed counts, generation-yield floors, and 100% byte-identical replay under the same version and seed. These are not aspirations. They’re the gates a spec must pass before it becomes a system a lab would pay for.

What this means

The difference between an environment labs trust and an environment labs return is the contract. A spec that fixes state, reward, and splits before code, freezes its corpus with checksums, and names what does not exist yet is the minimum bar. Everything else is a prototype.

FAQ

What is a checksum-frozen asset?

Every file in the case corpus (schematics, sensor traces, subledgers, oracle scripts) has a SHA-256 digest recorded in a manifest. If any file changes, the digest breaks and the corpus is invalid. This ensures deterministic replay: the same seed and version always produce the same episode.

Why does determinism matter so much?

Verifier calibration requires comparing the verifier’s scores against expert-adjudicated ground truth on identical episodes. If the environment can’t replay an episode byte-identically, there is no stable ground truth to calibrate against. Every measurement is of a slightly different episode.

What is CoreCraft?

CoreCraft is Surge AI’s EnterpriseBench: a synthetic PC-retailer support organization with 2,500-plus entities and 23 MCP tools, scored by LLM-adjudicated rubric criteria. It argues for chaotic, large-scale enterprise worlds. The counterpoint is that chaos without deterministic replay cannot calibrate a verifier.