RL environments just moved a 397B model. The training guide is public

The clearest proof yet that RL environments move frontier-scale models is now public. In a training guide published September 1, Mercor reports reinforcement-learning post-training of Qwen3.5-397B-A17B on 1,928 expert-created knowledge-work tasks, lifting Pass@1, the share of tasks solved on the first try, from 16.11% to 27.29% on a held-out benchmark of 480 realistic tasks. That is a 70 percent relative improvement at 397B-parameter scale, with no supervised warmup, using the open-source SkyRL framework. Our June census argued the field over-invests in measuring capability and under-invests in the environments that produce it. This is what the production side looks like when someone funds it: purchased expert tasks in, measured frontier capability out, method published.

Key Takeaways

• RL on 1,928 expert-written tasks moved a 397B model from 16.11% to 27.29% Pass@1 on 480 held-out tasks, a 70 percent relative gain.
• The smaller 35B run surpassed a frontier lab’s Opus 4.5 on the same benchmark, per the guide.
• The whole recipe is public and the training data is a purchasable dataset, which makes capability gain a supply-chain question rather than a lab secret.

What exactly was trained, and on what?

Two open-weight models, Qwen3.6-35B-A3B and Qwen3.5-397B-A17B, were post-trained with reinforcement learning and verifiable rewards, the RLVR recipe, directly on expert-created tasks: multi-file knowledge work across investment banking, management consulting, and corporate law, run inside a simulated company with documents, spreadsheets, and email. The training set is Mercor’s off-the-shelf dataset of 1,928 tasks, disjoint from the evaluation benchmark. No supervised fine-tuning warmup; the models learned from scored attempts alone.

Two details in the guide matter beyond the headline. The 35B run surpassed Opus 4.5 on the benchmark, a smaller open model out-scoring a frontier lab’s product on professional work after training on purchased tasks. And the gains held on evaluations the model was not trained for, including terminal-based agentic tasks, while general-reasoning scores did not regress, which is the same transfer pattern Surge documented in August when office-work training improved coding.

Why does one vendor result matter this much?

Because it converts an argument into a receipt. The economics of this market always assumed environments would eventually show up in frontier training runs; the guide documents them doing so, names the framework, and prices the input as a catalogued dataset. The pattern is industry-wide, not one company’s: in late August, AfterQuery served as sole data partner for Motif 3, a 314B-parameter open-weight model post-trained with reinforcement learning. Expert data vendors now sit inside frontier-scale training loops on two continents.

“Data is the most important ingredient in post-training.”

For the supply chain we track, that changes what a task is worth. A benchmark measures a model once. A training-grade environment, with calibrated grading of the kind labs actually trust, improves a model every time the loop runs.

What should a buyer check before believing an uplift number?

Three things, all of which this guide passes and most marketing posts fail:

1. Held-out evaluation. The 480 evaluation tasks are separate from the 1,928 training tasks, so the gain is not memorization.
2. Transfer checks. Improvements appeared on harnesses and task families outside the training set, and non-agentic reasoning did not regress.
3. A published method. Framework, model sizes, and data provenance are named, so the run can be reproduced or challenged.

What this means

Capability gain on professional work is now a purchasable, reproducible pipeline: expert tasks in, verified reward out, published method in between. The gap between 3,094 tracked benchmarks and 101 tracked environments, per the September 2 snapshot of our catalogue, is the investment mispricing this result exposes. The scarce asset is the training-grade environment, and results like this reprice it upward.

FAQ

What is Pass@1?

The share of tasks a model completes correctly on its first attempt. It is the strictest common headline metric for agent benchmarks because it disallows retries: 27.29% Pass@1 on 480 tasks means the model fully solved roughly 131 of them first try.

What is SkyRL?

An open-source reinforcement-learning training framework for long-horizon agents, built at Berkeley’s Sky Computing Lab with Anyscale, and used for both training runs in the guide. Its role matters because it makes the recipe reproducible outside the vendor that published it.

Does this mean any expert task set produces a 70 percent gain?

No. The gain came from tasks with verifiable rewards, expert-calibrated grading, and a held-out evaluation. Task volume without calibrated verification trains models to exploit the grader instead, which is its own failure mode.