Research

Training Scientific Judgment from Physical Experiments

PublishedAugust 6, 2026
UpdatedSeptember 9, 2026

TL;DR

We post-trained Qwen3.6-35B-A3B on campaigns built from recorded materials experiments, then added evidence guidance and test-time scaling. Agents chose which experiments to investigate, inspected the available evidence, and assessed likely outcomes of other experiments whose results remained hidden.

  • The final system became better at judging experimental outcomes and assigning confidence to those judgments, with 23% lower average log loss than the base system. On these campaigns, it scored above GLM 5.2 and DeepSeek V4 Flash 0731, approaching Gemini 3.6 Flash.
  • With trained weights fixed, evidence guidance and test-time scaling reduced overclaims from unreviewed X-ray diffraction fits, with none observed across 306 eligible belief assessments, compared with 33% before the inference changes.
  • Training combined supervised learning and reinforcement learning, progressing from valid actions to evidence use across short experimental campaigns. Recorded outcomes provided feedback on the agent’s assessments, belief updates, and expert-review decisions.

Scientific agents need to judge whether an experimental result supports a conclusion, when to ask for expert review, and what evidence to gather next. We studied these decisions after the research objective and candidate experiments were defined, using Qwen3.6-35B-A3B, an open model with 35B total parameters and 3B active parameters.

Our first SDL-1 study showed why this mattered. Models made claims beyond their evidence, and a better assessment of the current experiment often failed to improve their assessment of other experiments. The two moved in opposite directions in nearly half of scored updates, 145 of 293. Experiment selection also helped on some assignments and hurt on others.

Why train

Four attempts raised the Qwen base model’s valid-action coverage from 59% to 62%, with no additional valid actions recovered on the fourth attempt.

Bayesian optimization uses a predictive model to choose experiments for a defined objective. We train the policy around that optimizer to judge the evidence, revise its beliefs, and decide whether to continue or change direction. Bayesian optimization remains one of its tools.

The environment

The Precursor Genome records all 1,035 unordered pairs among 46 inorganic precursors, including X-ray diffraction, automated refinement, and expert phase assignments. SDL-1 turns this physical record into campaigns that an agent can replay under a fixed experiment budget.

The agent sees only the evidence it acquires. The evaluator keeps the recorded outcomes private and uses them to grade the agent’s decisions. Replay holds the physical record fixed while allowing the agent to inspect different evidence, choose another experiment, or revise different beliefs.

This lets us locate errors in the decision process and use them as training signals. New physical campaigns add outcomes to the corpus, while replay adds records of how agents used those outcomes.

The environment as a verifier
policy-visible evaluator-private
Policy
Visible evidenceDecisionAction
information boundary
Evaluator
Private outcomeScoreTraining signal

Results

The final system had 23% lower average log loss than the Qwen base system across the campaigns, with judgments of experimental outcomes and confidence that better matched the recorded results.

Log loss measures the probabilities assigned to recorded outcomes and penalizes confident wrong answers. SDL-1 summarizes this over the campaign as adaptive AUC, where higher is better and zero is best. The score improved from −1.893 to −1.466. The base system ran through OpenRouter and the final system through vLLM, making this a comparison of the complete systems, including training, inference changes, and runtime.

SDL-1 system comparisondescriptive comparison · system runtimes differ
Adaptive AUC
is the average score for held-out experimental outcomes across the campaign. Higher is better.
Selection lift
is the score advantage over matched random experiment selection.
Endpoint gain
is the reduction in held-out log loss from campaign start to finish.
Campaign assignmentsComparison systemsTrained systemBase systemselection-lift sign by assignment
Adaptive AUC ↑ · zoom −1.95 to −1.0
Claude Opus 4.8
−1.252
Detailed scores for Claude Opus 4.8
Adaptive AUC
−1.251921
Selection lift
+0.008578
Endpoint gain
+0.067590
Campaign assignment scores for Claude Opus 4.8
AssignmentAdaptive AUCSelection lift
1−1.217554−0.076677
2−1.246568+0.020187
3−1.351007+0.045810
4−1.172333+0.009796
5−1.292241+0.023560
6−1.231823+0.028795
GPT-5.6 Sol
−1.286
Detailed scores for GPT-5.6 Sol
Adaptive AUC
−1.286393
Selection lift
+0.056260
Endpoint gain
+0.043460
Campaign assignment scores for GPT-5.6 Sol
AssignmentAdaptive AUCSelection lift
1−1.196229+0.123215
2−1.334942+0.105326
3−1.511326−0.067968
4−1.043267+0.045058
5−1.407051+0.070699
6−1.225540+0.061231
Grok 4.5
−1.335
Detailed scores for Grok 4.5
Adaptive AUC
−1.334515
Selection lift
−0.037704
Endpoint gain
−0.057811
Campaign assignment scores for Grok 4.5
AssignmentAdaptive AUCSelection lift
1−1.241569−0.133077
2−1.324076−0.032488
3−1.500622−0.105694
4−1.172149+0.057321
5−1.432670−0.066052
6−1.336002+0.053765
Gemini 3.6 Flash
−1.433
Detailed scores for Gemini 3.6 Flash
Adaptive AUC
−1.432550
Selection lift
+0.005957
Endpoint gain
+0.075016
Campaign assignment scores for Gemini 3.6 Flash
AssignmentAdaptive AUCSelection lift
1−1.332559−0.020426
2−1.308397+0.099802
3−1.585200+0.041047
4−1.340646−0.038185
5−1.825953−0.069992
6−1.202542+0.023495
Ours (35B-A3B)Trained
−1.466
Detailed scores for Ours (35B-A3B)
Adaptive AUC
−1.466409
Selection lift
+0.009079
Endpoint gain
+0.022802
Campaign assignment scores for Ours (35B-A3B)
AssignmentAdaptive AUCSelection lift
1−1.459498+0.032298
2−1.522270−0.031706
3−1.441984+0.052024
4−1.445897−0.018633
5−1.473894+0.001671
6−1.454907+0.018819
GLM 5.2
−1.528
Detailed scores for GLM 5.2
Adaptive AUC
−1.527972
Selection lift
+0.157686
Endpoint gain
+0.056919
Campaign assignment scores for GLM 5.2
AssignmentAdaptive AUCSelection lift
1−1.444655+0.958079
2−1.627256+0.004538
3−1.544573−0.031797
4−1.558132+0.029733
5−1.564498−0.088891
6−1.428719+0.074451
DeepSeek V4 Flash 0731
−1.606
Detailed scores for DeepSeek V4 Flash 0731
Adaptive AUC
−1.606500
Selection lift
−0.195185
Endpoint gain
+0.291555
Qwen3.6-35B-A3B (base)Base
−1.893
Detailed scores for Qwen3.6-35B-A3B (base)
Adaptive AUC
−1.893100
Selection lift
−0.158200
Endpoint gain
+0.033900

* GLM 5.2’s selection lift is driven by one root with a lift of +0.958.

The final system scored 0.062 above GLM 5.2, the strongest complete open-weight entry in the original benchmark, and 0.034 below Gemini 3.6 Flash. The latter gap was smaller than SDL-1’s predefined minimum meaningful effect of 0.049.

With trained weights fixed, evidence guidance and test-time scaling reduced overclaims from unreviewed X-ray diffraction fits, with none observed across 306 eligible belief assessments, compared with 33% before the inference changes.

Assessments of held-out chemistry improved over the campaign, while experiment selection showed no reliable improvement. An audit of 468 update turns did not establish that newly acquired evidence reliably improved those held-out assessments.

We inspected the six scored assignments while revising inference, so this comparison measures development progress. A separate campaign showed the same direction of assessment improvement, with a held-out confirmation still to come.

Evaluation details

We evaluated six assignments, each with an agent-selected branch and a matched random-selection branch. All 12 branches completed, and all 234 eligible belief updates passed first-pass validity checks.

The measured unreviewed-refinement overclaim rate fell from 32.7% to 0%, with no flags across 306 eligible belief rows. Mean local sample log-loss improvement per update rose from 0.059 to 0.082. Held-out log loss decreased by 0.0228 from campaign start to finish, and mean experiment-selection lift reached 0.0091.

The expert-review example below shows how the agent waited for stronger evidence before changing its conclusion.

Expert review

The campaign tested whether Fe₂O₃ and NH₄H₂PO₄ formed Fe₂PH₆NO₇ after 60 minutes at 200 °C. An automated refinement reported two phases and a low fit residual, but the result had not been reviewed.

  1. 01

    Wait for evidence

    The trained Qwen model kept its prior belief because the unreviewed fit did not establish which phases were present.

  2. 02

    Request expert review

    It used one of two available reviews to distinguish target formation from a mixture without the target phase.

  3. 03

    Revise the conclusion

    Expert review accepted the refinement and found that the target had not formed. The agent updated the affected claims while preserving the others.

Read the model excerpts
01

Preserve

The refinement record provides Rwp and phase weights but is unreviewed. Per policy, unreviewed refinements and low Rwp values alone do not license reaction, target-presence, identity, quantification, or adequacy claims. An evidence-insufficient no-op preserves the full prior belief, and further evidence or expert review is required to advance the sample state.

02

Escalate

The unreviewed refinement shows two phases: Fe11.9208O18_167 (0.6135) and PH6NO4 (0.3865). The conflict is whether this is a single target phase with minor impurity or a failed/unreacted mixture. Expert review will adjudicate this.Uncertainty: The sample's reaction category, phase identity, and refinement adequacy are unlicensed due to an unreviewed refinement with low Rwp. Expected evidence: Requesting an expert review will license adequacy and phase claims (tier 1 or 2), providing the necessary expert judgment to update the sample belief. Transfer target: The refined reaction category and phase identity for this Fe2O3-NH4H2PO4 precursor pair will inform future forecasts for similar phosphate synthesis routes.

03

Update

Tier 1 review explicitly accepts the refinement and assigns quality tier 1, directly licensing adequacy, identity, and quantification. The identified phases indicate the target Fe2PH6NO7 did not form, supporting an unreacted classification. Unaddressed prior fields are preserved.

The agent assigned a 95% probability that expert review would change the conclusion. Local joint log loss fell 85%, from 1.4376 to 0.2138.

Training

A campaign does not contain one kind of decision. The policy must produce a valid action, decide how new evidence changes its beliefs, preserve claims that the evidence does not affect, and carry those choices into later experiments. One final score would hide which of these behaviors failed.

Each training stage continued from the previous checkpoint and expanded the decisions the policy learned to make.

Valid actions

We sampled several actions from each training state and trained on those that passed the frozen evaluator’s checks for valid evidence, provenance, and replay.

Most accepted actions left beliefs unchanged. We balanced these with physical and expert-reviewed updates, preservation of unaffected beliefs, and calibrated assessments so the policy also learned when to act.

Supervised training improved the starting policy but did not meet its predefined success threshold.

Learning from feedback

We used REINFORCE Leave-One-Out (RLOO) to compare actions taken from the same campaign state. Each action received credit for the later rewards it preceded, helping the policy learn which evidence choices and belief updates improved the outcome.

Clipped Importance Sampling Policy Optimization (CISPO) limited the influence of extreme sampling ratios during training. A comparison on fresh trajectories found small sample-to-train drift, and clipping was rarely active.

Learning across experiments

We extended training from individual updates to campaigns of one or two experiments. The policy chose evidence, allocated expert review, and revised its beliefs, with later assessment rewards providing feedback on earlier decisions.

Learning from recorded outcomes

The final stage held experiment selection fixed and trained belief updates against recorded outcomes. Rewards covered supported claims, accurate assessments, preservation of unaffected beliefs, and the value of expert review.

Recorded outcomes remained private to the evaluator, and deterministic checks rejected unsupported claims and invalid actions before semantic scoring.

Training details and diagnostics

An update required at least six of eight same-state groups to contain two or more valid terminal trajectories. Every selected batch passed this gate, and weaker groups still contributed after the batch passed.

The final stage combined six reward signals and a separate penalty for invalid actions.

Reward signalWeight
Evidence-licensed claims
Change only claims supported by acquired evidence		0.20
Recorded-outcome forecast
Assign probabilities that agree with the recorded physical result		0.20
Exact preservation
Keep unaffected beliefs unchanged		0.15
Correct no-change decision
Abstain when the evidence does not license an update		0.15
Expert-review value
Estimate whether expert review could change the decision		0.10
Future-chemistry forecast
Carry acquired evidence into later predictions		0.20
Invalid policy action
Reject an action that breaks the campaign contract		−1.00
Training dynamics

(a) Supervised negative log-likelihood

Each point averages eight equally weighted training examples. Each segment starts a new corpus, learning-rate schedule, and optimizer. Segment 3 also changes the instruction format.

batch mean, n=8 exampleswithin-segment median

S1 · updates 1–40n=40 · LR 1e-4

S2 · updates 41–70n=30 · LR 3.0e-5

S3 · updates 71–80n=10 · LR 0.5e-4

(b) Trajectory rewards by curriculum

Each curriculum shows 64 trajectory rewards from a separate population with its own reward definition.

trajectory reward, n=64IQRmedianmean
01Expert updatesn=64 · median −1.000 · mean −0.779
02No-change curriculumn=64 · median −0.434 · mean −0.624
03Physical updatesn=64 · median −0.650 · mean −0.406
04Exact preservationn=64 · median +0.220 · mean −0.140
05Future forecastn=64 · median −1.000 · mean −0.782

(c) Stability diagnostics by curriculum

Dots share a scale only within each column. Exact values are shown. The sampled-token count is the clipped-rate denominator.

Stability diagnostics for each reward curriculum
CurriculumKL v2 ×10⁻³EntropyClipped tokensSampled tokens
Expert updates1.4630.1481504,887
No-change curriculum1.6310.1643432,555
Physical updates1.4300.1550415,364
Exact preservation1.4860.1590412,631
Future forecast1.5320.1501498,033

(d) Training-signal coverage

A valid terminal completed all three protocol actions, ended the episode, and had no hard failure. A qualified group had at least two valid terminals. All accepted batches passed the batch-level gate.

Valid-terminal and qualified-group coverage for each accepted training batch
CurriculumValid terminalsQualified groups
Expert updates
28/64
8/8
No-change curriculum
37/64
8/8
Physical updates
28/64
7/8
Exact preservation
40/64
8/8
Future forecast
30/64
6/8
Exact data
Exact reward, stability, clipping, terminal, and group values by curriculum
UpdateCurriculumReward meanMedianIQRKL v2EntropyClippedValidGroups
1Expert updates−0.779−1.000−1.215 to −0.2681.463×10⁻³0.1481/504,88728/648/8
2No-change curriculum−0.624−0.434−1.057 to −0.2541.631×10⁻³0.1643/432,55537/648/8
3Physical updates−0.406−0.650−1.000 to +0.2531.430×10⁻³0.1550/415,36428/647/8
4Exact preservation−0.140+0.220−0.654 to +0.3481.486×10⁻³0.1590/412,63140/648/8
5Future forecast−0.782−1.000−1.243 to −0.2911.532×10⁻³0.1501/498,03330/646/8

Negative log-likelihood by update

010.5628020.3768030.4288040.4061050.3127060.2180070.2116080.3878090.2494100.4303110.3662120.3624130.2908140.2488150.1786160.3715170.2782180.3255190.3110200.2206210.2252220.2417230.1798240.1722250.1823260.1591270.1826280.1954290.1500300.2037310.2292320.1330330.1629340.2168350.1425360.1426370.0981380.1451390.1319400.1786410.2438420.1538430.2341440.1988450.2930460.2709470.2061480.1437490.1130500.2482510.1475520.1667530.3271540.2359550.2519560.2799570.3466580.1745590.1627600.1967610.2299620.2110630.1488640.1314650.1972660.2667670.3202680.1656690.2490700.1796710.3268720.2893730.2832740.2620750.1596760.2831770.1709780.2786790.2392800.3669
Panel A shows training-batch NLL on one shared y-scale. Panel B uses linear-interpolated quartiles and keeps the five curriculum populations separate. Panel C reports sample-to-train KL v2, token entropy, and exact CISPO clipping counts. The upper clip bound is 4.0, and the largest clipped rate is 6.9×10⁻⁶. Panel D reports coverage of the accepted batches.

Test-time scaling

Campaign traces showed that the trained model needed to apply its evidence rules consistently and make better use of related experiments when assessing a new outcome. We held its weights and runtime fixed while revising inference.

We added seven rules for evidence use and a completion rule requiring the model to use its full experiment budget. The rules guided when to change a claim, seek more evidence, or request expert review. Together the rules reduced the measured overclaim rate from 33% to 5%, with little improvement in how well the model assessed experimental outcomes. The comparison tested the rules as a group.

An offline comparison found that conditioning on recorded reactions involving the same precursors reduced log loss by 0.105 nats per item. We then asked the model to use those reactions as a starting point and adjust its assessment using evidence from the current campaign.

For test-time scaling, we sampled three independent assessments and averaged the probabilities they assigned to each outcome.

On the same trained weights and vLLM runtime, the full inference revision improved adaptive AUC from −1.514 to −1.466.

Limitations

SDL-1 begins with a defined objective and set of candidate experiments, and training covered at most two experiments. Research-question selection, optimal stopping, and downstream experimental utility remain unmeasured.

The traces also show that a valid experiment choice can lack a clear connection to available resources or the decision it is meant to resolve.

What’s next

Completed experiments can provide feedback on the decisions that led to a conclusion, giving us a way to train more reliable scientific behavior.

Scientific Autoresearch extends this work to agents that start with a research question and build a virtual laboratory to investigate it. They gather evidence and propose a physical experiment for the facility to review.

Those campaigns create new records of experiments, failures, and decisions. SDL-1 provides a way to turn completed physical experiments into training for the agents that plan the next ones.

Citation

@article{barnes2026physicalexperiments,
  author  = {Barnes, Jarrod},
  title   = {Training Scientific Judgment from Physical Experiments},
  journal = {Dynamical Systems},
  year    = {2026},
  url     = {https://dynamicalsystems.ai/blog/training-scientific-agency}
}