An agent's behavior comes from its model and the software around it. Tool handling, context reduction and execution rules can change what the model sees. Rebuilding that orchestration for training creates a risk that improvement is measured on a different system from the one users receive.
Microsoft Research's October 7 Agent Lightning v1.0 release describes training with the existing agent harness through a model API proxy. The proxy records calls and connects execution to training infrastructure. Microsoft reports a coding example in which Qwen3.5-9B improved from 41.8 to 56.4 percent Pass@1 on SWE-bench Verified using approximately 6,000 training samples. Primary source.
The unit of learning matters
The release highlights a statistical issue: one rollout can produce several samples as the harness changes context or makes additional calls. Treating each fragment as an independent episode can give some rollouts disproportionate weight. Its discussion of rollout-level handling addresses that mismatch.
For a developer, the broader lesson is to document how an interaction becomes training data. A reward attached to a completed task does not automatically identify which model calls deserve credit. That mapping should remain inspectable when orchestration becomes more complicated.
Test the complete system
The reported coding result establishes a result in the stated setup. It does not show that every harness, task or reward design will improve similarly. Teams adapting the framework should retain a held-out task set, freeze evaluation conditions and examine failed trajectories as carefully as accepted solutions.
Using the same orchestration in training and deployment could reduce duplicated engineering and make experiments easier to reproduce. It does not remove the need to reassess behavior when tools or context policies change. The next useful comparison is performance across harness versions and task distributions, with the complete configuration recorded.
Source organization: Microsoft. Source statements are attributed; operational implications and suggested evaluation criteria are editorial analysis.
References
Questions, news tips or corrections? Contact [email protected].
