Microsoft and Hugging Face report that reward-based fine-tuning improved Qwen3.8-27B’s repeatable success on database workflow tasks, while slightly reducing task coverage.
Microsoft and Hugging Face published a training stack on October 22, 2026, that uses database outcomes as reinforcement-learning rewards for AI agents. The approach builds on ThinkingBox, which evaluates whether an agent leaves business systems in the correct final state rather than judging success from its tool calls or claims.
In the reported experiment, the teams fine-tuned Qwen3.8-27B for 50 GRPO steps using a curated pool of 187 tasks. The model was then tested on ThinkingBox-Bench, a separate set of 507 tasks across five domains. Each task was attempted 20 times, producing 10,140 trials per model. The benchmark tasks were excluded from training, though the authors note that the training and evaluation sets share parts of the ThinkingBox framework and task style.
After fine-tuning, the model’s average success per attempt rose from 51.70% to 60.78%. More notably, the number of tasks it passed on all 20 attempts increased from 38 to 107. The number it solved at least once moved in the other direction, slipping from 453 to 446. The results therefore point to more consistent performance on some tasks, not broader coverage overall.
The reward signal comes mainly from executable checks of the final database state: correct outcomes receive a pass, while incorrect or missing changes fail. A smaller set of requirements that cannot be determined from database records can use an LLM judge. To choose training examples, the team first probed the base model and retained tasks where it sometimes succeeded, aiming to give training runs a useful mix of outcomes.
The released code covers task selection, rollouts, reward handling, and integration with the Verl training framework. The reference run used 24 H100 GPUs. The findings are specific to this experiment; they do not establish that the method will improve every agent or workflow. The authors emphasize measuring both how often an agent can solve tasks and how reliably it repeats those successes.