AWS detailed a supply-chain example using Amazon Bedrock AgentCore Evaluations to test agent helpfulness, business-rule compliance and explanation quality.
AWS published a reference implementation on October 5, 2026, showing how teams can assess a multi-agent supply-chain system with Amazon Bedrock AgentCore Evaluations. The example uses a fictional retailer facing inventory imbalances and delivery trade-offs; it is a demonstration, not a report of a deployed customer system.
Built with the Strands Agents SDK, the system has an orchestrator that delegates tasks to four specialists: optimization, distribution, routing and analytics. AWS describes these agents as running on AgentCore Runtime, with memory and observability enabled. The example connects them to mock APIs and supply-chain data sources to handle inventory allocation, distribution changes, routing options and diagnostics.
The evaluation approach separates three kinds of checks. Built-in evaluators provide baseline measures such as helpfulness, task success, instruction following and response relevance. Custom evaluators test whether outputs satisfy operational rules—for example, whether inventory recommendations respect budgets and warehouse capacity, or whether a route meets delivery-window and carrier constraints. Other checks cover data grounding, SQL correctness and whether the orchestrator combines agent results coherently.
A separate explainability layer checks whether agents state their rationale, identify supporting data, describe relevant constraints and trade-offs, explain tool use, and disclose assumptions when information is incomplete. AWS argues that measuring transparency independently from business accuracy can help distinguish a sound decision that is poorly explained from a clearly explained recommendation that violates a rule.
The post also describes using evaluations during development and in production. On-demand runs can support benchmarking, regression testing and CI/CD gates; online evaluations can sample production traces and send scored results to CloudWatch dashboards and alarms. The walkthrough focuses on on-demand testing, using 20 sample queries across the agents. AWS provides the implementation as a deployable example, including scripts for running sessions and evaluators.