Large Language Models

AWS Gives Teams More Control Over Custom LLM Training on Managed GPUs

AWS Machine LearningOct 25, 2026

Amazon SageMaker AI introduced training session APIs that let developers control custom model-training loops while AWS operates the GPU infrastructure.

Amazon Web Services announced Amazon SageMaker AI training session APIs on October 22, 2026, giving developers a way to define custom model-training procedures in Python while SageMaker operates the GPU infrastructure. The service is designed for teams that need more control over training algorithms than preconfigured fine-tuning offers, without taking on the work of provisioning and managing GPU clusters.

A session exposes operations for sampling model outputs, calculating gradients, updating weights and saving checkpoints. Developers provide the data, training loop and task-specific reward or scoring logic; SageMaker handles GPU allocation, model and Low-Rank Adaptation (LoRA) loading, distributed execution, fault recovery and checkpoint storage. Its three SDK clients organize service setup, training and sampling. Active sessions also provide an OpenAI-compatible chat-completions endpoint.

AWS demonstrated the approach by applying Group Relative Policy Optimization to OpenAI’s GPT-OSS 20B model on GSM8K math problems. In the reported example, 10 training steps using 40 problems raised held-out accuracy from 56% to 75%. A separate competitive-programming recipe trained a rank-32 LoRA adapter for 50 steps, using 128 problems and eight responses per problem at each step. AWS reported that training reward rose from 0.46 to 0.60, while accuracy on 200 held-out problems increased from 40% to 50.5%.

The API’s per-token billing applies to GPU-bound work, according to AWS; local tasks such as reward scoring do not accrue GPU charges while they run. That distinction may matter for experiments involving expensive local evaluation or tool-based rollouts, where a GPU cluster could otherwise remain idle during scoring.

The new option sits between SageMaker’s more managed customization APIs and its Training Jobs or HyperPod offerings, where users take on more infrastructure choices. AWS also released a cookbook with example recipes for supervised fine-tuning, reinforcement learning and preference alignment.

Industry Impact

More flexible managed model training

Cloud computing, AI and software development

Overview

SageMaker’s training session APIs give ML teams a middle ground between preset fine-tuning and managing their own GPU infrastructure. Developers can define training loops and scoring logic while AWS handles GPU allocation, distributed execution, recovery and checkpoint storage. This could make specialized training experiments more practical for software and AI teams, though customers still need to build and assess their own training procedures. Per-token billing for GPU-bound work, with local scoring excluded while it runs, may also affect how teams design evaluation-heavy workflows.

Key Impact Points

Teams can customize training algorithms without provisioning and operating GPU clusters themselves.

Local reward scoring does not accrue GPU charges while it runs, which may matter for evaluation-heavy experiments.

AWS’s examples show reported gains on held-out math and programming problems, but are limited to specific recipes and datasets.

Opportunities

Custom training workflows

Enterprise AI, cloud services and software development

Organizations that need training methods beyond preconfigured fine-tuning can use the APIs to build custom workflows while AWS manages GPU execution and session operations. Potential use cases include supervised fine-tuning, reinforcement learning and preference alignment, examples covered in AWS’s cookbook. ML teams can keep task-specific data, training loops and reward logic under their control, while avoiding direct cluster management. The per-token GPU billing model makes it important to estimate GPU-bound work separately from local scoring. The article supports an opportunity to improve training experimentation; it does not establish customer revenue outcomes or quantify cost savings.

Managed custom model-training workflows
Problem: Teams seeking training algorithms beyond preset fine-tuning may otherwise need to manage GPU infrastructure and distributed execution.
Solution: Use SageMaker training session APIs to provide custom Python training loops, data and scoring logic while SageMaker manages GPU allocation, execution, recovery and checkpoints.
Target Users: Data scientists and ML engineers building specialized model-training experiments.
Business Value: Adds control over training procedures without requiring teams to operate GPU clusters directly; local scoring does not incur GPU charges while it runs.
Revenue Potential: AWS bills GPU-bound work per token. The article does not quantify customer revenue potential or savings.

Reference link

https://aws.amazon.com/blogs/machine-learning/write-your-own-training-loop-without-managing-gpus-with-amazon-sagemaker-training-sessions

Image Source Link

https://docs.aws.amazon.com/images/sagemaker/latest/dg/images/training-plans/tp-create-studio-app.png

Community Discussion (0)

Loading discussions...