Ai2 says its budget-based scheduler delivered 98% of GPU hours owed to teams over 30 days while keeping cluster occupancy at 98%.
The Allen Institute for AI (Ai2) has replaced a priority-based GPU scheduler with a system that allocates GPU-time budgets, applies hierarchical fair-share scheduling, and allows workloads to be interrupted after declared minimum runtimes. In a post published October 9, 2026, Ai2 says the change helped distribute scarce compute among research teams while keeping its clusters occupied.
Ai2 operates clusters ranging from 88 to 1,024 GPUs, using NVIDIA H100, B200, and B300 accelerators. The institute says demand at any given time is typically two to three times available capacity. Under its previous system, optional preemption and priority labels contributed to GPU “squatting,” inflated priority claims, and lengthy discussions with job owners when machines needed maintenance.
In the new approach, managers assign GPU-time budgets through the research organization’s hierarchy. A fair-share scheduler compares groups’ recent usage with their allocations over a default seven-day window. Workloads request a minimum runtime: they receive protection from preemption for that period, after which the scheduler can rebalance capacity and requeue resumable jobs. Work not charged to a budget can use spare capacity but may be interrupted immediately.
During a 30-day test, Ai2 says teams received 98% of the GPU hours they were owed; 13 of 15 team allocations received at least 95%, and the lowest received 90%. Cluster occupancy remained at 98%, with 18% of delivered time classified as unallocated. On Ai2’s largest H100 cluster, median queue waits fell from five minutes to 24 seconds, while the 90th-percentile wait decreased from 2.8 hours to 1.8 hours. The organization also reports that repairs requiring human intervention dropped by 74% as unhealthy hosts could drain workloads after minimum runtimes.
Ai2 notes that the change created difficulties for interactive sessions that previously could remain active for a week. It is exploring a neighboring CPU-only cluster for development work and plans restorable sessions, while investigating whether the new policy could fragment capacity and delay the largest jobs.