NEW DELHI, October 16, 2026, 9:56 PM IST — AMD has introduced Helion, an open rack-scale blueprint for running AI agents, with separate designs for hyperscalers that already operate GPU clusters and enterprises that want agent execution and model inference in one rack.
The announcement matters because production AI agents do more than generate tokens. They also run tools, execute code, query databases, maintain state and coordinate many concurrent jobs. Those supporting tasks can consume substantial CPU, memory, networking and storage capacity even when the underlying model is hosted elsewhere. Helion is AMD’s attempt to turn that mixed workload into a configurable infrastructure unit rather than treating every agent task as a GPU problem.
What AMD confirmed
AMD said Helion is an Open Compute Project specification-based reference design, not a finished rack sold directly by AMD. The company plans to contribute the specification to OCP, while hardware partners including Cisco, Dell, HPE, Lenovo, Supermicro and Wiwynn develop systems based on it. Partner-built configurations are planned for availability in November 2026.
The CPU-only hyperscale design uses sixth-generation AMD EPYC 9006 processors and can scale to 32,768 Zen 6 CPU cores in one rack, according to AMD. It is intended to execute agent workflows while connecting to separate infrastructure for model inference.
The enterprise design mixes EPYC CPU blades, AMD Instinct MI350P GPU inference blades and general-purpose compute for databases, storage and technical workloads. AMD lists configurations with up to 16 MI350P GPUs and up to 102 terabits per second of networking bandwidth. The rack can be air-cooled or direct-liquid-cooled.
Why agent infrastructure is splitting into tiers
Model inference remains accelerator-heavy, but an agent’s full execution path is broader. A coding agent may spend part of a job calling a model and the rest cloning repositories, running builds, executing tests and waiting on external services. Other agents may perform retrieval, database operations or multi-step orchestration. A platform sized only around GPU utilization can leave those surrounding services under-provisioned or make expensive accelerators handle work better suited to CPUs.
Helion’s two configurations reflect that distinction. Hyperscalers can keep dense CPU capacity for execution separate from large inference clusters. An enterprise can instead place CPU execution, local GPU inference and supporting data services in the same rack. AMD also describes an integrated token router that can direct model calls to frontier models in the cloud or open-source models hosted locally.
That routing model is consistent with the hybrid architectures many teams are already building around retrieval-augmented generation and LLMOps: model endpoints are only one component of a larger production system that also needs identity, policy, observability, data access and rollback controls.
What platform and DevOps teams should evaluate
For infrastructure teams, Helion is most useful as a signal that agent capacity planning is becoming a rack-level problem. Teams evaluating similar systems should measure complete task throughput, not only model tokens per second. Useful production metrics include completed tasks per watt, queue time, CPU and accelerator saturation, tool-call latency, storage I/O, network congestion and the cost of failed or retried jobs.
Isolation is equally important. High-volume agents may run untrusted code, use privileged tools or access sensitive data. A dense rack increases the blast radius if sandboxing, workload identity, egress controls or secrets handling are weak. The system design therefore does not replace the controls described in GravityDevOps’ guide to AI agent security; it makes consistent policy enforcement and auditable execution more important.
Teams should also validate the operational boundary between local and remote inference. A token router can improve cost and placement flexibility, but it creates policy questions: which prompts may leave the environment, which model is approved for a given data class, what happens when a provider is unavailable, and how routing decisions are recorded. Those decisions should be testable configuration rather than hidden application logic.
Availability and open questions
AMD’s announcement establishes the reference architecture and partner list, but it does not provide independent performance results, rack-level power measurements or public pricing. The specifications describe maximum configurations, not guaranteed production throughput. Hardware availability is also forward-looking until OEM and ODM systems ship.
There is one additional standards detail to watch. AMD’s product page describes the rack as native OCP-compliant and identifies standards including ORv3, DC-MHS, DC-SCM, HPRv2, OCP NIC 3.0 and Caliptra. Some early independent coverage referred to ORv4. Buyers should verify the exact specification revision and supported management interfaces with the system vendor before procurement.
The larger direction is clearer than the benchmark picture: agent platforms are pushing infrastructure design beyond accelerator counts. If Helion systems arrive on schedule, the practical test will be whether an open, modular rack helps operators increase completed agent work without sacrificing isolation, observability or power efficiency.
Sources
AMD’s Helion announcement and product specifications; independent reporting from Data Center Dynamics, Tom’s Hardware and The Register; event context from the Open Compute Project.
