NEW DELHI, Aug. 25, 2026, 10:32 a.m. IST — NVIDIA has moved its Groq 3 LPX inference system into full production, turning a previously announced part of the Vera Rubin platform into hardware that cloud providers can begin deploying for latency-sensitive AI agents.
The milestone matters because agent performance increasingly depends on the infrastructure around the model. Long-running coding, research and operations agents repeatedly expand context, generate tokens, call tools and wait for results. Faster decoding can shorten those serial pauses, but it does not by itself make an agent more accurate, safer or cheaper to operate.
What NVIDIA confirmed
In an announcement published during Hot Chips, NVIDIA said Groq 3 LPX is in full production and is designed to work alongside Vera Rubin NVL72. Nebius is the first AI cloud named as an adopter, while Groq said separately that it will be among the early providers and is working with Dell Technologies on deployment.
A rack-scale LPX configuration can contain 256 LP30 accelerators linked directly to one another. NVIDIA positions the system as a specialized decode layer: Rubin GPUs handle broader model computation while the LPUs accelerate the sequential generation of output tokens. The company also described configurations that separate prompt processing from token generation or divide attention and feed-forward work between processors.
NVIDIA reported 3,431 output tokens per second for Gemma 4 31B with a 100,000-token input in testing attributed to Artificial Analysis. Its technical breakdown compares that result with 870 tokens per second for the fastest public endpoint in the test. The same post reports a median 4,767 output tokens per second on NVIDIA’s SPEED-Bench coding workload.
Those numbers are useful evidence that specialized decode hardware can reduce generation time under selected conditions. They are not yet a complete production comparison. The published material does not provide generally available cloud pricing, an application-level service-level agreement, broad model coverage or independently reproduced results across competing systems. Output speed also says nothing about whether a model chose the right tool, produced a correct patch or completed a workflow safely.
Why decode latency becomes an agent bottleneck
Traditional API comparisons often compress performance into one request. An agent creates a chain: read context, reason, emit a tool call, wait for the tool, inspect the result and continue. Each round can be blocked on first-token and output-token latency. The delays compound when a controller invokes sub-agents or runs several verification passes.
That makes prefill and decode different infrastructure problems. Prefill processes the prompt and accumulated context in parallel. Decode generates the response sequentially, one token after another. Separating those stages lets an operator scale them independently, but also introduces scheduling, routing, cache and failure-management work between hardware pools.
The architecture therefore shifts some optimization from model selection to systems design. Teams will need to measure how requests move between prefill and decode capacity, whether key-value caches stay warm, how retries behave, and what happens when the specialized tier is saturated or unavailable.
What developers and platform teams should watch
For developers, the near-term opportunity is faster interaction for workloads with substantial generated output: coding agents, multi-step research, simulation control and tool-heavy assistants. A faster decoder may make iterative agents feel more responsive and raise the number of completed workflows per unit of time.
Platform teams should resist translating the peak token figure directly into an application promise. Before changing an inference architecture, benchmark representative traces that preserve prompt growth, tool latency, concurrency, cancellation and retry behavior. Measure end-to-end task completion time and cost, not only output tokens per second.
Operational testing should also include fallback. If LPX capacity is a separate serving tier, routing policy needs a compatible GPU path when that tier is constrained. Model versions, quantization settings and sampling behavior must remain visible in telemetry so a hardware failover does not silently change output quality. For broader lifecycle guidance, GravityDevOps’ LLMOps overview explains why model serving, evaluation and observability belong in one operating process.
Agent teams should retain workflow-level evaluation as well. A coding agent that emits tokens four times faster can also produce a wrong change four times faster. Tests, policy checks, human approval for high-impact actions and audit trails remain the controls that determine whether speed becomes useful throughput. The same principle applies to retrieval: the GravityDevOps guide to retrieval-augmented generation shows why source quality and grounding matter beyond model response speed.
Availability is the next proof point
Nebius is the first cloud provider NVIDIA named for Groq 3 LPX, and Groq says it is working with Dell to bring the platform to its inference cloud. Neither announcement provides the pricing, regional footprint, model catalog or service guarantees teams would need for a procurement decision.
That leaves a clear dividing line between the confirmed production milestone and the practical availability developers care about. Full production means the hardware program has advanced; it does not mean any developer can provision a stable endpoint today.
An independent report from SiliconANGLE corroborated the production announcement and Nebius adoption. NVIDIA’s earlier Vera Rubin platform disclosure provides the architectural context, including the role of Groq 3 LPX inside a seven-chip platform. The latest news is the move into production and the first named cloud deployments, not a new claim that every agent workload will see the same gains.
The bottom line
Groq 3 LPX is a meaningful step toward disaggregated, workload-specific AI inference. It gives cloud operators a specialized option for the serial decode work that can make long-context agents slow. For engineering teams, the responsible response is to prepare trace-based evaluations and fallback designs, then wait for real endpoint pricing, availability and service data before treating the benchmark as an operating forecast.
