Build a Validated Kubernetes Runtime for NVIDIA Dynamo
According to NVIDIA Developer. NVIDIA introduces the AI Cluster Runtime (AICR) to package NVIDIA Dynamo and its supporting infrastructure into validated recipes.
- • NVIDIA introduced the AI Cluster Runtime (AICR) to package Dynamo and supporting infrastructure.
- • Dynamo 1.4.2 uses TCP for the request plane and ZMQ for KV-cache events.
- • AICR separates runtime setup from application deployment using a five-step flow.
- • The default AICR path no longer uses NATS in Dynamo 1.4.
AICR Infrastructure and Packaging
The NVIDIA DSX AI Factory Platform brings together technologies to help the ecosystem design, build, and operate more efficient AI factories, with NVIDIA Dynamo as one of them. To make management of infrastructure dependencies easier, the platform contains NVIDIA AI Cluster Runtime (AICR). It packages NVIDIA Dynamo and its supporting infrastructure into validated recipes defining configurations with pinned component versions.
Routing and KV-Cache Architecture
Dynamo 1.4.2 uses TCP for the request plane and ZMQ for KV-cache events, enabling both the native Dynamo Frontend router and the Gateway API Inference Extension (GAIE) / Endpoint Picker Plugin (EPP) path to make cache-aware routing decisions. Each vLLM worker engine emits raw KV-cache events through a local ZMQ publisher, which the Dynamo worker runtime subscribes to and republishes.
- 1. developer.nvidia.comOpen source