Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKS
According to AWS. AWS details the deployment and architecture of the NVIDIA Nemotron 3.5 Super VL multimodal model on Amazon EKS and SageMaker HyperPod EKS.
- • NVIDIA Nemotron 3.5 Super VL joins the existing Nemotron models available on AWS.
- • The model features 120B total parameters with 12B active parameters per token.
- • It supports text, image, document, and video understanding for agentic workflows.
- • Deployments are supported on Amazon EKS, EKS Auto Mode, and SageMaker HyperPod EKS.
Model architecture and capabilities
NVIDIA Nemotron 3.5 Super VL is an open multimodal model distilled from Nemotron 3 Ultra. It features 120 billion total parameters and activates 12 billion parameters per token using a hybrid Mamba-Transformer mixture-of-experts architecture. The model combines reasoning and tool use with text, image, document, and video understanding, allowing a single model to handle both text and visual tasks.
Deployment on Amazon EKS and SageMaker
The reference implementation runs on standard Amazon EKS, EKS Auto Mode, and SageMaker HyperPod EKS using NVIDIA Dynamo and vLLM. AWS provides guidance on cluster creation, model weights download, deployment, and performance benchmarking using tools like aiperf to measure metrics such as time to first token and throughput.
- 1. aws.amazon.comOpen source