# NVIDIA's NIM 2.0.12 Boosts AI Throughput and User Capacity > NVIDIA's NIM 2.0.12 optimizations allow businesses to serve 2.5x more users simultaneously, transforming AI application performance while maintaining responsiveness. Streamlined deployment and enterprise support position NIM as essential for scaling AI capabilities. **Source**: developer.nvidia.com | **Published**: 2026-09-10 | **Type**: case_study ## Key Facts - NIM 2.0.12 boosts throughput by 2.5x, enhancing user capacity to 50 TPS/user, revealing scalability potential. - Optimized configurations leverage GPU resources, indicating a competitive edge in AI deployment efficiency. - Performance gains from NIM suggest financial benefits through reduced operational costs and increased user engagement. - Regular updates and support from NVIDIA provide strategic stability, enhancing customer trust and retention. - The shift to validated performance engineering signals a market trend towards reliability in AI infrastructure. ## Summary \## Summary NVIDIA deployed its NIM (NVIDIA Inference Microservice) to enhance the performance of the Nemotron 3 Ultra model. The challenge was to optimize serving for agentic AI workloads while maintaining responsiveness. The solution resulted in a 2.5x increase in throughput, allowing organizations to serve more users effectively. \## Background NVIDIA, a leader in AI and GPU technology, operates in the technology sector with a significant global presence. Before deploying NIM, the company faced limitations in serving concurrent users on its GPU infrastructure, particularly for applications requiring high interactivity and extended response times. \## Challenge The specific problem was to improve the throughput of the Nemotron 3 Ultra model while ensuring that the application remained responsive for users. This was critical for agentic AI workloads that involve long prompts and context reuse, which can strain existing serving configurations. \## Solution NVIDIA implemented the NIM 2.0.12 optimized serving stack, which included a range of performance enhancements such as precision and autotuned model-aware kernels, parallel execution across multiple GPUs, and sophisticated memory and scheduling optimizations. These changes were packaged into a deployable microservice, allowing for a streamlined deployment process while maintaining flexibility for developers to benchmark against their own traffic. \## Results The NIM 2.0.12 optimized serving stack achieved a throughput of 1,997 tokens per second, which is 2.5 times higher than the baseline throughput of 718 tokens per second when NIM was not utilized. This significant improvement enabled organizations to serve up to 2.5 times more users at a rate of 50 transactions per second per user. \## Key Insights Organizations looking to enhance the performance of AI workloads should consider deploying optimized serving configurations that are validated for specific models and hardware. Leveraging tools like NVIDIA AIPerf can help in benchmarking performance against real-world traffic, ensuring that deployments meet user latency and throughput requirements. \## Customer Testimonial No direct quotes are provided in the source material. ## Entities - **Companies**: NVIDIA - **Products**: Nemotron 3 Ultra, NIM 2.0.12, NVIDIA AI Enterprise - **Technologies**: large language model, GPU, microservice, inference-stack, AIPerf ## Key Concepts performance engineering, production readiness, agentic AI workloads, throughput optimization, NIM optimizations, benchmarking, concurrency, model-aware kernels ## Definitions - **NIM**: NIM is a package from NVIDIA that provides model- and GPU-aware serving optimizations for deploying AI models. - **agentic AI workloads**: Agentic AI workloads refer to applications that require interactive and responsive processing of long prompts and extended responses. - **throughput**: Throughput is the amount of data processed by a system in a given amount of time, often measured in tokens per second for AI models. - **Pareto curve**: A Pareto curve is a graphical representation that shows the distribution of outcomes, helping to identify optimal points for performance metrics. - **CVE handling**: CVE handling refers to the process of managing and addressing Common Vulnerabilities and Exposures in software systems. ## Use Cases - Deploying AI models with optimized performance on GPU infrastructure - Benchmarking AI model performance using representative traffic - Serving multiple concurrent users effectively in real-time applications - Utilizing NIM for improved throughput in agentic workloads - Running performance sweeps with NVIDIA AIPerf - Selecting optimized profiles for specific hardware configurations ## Frequently Asked Questions **What is NIM and how does it benefit AI deployments?** NIM is NVIDIA's package that optimizes model and GPU serving for AI applications. It provides validated configurations that enhance performance and ensure production readiness, allowing organizations to serve more users effectively. **How can I benchmark my AI model using NIM?** You can benchmark your AI model by replaying representative traffic with NVIDIA AIPerf. This tool allows you to measure performance and identify the Pareto point that meets your application's latency and throughput targets. **What are agentic AI workloads?** Agentic AI workloads are applications that require handling long prompts and delivering extended responses interactively. These workloads benefit significantly from optimizations provided by NIM. **What is the significance of the 2.5x throughput improvement?** The 2.5x throughput improvement indicates that the NIM 2.0.12 optimized serving stack can handle significantly more user requests compared to the baseline, enhancing the efficiency of AI applications. **How do I get started with Nemotron 3 Ultra NIM?** To get started, download the Nemotron 3 Ultra NIM 2.0.12 from NGC, run it on your NVIDIA GPU infrastructure, and use AIPerf to replay traffic and select the optimal performance settings for your application. ## Links - [Read on Welcome.AI](https://welcome.ai/content/nvidias-nim-2012-boosts-ai-throughput-and-user-capacity) - [Original source](https://developer.nvidia.com/blog/how-full-stack-nim-optimizations-deliver-2-5x-more-users-on-nemotron-3-ultra/) - [NVIDIA](https://welcome.ai/company/nvidia): Featured company --- Source: Welcome.AI | https://welcome.ai/content/nvidias-nim-2012-boosts-ai-throughput-and-user-capacity