# MLPerf Inference v6.1 Shows Benefits of Multi-Vendor AI Accelerators

> The launch of MLPerf Inference v6.1 showcases a pivotal shift in AI data centers, highlighting the first-ever multi-vendor accelerator benchmarking. This innovation empowers organizations to optimize performance across diverse hardware, driving efficiency and resilience.

**Source**: blogs.cisco.com | **Published**: 2026-09-21 | **Type**: article

## Key Facts

- Multi-vendor setups yield 104.7% throughput, enhancing resource utilization and flexibility.
- Ethernet fabric reduces supply chain risks, promoting resilience against component shortages.
- PD-disaggregation optimizes performance by leveraging strengths of different accelerators.
- Superlinear results indicate strategic advantages in combining architectures for efficiency.
- Organizations can scale AI infrastructure incrementally, avoiding costly ecosystem lock-in.

## Summary

The recent release of the MLPerf Inference v6.1 results marks a significant advancement in the AI data center landscape, showcasing the industry's first multi-vendor accelerator benchmarking. This development, achieved through a distributed inference service that integrates NVIDIA H200 and AMD Instinct™ MI350X systems over a Cisco Silicon One™ Ethernet fabric, signals a pivotal shift in how organizations can optimize their AI infrastructure. As AI applications evolve to demand more resources, this approach allows for greater flexibility and efficiency in managing diverse hardware capabilities.

The AI data center market is currently experiencing a transformation driven by the increasing complexity of AI workloads. Unlike traditional large language model (LLM) applications, newer AI systems require more tokens and continuous capacity expansion. This trend necessitates a shift from reliance on single-vendor ecosystems to a more heterogeneous infrastructure. Companies are now compelled to blend accelerators from various manufacturers and generations, enabling them to maximize resource utilization and mitigate risks associated with supply chain disruptions and rising costs.

Cisco's demonstration of a distributed inference service illustrates the practical benefits of this multi-vendor approach. By orchestrating workloads across different accelerator types with minimal cross-architecture overhead, organizations can achieve improved unit economics. The MLPerf results revealed that the combined service outperformed the individual systems, achieving 104.7% of the expected throughput without incurring penalties for cross-architecture processing. This efficiency is crucial as it allows businesses to scale their AI capabilities without being tethered to a single vendor's technology.

The architecture of AI inference is complex, involving distinct phases such as prefill and decode. Cisco's service employs both whole-request in-flight batching and prefill-decode disaggregation to enhance performance. This flexibility allows organizations to assign the most suitable accelerators to each phase based on their strengths, optimizing resource use and improving overall throughput. The ability to separate these phases and manage them independently is particularly important as model sizes and request volumes grow.

The strategic implications of this shift are profound. Organizations that invest in a robust Ethernet fabric, like Cisco's, can maintain a competitive edge by integrating existing hardware with new accelerators. This approach not only preserves capital investments but also enhances the agility of AI operations. As the demand for AI capabilities escalates, the ability to adapt and scale infrastructure efficiently will be a key differentiator in the market.

The success of Cisco's multi-vendor MLPerf submission demonstrates that the network infrastructure is integral to AI performance rather than merely a conduit for data transfer. With Ethernet serving as the common fabric, organizations can effectively manage the complexities of heterogeneous systems, ensuring that they can leverage the full potential of their AI investments. This evolution towards a more flexible infrastructure is essential for companies aiming to stay ahead in an increasingly competitive landscape.

Looking ahead, the trajectory of AI infrastructure will likely focus on maximizing the utility of existing assets while incorporating new technologies. As organizations embrace heterogeneous systems, the emphasis will shift towards creating adaptable, resilient data centers capable of meeting the demands of advanced AI applications. The strategic integration of diverse accelerators will not only enhance performance but also foster innovation and responsiveness in a rapidly changing market.

## Entities

- **Companies**: NVIDIA, AMD, Cisco
- **Products**: H200, Instinct MI350X, Silicon One G200, MLPerf Inference
- **Technologies**: Ethernet, RDMA, KV-cache
- **People**: Alan Weckel

## Key Concepts

multi-vendor accelerators, distributed inference service, heterogeneous AI infrastructure, PD disaggregation, model parallelism, data-parallel serving, network latency, unit economics

## Definitions

- **MLPerf Inference**: A benchmarking suite designed to evaluate the performance of machine learning inference systems.
- **PD disaggregation**: A method that separates prefill and decode phases in inference to allow independent scaling of each.
- **KV-cache**: A key-value cache used to store intermediate data during the inference process.
- **Ethernet fabric**: A network architecture that uses Ethernet technology to connect various components in a data center.
- **superlinear throughput**: A performance outcome where the combined throughput of a system exceeds the sum of its individual components.

## Use Cases

- Orchestrating workloads across mixed-accelerator pools
- Scaling inference services for large language models
- Improving supply-chain resilience in AI infrastructure
- Optimizing resource allocation in heterogeneous systems
- Maintaining performance while integrating new hardware
- Enhancing unit economics in AI data centers

## Frequently Asked Questions

**What is the significance of the MLPerf Inference v6.1 results?**

The MLPerf Inference v6.1 results demonstrate the effectiveness of a multi-vendor accelerator approach, allowing organizations to combine different hardware for improved performance without being locked into a single ecosystem.

**How does PD disaggregation improve inference performance?**

PD disaggregation allows the prefill and decode phases of inference to scale independently, optimizing resource use by assigning the most suitable accelerators to each phase based on their strengths.

**What role does Ethernet play in this architecture?**

Ethernet serves as the common fabric that connects various accelerators, enabling efficient data transfer and reducing latency, which is crucial for maintaining high performance in a distributed inference service.

**What are the benefits of using heterogeneous AI infrastructure?**

Heterogeneous AI infrastructure allows organizations to leverage existing hardware while integrating new accelerators, enhancing flexibility, resilience, and efficiency in data center operations.

**How does the Cisco Silicon One G200 contribute to the system's performance?**

The Cisco Silicon One G200 provides a high-bandwidth, low-latency Ethernet path that facilitates the quick transfer of KV-cache data between different serving groups, which is essential for maintaining optimal performance in the inference process.

## Links

- [Read on Welcome.AI](https://welcome.ai/content/mlperf-inference-v61-shows-benefits-of-multi-vendor-ai-accelerators)
- [Original source](https://blogs.cisco.com/datacenter/industrys-first-mlperf-inference-benchmarking-across-multi-vendor-accelerators)
- [Cisco](https://welcome.ai/company/cisco): Featured company

---

Source: Welcome.AI | https://welcome.ai/content/mlperf-inference-v61-shows-benefits-of-multi-vendor-ai-accelerators