networkZONE Products for the week of May 26, 2007
Mellanox Technologies Says…
Mellanox Delivers Single-Chip 10 Gigabit Ethernet Adapter
Mellanox Technologies Ltd. has announced the immediate availability of ConnectX EN Dual-Port 10 Gigabit Ethernet adapter chips and NICs (Network Interface Cards) which optimize data center productivity with leading throughput and latency performance. Applications such as clustered databases, web infrastructure and virtualized solutions that utilize ConnectX EN deliver an enhanced user experience while reducing data center infrastructure power and complexity as compared to multiple, slower speed Gigabit Ethernet and Fibre Channel connections. Because no external memories or interface support devices are required, ConnectX EN is the most integrated 10 Gigabit Ethernet adapter - ideal for low power blade servers and volume LAN on Motherboard designs.
"Our performance leading ConnectX EN adapters deliver optimized 10 Gigabit Ethernet I/O services that easily scale with multi-core CPUs and virtualized server and storage architectures," said Eyal Waldman, chairman, president and CEO of Mellanox Technologies. "Mellanox's extensive experience with high-performance InfiniBand connectivity and broad range of industry relationships has enabled us to also provide Ethernet products which achieve unprecedented integration and performance that support the needs of our customers."
"As the volume market leader in high speed interconnects, specifically 10 and 20Gb/s InfiniBand products, Mellanox is well positioned to understand the challenges with driving 10 Gigabit Ethernet products to volume," said Bob Wheeler, Senior Analyst for The Linley Group. "The feature set of ConnectX EN is clearly designed to take advantage of the growing multi-core CPU and virtualization trends in the industry, and the simplicity of the single-chip design is ideal for volume applications."
Key features of the ConnectX EN 10 Gigabit Ethernet adapter products include:
- Leading performance: ConnectX EN has achieved an industry leading 17.6Gb/s bi-directional throughput and TCP/IP latencies of less than 7 microseconds. Uni-directional traffic is line rate in either direction.
- Multi-core optimizations: Whether it's Microsoft Windows using RSS (Receive-Side Scaling), Linux, or virtualized environments such as VMware, ConnectX EN spreads traffic evenly across computing cores or virtual machines to enable dynamic rebalancing and optimized application performance.
- Virtualization enhancements: ConnectX EN provides a comprehensive set of features that complement Intel and AMD virtualization technologies, optimize traffic flow and accelerate I/O traffic processing in virtualized server deployments. Support for current and future VMware and Xen based solutions provides investment protection and guards against obsolescence.
- Leading integration: ConnectX EN is a single chip design and does not require external memories or interface support devices (for CX4 copper cables or XFP optical fiber modules). This translates to reduced power, board space and cost, as well as higher reliability. The small silicon adapter package (21mm x 21mm) is well suited for embedded and volume LAN on motherboard designs.
- Class Based Flow Control: ConnectX EN sets the stage for the emerging new generation of Ethernet standards by supporting an extension to the 802.3x IEEE PAUSE standard which enables finer grain traffic rate control, enabling true realization of converged wire benefits without risking transmission interruption of critical traffic.
Software Compatibility
ConnectX EN is supported with standard Microsoft Windows, Linux and other popular operating system TCP/IP and iSCSI protocol stacks which enables seamless networking and storage application integration without the need to replace standard OS components with proprietary software patches. Host-resident TCP/IP support under Linux enables the open source community to stand behind the protocol stack implementation, and any code updates can be quickly supported. ConnectX EN also supports additional enhancements through software upgrades that address the evolving demands on data center connectivity, including support for RDMA-enabled protocol stacks developed by the OpenFabrics Alliance. This unique capability, broadly deployed on performance-leading InfiniBand fabrics today, further accelerates the performance of clustered high-performance computing and data center applications over 10 Gigabit Ethernet networks.
EN-Genius Says . . .
I don't review many board-level products, but that may change since I suspect that the complexities of 10GbE technology and multi-core, Virtual Machine environments will have many designers choosing to buy rather than design their NICs. Over the next few months, I'll try to take a look at a few representative examples to give you at least a feel for what's out there. Mellanox 's ConnectX EN dual-port 10GbE NIC seemed like a good place to start because they brewed their own silicon for the card and because of its extensive support for multi-processor, virtualized environments. Of course, the interesting contrast between its architecture and that of Broadcom 10 GbE converged NIC chip that I looked at recently (in May 2007) also made a quick tour of the ConnectX EN seem like a worthwhile exercise.
As the release above notes, the ConnectX EN is a dual-port 10G port, iSCSI-capable NIC that supports both fiber and copper interfaces in multi-CPU/multi-OS environments. It's designed to do efficient hardware-based load balancing in multi-core servers by monitoring incoming traffic and directing it to the proper CPU. In Windows applications, it works with Microsoft Receive Side Scaling architecture by looking at the address in the TCP field and doing the hashing needed to boil it down to one of eight fields used as an indirect core direction pointer. A similar scheme is employed for Linux and VMWare operations. By handling this at the hardware level, the ConnectX EN is able to minimize message handoff, context switching and cache misses so your cores get to spend more time doing useful work. Mellanox also says that the application software it provides has hooks in it that can also be used for load balancing.
The Mellanox press release does a pretty good job of outlining the ConnectX EN virtualization capabilities but it's worth noting that you'll be getting technology that's very solid because much of it has already been developed and proven in their InfinBand products. This includes the ability to do DMA re-mapping which drops data directly into a Virtual Machine (VM) memory space (see Fig. 1). Doing this saves many extra read/write or DMA cycles usually required to double-copy the block first into the main OS storage space and then into the VM. The card processing engine also has hardware support for Hypervisor offload, a feature (scheduled to be available for VMWare some time this year) that will perform messaging between up to 32 VMs in hardware instead of burdening the Hypervisor. Mellanox says it is working closely with the VMWare community and running the NIC on beta versions of the software in order to guarantee it will work with the production version of the HV offload.
 Speaking of soon-to-be released features, the NIC is also designed to support the new Per-Priority Pause protocol being developed to support Converged, Enhanced Ethernet (the industry new buzzword for Converged Ethernet with built-in priority queuing). It's designed to support converged network tunneled services with a mechanism similar to IB protocol's per-lane credit-based flow control that pauses other services to let high-priority storage and control traffic cut to the head of the queue. They're claiming that the current product has pre-standards support that's interoperable with several undisclosed partners. When pressed, Mellanox could not guarantee that the current hardware can be upgraded to comply with the final CCE standards with a software patch but they seemed pretty confident that it is close enough to not cause any significant problems.
As far as performance goes the card is not quite line rate speed, but comes pretty close under ideal conditions. In half-duplex mode, a single 10G port supports 9.9 Gbit/s when it is pushing long packets. It slows down a bit in full-duplex mode with 17.6 Gbit/s worth of throughput (8.8 Gbit/s in each direction). Mellanox has not completed the tests for dual port operation yet but they expect to see the card be able to at least deliver enough throughput to exceed the maximum capacity of the 8x PCIe host interface.
At these speeds, efficient PCIe, DMA, and offload capabilities become the defining factor in a 10G NIC's performance and it looks like Mellanox 10G & 20G experience with IB adapter cards has smoothed most of the speed bumps for Ethernet. That's why I was so intrigued when I learned that they chose to use a stateless TCP offload scheme rather than the TOE (TCP Offload Engine) technology favored by several manufacturers, including Chelsio 10GbE adapter and Broadcom current 10GbE converged NIC chip.
Mellanox says that they chose the stateless approach because it is more economical for all involved to let the natural evolution of CPUs supply the extra MIPS required for a partially-offloaded TCP termination than to build more specialized, power-hungry hardware on their own chip. The board's power consumption figures certainly seem to bear this out with a typical consumption of 5.7 W/port when driving CX4 PHY connections up to 20 m. It draws a bit more when driving XFP optical modules at 7.8 W/port for short-range and 8.6 W / port for long-range modules.
Mellanox also claims that processing the actual TCP headers on the host doesn't create the bottleneck alleged by some manufacturers, although it does require more memory than a full offload solution does. To get around the problem, the NIC controller splits its storage between a small on-chip memory and the host system memory -- just like most other NICs on the market today.
Whether or not it's actually more efficient, stateless offload certainly lets Mellanox avoid the modifications to the standard TCP/IP stack and other complications that arise when you don't terminate a connection at the host processor. They also claim that the Linux community prefers to keep the TCP stack on the host because it allows them to use its collective expertise to quickly develop patches for security holes and block exploits rather than wait for each individual silicon/card provider to do their job for their particular pieces of hardware.
It will be a while until the stateless Vs TOE offload debate is settled but, for the moment, Mellanox's dual-port 10GbE solution certainly provides some good arguments for using a stateless approach because of its price, performance and power consumption.
All the products are in production. The ConnectX EN adapter chip, with integrated XAUI, CX4, KX4 and XFI interfaces is volume priced at $182. The ConnectX EN adapter NICs supporting dual CX4 copper 10GigE interfaces is volume priced at $405. The ConnectX EN adapter NICs that support fiber SR and LR XFP module connectivity have pricing available upon request. Mellanox also hinted that future versions of ConnectX EN (probably Q4 2007) will allow users to support both 10G Infiniband and 10G Ethernet on the same card.
ConnectX EN Dual Port 10-Gigabit Ethernet NIC Product Page
Dual Port 10-Gigabit Ethernet NIC Product Brief
ConnectX EN Dual Port 10-Gigabit Ethernet IC Product Page
Dual Port 10-Gigabit Ethernet IC Product Brief
|
