Interviews

posted on 25 May 2008 13:02

Exanodes, a supplier of networked storage using VMware server's own direct-attached storage, says ussrs are seeking a better alternative to Fibre Channel and IP storage area networks (SANs) and its Seanodes product could be it.

B&F asked the company's Chief Technology Officer, Christophe Guittenit, a set of questions designed to elicit the background to this and provide a fuller picture of how Exanodes' technology could provide that better alternative.

B&F: Where is the evidence that "50 percent of respondents are seeking a better alternative to Fibre Channel and iSCSI SANs and NAS." What was/were the survey questions here?

Christophe Guittenit: The survey question was:

4. What could change your mind concerning external storage acquisition?

a.cost (71,1%)

b.complexity (40,3%)

c.competencies (14,1%)

d.floor space (8,7%)

e.power consumption (17,4%)

f.a better alternative (52,3%)

Comment: Saying you would do something different if a better alternative were presented is not exactly 'seeking' and doesn't explore what 'better' means but let's cede the point and move on.

B&F: Isn't aggregating the storage of a group of physical servers running VMware into a shared storage pool for the virtual machines in those servers adding complexity to those servers?

Christophe Guittenit: Generally, no. Exanodes can run on very basic commoditized servers: i.e. one disk drive (or a disk partition or any kind of block-based storage), 1GB RAM, one CPU core, and one Gig Ethernet NIC.

Exanodes has been designed to have a very small footprint, because Exanodes shares CPU, memory and network with the virtual machines running applications.

With Vmware ESX server, Exanodes runs inside its own VM and so the user can setup the resource percentage (CPU, memory and network) he wants to provide to the Exanodes' VM using the VI3 (Virtual Infrastructure 3) user interface. Resources allocated for Exanodes and resources allocated for applications are defined and enforced by VI3.

B&F: Doesn't it tend to overload the network links between those servers as the network has to carry normal traffic and storage traffic, with consequent slower communications?

Christophe Guittenit: If Exanodes and VMs share the same network links, the user can use the VM Manager interface to set up the portion of network bandwith he wants for Exanodes and the portion he wants for the VM.

The user can also choose to separate the physical network for applications and the physical network for storage. This adds a little more complexity to the server, but most servers have several Gig
Ethernet NICs (as) standard.

B&F: Doesn't it make it harder to move virtual machines from a failed physical server to a new one as all the storage connections from the failed server have to be moved too, and they might be to a group of physical servers instead to one physical storage resource?

Christophe Guittenit: No, no connections are moved.

With Exanodes, all the connections to the virtual storage (LUNs) are local. It is not like with iSCSI where the initiator has to connect to a distant target. Accessing Exanodes storage is exactly like accessing a local disk drive. The iSCSI initiator and target are purely and simply removed from the storage software stack.

However, Exanodes still needs to access the physical storage that is generally outside the local physical server. It uses our simple proprietary protocol and an internal TCP socket. These internal connections are always up and running in Exanodes.
In short, with Exanodes, access to virtual storage is local, and access to physical storage is remote. A virtual machine accesses the virtual storage so when it moves, there's no change in the connections, the virtual storage stays local.

In comparison with traditional clustered iSCSI storage systems, where access to virtual storage (LUNs) is remote and access to physical storage is remote, Exanodes reduces by 50% the network
bandwidth needed for storage.

B&F: In the Seanodes setup the storage has multiple 'controllers', i.e. the CPU per physical server providing some of the shared aggregated storage pool. Each controller is doing its normal PC server work as well as looking after its part of the storage pool. Doesn't this make its responsiveness to storage requests slower than it would otherwise be?

Christophe Guittenit: Yes, the storage requests are a bit slower because of the switching between the "Exanodes process" and "the application VMs", but there can be no deterioration of the storage latency as this deterioration depends on many factors: the I/O workload, the hypervisor, the O/S running the virtual machine, (and) the disk drive response time.

On some workloads we couldn't have measured a difference in performance between virtual environnements and physical ones. It's important to bear in mind that the latency of disk drives is considerable in comparison with the CPU switching time because processes waiting for I/O are scheduled in priority in most O/S schedulers.

B&F: How does the Seanodes product compare to LeftHand Networks iSCSI SAN, particularly its Virtual Storage Appliance which also aggregates storage on physical servers running VMware?

Christophe Guittenit: The major difference is that the access to LUN is local and does not require iSCSI: it's exactly like accessing a local disk drive. It simplifies storage management drastically
and halves the network bandwidth required to complete an I/O request. (With LeftHand, to complete a request, there is a network transfer from the iSCSI initiator to the iSCSI target and a network transfer from the iSCSI target to the server hosting the physical drive needed for the request).

Moreover because the access to LUN is local, the I/O workload is more evenly balanced with Exanodes; a physical server can't be overloaded because it hosts a LUN that is under intense access by other servers in the infrastructure.

Moreover (again), Exanodes has been designed for Shared Internal Storage: so it's very scalable (up to 128 servers inside the same cluster); has a small footprint; can manage the heterogeneous capacities of physical disks drives; and has a self-healing capability to support up to 16 consecutive servers' failures. In short, Exanodes has been designed from scratch to aggregate a large number of disks disseminated across a lot of unreliable heterogeneous servers.

Exanodes can also run outside a VM, in a stand-alone mode and is compatible with Virtual Machine Monitors other than Vmware. For example, some early adopters are testing it on Xen-based VMs.

B&F: How does the Seanode's product interoperate with VMware's Site Recovery Manager?

Christophe Guittenit: At the moment, it doesn't.

B&F: Does the Seanodes' use of spare storage capacity on physical servers running VMware limit its maximum capacity and make physical storage provisioning more difficult as disks have to be added to a physical server running VMware?

Christophe Guittenit: It's up to the user. He can choose the best server for his needs. For large storage configurations, some servers on the market can handle 16 hotswap disk drives on 3U, or even
48 drives on 4U. The user can also choose the network interconnect he prefers: 1 GE, GE with channel bonding, 10 GE, or IB (InfiniBand) if he needs higher performance.

Exanodes can handle the on-line addition of a new server, or some new disks inside a server, and Exanodes supports heterogeneous configurations. The user can choose to add the servers he needs at the time he needs them: more CPU cores, more disk drives, internal RAID/JBOD, it's up to him.

Comment: The Exanodes technology certainly seems worth examining if you are a VMware shop and don't want the complexity and expense of Fibre Channel and appreciate staying inside the VMware environment. The avoidance of iSCSI work removes that burden from the storage work and the picture painted by Christophe Guittenit is coherent.

Take a look-see at Seanodes; it could be what you need.

^{[Chris Mellor.]}