Server Virtualization Blog - A SearchServerVirtualization.com blog

Server Virtualization Blog:

 

A SearchServerVirtualization.com blog


A server virtualization blog covering virtual machine (VM) management and administration, VMware, Xen, Microsoft, server consolidation and hardware, backup and disaster recovery, VDI (virtual desktop infrastructure) and more.

VMware virtualization used to cut costs at 900 universities

Attention, college students: your tuition may soon decrease!

Well, maybe not. However, VMware Inc. reported today that 900 universities including top tier schools such as Harvard and Yale are saving big bucks using VMware Inc. virtualization.

Many renowned universities that have deployed VMware to reduce capital and operating costs, increase application and system uptime, decrease power consumption and improve disaster preparedness include Cambridge, Princeton, Stanford, Purdue, the University of Maryland, the University of Auckland, and the University of California campuses at Berkeley, Los Angeles and San Diego.

These schools and hundreds more around the world are running their mission-critical enterprise applications, database systems, and education-specific applications such as CollegeNET and the Blackboard Academic Suite in VMware virtualized environments, the company reported.

Others are using VMware for disaster recovery (DR).

Bowdoin College in Maine partnered with Los Angeles-based Loyola Marymount University to build a co-located datacenter for cross-country DR. By partnering and using VMware to create back-up systems, the schools have achieved higher availability and better load balancing, with more than 70% of their environment virtualized and more than 100 virtual machines (VM). They are saving $15,000 in annual server maintenance and have avoided $500,000 in hardware costs, according to VMware.

Ohio State University has been a VMware virtualizatiton customer since 2003 when the College of Humanities needed to upgrade its IT infrastructure and found there was not enough room to expand. After deploying VMware virtualization, the College was able to meet its upgrade needs with 54 VMs running on three physical host servers. The college avoided $160,000 in hardware costs and cut server provisioning time down from three weeks to five minutes, and the IT staff can now manage all of its VMware VMs from a single console.

Clearly, the education sector is a strong market for VMware, as there are now 900 universities and colleges using the virtualization platform. Because of this, VMware created a free online tool called VMware Academic Program staffed with IT professionals from higher education facilities to answer questions about overall IT best practices. In addition to these experts, the site also includes case studies to help understand how others have implemented VMware.

Intel Premier IT Professional series provides virtualization resources

New vendors, strategies, technologies and capabilities seem to present themselves daily to the virtualization administrator and manager. One resource that can help is the Intel Premier IT Professional (IPIP) community.

Today I had the opportunity to attend the IPIP event here in Columbus, Ohio. The meeting provided a great vendor-independent view of virtualization products that revolve around Intel technologies. Planning your virtualization hardware environment is critical to the decisions that will be made in your current and future virtualization implementations.

Between now and the end of the year, Intel is conducting ten more of these events throughout North America. The agenda of these events includes sessions in the following areas:

  • Intel product roadmap
  • Client virtualization strategies
  • Consolidation efficiencies through virtualization
  • Application virtualization strategies

One important advantage to attending the events is that you can have access to non-disclosure information about the processor product line, a key planning part of virtual environments. But the live events are only the tip of the iceberg. On the IPIP website, members can access case studies, presentations, videos and white papers anytime. Also, every page on the IPIP site has a popularity tag that content of all types can be viewed from the tags.

The best part of these resources is that they are free. Check out the Intel Premier IT Professional website and register for an event in your area.

Virtualization performance benchmarks needed ASAP, vendors say

Big players in the virtualization world griped about the absence of performance benchmarks for virtual machines on CIO Talk Radio yesterday and discussed some of the issues surrounding virtualization standards.

Guests on the show included: Simon Crosby, Chief Technology Officer of the Virtualization and Management Division of Citrix; Tom Bishop, Chief Technology Officer, of BMC Software; Dr. Tim Marsland, Sun Fellow, Chief Technology Officer, for the Software Organization at Sun Microsystems Inc.; and Brian Stevens, Chief Technology Officer and Vice President of Engineering at Red Hat.

The glaring ommission in this lineup: VMware, Inc.

The panelists on CIO Talk Radio didn’t mention VMware by name, but did complain that some companies aren’t being open with their performance data, thus prohibiting the virtualization industry from publishing comparative performance data.

VMware’s licensing agreement for ESX allows users to conduct internal performance testing and benchmarking studies, and allows those users (and not unauthorized third parties) to publish or publicly disseminate the data provided that VMware has reviewed and approved of the methodology, assumptions and other parameters of the study.

Users that have published benchmark data, like Sr. Systems Engineer Mark Foster did on his blog, have had to unpublish results because of VMware’s stipulations.

VMware introduced its own free benchmarking tool, VMmark, last year for certain applications.

Meanwhile, the SPEC Virtualization Committee has been working to create standard benchmarks for VMs. The committee’s goals are to deliver a benchmark that will model server consolidation of commonly virtualized systems such as application servers, web servers and file servers; provide a means to compare server performance while running a number of VMs; and produce a benchmark designed to scale across a wide range of systems.

SPEC expects these benchmarks to be available by the end of this year, but the timeline is not set in stone, according to the website.

Sun’s Marsland said benchmarking progress has been slow because there isn’t an easy way to define a workload, and a large number of benchmarks are required.

“We are talking about a virtual computer, with lots of aspects that need to be benchmarked,” Marsland said. “Every component that gets virtualized needs to be benchmarked.”

Having an open, standardized way of benchmarking is expected to push virtualization further into the mainstream because it will eliminate false perceptions about performance, panelists said. For instance, “there is the thought that I/O intensive workloads can not be virtualized, and the absence of benchmarks prevents us from proving otherwise. It is important for us to have good benchmarks out there,” one panelist on the show said.

Though users look at benchmarks, this type of data is most useful to vendors and OEMs who can use the performance standards to improve the technology, and of course, market their products.

“More open scrutiny of performance results will help us to improve as an industry overall,” Bishop said. “There are ways to measure performance in non-virtual environments, and people are adapting those techniques to get the most out of their virtualized environments.”

In terms of application performance in virtual environments, the issues differ depending on the data center infrastructure. The network, the servers and the storage all affect performance, said Stevens of RedHat.

“The areas that have to progress are around I/O. Intel and AMD are improving around page tables, and we will see improvements around I/O adapters soon,” Stevens said.

Another problem with virtualization? There are support challenges. If an application running in a VM starts acting wacky, the application vendor may not support it, Crosby said.

Licensing and support in virtual environments has been a major gripe with Oracle, for example, which does not support running its applications with VMware.

“It is a reasonable concern…right now there is irrational market based control. Some folks are abstaining from supporting certain apps [in virtual envionments]. As customers demand support, things will hopefully get rational, by next year I hope,” Crosby said.

Virtual machines bleed money

This blog post was written by Megan Santosus, Features Writer.
A recent white paper published by Embotics Corp. on the hidden costs of virtual machines (VMs) paints just the kind of picture one might expect from a vendor of VM lifecycle management software. According to the paper, an IT shop with 150 virtual machines will typically spend between $50,000 and $150,000 on VMs that are redundant. Those costs stem from four areas: infrastructure (processing, storage, memory and the like); management systems (backup, change and configuration management, etc.); server software (licenses for operating systems and applications); and administration (labor and training). David Lynch, Embotics’ vice president of marketing, said that it’s not unusual for customers to discover that half of their VMs are redundant.
Are VMs really sieves leaking that much money?

Todd Monahan, data center manager at Alcatel/Lucent’s Ottawa, Ont., facility, (and an Embotics customer, although he didn’t talk about his own company’s experience), finds the white paper’s conclusions on the money, so to speak. Monahan estimates that typical licensing costs incurred by a data center for his size – 500 servers split 50:50 between physical and virtual boxes – to break down per machine as follows: Monitoring, $250 to $300; backup, $600 to $700; and operating system for standard Windows $600 to $700. Add on the application licensing costs that vary widely, and you’ve got quite a bit more than chump change at stake.

And as for half the number of VMs being unnecessary, that resonates with Monahan as well.

“It’s so easy to create VMs when you go through a consolidation exercise,” Monahan said. “And because you can’t see them, it really becomes an issue of out of sight, out of mind.”

Storage utilization is a new battle

I was recently asked, “do you have any visibility of the storge utilization you provide your virtual machines?” I stopped, thought about it and said “no”. However, in my situation, this is not yet a problem.

A pitfall for most enterprise server virtualization strategies is in a reservation for storage, regardless of what the virtual machine has written on the virtualized filesystem to the defined maximums. For example, if I have a base installation of a Windows Server 2003 system, the footprint as I do my server builds will be around 5 GB. My standard build allocation is 32 GB. This makes this system only 15.6% utilized from inception. This rule of thumb applies to most servers, and a standard build has 32 GB as an accepted footprint per system. 

Excluding backend storage virtualization and de-duplication strategies, what about systems that have a storage footprint larger than 32 GB? Well, luckily we’ve been down this path before:

The storage is the storage, virtual or physical.

Managing the percentage of utilization for shared storage should be a task of continuing diligence. I don’t (yet) have a large number of virtual servers with a footprint above the standard build, these systems face the same battles we have had for years with general purpose servers.  As an example, take a main file and print server that is 2 TB on a general purpose server: It will be about 2 TB on a virtual server as well from the storage perspective. For large storage footprints using iSCSI or storage-area network (SAN) technologies, the difference in configuration is minimal.

However, how do we address the first question about under-utilized storage footprints for the virtualized systems? Is it best to look only at operating system metrics? That may be an adequate solution for each operating system, but the aggregation will be from different sources and outputs. What are you doing to address storage utilization when you are not using storage virtualization?

Virtual environment architecting requires network zone placement

Almost every virtualization admin that I interact with has materially changed their strategy at some point with their first generation of server virtualization before the entire project is complete. Among the strategy changes are those related to network zoning, which will become a more important consideration as organizations approach higher levels of virtualization.

Specifically, the placement of external facing systems on the same virtual host as systems which house internal systems can put both sides of the network at risk if a compromise is made to the hypervisor from the external facing systems. This becomes especially important as the virtual appliance space allows organizations to easily consider firewall, intrusion detection, VPN and other external facing roles to be placed into the virtual environment as well as the frequent goal to virtualize everything.  

A more isolating strategy creates a separate environment with hosts dedicated to hosting virtual machines (VMs) that are external facing and not simultaneously host VMs on the internal network. While the hosts may be connected both to the internal and external networks in a DMZ network role, a compromise to the hypervisor or host system would not have as direct of an impact to the VMs running only on the internal networks. This also helps in emergency remediation by allowing a virtual host to be fully isolated or powered off until the issue is identified without impacting the internal network VMs.

When planning your next generation of server-side virtualization, consider the risks of placing internal and external network zones on resources that may contain external facing and internal only VMs. This type of architecture can bake in some inherent security into your environment that may save the day in the event of a zero-day vulnerability situation that affects the guest operating system or the virtualization hypervisor.

VMware pushes desktop virtualization on management and security benefits

VMware Inc. Senior Director of Enterprise Desktops Gerald Chen visited our office on Tuesday morning to discuss the different types of desktop virtualization and answer common questions about Virtual Desktop Infrastructure (VDI), for example, how it differs from terminal services and cost issues.

Here’s how VDI works: each end user gets a virtual machine (VM) that is deployed from a server in the data center directly to a PC, laptop or thin client computer. Each VM is customizable, so all of the user’s settings are saved and re-booted each time the user signs in, Chen said.

When a user logs off for the day, their VM goes idle, and wakes back up when the user logs into their system again, according to Chen. Chen believes that the advantage of VDI is that sensitive data is not being stored on desktops, which can easily be lost or stolen, and these virtual desktops are easier to manage than physical ones.

“VDI is great for industries like health care that are really concerned about information security and compliance. The real value though, is in management. All of the information is safe in the data center, and centrally managed through Virtual Infrastructure,” Chen said. “For instance, if you have 100 new employees who need desktops, you can deploy a VM for each of them in just minutes, and manage all of them centrally.”

VDI is different from Sever Based Computing (SBC) systems like Citrix Systems Inc.’s XenApp in that VDI is connects a single user to a single operating system (OS), instead of having multiple users share one OS.

“Not every application likes to share an OS, and there is also bad isolation; if one application crashes, everyone sharing that OS crashes as well. Those desktops can’t be customized either. It is a locked environment.”

Chen went on to explain that with VDI, four to ten VMs per server core are supported, so a server with one quad-core processor can, theoretically, house 40 VMs. Of course, that varies depending on things like workload, applications and memory. If the VMs become too heavy for the server to handle, management features in VI3 intervene. VMotion can move live VMs from one server to another when capacity issues arise, as can Dynamic Resource Scheduler, which allocates and balances computing resources as needed using VMotion.

Desktop virtualization case study
As VMware announced customer case studies in February, including one at Huntsville Hospital in Huntsville, Alabama.

The hospital needed to implement a new medical information application throughout its network while protecting HIPAA-related data. Deploying hosted desktops on VMware, the hospital could lock down sensitive patient data and reduce the cost and complexity of desktop management.

They used combinations of thin clients and blade servers to access the centralized virtual desktops, and in turn, reduced power consumption across the hospital by 78%, improved longevity with lower hardware maintenance needs and made wireless thin clients on wheeled carts available to hospital staff. Also, doctors can remotely access their VMs through the Internet using a web browser when necessary.

The downside to desktop virtualization
While the benefits are clear, there are some downsides to desktop virtualization: extra storage and initial cost.

Chen told SearchServerVirtualization.com that VMware is working on reducing image sizes and has designed a way to keep only one copy of files that are identical among many users, like icons and other graphics, to reduce the amount of storage necessary.

The cost of implementing desktop virtualization turns users off. According to Ars Open Forum blogger ‘Bright Wire,’ the cost and the magnitude of system upgrades required is not worth the benefits.

“The cost of deploying virtual desktops is massive,” Bright Wire wrote. “You will need to re-gear your existing desktops to run the virtual or you will need vendor equipment that costs twice as much as a new desktop. Either way, the cost is big in manpower. On top of that, your infrastructure will need serious review.”

According to VMware’s product specifications, local desktop virtualization requires a 500 MHz or faster processor with recommended 256 MB of memory, though Forrester reports that PCs must be faster and have more RAM to work efficiently.

“In addition you need to look into the server infrastructure,” Bright Wire said. “You are talking about needing a lot of iron on the backside to handle the needs of the server to supply two to 16 desktops. All this adds up quickly and can easily swamp a datacenter.”

As for pricing complaints, VMware is used to hearing them and holds firm to the ‘you get what you pay for’ mantra, saying the management benefits are worth the price.

The company charges $150 per concurrent user plus additional costs for support, either Gold or Platinum levels. Both bundles include VMware Infrastructure Enterprise Edition for VDI (which consists of VMware ESX Server 3.5 and VirtualCenter 2.5) and the VMware Virtual Desktop Manager 2. The VMware VDI Starter Edition, which enables 10 virtual desktops, has a list price of $1,500. The VMware VDI Bundle 100 Pack, which enables 100 virtual desktops, has a list price of $15,000.

The market indicates a demand for desktop virtualization, as a number of other vendors also entered the desktop virtualization space including Sun Microsystems Inc., Citrix., Pano Logic Inc. and Symantec. Chen would argue that many customers come for reduction in hardware but stay for the management applications.

“Reducing hardware costs is not a reason to use VDI, it is management. We have customers who have seen 40% to 50% ROI in terms of management costs and the amount of time it frees up.”

Burning in virtual server RAM prevents headaches

When system administrators receive new servers, they are often anxious to get them unpacked, in the rack and loaded up with ESX so they can start creating virtual machines. But an important first step should be done before proceeding with virtualization software installation on the server: always burn-in the memory to test for defective memory modules.

Defective memory will usually be unnoticed in a newly-deployed server and it may be months before signs of defective memory start to show. In one group of five HP servers, I had to replace seven memory DIMMs over an 18 month time period. Most of these were eventually detected by HP’s Insight Manager agents that reside on the server, but two of them caused hard server crashes of VMware ESX servers commonly known as a PSOD (Purple Screen of Death). A PSOD on one of your production servers, loaded up with important virtual machines, is never a good thing. You can reduce your chances of this happening by burning in your memory.

Most servers do a brief memory test on startup as part of their POST procedure. This is not a very good test and will only detect the most obvious of memory problems. A more thorough test checks the interaction of adjacent memory cells to ensure that writing to one cell does not overwrite an adjacent cell.

A good, free memory test utility is available, called Memtest86+, that performs many different tests to thoroughly test your servers memory. You can download it as a small 2MB ISO file that can be burned to a CD and booted on your new server. Let the memory burn-in for at least 24 hours (the longer the better though). Memtest86+ will run indefinitely and the pass counter will increment as all of the tests are run. The more RAM you have in your system, the longer it will take to complete one pass. A system with 32GB will generally take about one day to complete. Memtest86+ not only tests your system’s RAM but also the CPU L1 and L2 caches. Should it detect an error, the easiest way to identify the memory module that caused it is to simply remove a DIMM and run the test again and repeat until it passes. Documentation on Memtest86+ includes troubleshooting methods, detailed test descriptions and the causes of errors.

If you already have ESX servers running and want to test their memory, you can use the little known Ramcheck service to do this while ESX is running. This service is non-disruptive and runs in the background consuming minimal CPU cycles.

The extra time you spending testing memory before deploying servers helps eliminate potential problems down the road.

Citrix XenServer now shipping in Dell PowerEdge servers

Citrix Systems, Inc.’s XenServer hypervisor is now shipping in Dell PowerEdge servers, following the partnership accouncement in October 2007.

With Dell, initial products available worldwide include the Citrix XenServer Dell Express Edition and Citrix XenServer Dell Enterprise, both of which include Dell’s management software, Dell OpenManage System Management. Express Edition is a free download that can be upgraded to Enterprise edition. 

By factory-integrating the Citrix XenServer hypervisor into Dell PowerEdge platforms, users can deploy virtual machines (VMs) when they start up their systems for the first time. Also, the XenServer Dell Enterprise Edition does not require additional management licenses or hardware. Also, upgrades for features like live migration on Dell’s MD3000 direct attached storage arrays can be made easily, by imputing a license key.

In March, Hewlett-Packard began shipping XenServer embedded in ProLiant servers. HP’s servers also have specific versions of XenServer called HP Select Edition, which differs from traditional XenServer in that it is tied into HP management tools, like HP Insight Control and HP Integrated Lights-Out for remote server management, according to a Citrix spokesperson.

In light of its partnerships with HP and Dell, Citrix simplified its licensing model recently to per-server, instead of per core, as reported on SearchServerVirtualization.com. This way, users can deploy an unlimited number of virtual machines or guest operating systems on each physical server for a single price, regardless of whether it has one, two or four CPU sockets.

Choosing your next virtualization project

For organizations with an established server virtualization environment, future virtualization projects are looming on the horizon. Whether it is desktop or application virtualization, much deliberating will undoubtly be given to the best product for the new virtualization endeavor — as it should.

The next wave of virtualization projects should always be best of breed for the requirements and functionality you require for your particular environment. For example, say you’re an organization with a successful VMware-based server virtualization environment using VirtualCenter and ESX 3. Does this mean that VMware Virtual Desktop Infrastructure (VDI) is the default selection for a virtualized desktop project? Don’t be fooled into thinking that a single-vendor environment is going to translate into an efficient one.

Identify the best solution, even if you can’t afford it. That also includes your host environment hardware for the next virtualization project. Your next virtualization project may require a decision between blades versus general purpose servers for virtual hosts. Taking the time and effort to identify the best solution after making full comparisons for of potential environments will also prepares you for any unforeseen element in post-implementation inquiry.

Make no mistake, there are plenty of advantages to going with what’s familiar: Price discounts, vendor relationships and non-disclosure access are all strong reasons to select the same vendor, but only after due diligence in your decision process should you make another commitment.