VMsafe, the new security technology created by VMware Inc., gives virtualization users the ability to monitor and secure virtual machine resources in ways never before possible.

After I wrote a short article on VMsafe last week, I received feedback from Burton Group analyst Chris Wolf, who was at the VMworld conference in Cannes, France. His comments weren’t included in the story, but they put things in perspective, so here they are:

“VMsafe is a very important technology in my opinion, as it changes how virtual environments are secured. Today, security appliance virtual machines (VMs) typically monitor other VMs by connecting to them over a virtual switch. The result is virtual network monitoring that resembles physical network monitoring,” Wolf said. “The current model is fine until VMs begin to dynamically move across a virtual infrastructure. Dependent security appliances always have to follow their associated VMs as a VM is relocated. This can complicate the live migration and restart processes.”

“With VMsafe, you would typically configure a security appliance per physical host, such as an [intrusion prevention system] virtual appliance. The security appliance vendor would leverage policies to determine what to monitor (such as by VM, virtual switch, subnet, etc). With VMsafe, the appliance can connect to any virtual switch by accessing it through the hypervisor; you no longer have to configure a special promiscuous port group for network monitoring,” Wolf said. “With security configured at the host level with no direct attachment to virtual networks, VMs can move freely about the virtual and physical infrastructure and still have their related security policies enforced.”

Wolf continued, “VMsafe also provides the framework for offloading many security activities to special-purpose security VMs, including roles such as antivirus monitoring. As we move to an automated or dynamic data center, having special-purpose security appliances that are capable of enforcing security policies at the hypervisor level can ease security management in an environment that will be constantly changing. Sure, it’s possible to enforce security policies with special purpose network-based appliances, but such configurations would be substantially more complex to deploy and manage than comparable solutions based on VMsafe technology.”

Fortunately for me, my job never requires me to determine vendor awards. However, Alex Barrett and the SearchServerVirtualization.com staff aren’t so lucky. While it’s great to have the power to name Products of the Year, it also means that you’re stuck hearing complaints from everyone that wasn’t named. In case you missed it, Alex recently published the SearchServerVirtualization 2007 Products of the Year.

I think that Alex and the editorial staff did a great job with selecting products, but thought that I would take a moment to highlight some vendors with excellent products that did not make the list. After all, it’s just as much fun to debate the vendors that were not recognized as it is for those who were.

VMware

Yes, VMware’s on the list, but at the same time they’re not on the list. If you didn’t notice, VMware ESX Server 3.5 is nowhere to be found in the article. The SearchServerVirtualization.com editors informed me that ESX 3.5 missed the cutoff date for award consideration (November 30th), and therefore wasn’t eligible. Editors do need time to work with a released product in order to make a fair judgment, so I understand the reasoning for the cutoff. Still, ESX 3.5 was a significant release from VMware, with features such as Storage VMotion adding significant value to VMware deployments.

Novell

Novell quietly had a great 2007, from a virtualization product perspective. Novell was right behind Citrix/XenSource in achieving Microsoft support for their Xen-based virtualization platform, and was pushing the innovation envelope throughout the year. Novell was the very first virtualization vendor to demonstrate N_Port ID virtualization (NPIV) on their Xen platform. Novell was even showing their work with open virtual machine format (OVF) last September at their booth at VMWorld. When you factor in Novell’s work with their heterogeneous virtualization platform management tool, ZENworks Virtual Machine Manager, you’re left with a pretty nice virtualization package. The vendors mentioned in the virtualization platform category (VMware, Citrix/XenSource, SWsoft) are all worthy of recognition, and I think it’s equally fair to recognize Novell’s work in 2007 as well. Perhaps Novell’s heavy lifting in 2007 will result in recognition in 2008; however, it’s safe to say that Novell is going to have some stiff competition from VMware, Citrix/XenSource, Microsoft, Sun, Parallels, and Virtual Iron.

Symantec

I thinks it’s hard to leave Symantec Veritas NetBackup 6.5 out of the discussion. In fact, amongst backup products, I’d list them as first, right alongside CommVault. Symantec was the first major backup vendor to announce support for Citrix XenServer backup, while all other backup products officially supported one virtualization platform - VMware ESX Server. The NetBackup team was also very innovative with VMware Consolidated Backup (VCB), as NetBackup 6.5 includes the capability to perform file level recoveries of VCB image level backups. Typically, a backup product performs two VCB backup jobs - an image level backup for DR purposes, and a file level backup for day-to-day recovery tasks. NetBackup 6.5 provides the ability to do this in a single pass, which I found to be pretty innovative. Factor in Data-deduplication (extremely valuable considering the high degree of file redundancy on VM host systems), also available in NetBackup 6.5, and it’s hard to see how NetBackup could be ignored.

SteelEye

SteelEye is another vendor in the data protection category that I’m surprised did not make the list. VMware HA by itself will not detect an application failure and initiate a failover job as a result, as it’s primarily designed to monitor and react to hardware failures and some failures within the guest OS. SteelEye LifeKeeper, on the other hand, provides automated VM failover in response to application and service failures (in addition to guest OS and physical server failures). Many failures are software-specific, and products that can automate VM failover or restarts in response to software failures go far to improve the availability of VMs in production.I’m limiting my comments only to the award categories, hence I’m only listing some of the products I’ve worked with in 2007 that fit into one of the SSV categories. I hope that for the 2008 awards, we’ll see a higher number of award categories, so all products in the virtualization ecosystem are represented.

Do you agree with editors’ choice of winners? Which deserving vendors do you feel were left off the list? I’d love to hear your thoughts.

Note: Reposted with the author’s permission from Burton Group’s Data Center Strategies blog.

If you haven’t seen Mike DiPetrillo’s latest blog, “VMware Patch Tuesday,” it’s definitely worth a few minutes of your time. Mike’s post contrasts patch management on the ESX hypervisor with that of competing platforms. I think the picture DiPetrillo paints is much darker than reality (at least with Windows hosts) being that a given Windows Server 2003 host will not require every available patch (many are service-specific) and since not all updates require a reboot. The patch reboot requirements will further diminish in Windows Server 2008 thanks to hot patching support.

That being said, Mike’s latest post is about much more than VMware’s patch management strategy. Instead, consider it the start of the VMware Offensive. In 2007, VMware for the most part smiled and waved at their competition. That’s not going to be the case in 2008. Citrix, Microsoft, Novell, SWsoft, Sun, Oracle, and Virtual Iron all have plans to chip away at VMware’s market share, and rather than ignoring their competitors, I expect VMware to be much more aggressive at highlighting what makes their approach to virtualization different from the competition.

Read the rest of this post at Burton Group’s Data Center Strategies blog.

Following the launch of the article “VMware dispels virtualization myths (sort of),” VMware emailed me to correct some issues about virtual machine security.

According to VMware, an “incorrect statement” was made by Burton Group Analyst Chris Wolf, who, like all of the engineers at VMware he’s spoke with, he thought to be correct.

In the article, Wolf said, “one significant issue with virtual machine security is with virtual switch isolation. The current all-or-nothing approach to making a virtual switch ‘promiscuous’ in order to connect it to an IDS/IPS is not favorable to security.”

For example, “if you connect an IDS appliance to a virtual switch in promiscuous mode,” Wolf said, “not only can the IDS capture all of the traffic traversing the switch, but every other VM on the same virtual switch in promiscuous mode could capture each other’s traffic as well.”

This statement ruffled some feathers at VMware, and they quickly emailed me and Burton to “educate us” and the VMware community that in fact, VMware allows (and encourages) users to configure only the ports they need to be promiscuous as such. This is not a per vswitch setting, but rather a per portgroup setting. The way to configure a vswitch for IDS/IPS is to create a separate portgroup from those used for normal VMs and configure it for “Promiscuous Allowed,” a VMware spokesperson said.

After testing this out in his own lab, Wolf said it is really an easy solution, because the architecture is already there.

“At the switch level, promiscuous mode is an all or nothing configuration. VMware doesn’t argue this. However, a way around this issue is by configuring a separate port group on a virtual switch just for the IDS and making the port group promiscuous. That allows the IDS to monitor the vswitch traffic and still keep all other traffic isolated,” Wolf learned from VMware.

“So, with the port group feature it isn’t all or nothing, it can be granular,” Wolf said. That said, “Vmware’s own team wasn’t even aware of this,” therefore it’s unlikely many VMware administrators are either, he said.

So the record stands corrected. “The option of making a virtual switch ‘promiscuous’ in order to connect it to an IDS/IPS is not favorable to security and should never be used,” Wolf said. Instead, administrators should create a dedicated port group on the switch for the IDS and only make the IDS port group promiscuous. This would allow the IDS to monitor all unicast traffic on the switch while preventing all other VMs on the virtual switch from seeing each other’s unicast traffic.”

Today, XenSource became the first vendor to officially announce support of an embedded hypervisor.  Specific independent hardware vendors (IHVs) are not mentioned in the announcement, but I expect that we will all hear more details soon enough. While VMware has been rumored to have something cooking for months, they have yet to make any official announcement about their plans for an embedded hypervisor. Considering that VMworld starts next week, the timing of the XenSource announcement should not be considered coincidental.

So if you believe the VMware rumors and expect them to take a public stance at VMworld, that places two vendors in the embedded hypervisor club. If you think for a second that Microsoft will also not be a member, you’re fooling yourself. Next year, I expect that IHVs will ship an embedded Windows Server Virtualization Service running on the Windows Server 2008 Core OS.

So by the end of 2008, as I see it, three virtualization platforms will be available to ship on server hardware. Organizations deploying SAN-based virtualization solutions will be able to purchase servers with no internal hard disks and a hypervisor that resides in flash memory. To me, the movement to hypervisors that ship on the bare metal may impact how organizations purchase servers in the future. Instead of selecting an OS, they can pre-order a server with an embedded hypervisor.

So based on the XenSource announcement coupled with the fact that VMware and Microsoft embedded hypervisors, in my opinion, are foregone conclusions, we now know of three vendors in the embedded hypervisor club. Club meetings will likely be hosted by HP, IBM, and Dell servers, at a minimum (I’m basing my speculation on server market share). Now the question that we must consider is what will the impact be for virtualization vendors that are not yet in the club. Those vendors include Virtual Iron, Red Hat, and Novell. Membership may not guarantee success, but it sure doesn’t hurt either.

Chris Wolf, Burton Group senior analyst, analyzed Citrix’s acquisition of XenSource in a recent Burton Group blog post. He sizes up the situation, saying:

“While having the technology is one thing, bringing it to market is an entirely separate issue. This is where the Citrix acquisition makes great sense for XenSource. Financially fueled by Citrix, XenSource now has the financial clout, sales, and channel resources to go after the large stake of unclaimed virtualization market share in the enterprise. Don’t get me wrong. This will not be easy, as Citrix and XenSource are competing against powerhouse vendors with strong sales, channel, and IHV partnerships. VMware, Microsoft, Red Hat, and Novell are well established in the enterprise, and are all looking to add to their share of the market. Virtual Iron has been making a lot of noise in the SMB space lately, and they should see the explosion of the XenSource sales channel as a serious threat.”

Wolf sees the acquisition as a win for Citrix and Xen and for users, too.

“In the coming months and years, we should expect to get enterprise-class virtualization technologies at lower costs, with more features, and a motivated group of vendors that are eager to push innovation to remain competitive.”

Read his blog in its entirety on the Burton Group Data Center Strategies blog.

XenSource recently announced a partnership with Symantec that paves the way for Veritas Storage Foundation to be embedded in XenEnterprise 4.0, expected to ship Q307. Note that the OEM includes a fully licensed, unrestricted version of Storage Foundation. The majority of enterprises today rely on Veritas backup and storage management tools, so it makes perfect sense that XenSource would partner with Symantec to build out a more robust storage architecture for XenEnterprise virtualization platforms. By embedding Storage Foundation in XenEnterprise, storage resources will be able to be managed transparent to their dependent VMs. So XenEnterprise will support connecting VMs to disparate storage targets (FC, iSCSI, NAS, etc.), multipath, and relocation to storage resources as needed, without impacting VM availability.

If you’re already a Veritas shop, this announcement should come as significant news. As a result of the XenSource - Symantec partnership, organizations using Veritas Storage Foundation will be able to manage XenEnterprise storage resources using their existing management toolsets. Furthermore, the partnership is also going to result in certified NetBackup solutions for XenSource platforms. Many backup vendors are still sorting out their VMware backup solution set, while Symantec is steaming ahead by adding XenSource to its already supported VMware and Microsoft virtualization backup solutions. There’s a big difference between a “we support VMware and Xen backup” marketing check box, and a robust and well documented solution set for virtual machine data protection and recovery. Symantec clearly gets it. For example, NetBackup 6.5 is the first backup platform to support recovering VM images or individual files from a single VMware Consolidated Backup (VCB) job.

The OEM agreement may also impact organizations that are required to certify their storage management solutions with every new version release. By using a single storage management infrastructure for both server and virtualization platforms, re-certification of storage management following virtualization platform updates will be easier than on virtualization platforms using a proprietary storage management architecture.

Storage management, high availability, and backup support have been three key issues that have stalled XenSource’s assault on the enterprise. All three of these issues will be solved in XenEnterprise 4.0 as a result of the XenSource - Symantec partnership. With Storage Foundation embedded in XenEnterprise, organizations that do not run Symantec (Veritas) software will still be able to take advantage of the new storage features and manage them using their XenEnterprise management tools. High availability and dynamic VM failover will be included as well. Inclusion of high availability into their virtualization architecture will place XenSource in the high availability virtualization club that now includes VMware, Microsoft, Virtual Iron, Novell, and Red Hat.

When virtualizing mission critical systems, I have long viewed high availability and certified backup support as requirements, and have recommended that virtualization platforms devoid of these features remain relegated to training, test, and development environments. With the upcoming release of XenEnterprise 4.0, XenSource appears to be on the verge of crossing the chasm to join the enterprise virtualization elites such as VMware.

Chris Wolf
Senior Analyst, Burton Group
Note: This post also appears on the Burton Group Data Center Strategies blog.

Burton Group analyst Chris Wolf shared some good advice about
consolidating servers with virtualization in our recent interview. Here are some quick tips gleaned from our conversation and some more-info links and questions for you about these topics.

Making a pitch

Make these key points when pitching server consolidation via virtualization to upper management:

• Virtualization is a means to running fewer physical servers and, thusly, consumer less power in the data center.
• With fewer physical servers, hardware maintenance and upkeep costs go down.
• Virtualization increases server availability via dynamic failover enacted at the virtual machine level. So, any application oncan support high availability, and that is a big difference with virtualization compared to traditional clustering solutions.

(Have you made this pitch? What did you say? What were the results? Let me know in the comments below or by writing to jstafford@techtarget.com.)

Converting multi-purpose servers to VMs

Watch out. This is tricky territory, says Wolf.

“When I have multi-purpose servers, I generally want to take each application or service on that server that I need and run it as its own VM instance. So, in those cases, you are better off manually reprovisioning those services as separate virtual machines again; because in a dynamic failover environment, the VM itself is the point of failover. So, if I have a multi-purpose server, if I am looking at failover, every application on that server is going to be off-line for the period of the failover. If I have a single application per virtual machine, if the VM fails over now, only a single application would be down.”

(Wolf talks more about this process in the interview. Has anyone out there tackled multi-purpose server-to-virtualization conversions? If so, please share your experiences with me at jstafford@techtarget.com.)

Physical-to-virtual (P2V) migration

There are several approaches, says Wolf. Some common practices that work in small environments — such as manually staging a VM and migrating the data and relying on a backup product to help with the migration — are not a good fit for larger data center environments. When migrating many servers, use a product designed for that job to do do a hot clone of a virtual machine.

“Not only does it let me move each VM in a live state, I can schedule when the VMs get converted so I can do a conversion during off-business hours.”

More P2V info can be found here:

SSV’s P2V news and expert advice;
Measuring the success of your server consolidation project.

Got other good P2V links or advice? Let me know: jstafford@techtarget.com.

A couple of weeks ago I spoke with Alex Barrett regarding what I though was a talk on the direction of the server virtualization landscape. Our conversation resulted in her article “Xen virtualization will catch up to VMware in 2008.” After reading the article, I was a little surprised at how some of my words were quoted out of context and wanted to offer my take on the virtualization market and its future direction.

VMware’s Role in Shaping the Future

Many of VMware’s competitors have based their product development road map on VMware’s VI 3 feature set. When I state that Xen platforms can catch-up to VMware’s VI3 features by mid 2008, I mean just that. By this time next year, several Xen vendors will offer mature dynamic failover (comparable to VMware HA) and live migration (comparable to Vmotion) solutions. In doing so, Xen platforms will offer the features that today’s enterprise environments are demanding. Virtual Iron has been very aggressive with their development roadmap and XenSource is working hard as well.

Still, in order to “catch up,” one would have to assume that VMware is sitting on their hands, which of course if far from the case. So will the Xen vendors be caught up to VMware next year? I don’t think so. Will they offer the features and maturity that allow them to be observed as an alternative in the enterprise? Yes.

However, looking into my crystal ball, I see the next generation VMware virtual infrastructure architecture as once again raising the bar. VMware’s ESX hypervisor will have a smaller footprint and improved security. Features that are important in the enterprise, including dynamic VM failover and backup will see significant improvements. You should also to see the complexity of storage integration reduced as well. Technologies such as N_Port ID Virtualization (NPIV) and the proliferation of iSCSI will significantly ease VM storage integration and failover.

I also expect to see more leadership from VMware in the following areas:

• Virtual network security, including monitoring and isolation
• Storage virtualization - development of consistent standards and best practices for integration between server and storage virtualization platforms
• Centralized account management and directory service integration (this is one of my VCB pet peeves)
• Virtual desktop management

Keep in mind that oftentimes many VMware Workstation features find their way into ESX as well. So you should expect some of the new Workstation 6 features to play a part in the next ESX Server product release.Record/replay, is one of my favorite new features, and has numerous uses for testing, troubleshooting, and security auditing.

As the market leader, we should all expect VMware to continue to provide leadership in virtualization innovation, and I don’t expect that to subside.

Virtualization and Security

Security has been getting much more attention lately and will continue to do so in coming years. My recent article “Virtual Switch Security” outlined some of the current weaknesses regarding Layer 2 traffic isolation in some virtual switches. Virtual switches need to improve their default isolation as well as manageability. Port mirroring is an important feature in virtual switches and will be needed for integration with intrusion detection and prevention systems. However, administrators need to be able to control port mirroring within a virtual switch and in turn enable or disable port mirroring on specific ports as needed. VLAN integration is and will remain a concern for virtual switches and vendors that do not offer 802.1Q VLAN support will remain at a disadvantage.

Intrusion detection is becoming more of a concern for numerous organizations, and the uptake of virtualization support by many security ISVs is evidence of that. For example, Catbird’s V-Agent can be used to quickly add an IDS to existing virtual networks.

Hypervisor security is naturally important as well. If you would like to see some of the issues out there today, take a look at Harley Stagner’s excellent article on preventing and detecting rogue VMs. The blue pill attack has also received considerable interest. For more information on blue pill, take a look at Joanna Rutkowska’s presentation “Virtualization - the other side of the coin.”

The security concerns relating to virtualization are no more scary than what we already see with existing operating systems and applications. While security concerns should not prevent you from implementing virtualization, you cannot ignore security either. Hypervisors and management consoles (such as the ESX console which uses a Red Hat-based kernel) still must be managed and updated like all other server operating systems.

To validate the security of their architectures, you should expect virtualization vendors to obtain EAL certification for their respective platforms.

Standards

At the moment, standards are more on my wish list than an actual prediction. I’m hopeful that we will see a common virtual hard disk format within the next 2-5 years. Doing so could provide virtual machine portability amongst all server virtualization platforms and make it considerably easier for ISVs to package and deploy virtual appliances. Administrators would be free to choose their preferred virtualization platform and run virtualization systems on that platform regardless of the virtualization engine that may have packaged a particular VM.

Management standards would also go far in easing virtualization deployments and management. Common APIs for management and backup would allow any third party management or backup tool vendor to support all major virtualization platforms. With industry support of the DMTF System Virtualization, Partitioning, and Clustering (SVPC) Working Group, realization of standardized virtualization management can become a reality.

Emerging Architectures

Application and OS virtualization, fueled by vendors such as SWsoft, Sun, DataSynapse, and Trigence, will continue to add to the virtualization mix in the enterprise. Down the road, application virtualization can significantly ease application deployment by allowing ISVs to package their applications in virtualized containers, thus far reducing application deployment complexity. These technologies run alongside server virtualization deployments today, and it’s very likely that they may be deployed within server virtualization frameworks down the road.

Much work still remains in aligning the non-virtualized industry with the virtualized world. Both application and OS vendors need to be clear on their virtualization licensing terms, with licensing models that support virtualization that are either based on physical or virtual resources. Hybrid licensing that includes terms for virtualization and restrictions on relocation of VMs to other physical resources impedes virtualization adoptions and adds unnecessary confusion. In 2005 Microsoft added a needed jolt to virtualization by being the first vendor to define product licensing in support of server virtualization. Today they need to go further and set the gold standard for licensing of operating systems and applications inside virtual environments. That model should be clear and concise, with simple terms for virtual machines and without limits on portability. “Buffet” style licensing that provides for unlimited VMs on a physical host is ideal as well. Choices and rules are good, but let’s not get carried away. In terms of licensing, less is more. If Microsoft gives us a simple licensing model, many other industry vendors will follow.

Virtualization’s future holds plenty of promise, and we’ll all be the beneficiaries of that promise.

Following Simon Crosby’s release of a XenSource performance benchmark, I began to needle the folks at Virtual Iron about publishing a benchmark of their own. In short time, Chris Barclay, Virtual Iron’s Director of Product Management, sent me some numbers with his blessing to make public.

Their benchmark was based on the Windows Server 2003 OS running on an Intel Xeon 2.66GHZ dual socket/dual core server, with a 1333MHz FSB and 4GB of DDR2 667MHz RAM. For their tests, 1GB of RAM was allocated to the OS and the VM connected to raw SAN storage. So the test environment, in my opinion, is very fair.
Now onto the results…

So overall the Windows Server 2003 VM was able to perform at or below a 3% performance degradation. The Virtual Iron tests followed the same benchmark pattern used by VMware. If you would like to see the VMware results and also get more detail on what each individual benchmark is testing, take a look at VMware’s document “A Performance Comparison of Hypervisors.” Keep in mind that the Xen performance numbers in the VMware paper are under significant debate, with most of us (myself included) seeing Simon Crosby’s Xen benchmark numbers as being more accurate.

Throughput degradation has been very important in many of the virtualization projects that I have been involved with, so having some hard numbers for performance comparison between VMware, XenSource, and Virtual Iron is extremely helpful. I’m hopeful that we’ll see a similar benchmark from Microsoft once the Windows Server Virtualization (WSV) service is available in Longhorn Server, or even for Microsoft Virtual Server 2005 R2 SP1 for the time being. If not, I’ll churn WSV or Virtual Server through the VMware benchmarks and post some numbers myself.

~Chris Wolf