Tag Archives: XCP-ng

Staying Ahead in the Virtualization Landscape: Late Spring 2024

Posted on by

Staying Ahead in the Virtualization Landscape: Late Spring 2024

At the end of our “Life after VMware” series, we started to get comments and suggestions that we should regularly release videos on the state of the virtualization landscape since it’s changing so rapidly that it’s hard for a lot of people to keep up. So, we’ll be doing these videos every few months to check in and see what’s changed and evolved in the virtualization landscape and hopefully keep everyone up to date as well. Let’s get to it!

What’s New in Virtualization?

Hey there homelabbers, self-hosters, IT pros, and engineers, Rich here. Let’s get down to business on what’s happened lately in terms of features, changes, updates, and other news in the infrastructure virtualization space. I will endeavor to provide you with useful information in these videos from your favorite hypervisors like XCP-ng, Proxmox, Hyper-V, Nutanix, and yes…even VMware by Broadcom. If I miss anything, or if you have any options/thoughts/etc. toss them in the comments below and share the knowledge with others, since we’re all in this together!

XCP-ng News and Updates:

May has been a relatively quiet month in terms of news out of Vates and XCP-ng; however, April was pretty busy. XCP-ng announced the availability of its first SMAPIv3 driver in preview, specifically designed for ZFS-based storage repositories. This driver, included in XCP-ng 8.3, allows users to create and manage ZFS storage repositories with greater flexibility and efficiency. April also saw a security update for XCP-ng 8.2 that fixed host crashes, corrected logic to prevent unauthorized memory access by attackers, and mitigated Native Branch History Injection, which is an evolution of the Spectre vulnerability. If you haven’t patched, get on it.

One thing to note, the SMAPIv3 driver is only available in the beta XCP-ng 8.3, and not XCP-ng 8.2 LTS. Now, let’s dig into Proxmox.

Proxmox News and Updates:

Proxmox has released version 8.2 of its Virtual Environment, featuring an import wizard for migrating VMware ESXi guests, automated bare-metal installation, and a backup fleecing feature to improve VM performance during backups. This update also includes a modernized firewall using nftables, device passthrough via GUI, advanced backup settings, and support for custom ACME-enabled Certificate Authorities. The release enhances ease of use and performance, making it simpler to manage virtual environments. And anything that makes that GUI better is worth it in my book – go get it installed!

In other huge news, Veeam announced they will officially support Proxmox VE. Support for this virtualization platform has been a popular request from Veeam’s existing small and medium-sized business (SMB) customers and service providers. This is big news in terms of opening up Proxmox VE to businesses who rely on Veeam for their backup, disaster recovery, and immutability strategies and will go a long way to making Proxmox a viable alternative virtualization platform.

Two giant leaps forward for Proxmox – first, the release of 8.2 in late April brought with it their import wizard for VMware ESXi, something we’ll be trying out in the future, so look for videos on that. And then the announcement from Veeam that they’ll be releasing native support for PVE. This is a big win for Proxmox and Veeam.

Nutanix News and Updates:

The biggest news landed on May 21st. Nutanix and Dell announced a collaboration to enhance hybrid multicloud solutions, aiming to streamline IT operations and improve resiliency. The partnership will introduce two key solutions: a hyper-converged appliance combining Nutanix Cloud Platform and Dell servers, and Nutanix Cloud Platform for Dell PowerFlex, allowing independent scaling of compute and storage. Put simply – this is the first time we’ve seen Nutanix decouple storage from the compute stack.

Just in case I’m using too much business-speak here, let me clarify. When you go with Nutanix, you buy into hyperconverged 100%. You don’t get to use your sexy Pure Storage array or any SAN for that matter for your virtualization in Nutanix – but this announcement is saying, that’s not always the case now, at least if you buy into Dell PowerFlex running Nutanix on it. It looks like we might be seeing the ice thaw on Nutanix’s no SAN or external storage doctrine, and believe me folks, that’s really big news!

In other news, AHV, or Acropolis Hypervisor, turns 10 years old this year. AHV still looks to be running on top of CentOS7 though, and with the official end of CentOS coming later this year, I suspect we’ll see something from Nutanix in the near future announcing what base OS AHV and AOS as a whole moves to.

I’ve been getting a lot of questions and comments about Nutanix’s use of CentOS, and the short answer is I don’t know when they’ll move or what they’ll be moving their platform to, but I suspect we’ll see a lot of buzz about it soon. If any of you know and can share, drop it in a comment below.

Hyper-V News and Updates:

Hyper-V rarely sees a lot of big independent announcements since it’s always coupled with Windows Server announcements. So, the best I have for you was this announcement about Windows Server 2025 preview and a note about the massive improvements to performance and scalability in Server 2025. Stating:

Windows Server 2025 Hyper-V Virtual Machine Maximums:

•Maximum Memory per VM: 240 Terabytes* (10x previous)

•Maximum Virtual Processors per VM: 2048 VPs* (~8.5x previous)

*Requires Generation 2 VMs

Microsoft is on a mission to bridge the gap between on-premise virtualization and hybrid cloud deployments, and Hyper-V is playing a pivotal role in that. Server 2025 obviously isn’t out yet, but it’s good to see that Microsoft is continuing to develop and improve Hyper-V.

VMware by Broadcom News and Updates:

First, let’s start with some good news. On May 13th VMware announced that Fusion Pro and Workstation Pro are now available for free for personal use. Both software can be used free for personal use, but commercial use still falls under a subscription contract. Fusion Pro is desktop virtualization for MacOS, both Intel and ARM CPU, and Workstation Pro is for x86 Windows environments.

While it’s always nice to get something for free, this freebie has felt like a bit of a consolation prize for the loss of the ESXi hypervisor which many of us have been using in our homelabs for over a decade. Whether this fills the hole left by the loss of ESXi or not will remain to be seen, typically trading a dedicated type-1 hypervisor for a type-2 software package brings poorer performance and user experience, but having a good desktop hypervisor is always a good thing.

In more negative news, Computershare, a large financial products and services company in Australia announced that it’s pulling VMware out of their organization to the tune of 24 THOUSAND VMs. The significant licensing cost increase has forced the company to reconsider its virtualization strategy, moving away from VMware to more cost-effective alternatives which, at least, from the article, looks to be Nutanix AHV.

We’ve been hearing from Broadcom from the beginning that their strategy has been to focus on big enterprises, and news like this doesn’t reflect well on the realities of Broadcom’s strategy. At a minimum, this is a bit of egg on the face of Broadcom and Hok Tan, but big news like this also calls into question Broadcom’s indifference to its customer’s complaints is netting higher attrition than expected.

Two more things worth mentioning this round, on May 5th Broadcom shutdown the VMware customer connect website. Now all support for VMware customers, including accessing your entitlements like keys, downloads, and so on, have to be done via the Broadcom support portal. Hopefully, you created your account ahead of the cutover. Also, Broadcom has moved VMware’s knowledge base, which were fully public-facing, to within the Broadcom support portal. I suspect this will have a significant effect on google search results when looking for answers to issues online.

This is my personal take here, but the Broadcom support portal is a nightmare to find things that are VMware related. For example, when you do get logged in, you’re dropped into the ‘Mainframe Software’ section, which has nothing to do with VMware and is confusing. Broadcom knows what I’m licensed for; they should put me in that area when I log in. To get to the VMware downloads, one has to head over to the ‘VMware Cloud Foundation’ section, which again is also confusing since I’m not a VCF customer. It’s a pretty terrible user experience, but then again – so is dealing with VMware by Broadcom.

Tell us what you think!

This is new for us here at 2GT, and we want to make sure we’re covering the hypervisors you care about for your homelab, and your job so please don’t hesitate to let me know what we’re missing and what more you’d like to see!

Watch the video here:

Life After VMware: A Comprehensive Roundup of Alternative Hypervisors

Posted on by

How did we even get here?

Welcome to the final-final video on evaluating your options if you’re coming from the world of VMware and ESXi. This video is my opus, my coup de grâce, the finale of this series that has taken up practically four solid months of my time. In this video, I’m going to attempt to aggregate and summarize as much of the information from the last four videos as I can, put a nice little bow on it, and hand it over to you.

In the last four videos, I looked at XCP-ng, Proxmox, Hyper-V, and Nutanix as alternatives to VMware ESXi and vCenter in your homelab and for your business. All four platforms have pros and cons, so I’m going to attempt to compare them based on their features and limitations in the hopes that this summary gives you some ideas on which direction you want to take with your gear or business.

If you haven’t watched the videos, I encourage you to do so to understand the depth of research that went into this summary. So with that, it’s time to commit death by PowerPoint! Let’s get to it!

Can These Products Replace VMware?

For the most part, all of these hypervisors will replace VMware ESXi and even vCenter, but the devil, friends, is always in the details. Let’s dig in.

  • XCP-ng: Yes, hands-down. It’s the most analogous to ESXi and vCenter and is free.
  • Proxmox: Yes, absolutely, with vCenter-equivalent features, added LXC container support, and, as a friend in our Discord is fond of saying, it will run on a potato, and is also free.
  • Hyper-V: Mostly yes. With the exception of some Linux OS compatibility, it will serve Windows shops well and has vCenter-equivalent features.
  • Nutanix: Yes, best suited for VMware users and businesses already invested in HCI or hyper-converged infrastructure.

All of these hypervisors will run VMs without issue, and with the exception of Hyper-V, are either free or have a free version you can run in your homelab or use to personally evaluate for your business. Now, let’s talk a bit about their underlying operating systems and how they’re deployed.

Underlying Operating Systems and Deployment

  • XCP-ng: Based on the Xen Hypervisor, it is entirely open-source. A standard deployment consists of one or more XCP-ng hosts for running virtual workloads, and either the Xen Orchestra Appliance (XOA) or a deployment of Xen Orchestra (XO) to manage XCP-ng.
  • Proxmox: Based on Debian with a customized Linux kernel, uses KVM for running VMs and LXC for running Linux containers. Deployment consists of one or more independent Proxmox hosts, each having their own web-based management consoles.
  • Hyper-V: A component of Windows Server, deployed after a complete setup of the Windows Server OS. Because Microsoft offers both a Core and the full Desktop experience version of Windows Server, footprints of a Hyper-V deployment can be dramatically different. Hyper-V is entirely closed source.
  • Nutanix: Comprises several components. AHV (Acropolis Hypervisor) is based on CentOS 7 and uses KVM for virtualization. A typical deployment involves the hypervisor followed by the Controller Virtual Machine (CVM), responsible for all management aspects of Nutanix.

Storage and Supported Storage Types

  • XCP-ng: Supports local storage, NFS, iSCSI, and HCI storage using XOSAN.
  • Proxmox: Supports local storage, NFS, Ceph for HCI storage, and many other storage formats.
  • Hyper-V: Supports all the same storage types as Windows OS, including local storage and shared storage via iSCSI.
  • Nutanix: Only supports hyper-converged storage from within the cluster itself. No external storage access is supported.

Backup Solutions

  • XCP-ng: Built-in backup and restore functionality, with third-party support from Commvault and talks of Veeam ongoing.
  • Proxmox: Native backup and restore functionality through Proxmox Backup Server, with Veeam integration announced.
  • Hyper-V: Supported by all major backup vendors, including Veeam, Commvault, Rubrik, and others.
  • Nutanix: Native support for Veeam, Rubrik, Commvault, HYCU, and others.

Live Migration, Workload Balancing, and High Availability

Live Migration

All hypervisors support live migration of virtual machine workloads between different hosts in a cluster, with the exception of Proxmox and LXC containers, which must be shut down before migrating.

Automated Workload Balancing

  • XCP-ng: Automatically migrates virtual machine workloads between hosts to balance CPU load.
  • Proxmox: Does not have built-in automated workload balancing but can use community scripts. Automated workload balancing is on their roadmap.
  • Hyper-V: Supports workload balancing for both RAM and CPU utilization using Hyper-V Failover Cluster Manager.
  • Nutanix: Natively supports workload balancing across the cluster.

All four hypervisors also support high availability as a core feature of their clustering and will restart VMs on different hosts in the cluster if a host fails or goes offline.

User Interface and Experience

  • XCP-ng: Xen Orchestra GUI is functional but feels dated. A refreshed UI/UX is in development.
  • Proxmox: The UI/UX is cluttered and needs improvement, though it has great graphing and extensive functionality.
  • Hyper-V: The worst experience, using the Microsoft Management Console framework, which is dull and fragmented.
  • Nutanix: Prism Element and Prism Central provide a clean, elegant, and highly functional user experience.

Minimum Hardware Requirements

  • XCP-ng: Requires a 64-bit x86 CPU (1.5GHz minimum, 2GHz recommended), 2GB of RAM (4GB recommended), and 46GB of disk space (70GB recommended).
  • Proxmox: Requires a 64-bit x86 CPU, 1GB of RAM (2GB recommended), and does not list a minimum storage requirement.
  • Hyper-V: Requires a 64-bit x86 CPU (1.4GHz minimum), 2GB of RAM, and 32GB of storage space for the OS install.
  • Nutanix: Requires Intel Sandy Bridge or newer, or AMD Zen or newer, 32GB of RAM, and specific storage requirements for cold and hot storage tiers.

Cost and Support

  • XCP-ng: Vates offers VMS Pro ($1000 per host per year) and VMS Enterprise ($1800 per host per year).
  • Proxmox: Four support tiers: Community (€100 per socket per year), Basic (€340 per socket per year), Standard (€510 per socket per year), and Premium (€1020 per socket per year).
  • Hyper-V: Pricing depends on the Windows Server version (Datacenter: $6,155 USD, Standard: $1,069 USD). Support is additional.
  • Nutanix: Pricing is not publicly disclosed, sold through VARs, and available as a turn-key deployment or software-only solution.

Final Thoughts

In just the last four months, the hypervisor and on-premise virtualization space has seen incredible changes. The market has responded quickly to Broadcom’s acquisition of VMware, with third-party backup solutions now supporting XCP-ng and Proxmox. Imagine what this space will look like in a year from now!

Choosing the right hypervisor depends on your specific needs and priorities:

  • XCP-ng: Best for those familiar with VMware, offering a similar deployment and management experience.
  • Proxmox: Ideal for running older hardware, needing LXC container support, and those comfortable with Linux.
  • Hyper-V: Suitable for Windows-centric environments where familiarity with Windows Server is a priority.
  • Nutanix: Offers a stellar user experience but is limited to hyper-converged infrastructure.

Watch the video here:

Evaluating XCP-ng as an Alternative to VMware ESXi

Posted on by

Introduction

With Broadcom’s acquisition of VMware, many homelab enthusiasts and small to medium businesses are reconsidering their virtualization options. VMware’s uncertain future has prompted a search for reliable, open-source alternatives. One such option is XCP-ng, a powerful and cost-effective alternative to VMware ESXi.

Background on XCP-ng

XCP-ng originated from the Xen Cloud Platform and Citrix XenServer. Citrix’s decision to open-source XenServer aimed to cut costs and compete with VMware and Microsoft. However, users were reluctant to pay for maintenance and support, leading Citrix to reintroduce limitations on the free version by 2017. This shift catalyzed the birth of XCP-ng, officially released on March 31, 2018. Since then, XCP-ng has grown, offering multiple versions and an active community.

Comparing XCP-ng and ESXi

Architecture

Both XCP-ng and VMware ESXi are type-1 hypervisors. VMware ESXi is lightweight, running from RAM after boot, with a closed-source kernel. XCP-ng, based on the Linux kernel, also uses disk storage for operational needs after booting.

Performance

Performance between XCP-ng and ESXi is nearly equivalent, though historical reports indicated I/O performance issues with XCP-ng under certain workloads. However, modern updates have largely mitigated these differences. It’s essential to consider the specific workloads and configurations when evaluating performance.

Usability

VMware ESXi excels in usability with a built-in web-based HTML5 GUI, allowing complete management of a single host without additional steps. This intuitive interface simplifies tasks like building and managing VMs, configuring vSwitches, and handling datastores. In contrast, a fresh deployment of XCP-ng lacks a local web GUI for host management. Instead, users must deploy XenOrchestra (XOA), which, while offering a rich feature set, adds complexity to the initial setup.

Features

ESXi’s advanced features, including distributed resource scheduling (DRS), high availability (HA), fault tolerance, vMotion (live migration of VMs), storage vMotion, and API control, require additional licensing. These features are robust but come at a significant cost. XCP-ng, on the other hand, includes clustering, live migrations, VM backup functionality, and automation via API calls out-of-the-box for free. With the exception of DRS, XCP-ng’s feature set closely matches that of VMware’s licensable features, providing a compelling alternative for users on a budget.

Scalability

ESXi is renowned for its scalability, capable of managing thousands of VMs and extensive clusters. It is used in some of the largest virtual environments globally, thanks to its robust architecture and comprehensive management tools. XCP-ng also supports large environments, though its adoption in complex setups is less common compared to ESXi. Xen Orchestra’s concept of pools (analogous to VMware’s clusters) allows for the management of large collections of hosts and VMs, supporting significant scaling needs.

Support

VMware offers extensive professional support, training, certifications, a well-maintained public knowledge base, and a large community. However, with Broadcom’s acquisition, the future of this support structure is uncertain. XCP-ng relies more on community support, which is active and growing. Professional support services are available through vendors like Vates, the company actively developing XCP-ng. This blend of community-driven and professional support ensures users can find help when needed.

Cost

VMware’s recent licensing changes make it less accessible to smaller users. The ESXi free version has limitations, and additional features require costly licenses. XCP-ng is entirely free and open-source, making it more cost-effective, especially for smaller deployments. Professional support services are available for a fee, but overall, XCP-ng remains budget-friendly, providing significant value without compromising functionality.

Real-World Comparison

Console and Management Interface

ESXi: Provides basic host information and management functions through a console, with extensive management via a web-based GUI. The ESXi console offers details about the physical host, including version information, hardware specifications, and management interface settings. Basic management functions like configuring the management interface, enabling SSH, and managing host power states are available from the console.

XCP-ng: Offers comprehensive host management from the console, including VM management, storage, and network configuration, surpassing ESXi in console capabilities. XCP-ng’s console allows users to start and stop VMs, manage storage, join or leave resource pools, and view detailed hardware information. This robust console management is a significant advantage for users who prefer direct control over their hosts.

Web Management Interface

ESXi: The web UI offers a polished experience with detailed host, VM, storage, and network management. The interface is intuitive, providing a clear overview of the host’s state, usage, vSwitch and port group configurations, datastores, and system information. Detailed tabs allow for in-depth management of virtual machines, storage systems, and network configurations.

XCP-ng (XOA): The interface feels cluttered but provides extensive functionality, including managing multiple hosts, pools, and VM backups. The XOA dashboard offers an overview of pools, hosts, and VMs, with detailed statistics and performance graphs available through various tabs. While the interface is less polished than ESXi’s, it includes advanced features like VM backup and restore, which are not available in ESXi’s free version.

Detailed Evaluation

User Experience

Xen Orchestra, as an interface, leaves much to be desired in terms of aesthetics and user experience. The first impression is that it looks like many other generic open-source web GUIs, with wasted space and sections that feel disorganized. Despite its functional capabilities, the user experience could be significantly improved to make it more enjoyable to use. The organization of information and the presentation in XOA often feels like an afterthought compared to VMware’s polished interface.

Premium Features and Paywalls

One of the frustrations with XOA is the seemingly arbitrary paywalls for certain features. For example, visualizations and statistics on the dashboard are behind a premium license, while similar information is accessible in other sections. Host updates require a paid license, though XOA updates do not. These inconsistencies can be annoying, especially for users accustomed to fully-featured open-source solutions. However, it’s worth noting that compiling Xen Orchestra from source can unlock these features without additional cost, though this approach requires technical proficiency.

Functionality and Configurability

Despite these drawbacks, XCP-ng offers impressive functionality. The extensive host management capabilities from the console, the ability to create pools of hosts, live migrate workloads, enable high availability, and built-in VM backup functionality are standout features. XCP-ng’s native backup capabilities eliminate the need for third-party software, providing a significant advantage over VMware, which requires additional licensing for similar features.

Network Configuration

ESXi’s virtual network configuration is more complex and offers greater configurability, with visual representations of how VMs connect to virtual networks. XCP-ng’s approach is simpler but effective, with PIFs (physical interface configurations) analogous to port groups and vSwitches in ESXi. Private networks in XCP-ng provide segmented internal communication for VMs within a host, adding flexibility to network configurations.

Conclusion

After extensive use, XCP-ng proves to be a robust and feature-rich alternative to VMware ESXi. It meets the general needs of host and VM management and offers many advanced features without additional cost. The user experience and interface design could benefit from improvements, but the core functionality and configurability make XCP-ng a compelling choice.

For those seeking a cost-effective, open-source virtualization solution, XCP-ng is a strong contender. It offers significant value, especially for homelab enthusiasts and small to medium businesses with budget constraints. The next step in our exploration will be evaluating Proxmox, so stay tuned for that assessment.

Watch the video here!