Staying Ahead in the Virtualization Landscape: Late Spring 2024

Staying Ahead in the Virtualization Landscape: Late Spring 2024

At the end of our “Life after VMware” series, we started to get comments and suggestions that we should regularly release videos on the state of the virtualization landscape since it’s changing so rapidly that it’s hard for a lot of people to keep up. So, we’ll be doing these videos every few months to check in and see what’s changed and evolved in the virtualization landscape and hopefully keep everyone up to date as well. Let’s get to it!

What’s New in Virtualization?

Hey there homelabbers, self-hosters, IT pros, and engineers, Rich here. Let’s get down to business on what’s happened lately in terms of features, changes, updates, and other news in the infrastructure virtualization space. I will endeavor to provide you with useful information in these videos from your favorite hypervisors like XCP-ng, Proxmox, Hyper-V, Nutanix, and yes…even VMware by Broadcom. If I miss anything, or if you have any options/thoughts/etc. toss them in the comments below and share the knowledge with others, since we’re all in this together!

XCP-ng News and Updates:

May has been a relatively quiet month in terms of news out of Vates and XCP-ng; however, April was pretty busy. XCP-ng announced the availability of its first SMAPIv3 driver in preview, specifically designed for ZFS-based storage repositories. This driver, included in XCP-ng 8.3, allows users to create and manage ZFS storage repositories with greater flexibility and efficiency. April also saw a security update for XCP-ng 8.2 that fixed host crashes, corrected logic to prevent unauthorized memory access by attackers, and mitigated Native Branch History Injection, which is an evolution of the Spectre vulnerability. If you haven’t patched, get on it.

One thing to note, the SMAPIv3 driver is only available in the beta XCP-ng 8.3, and not XCP-ng 8.2 LTS. Now, let’s dig into Proxmox.

Proxmox News and Updates:

Proxmox has released version 8.2 of its Virtual Environment, featuring an import wizard for migrating VMware ESXi guests, automated bare-metal installation, and a backup fleecing feature to improve VM performance during backups. This update also includes a modernized firewall using nftables, device passthrough via GUI, advanced backup settings, and support for custom ACME-enabled Certificate Authorities. The release enhances ease of use and performance, making it simpler to manage virtual environments. And anything that makes that GUI better is worth it in my book – go get it installed!

In other huge news, Veeam announced they will officially support Proxmox VE. Support for this virtualization platform has been a popular request from Veeam’s existing small and medium-sized business (SMB) customers and service providers. This is big news in terms of opening up Proxmox VE to businesses who rely on Veeam for their backup, disaster recovery, and immutability strategies and will go a long way to making Proxmox a viable alternative virtualization platform.

Two giant leaps forward for Proxmox – first, the release of 8.2 in late April brought with it their import wizard for VMware ESXi, something we’ll be trying out in the future, so look for videos on that. And then the announcement from Veeam that they’ll be releasing native support for PVE. This is a big win for Proxmox and Veeam.

Nutanix News and Updates:

The biggest news landed on May 21st. Nutanix and Dell announced a collaboration to enhance hybrid multicloud solutions, aiming to streamline IT operations and improve resiliency. The partnership will introduce two key solutions: a hyper-converged appliance combining Nutanix Cloud Platform and Dell servers, and Nutanix Cloud Platform for Dell PowerFlex, allowing independent scaling of compute and storage. Put simply – this is the first time we’ve seen Nutanix decouple storage from the compute stack.

Just in case I’m using too much business-speak here, let me clarify. When you go with Nutanix, you buy into hyperconverged 100%. You don’t get to use your sexy Pure Storage array or any SAN for that matter for your virtualization in Nutanix – but this announcement is saying, that’s not always the case now, at least if you buy into Dell PowerFlex running Nutanix on it. It looks like we might be seeing the ice thaw on Nutanix’s no SAN or external storage doctrine, and believe me folks, that’s really big news!

In other news, AHV, or Acropolis Hypervisor, turns 10 years old this year. AHV still looks to be running on top of CentOS7 though, and with the official end of CentOS coming later this year, I suspect we’ll see something from Nutanix in the near future announcing what base OS AHV and AOS as a whole moves to.

I’ve been getting a lot of questions and comments about Nutanix’s use of CentOS, and the short answer is I don’t know when they’ll move or what they’ll be moving their platform to, but I suspect we’ll see a lot of buzz about it soon. If any of you know and can share, drop it in a comment below.

Hyper-V News and Updates:

Hyper-V rarely sees a lot of big independent announcements since it’s always coupled with Windows Server announcements. So, the best I have for you was this announcement about Windows Server 2025 preview and a note about the massive improvements to performance and scalability in Server 2025. Stating:

Windows Server 2025 Hyper-V Virtual Machine Maximums:

•Maximum Memory per VM: 240 Terabytes* (10x previous)

•Maximum Virtual Processors per VM: 2048 VPs* (~8.5x previous)

*Requires Generation 2 VMs

Microsoft is on a mission to bridge the gap between on-premise virtualization and hybrid cloud deployments, and Hyper-V is playing a pivotal role in that. Server 2025 obviously isn’t out yet, but it’s good to see that Microsoft is continuing to develop and improve Hyper-V.

VMware by Broadcom News and Updates:

First, let’s start with some good news. On May 13th VMware announced that Fusion Pro and Workstation Pro are now available for free for personal use. Both software can be used free for personal use, but commercial use still falls under a subscription contract. Fusion Pro is desktop virtualization for MacOS, both Intel and ARM CPU, and Workstation Pro is for x86 Windows environments.

While it’s always nice to get something for free, this freebie has felt like a bit of a consolation prize for the loss of the ESXi hypervisor which many of us have been using in our homelabs for over a decade. Whether this fills the hole left by the loss of ESXi or not will remain to be seen, typically trading a dedicated type-1 hypervisor for a type-2 software package brings poorer performance and user experience, but having a good desktop hypervisor is always a good thing.

In more negative news, Computershare, a large financial products and services company in Australia announced that it’s pulling VMware out of their organization to the tune of 24 THOUSAND VMs. The significant licensing cost increase has forced the company to reconsider its virtualization strategy, moving away from VMware to more cost-effective alternatives which, at least, from the article, looks to be Nutanix AHV.

We’ve been hearing from Broadcom from the beginning that their strategy has been to focus on big enterprises, and news like this doesn’t reflect well on the realities of Broadcom’s strategy. At a minimum, this is a bit of egg on the face of Broadcom and Hok Tan, but big news like this also calls into question Broadcom’s indifference to its customer’s complaints is netting higher attrition than expected.

Two more things worth mentioning this round, on May 5th Broadcom shutdown the VMware customer connect website. Now all support for VMware customers, including accessing your entitlements like keys, downloads, and so on, have to be done via the Broadcom support portal. Hopefully, you created your account ahead of the cutover. Also, Broadcom has moved VMware’s knowledge base, which were fully public-facing, to within the Broadcom support portal. I suspect this will have a significant effect on google search results when looking for answers to issues online.

This is my personal take here, but the Broadcom support portal is a nightmare to find things that are VMware related. For example, when you do get logged in, you’re dropped into the ‘Mainframe Software’ section, which has nothing to do with VMware and is confusing. Broadcom knows what I’m licensed for; they should put me in that area when I log in. To get to the VMware downloads, one has to head over to the ‘VMware Cloud Foundation’ section, which again is also confusing since I’m not a VCF customer. It’s a pretty terrible user experience, but then again – so is dealing with VMware by Broadcom.

Tell us what you think!

This is new for us here at 2GT, and we want to make sure we’re covering the hypervisors you care about for your homelab, and your job so please don’t hesitate to let me know what we’re missing and what more you’d like to see!

Watch the video here:

Evaluating Nutanix as an Alternative to VMware ESXi

Welcome, friends! If you can believe it, this is the fourth installment of our series on evaluating alternative hypervisors now that Broadcom is making VMware untenable for homelabs and small to medium businesses. In the last three videos, we focused on XCP-ng, Proxmox, and Hyper-V as alternatives to ESXi. If you haven’t seen those videos, check them out.

Introduction to Nutanix

Now it’s time to focus on the fourth most requested hypervisor, Nutanix. Let’s get to it!

Hey there, homelabbers and self-hosters, Rich here. This video is the fourth in our series focused on evaluating your options if you’re coming from the world of VMware and ESXi, with this video, in particular, being focused on Nutanix AOS. As always, I’ll be looking at this from a VMware perspective and sharing my thoughts and opinions along the way. Let’s start with some background on Nutanix and the history of the product.

The Nutanix Origin Story

2009: Nutanix was founded by Dheeraj Pandey, Mohit Aron, and Ajeet Singh with the vision of simplifying data centers by integrating compute, storage, and virtualization into a single solution.

2011: Nutanix launched its first product, Nutanix Complete Cluster, marking its entry into the hyper-converged infrastructure (HCI) market. This innovative solution combined compute and storage into a single, scalable appliance.

2015: Nutanix introduced its own hypervisor, AHV (Acropolis Hypervisor), based on KVM and running on CentOS 7. AHV offered a cost-effective alternative to VMware and Hyper-V, emphasizing software-defined solutions.

2016: Nutanix went public with one of the most successful IPOs of the year, underlining the company’s growth and the market’s confidence in its technology.

2017 & 2018: Nutanix broadened its portfolio, introducing Nutanix Calm for application automation and orchestration and Xi Cloud services for hybrid cloud environments. The company also acquired several firms to enhance its cloud capabilities.

2020: Nutanix transitioned to a subscription model for licensing, intending to provide more flexibility to customers and predictable revenue for investors. However, this shift resulted in lower upfront revenue compared to traditional licensing. 2020 also saw a leadership change, with co-founder Dheeraj Pandey stepping down as CEO, succeeded by Rajiv Ramaswami.

2023 and Beyond: Nutanix continues to expand its cloud offerings, introducing new services to support multi-cloud and hybrid cloud strategies. With VMware’s recent ownership change, Nutanix is poised to gain market share, especially through partnerships like the one with Cisco.

Comparing Nutanix to VMware ESXi

Architecture

Both Nutanix and VMware ESXi are type-1 hypervisors but differ in deployment methodologies. ESXi is lightweight, running from RAM with a closed-source kernel. Nutanix AOS and AHV are based on CentOS 7, using disk storage for various operational needs, with closed-source intellectual property.

Performance

Performance between the two hypervisors is nearly equivalent. ESXi supports up to 896 logical CPUs and 24TB of RAM per host, while Nutanix AHV does not have published maximums for CPU cores or RAM.

Usability

  • VMware ESXi: Uses a web-based HTML5 GUI for host and VM management, with extended functionality requiring vCenter.
  • Nutanix: Uses Prism Element, an HTML5 management interface, supporting host and VM management, clustering, live migrations, and DR functionality. Prism Element can manage all aspects of HCI, similar to vCenter but without the need for additional software.

Features

  • VMware ESXi/vCenter: Offers advanced features like distributed resource scheduling, high availability, fault tolerance, vMotion, and API control, requiring additional licensing.
  • Nutanix: Provides clustering, live workload migrations, high availability, hyper-converged storage and networking, API control, and workload balancing out of the box.

Scalability

  • VMware ESXi: Manages thousands of VMs, scaling clusters up to hundreds of hosts with many thousands of VMs.
  • Nutanix: Supports up to 32 nodes per cluster, with the only limit on VMs being the physical CPU and memory. Nutanix is designed solely for HCI and does not support external SAN or NAS storage.

Support

  • VMware: Offers extensive professional support, training, certifications, and a large community.
  • Nutanix: Provides professional support, training, certifications, a public knowledge base, and a large community.

Cost

  • VMware ESXi: Recent licensing changes make it less accessible, with the free hypervisor no longer available as of February 12, 2024.
  • Nutanix: Offers subscription-based licensing on a per-core basis. Nutanix CE (Community Edition) is available for free, allowing homelab users to experiment and learn with some limitations.

Real-World Examples and Interface Comparisons

Console Interfaces

  • VMware ESXi Console: Provides basic host information and minimal management functionality.
  • Nutanix AHV Console: A typical Linux console with most management done via the web GUI.

Management GUIs

  • VMware ESXi: Features a web management UI with detailed host, VM, storage, and networking information.
  • Nutanix Prism Element: An all-encompassing management solution for HCI, similar to vCenter, but integrated into the Nutanix platform.

Likes and Dislikes

What I Like

  • User Interface: Nutanix Prism Element is well-designed with clear overviews and detailed sub-tabs. It includes a fun touch with a built-in web-based game called 2048.
  • Detailed Stats and Logging: Nutanix provides deep insights into performance and operations, rivaling what you’d need VMware Aria Operations for in a VMware environment.

What I Dislike

  • No Support for External SAN Storage: Nutanix’s strict HCI approach limits flexibility for those with traditional SAN setups.
  • Lack of Passthrough: Nutanix doesn’t support passthrough for devices, making it less suitable for certain advanced setups.
  • Complex Setup: Installing and setting up Nutanix AHV and AOS involves multiple steps and requires SSH and command-line interaction.
  • High Resource Requirements: The CVM (Controller Virtual Machine) has significant RAM requirements, starting at 16GB and potentially up to 64GB, which can be burdensome for smaller setups.

Conclusion

Nutanix offers a robust alternative to VMware ESXi, especially for businesses already invested in HCI. However, certain limitations, like no external SAN support and complex setup, may be deal-breakers for homelab enthusiasts. Despite these caveats, Nutanix is a strong contender in the hyper-converged infrastructure space.

 

Watch the full video here:

Evaluating Hyper-V as an Alternative to VMware ESXi

As Broadcom’s acquisition of VMware makes VMware ESXi increasingly untenable for homelabs and small to medium businesses, it’s time to consider alternative hypervisors. This is the third installment in our series, following our evaluations of XCP-ng and Proxmox VE. Now, let’s delve into Microsoft Hyper-V, the third most requested hypervisor.

The Hyper-V Origin Story

Microsoft Hyper-V first launched on June 26, 2008, as part of an update to Windows Server 2008, marking Microsoft’s entry into the hardware virtualization space. Later that year, on October 1, Microsoft released the stand-alone Hyper-V server, built on Windows Server 2008 Core, which was freely available.

Key Milestones:

  • 2009: Windows Server 2008 R2 brought live migration of VMs and expanded support for operating systems.
  • 2012: Windows Server 2012 introduced Hyper-V Replica for disaster recovery, a new virtual hard disk format, and network virtualization.
  • 2016: Windows Server 2016 added support for Docker and containers, nested virtualization, and shielded VMs for improved security.
  • 2019: Windows Server 2019 focused on cloud and hybrid cloud environments, integrating Hyper-V with Azure services.

Although Microsoft announced that Hyper-V Server 2019 would be the final stand-alone version, Hyper-V continues to be a component of Windows Server, with support for Hyper-V Server 2019 ending in 2029.

Comparing Hyper-V to VMware ESXi

Architecture

Both Hyper-V and ESXi are type-1 hypervisors but differ in deployment. ESXi is lightweight, running from RAM with a closed-source kernel, while Hyper-V, part of Windows Server, varies in footprint depending on whether the Core or Desktop version of Windows Server is used. Hyper-V is also closed-source.

Performance

Performance between Hyper-V and ESXi is nearly equivalent. ESXi supports up to 896 logical CPUs and 24TB of RAM per host, whereas Hyper-V supports up to 512 logical CPUs and 48TB of RAM in Windows Server 2022.

Usability

  • VMware ESXi: Uses a web-based HTML5 GUI for host and VM management, with extended functionality requiring vCenter.
  • Hyper-V: Lacks a web-based management interface and relies on various Windows OS or PowerShell tools. Management can be done using Hyper-V Manager, Failover Cluster Manager, or SCVMM.

Features

  • VMware ESXi/vCenter: Offers advanced features like distributed resource scheduling, high availability, fault tolerance, and vMotion, requiring additional licensing.
  • Hyper-V: Nearly feature-complete with VMware, supporting clustering, live migrations, high availability, and workload balancing out of the box.

Scalability

  • VMware ESXi: Manages over a thousand VMs per host, scaling up to 2500 hosts with vCenter.
  • Hyper-V: Supports over a thousand VMs per host and up to 64 nodes per cluster with up to 8,000 VMs.

Support

  • VMware: Offers extensive professional support, training, certifications, and a large community.
  • Hyper-V: Provides support through Microsoft’s website, community forums, and direct contact. Windows Server certifications are available, but none specifically for Hyper-V.

Cost

  • VMware ESXi: Recently eliminated its free hypervisor, increasing costs.
  • Hyper-V: Licensing costs depend on the Windows Server version. Windows Server 2022 Datacenter costs $6,155 USD, supporting unlimited Windows VMs, while the Standard edition costs $1,069 USD for up to two Windows VMs. There are no licensing limits for Linux VMs.

Real-World Examples and Interface Comparisons

Console Interfaces

  • VMware ESXi Console: Provides basic host information and minimal management functionality.
  • Hyper-V: Integrated into Windows Server, with no discrete console.

Management GUIs

  • VMware ESXi: Features a web management UI with detailed host, VM, storage, and networking information.
  • Hyper-V Manager: An MMC-based console for managing Hyper-V, offering basic VM and host management functions.

Likes and Dislikes

What I Like

  • Hyper-V is nearly feature-complete with VMware ESXi.
  • Integration with Windows Server simplifies management for Windows-centric environments.

What I Dislike

  • User Experience: Hyper-V Manager’s MMC interface is outdated and lacks detailed host statistics and VM performance metrics.
  • Linux Compatibility: Hyper-V’s support for Linux is inconsistent, making it less suitable for homelabs focused on experimentation and learning.
  • Limited Management Tools: Hyper-V Manager is the default but suboptimal tool for managing Hyper-V, with better alternatives like Windows Admin Center or SCVMM requiring additional installation or configuration.
  • Perception: Hyper-V often feels like an afterthought by Microsoft, driven more by cost considerations than superiority as a virtualization platform.

Conclusion

While Hyper-V can run VMs and may suit Windows-centric environments, its limitations and inconsistent Linux support make it less ideal as a replacement for ESXi. For those seeking a hypervisor for homelabs or diverse environments, Hyper-V may not be the best choice.

If you have thoughts or suggestions for other hypervisors to review, let us know in the comments or join our Discord community!

Watch the full video here:

Evaluating Proxmox as an Alternative to VMawre ESXi

Evaluating Proxmox VE as an Alternative to VMware ESXi

With Broadcom’s acquisition of VMware, many homelab enthusiasts and small to medium businesses are seeking alternatives to VMware ESXi. Today, we delve into Proxmox VE, comparing it to our once-beloved VMware. If you missed our previous video on XCP-ng, check it out here.

The Proxmox Origin Story

Proxmox was founded in 2005 by Linux developers Dietmar and Martin Maurer. Their company, Proxmox Server Solutions GmbH, has evolved into three main products: Proxmox Virtual Environment (PVE), Proxmox Backup Server, and Proxmox Mail Gateway. Proxmox VE, first released in 2008, aims to provide an easy-to-use, scalable virtualization platform managing both virtual machines and containers in a single system.

Growth and Community Support

Over the years, Proxmox VE has introduced features such as live migration, high availability, and software-defined storage and networking. A robust community supports its development, and Proxmox Server Solutions GmbH offers enterprise support subscriptions, providing professional support and exclusive repository updates.

Proxmox VE vs. VMware ESXi: Key Comparisons

Architecture

  • VMware ESXi: A lightweight, type-1 hypervisor running from RAM with a closed-source kernel.
  • Proxmox VE: Based on Debian Linux, it continues to use disk storage for various operational needs post-boot.

Performance

Both hypervisors offer nearly equivalent performance. However, ESXi’s limits for RAM and hosts per cluster are well-defined, whereas PVE does not specify such limits.

Usability

  • VMware ESXi: Features a basic HTML5 GUI for host and VM management, with extended functionality requiring vCenter.
  • Proxmox VE: Offers a built-in management interface that includes clustering, live migrations, and backup functionality without additional software.

Features

  • VMware ESXi/vCenter: Requires additional licensing for advanced features like distributed resource scheduling, high availability, fault tolerance, and vMotion.
  • Proxmox VE: Provides clustering, live workload migrations, high availability, and workload balancing out of the box for free.

Scalability

  • VMware ESXi: Known for managing thousands of VMs, scaling clusters with vCenter.
  • Proxmox VE: Supports clustering and can manage large collections of hosts and VMs, though less commonly used in very large environments.

Support

  • VMware: Offers extensive professional support, training, certifications, and a well-maintained public knowledge base.
  • Proxmox VE: Relies on community support but also offers professional support services at different levels through Proxmox Server Solutions GmbH.

Cost

  • VMware ESXi: Recently eliminated its free hypervisor, making it less accessible.
  • Proxmox VE: Free and open-source, with professional support available for a fee. Premium support is currently priced at €1020 per socket, per year.

Real-World Examples and Interface Comparisons

Both ESXi and Proxmox VE feature built-in web management GUIs. Let’s compare their consoles and management interfaces:

Console Interfaces

  • ESXi Console: Provides basic host information and minimal management functionality, expecting most tasks to be managed via the web GUI.
  • Proxmox VE Console: Sparse, with most management done via the web GUI, lacking a text-based menu for easier troubleshooting.

Management GUIs

  • VMware ESXi: The web GUI offers a dashboard for host state, usage, and configuration, with detailed tabs for VMs, storage, and networking.
  • Proxmox VE: The GUI is more complex, reminiscent of vCenter, with extensive features for host and VM management, clustering, storage, and network configuration.

Creating Virtual Machines

Creating VMs in Proxmox is straightforward, with a wizard guiding you through the process. Unlike VMware, Proxmox supports various hardware emulation options and has native TPM support for VMs.

Pros and Cons of Proxmox VE

Pros:

  • Comprehensive feature set without additional costs.
  • Built-in clustering, live migration, and high availability.
  • Native VM backup functionality.
  • Supports both VMs and LXC containers.

Cons:

  • Busy and complex UI with steep learning curve.
  • Constant reminders about the lack of a subscription.
  • No built-in menu interface for console management.
  • Complicated upgrade process.
  • Lack of an easy migration path from VMware ESXi.

Conclusion

Proxmox VE presents a viable alternative to VMware ESXi, offering a feature-rich, cost-effective solution for homelab enthusiasts and businesses alike. While there are areas for improvement, its robust capabilities make it worth considering for your virtualization needs.

If you’re exploring different hypervisors and have a specific one in mind, let us know in the comments or join our Discord community!

Watch the Video Here:


Feel free to share your thoughts or ask any questions in the comments below. Until next time, happy homelabbing!