On 22 June 2023, Proxmox VE 8.0 was released, the new major version of the popular open source virtualisation platform. It is based on Debian 12 “Bookworm”, released on 10 June, so as usual Proxmox shipped the new release barely two weeks after Debian. In this article, we sum up what has changed and how best to upgrade an existing Proxmox VE 7.4 environment.
If you are just getting to know the platform, in an earlier article we explained why we think Proxmox VE is the best hypervisor for small businesses.
The main components of Proxmox VE 8.0
| Component | Proxmox VE 7.4 | Proxmox VE 8.0 |
|---|---|---|
| Base system | Debian 11 “Bullseye” | Debian 12 “Bookworm” |
| Kernel (default) | 5.15 | 6.2 |
| QEMU | 7.2 | 8.0.2 |
| LXC | 5.0.2 | 5.0.2 |
| ZFS | 2.1.9 | 2.1.12 |
| Ceph | Quincy 17.2.5 / Pacific 16.2.11 | Quincy 17.2.6 |
What’s new?
The release notes are long; these are the changes that affect most users:
- Ceph Enterprise repository. The Ceph Quincy integrated into Proxmox now also has a stable enterprise repository, which any Proxmox VE subscription gives access to. Until now, Ceph packages only came from the no-subscription and test repositories, which are open to everyone.
- Scheduled directory synchronisation. Users and groups of LDAP and Active Directory realms can now be synchronised automatically at set intervals; there is no need to start it by hand.
- Resource mappings for PCI and USB devices. Passthrough devices can be named and tied to permissions at cluster level. The mapping is valid across nodes too, so virtual machines that use a passthrough device can also be migrated, while shut down, to another node that has the same kind of device.
- Network permissions. The new
SDN.Usepermission controls which bridge or VNet, and even which VLAN, a user can attach their virtual machines to. - New default CPU type. The CPU type of new virtual machines created in the web interface is
x86-64-v2-AESinstead ofkvm64. This also passes through the SSE4.1, SSE4.2 and AES instructions, so systems compiled for the x86-64-v2 level – such as AlmaLinux 9 and RHEL 9 – boot without any extra settings, and encryption is much faster too. We wrote about the release of AlmaLinux 9 on the blog of our hosting brand, aWh. The settings of existing machines do not change. - Text-based installer. Besides the graphical mode, the installer ISO now also has a text (TUI) mode, which helps on very new or very old hardware if the graphical installer does not start.
- Two-factor login protection. After too many failed second-factor attempts, the account is locked, and unlocking it needs a recovery key or an administrator.
What to watch out for before upgrading
The “Known Issues & Breaking Changes” section of the release notes is required reading. A few points that are easy to run into:
- Browser. The web interface needs at least Firefox 79, Chrome 85 or Safari 14.
- Storage directories. If you pointed several content types to the same directory on a directory storage (
content-dirsor a symbolic link), activation fails with an error. Themkdiroption is deprecated; usecreate-base-pathandcreate-subdirsinstead. - Permissions. The
Permissions.Modifyprivilege has been removed from thePVEAdminandPVESysAdminroles. If you relied on it, you need a custom role. - Containers. The
lxc.id_mapkey now counts as an error (uselxc.idmapinstead), and because of the new version of lxcfs, every container has to be restarted after the upgrade. - Third-party storage plugins: if you use one, wait until the developer releases a version compatible with Proxmox VE 8.
Upgrading from Proxmox VE 7.4 to 8.0
Proxmox has published a detailed upgrade guide. Here is the gist:
Prerequisites:
- every node runs the latest packages of the 7.4 branch,
- if you use Ceph, it is already on Quincy (17.2) – if you are on Pacific, upgrade that first,
- a fresh, tested backup of every virtual machine and container (for example, with Proxmox Backup Server),
- at least 5 GB of free space on the root file system,
- console access (IPMI, iKVM or physical) in case the network does not come up.
The steps on one node:
# 1. Checker script (does not change anything)
pve7to8 --full
# 2. Update the 7.4 packages
apt update && apt dist-upgrade
pveversion
# 3. Switch the Debian repositories to Bookworm
sed -i 's/bullseye/bookworm/g' /etc/apt/sources.list
# 4. Proxmox repository (with a subscription)
echo "deb https://enterprise.proxmox.com/debian/pve bookworm pve-enterprise" \
> /etc/apt/sources.list.d/pve-enterprise.list
# or without a subscription:
# echo "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription" \
# > /etc/apt/sources.list.d/pve-no-subscription.list
# 5. The upgrade
apt update && apt dist-upgrade
# 6. Check again, then reboot
pve7to8 --full
reboot
Look through the other files in /etc/apt/sources.list.d/ too, and change or disable the lines pointing to Bullseye there as well; if you use Ceph, the Ceph repository also has to be switched to Bookworm. When the upgrade asks questions (about modified configuration files), read which file it is about; for /etc/issue, you can safely keep the existing one (this is the default “No” answer), because Proxmox regenerates it at boot anyway.
In a cluster, upgrade one node at a time. Migrate the virtual machines off it first; migrating from an older version to a newer one works, but the other way round is not guaranteed. Only move on to the next node once the previous one has rebooted and everything is in order.
After the reboot, clear your browser cache, because the old web interface’s JavaScript files can cause strange errors.
Network. Because of the new kernel, the names of network cards may change (for example, enp3s0f0 instead of eno1). This is why you need console access: if the node does not come up on the network, you have to correct the names to the new ones in /etc/network/interfaces.
Upgrade now, or wait?
In line with Debian 11 security support, Proxmox VE 7 gets updates until July 2024, so there is no need to rush. This is what we recommend:
- For a new installation, start with 8.0; there is no point putting a machine you are setting up now on the 7 branch.
- A test environment or a less critical node is fine to upgrade now, so you get to know the changes.
- For production, multi-node clusters, it is worth waiting a few weeks for the first point releases, and meanwhile running through the upgrade in the test environment – especially if you use Ceph, passthrough devices or a third-party storage plugin.
- On old hardware (roughly machines older than ten years), Proxmox itself notes that the new kernel is less tested there, so test the upgrade especially thoroughly on such machines.
If you are also planning new disks or a new server when upgrading to 8.0, our article on RAID levels sums up which storage layout is good for what, ZFS included.
We help with installing, upgrading and operating Proxmox VE clusters; you can read more about this on the Infrastructure and operations page.