When you install a new Proxmox server, one of the first decisions – and later one of the hardest to change – is the choice of Proxmox storage. The installer offers ext4 with LVM-thin, ZFS and Btrfs, and in a cluster there are also Ceph and external NFS or iSCSI storage. They all work, but they differ in what they can do with snapshots, live migration and high availability (HA), and they need very different hardware.
In this article, based on Proxmox VE 9.2 released on 21 May (Debian 13.5, ZFS 2.4, and by default Ceph 20.2 “Tentacle” on new installations), we compare the most common storage types and show what we recommend for one-, two- and three-server environments.
LVM-thin: the simple default
If you choose ext4 or XFS in the installer, Proxmox creates an LVM-thin storage called local-lvm on the rest of the system disk. The virtual machines’ disks are logical volumes on it, with thin provisioning: a 100 GB virtual disk only takes up as much space as the machine has actually written.
- Advantage: fast, needs little memory and CPU, and supports snapshots.
- Disadvantage: it has no redundancy of its own, so it needs hardware RAID underneath to protect against disk failures. It isn’t shared, and Proxmox’s replication feature doesn’t work with it either.
It is a good choice if you are dealing with a single server that already has a reliable RAID controller with a battery-backed cache.
ZFS: local storage with lots of extras
ZFS is a volume manager and a file system at the same time: it manages the disks itself, protects the data with checksums, compresses, and takes snapshots in an instant. The Proxmox installer can also install directly onto ZFS, as a mirror (RAID 1/10) or in RAID-Z.
The RAID-Z levels are the equivalents of RAID 5 and 6 (we wrote about the basics in our article on RAID levels), but we rarely choose them for virtual machines. According to the Proxmox documentation, in terms of IOPS a RAID-Z vdev behaves roughly like a single disk, whereas mirrored vdevs are better suited to VM workloads. So from four disks we usually build two mirrored pairs (RAID 10).
A few things to watch out for:
- Memory. The ZFS cache, the ARC, lives in RAM. Since Proxmox VE 8.1 the installer limits the ARC to 10% of memory, at most 16 GiB; as a rule of thumb it needs at least 2 GiB plus 1 GiB per terabyte stored. You have to account for this when planning the virtual machines’ memory.
- Hardware. A RAID controller with its own cache doesn’t belong under ZFS; you need an HBA or a controller set to “IT mode”, and the documentation recommends ECC memory.
- SSDs. In our experience, ZFS’s write pattern wears out cheap consumer SSDs quickly, and they are slow with synchronous writes; in servers we use enterprise SSDs with power loss protection (PLP).
ZFS’s biggest advantage in a cluster is storage replication: on a schedule (every 15 minutes by default, but as often as every minute), Proxmox copies the changes to the machines’ disks to a ZFS pool with the same name on another node. This way the machine can be migrated quickly, and HA can also restart it on the other node – losing the data written since the last replication.
Ceph: distributed storage for three or more nodes
Ceph builds a single, shared and replicated storage from the nodes’ local disks (OSDs). By default every piece of data exists in three copies, on three different nodes, so even the loss of a whole server causes no data loss, and HA restarts the machines immediately, from the most recent state.
This comes at a price. According to Proxmox’s Ceph guide:
- you need at least three servers, preferably identical ones, and at least 12 OSDs in total are recommended, evenly distributed,
- Ceph traffic needs a dedicated network of at least 10 Gbit/s, and 25 Gbit/s for high-performance systems,
- at least 8 GiB of memory per OSD is recommended for good performance, and at least one CPU core for each Ceph service,
- here too you need an HBA instead of a RAID controller, and in small systems SSDs for faster recovery.
Because of the threefold replication, only about a third of the raw capacity is usable. With three to five nodes, the machines can also be connected without a switch, in a full-mesh network.
Shared NFS and iSCSI
If you already have a separate storage device (NAS or SAN), Proxmox can also access it over NFS, CIFS or iSCSI. The storage is visible from every node, so live migration and HA are simple.
- NFS: the disks are qcow2 files, and snapshots are possible, but according to the documentation internal qcow2 snapshots are slow on network storage, and on a large disk they can take minutes or even hours.
- iSCSI or Fibre Channel with LVM: block-level, shared storage. Snapshots were missing for a long time; since Proxmox VE 9.0 they are possible in “snapshot as volume chain” mode, but still as a technology preview.
- ZFS over iSCSI: shared and snapshot-capable, but it only works with certain storage systems that run ZFS.
The drawback of external storage is that if it isn’t redundant, it becomes the single point of failure itself: if it goes down, every machine goes down.
What can each Proxmox storage type do?
| LVM-thin | ZFS (local) | Ceph RBD | NFS | iSCSI + LVM | |
|---|---|---|---|---|---|
| Shared | no | no | yes | yes | yes |
| Snapshot | yes | yes | yes | yes (qcow2) | technology preview |
| Live migration | with disk copy, slow | fast with replication | immediate | immediate | immediate |
| HA | not in practice | with replication, with data loss | yes | yes | yes |
| Replication | no | yes | built in (3 copies) | depends on the storage | depends on the storage |
| RAM/CPU requirements | low | medium (ARC) | high | low | low |
| Minimum | 1 node | 1 node | 3 nodes | 1 node + storage | 1 node + storage |
| Typical use | one server with hardware RAID | 1–2 nodes, small cluster | 3+ node HA | existing NAS | existing SAN |
What do we recommend?
One server: a ZFS mirror with two enterprise SSDs, or RAID 10 from four disks if you need more space. If the machine already has good hardware RAID, LVM-thin is fine too. Whichever you choose, keep the backups on a separate machine: RAID and snapshots are not backups.
Two servers: a ZFS pool with the same name on both nodes, storage replication for the important machines, and a QDevice running on a third machine for quorum. This is cheap, but asynchronous: it has to be acceptable that a few minutes of data may be lost in an outage. We described the details in our article on the Proxmox HA cluster. We don’t build Ceph on two nodes.
Three or more servers: if you have the 10/25 Gbit/s network, several enterprise SSDs per node and enough memory, Ceph gives the best availability. If the hardware falls short of that, ZFS replication or redundant external storage is more realistic.
Storage is hard to change later: the machines have to be moved, which means downtime and a lot of data transfer. So it’s worth planning for the needs of the next three to five years from the start. If you are still on Proxmox VE 8, remember that its support is scheduled to end in August 2026; we wrote about the new features of major version 9 when Proxmox VE 9.0 was released.
We help with planning Proxmox servers and clusters, and with choosing and building their storage; you can read more on our Infrastructure and operations page.