cpKamra

Backup for cPanel servers

cpKamra is a backup system for cPanel servers that runs on your own infrastructure. If a server can also delete its own backups, a break-in can take its backups with it. That is why, in cpKamra, cPanel servers can only append: every account gets its own restic repository, and old backups can only be deleted by the maintenance node running on the storage side.

The other goal is that getting a lost file back should not depend on the sysadmin. Users restore files, folders, mailboxes, databases or DNS zones themselves in cPanel, and the sysadmin restores whole accounts in WHM, even from another server’s backup.

The system was built for hosting providers and anyone running several cPanel servers, but for one or two servers there is also a simpler mode without a controller. We develop cpKamra, and it is free to use. The programs’ technical name is cpbackup (cpbackupd, cpbackupctl).

Architecture

cPanel servers (cpbackupd + cPanel and WHM plugin)
   │  mTLS: job assignment, status, key escrow
   ├──────────────────────────────▶ controller + MariaDB ◀── dashboard (admins)
   │                                       ▲ mTLS
   │  restic, append-only                  │
   ▼                                       │
storage server (rest-server) ◀──── maintenance node: retention, prune, checks
  • Agent (cpbackupd): runs on every cPanel server and backs up the accounts every night. The cPanel and WHM plugins are installed with it, and its command-line tool is cpbackupctl.
  • Controller: runs on a separate machine, with MariaDB. It limits concurrent jobs per storage target so that a large number of servers doesn’t overload the backup target, collects the status of the backups, holds the repository keys in escrow, and runs the dashboard. If it can’t be reached, the agents carry on from their local queue after ten minutes.
  • Storage server: restic rest-server with the --append-only --private-repos flags, so each cPanel server sees only its own repositories, and can only write to those. S3-compatible storage can also be used, with a key that has no delete permission.
  • Maintenance node (cpbackup-maint): runs next to the storage, and it is the only component with delete permission. It does the retention-based cleanup, the prune, the regular restic check verification and the deletion of removed accounts’ backups, and if you want, it also copies every repository to a second target.

Single-server mode. For one or two servers that back up to any SFTP or S3-compatible storage, even a NAS, you don’t need a controller or a maintenance node: the agent does the maintenance itself. The price is that the agent needs delete permission, so the backups have to be protected on the storage side (for example with snapshots or S3 versioning). You can move to the full system later without moving any data.

What it backs up

An account’s backup consists of several restic snapshots:

  • Files: the entire home directory, including the websites and email (with the mailbox passwords).
  • Account metadata: cPanel’s own pkgacct export (settings, database privileges and schemas, email, SSL, cron and FTP settings) and the DNS zones of the domains.
  • Databases: each MySQL/MariaDB database goes into a separate snapshot, as a consistent dump, without temporary files.

The server’s own configuration (/etc, /var/cpanel, /var/named, crontabs) goes into a separate repository, which helps when rebuilding a lost server.

Exclusions can be set with patterns; a recommended list (caches, logs, temporary files) can be loaded in WHM. The system rejects a pattern that would exclude an entire home directory.

On a Linux 5.9 or EL9 kernel, a fanotify-based change tracker watches the home directories, under CageFS too. An unchanged account skips the file backup, but its databases and metadata are still backed up; a full scan runs every week. More frequent database backups can also be scheduled. The default retention is 7 daily and 4 weekly snapshots, adjustable per server and per account.

Restore

In the cPanel plugin (Files → Backup & Restore), users can:

  • restore a file or folder by merging, by overwriting or into a separate folder, or download it as an archive;
  • restore a mailbox, even a deleted one, together with its old password;
  • replace a database with the backed-up version (a safety copy is made first, and the original is put back if something fails), restore it into a new database, or download the SQL dump;
  • compare a DNS zone with the live one and restore it;
  • restore account settings one by one: cron jobs, email forwarders, SSL certificates, subdomains, addon and alias domains;
  • start a backup themselves with the Back up now button, for example before a CMS update (one per day by default).

Users see only their own account’s backups, and files are written back with the user’s own privileges.

In the WHM plugin, root and resellers (the latter for their own accounts) can do the same, and they can also see the queue and the logs. Root can restore an account in full even after it has been deleted from the server. An account can also be brought over from another server: the maintenance node copies the repository into the target server’s area, and the servers still can’t see each other’s backups.

Security

  • mTLS: the cPanel servers and the maintenance node connect with a client certificate issued by the controller’s own certificate authority. The certificates renew themselves and become invalid immediately when revoked.
  • One-time registration token: bound to one server, valid for 24 hours by default, and it contains the fingerprint of the controller’s certificate authority, so a machine in the middle can’t answer on the controller’s behalf.
  • Key escrow: the agent places the repository passwords in escrow, encrypted with the controller’s public key, so the backups can still be opened after the cPanel server is lost. They can only be unlocked on the controller machine, and every unlock is recorded in the audit log.
  • Append-only storage: the cPanel server can’t delete or overwrite a backup. The maintenance node stops automatic deletion if many accounts disappear from a server at once.
  • Unusual backups: if an unusually large share of the files changed in a backup (suspected ransomware), an alert is sent, and you can ask for nothing to be deleted from the account’s backups in the meantime.
  • Dashboard: admin and read-only roles, two-factor authentication, login rate limiting; every action is recorded in the audit log.

Operations

The dashboard, available in Hungarian and English, shows in one place the servers, the last backup of each account, the queue, the storage targets, the backups of removed accounts, the retention rules and the audit log.

An alert is sent to a webhook (Slack- and Mattermost-compatible) or by email if a server doesn’t check in, a backup fails, there has been no good backup for 36 hours, or a repository check finds an error. You get one notification per problem, and one when it is resolved.

The controller shows when the servers or the maintenance nodes are running a different release, but it doesn’t block anything. When upgrading, the controller goes first, then the maintenance nodes, and finally the cPanel servers; on these, the same install.sh that installs also upgrades, and it doesn’t touch the settings or the keys.

The controller (with MariaDB) and the maintenance node can be installed as a Docker image or on a virtual machine, and the agent from a tar.gz package.

Requirements

  • cPanel & WHM with the Jupiter theme, on AlmaLinux, CloudLinux or RHEL 8/9, on an x86_64 or aarch64 processor, with root access.
  • restic 0.19.1 or later (the installer takes care of it).
  • For the controller and the storage server: Linux with Docker Engine and Compose, or a virtual machine.
  • For the change tracker: a Linux 5.9 or EL9 kernel; without it, a full scan runs every night.
  • In single-server mode: any SFTP or S3-compatible storage, even a NAS.