Commit Graph

87 Commits

Author SHA1 Message Date
Wolfgang Bumiller
ece740308f support for encrypted pbs backups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-10 14:23:17 +02:00
Stefan Reiter
fb9f512c19 backup: detect PBS features and use only supported
If 'query-proxmox-support' is not known to QEMU, assume that no other
features are supported either.

If 'pbs' is not supported at all, error out with a nice message.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 13:44:22 +02:00
Thomas Lamprecht
91c9f3b062 vzdump: log: small fixups
reuse can also come from the current backup - so drop the "from last
backup" as this can be very confusing if one reads it after making
the first backup ever, with no last backup existing.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-07 17:32:27 +02:00
Thomas Lamprecht
2098f2ff87 vzdump: fix variable redeclaration warning
happened due to moving the code from another scope which had no $res,
and not noticing as it was still working after all.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-07 08:41:30 +02:00
Thomas Lamprecht
b4be9c0207 backup: PBS: save backup task size
normally this is done centrally in the managers code, but we do not
have the info for PBS there.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 21:59:50 +02:00
Thomas Lamprecht
6cdb568c33 backup: query status: use latest up-to-date info when finished
we used the info from the previous loop for some summary logs, fix
that.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 21:59:50 +02:00
Thomas Lamprecht
4d159c24d9 backup: restructure log: use human-readable sizes and include PBS info
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 21:59:50 +02:00
Thomas Lamprecht
bbdf1ea5c8 backup: rename confusing incremental param to use-dirty-bitmap
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 20:33:13 +02:00
Stefan Reiter
69e6289470 enable dirty-bitmap incremental backups for PBS
The $total != $transferred check is changed to a log, as QEMU reports
only actually transferred bytes, and it is indeed correct for
incremental backups to have differing values from $total.

The 'incremental' parameter is always set, QEMU will figure out if it should
re-use an existing bitmap or create a new one on its own.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-29 13:15:47 +02:00
Dietmar Maurer
f57666e949 avoid backup command timeout with pbs 2020-06-26 09:30:31 +02:00
Thomas Lamprecht
5a92276e2d trivial whitespace followup fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-24 10:59:05 +02:00
Aaron Lauterer
185df962a5 vzdump: move include logic for volumes to method
Move the logic which volumes are included in the backup job to its own
method and adapt the VZDump code accordingly. This makes it possible to
develop other features around backup jobs.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-06-24 10:56:57 +02:00
Dominik Csapak
77b24c6267 fix pbs backups of templates/vms without disks
fingerprint is now given with an environment variable not as parameter

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-03 14:36:31 +02:00
Fabian Grünbichler
690748632b vzdump: use new 'pbs' option
instead of storage config to determine whether we are in 'PBS mode'

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-05-06 16:20:46 +02:00
Fabian Grünbichler
6bb12239cc vzdump: fix template backup to stdout
redirecting to the saved STDOUT in case of a template backup or a VM
without any disks failed because of the erroneous '=':

Backup of VM 123123 failed - command '/usr/bin/vma create -v -c [...]' failed:
Bad filehandle: =5 at /usr/share/perl/5.28/IPC/Open3.pm line 58.

https://forum.proxmox.com/threads/vzdump-to-stdout.69364

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-05-06 14:23:04 +02:00
Thomas Lamprecht
1bd6fe132f vzdump: assemble: fix skipping all pending and snapshot config entries
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-04 17:29:09 +02:00
Thomas Lamprecht
cb521f2d88 backup: never try to freeze in stop mode backup
VM was can be true for stop mode backup, we cannot check the "is VM
currently running" as that doesn't tells us anything (could be the
backup process), so check the mode also..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-03 11:13:38 +02:00
Fabian Ebner
912792e245 Switch to using foreach_volume instead of foreach_drive
It was necessary to move foreach_volid back to QemuServer.pm

In VZDump/QemuServer.pm and QemuMigrate.pm the dependency on
QemuConfig.pm was already there, just the explicit "use" was missing.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-04-08 22:11:54 +02:00
Fabian Grünbichler
0c498cca36 vm_start: condense signature
as preparation for refactoring it further. remote migration will add
another 1-2 parameters, and it is already unwieldly enough as it is.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-04-01 17:42:15 +02:00
Thomas Lamprecht
e485854100 vzdump: fix up PBS plugin
fixes commit 0b2f574b4c

enforce_vm_running_for_backup is now witout return value, for the PBS
I forgot to remove an now outdated call to handle_vm_powerstate, drop
that.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-26 08:51:29 +01:00
Thomas Lamprecht
e8705fc59e vzdump archive_vma: move getting QMPClient nearer to using sites
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:48:41 +01:00
Thomas Lamprecht
d11e91d2de vzdump: indentation and code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:46:49 +01:00
Thomas Lamprecht
0b2f574b4c vzdump: factor out VM powerstate handling and qmp backup cancel
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:41:43 +01:00
Thomas Lamprecht
1ece829a39 vzdump: factor out QGA FS freeze/thaw
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:37:20 +01:00
Thomas Lamprecht
02da0c65bb vzdump: factor out fork_compressor_pipe
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:34:31 +01:00
Thomas Lamprecht
81dcd4799f vzdump: factor out _get_task_devlist
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:32:00 +01:00
Thomas Lamprecht
09eb196b9e vzdump: refactor and cleanup code in query_backup_status_loop
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 18:26:34 +01:00
Thomas Lamprecht
f301bc0d34 vzdump: use QemuConfig directly to set/unset locks
no need to fork/exec a process for that..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 15:58:43 +01:00
Thomas Lamprecht
cd544fb2b9 vzdump: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 15:58:13 +01:00
Thomas Lamprecht
a113a58a64 vzdump prepare: refactor and code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-19 11:48:19 +01:00
Dietmar Maurer
f6168f1a6a archive_pbs: call backup_cancel only when job is really started 2020-03-11 11:47:24 +01:00
Dietmar Maurer
d2cc2cbe69 archive_vma: call backup_cancel only when job is really started 2020-03-11 11:47:24 +01:00
Dietmar Maurer
c5983223b0 add support for proxmox backup 2020-03-11 11:47:24 +01:00
Stefan Reiter
1249d579ea vzdump: exclude efidisks from backups of non-OVMF machines
Machines running with SeaBIOS don't have the efidisk attached, so QEMU
cannot back it up and fails with "unknown drive".

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-02-17 15:47:04 +01:00
Thomas Lamprecht
82b25dbc0c vzdump: log QGA FS freeze/thaw try in task log
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-27 20:20:30 +01:00
Stefan Reiter
3392d6cacf refactor: extract QEMU machine related helpers to package
...PVE::QemuServer::Machine.

qemu_machine_feature_enabled is exported since it has a *lot* of users
in PVE::QemuServer and a long enough name as it is.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-11-20 16:29:23 +01:00
Stefan Reiter
0a13e08ec2 refactor: create QemuServer::Monitor for high-level QMP access
QMP and monitor helpers are moved from QemuServer.pm.

By using only vm_running_locally instead of check_running, a cyclic
dependency to QemuConfig is avoided. This also means that the $nocheck
parameter serves no more purpose, and has thus been removed along with
vm_mon_cmd_nocheck.

Care has been taken to avoid errors resulting from this, and
occasionally a manual check for a VM's existance inserted on the
callsite.

Methods have been renamed to avoid redundant naming:
* vm_qmp_command -> qmp_cmd
* vm_mon_cmd -> mon_cmd
* vm_human_monitor_command -> hmp_cmd

mon_cmd is exported since it has many users. This patch also changes all
non-package users of vm_qmp_command to use the mon_cmd helper. Includes
mocking for tests.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-11-20 16:29:23 +01:00
Thomas Lamprecht
317c55c2d1 backup: is IOThread comaptible: only check running VMs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-23 11:28:51 +02:00
Thomas Lamprecht
48343b3f1d add runs_at_least_qemu_version to check if we can backup IOThread disks
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-23 10:47:45 +02:00
Thomas Lamprecht
d610b14591 [no-change] sort and group module use
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-23 10:36:46 +02:00
Thomas Lamprecht
8266bc59db Revert "fix #1071: VMs with IOThread enabled disks can now be backed up"
This reverts commit 6b4b369fe3.
2019-10-23 09:31:51 +02:00
Thomas Lamprecht
6b4b369fe3 fix #1071: VMs with IOThread enabled disks can now be backed up
Thanks to Dietmars patch[0] those VMs can now be backed up
successfully, so remove this aborting check.

[0]: https://git.proxmox.com/?p=pve-qemu.git;a=commit;h=69cb18950a705b54f438f4659b603b3f52901c2f

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-By: Dominik Csapak <d.csapak@proxmox.com>
2019-10-22 11:53:58 +02:00
Aaron Lauterer
a022e3fdab tree-wide trailing whitespace cleanup
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2019-09-25 16:55:53 +02:00
Thomas Lamprecht
96f00fd866 fix #503: print correct offline time for stop mode backups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-15 16:28:15 +02:00
Wolfgang Bumiller
f0f30448f9 fix #1779: vzdump: ensure guest-fsfreeze-thaw is called on error
as QMPClient's queue_execute can throw an error

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-05-23 11:18:28 +02:00
Dominik Csapak
a2fab11a46 fix #1570: fix template backup with pigz
when using pigz the resulting commandline would look like:

pigz -p 4>file

which resulted in pigz erroring out because it got no parameter for -p
(because the shell interpreted the 4>file as a file descriptor)

this patch adds a space so that the resulting line is
pigz -p 4 > file

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2017-12-06 14:54:00 +01:00
Dominik Csapak
1a988fd2e8 check if ga runs before a fsfreeze-freeze/thaw
since the guest-fsfreeze-freeze command has a timeout of 1 hour,
we want to check if the guest-agent even runs before executing that,
or else we wait 1 hour and then continue

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2017-11-10 11:20:55 +01:00
Emmanuel Kasper
76312b862b add description of read/writes statistics in vzdump output
since this statistics are note really described, sometimes users
miss that their slow backups are caused by slow reads
2017-05-03 09:14:50 +02:00
Emmanuel Kasper
0db93c2d8a Display volume size in log when doing a volume backup
Output before this patch
INFO: include disk 'scsi0' 'file=pve4tank:vm-402-disk-1'

Output after this patch:
INFO: include disk 'scsi0' 'file=pve4tank:vm-402-disk-1' 64G

we're mainly intersted by the volume size here, it was requested in #351
2016-12-22 10:25:13 +01:00
Stefan Priebe
e281273810 VZDump/QemuServer: set bless clas correctly
Signed-off-by: Stefan Priebe <s.priebe@profihost.ag>
2016-11-17 09:02:47 +01:00