The `guest include` logic handling `all` and `exclude` parameters was in
the `PVE::VZDump->exec_backup()` method. Moving this logic into the
`get_included_guests` method allows us to simplify and generalize it.
This helps to make the overall logic easier to test and develop other
features around vzdump backup jobs.
The method now returns a hash with node names as keys mapped to arrays
of VMIDs on these nodes that are included in the vzdump job.
The VZDump API call to create a new backup is adapted to use the new
method to create the list of local VMIDs and the skiplist.
Permission checks are kept where they are to be able to handle missing
permissions according to the current context. The old behavior to die
on a backup job when the user is missing the permission to a guest and
the job is not an 'all' or 'exclude' job is kept.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
As a first step to make the whole guest include logic more testable the
part from the API endpoint has been moved to its own method with as
little changes as possible.
Everything concerning `all` and `exclude` logic is still in the
PVE::VZDump->exec_backup() method.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
this unifies the logic into a single place instead of all over this
module and the plugins.
it also fixes tons of 'uninitialized value' warnings when backing up
with --dumpdir but no --storage set, since the existing conditions for
PBS targets are missing a definedness check.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
This patch adds the zstd to the compression selection for backup on the
GUI and add .zst to the backup file filter. Including zstd as package
install dependency.
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
When creating a backup the log part can make the mail too big to be
transferred. To ensure delivery, two measures are taken:
1. Always omit the status lines
2. Omit the whole log part if a mail becomes (too) big
Additionally, add a check for missing log files.
Co-developed-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
This moves the cfs register code for vzdump.cron to the
pve-guest-common package. Therefore, it relies on the corresponding
patches in pve-guest-common and pve-docs as build dependencies.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
make the message a bit more informative (with help from thomas), namely
mentioning the ability to change/increase the limit.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
This refactors things a bit to avoid having the same two lines in 3
places and allows the plugin to set the "guest was resumed" time
stamp at the point it really was resumed, not only once the backup
completed (see #503). Further, if a plugin prints it's own
"resumed/running after X seconds" message, it can unset the vmstop
time and thus avoid printing the message twice.
related to a fix for #503
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Adding timestamps to the log messages facilitates troubleshooting.
We only log this in the task log, as the syslog and the backup log
(stored together with the backup on the target storage) already have
date/time prefixed, so only the task log missed this info in case of
multiple backups tasks for a single job.
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
we already use POSIX strftime a lot in the stack, so nothing new,
also just use the perl built in localtime
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
while this is unlikely to cause any problems, it is unnecessary to
substract 1900 here - timelocal handles 4-digit years perfectly well.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
besides the cleanup purpose this fixes an actual problem, perls POSIX
module has the following caveat:
`Everything is exported by default (with a handful of exceptions).
This is an unfortunate backwards compatibility feature and its use is
strongly discouraged. You should either prevent the exporting (by
saying "use POSIX ();", as usual) and then use fully qualified names
(e.g. "POSIX::SEEK_END"), or give an explicit import list. If you do
neither and opt for the default (as in "use POSIX;"), you will import
hundreds and hundreds of symbols into your namespace.'
see `perldoc POSIX`
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
we copied our log method over there to resolve a cyclic dependency,
now use it here to reduce code duplication.
As we are below pve-guest-common in the dpendency hierachy we may use
it, but it does not may (or at least should) use us.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
A "sysconf(84)" is not really descripitve. There is no POSIX const
entry for _SC_NPROCESSORS_ONLN (number of processors online) as its
not standard but an adoption of glibc.
Better just use our proven ProcFSTool's cpuinfo method for getting
the desired information.
This is kept only for backwards compatibillity as we guaranteed that
we use #cpus/2 if 'pigz' is set to 1, else I would have removed this
alltogether and just directly passed 'pigz' to the command...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
when a vzdump script is set and the backup fails early (eg.
when exceeding the number of backups) run_hook_script()
showed an uninitialized value error trying to use
$task->{mode} which is set only after prepare() was called.
This sets $task->{mode} early, still updated later to $stop
if !$running, and changes the condition for whether
cleanup() should be called to not use $task->{mode} (which
makes no real sense anyway) to using the $cleanup hash like
the rest of the code.
VZDump->new() dies when a tmpdir or dumpdir is configured
but does not exist. At this point the error is not being
reported via email.
This also moves the instantiation of VZDump into the worker
since new() can now call sendmail() on error.
Additionally rather than only showing a single error if both
tmpdir and dumpdir don't exist, both are included in the
message.
rationale is the following:
we had this as default before commit
1ab98f059e
but wrongly documented
since we set the compression mode explicitly via the gui,
we can set the default to 0, which makes some operations more
intuitive (e.g. with the -stdout parameter it makes no sense
to compress it by default)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Not all document default values matched their actual
default, fix this by not maintaining two separate lists of
default values.
Note that this changes:
- the default compression from none to the documented LZO
- the documented default mode from stop to snapshot
The exclude list is built from both the command line
parameters and the config. If no excludes are passed on the
command line we previously copied the defaults to it,
effectively passing them twice.
Call split_args() on the 'exclude-path' option read from
/etc/vzdump.conf
With the move to the generic JSONSchema::parse_config() in
commit cc61ea36 the call to split_args() for 'exclude-path'
was dropped breaking the 'exclude-path' option.
We're replacing the use of 'find' in the backup code for
various reasons, so we cannot use find-specific options
anymore. Besides, it's up to the plugin to create the
archive and there's no guarantee using 'find' even makes
sense. Even in the current code it's not being used when
hitting the rsync case anyway.
Now using simple glob patterns as that is more widely
supported.
Instead of a lot of hardcoded if's use JSONSchema::parse_config to
parse and validate vzdump.conf. To do that $confdesc was extended
to match a valid schema.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Added an option to disable a backup job while preserving its
settings. When a job is disabled a '#' is added in front of the vzdump
command in the vzdump.cron file. So the cron job still fires,
but only a comment gets executed. If we would comment out the cron job
a bit more changes to the parser would be needed.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Added a vzdump.conf option to controll gzip compression.
When 'pigz' (defaults to 0) is >0, pigz support is enabled.
When the pigz option equals 1 pigz uses #cores/2 threads,
else it spawns N threads. To use it select gzip in the web
interface and set the aproppriate option in /etc/vzdump.conf
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
* do not unlink $pidfile inside stop_running_backups to avoid race conditions
* use pve UPID to save pid (see PVE::Tools::upid_decode)
* allow to pass -stop parameter to nomal backup job
* simple return 'OK' instead of calling exit() inside API call.
* rename stop_all_backups to stop_running_backups
This fix include a new function of vzdump.
Now you can call vzdump -stop and the backup will be aborted.
Also if the pve-manager init script stop the process, vzdump -stop will called.
Signed-off-by: Wolfgang Link <wolfgang@linksystems.org>
The maxfiles flags is now a storage related flag. So we set that on
the storage definition, and removed it from the scheduled backup GUI.
We use '--remove 0' for 'Backup now', so the user need to remove
old backups himself.
